Skip to main content
Realsee
Dashboard
Sign out
Documentation
Docs
/
Five SDK

Tagging

Place interactive tags in a Realsee 3D space with Five SDK in React function components and Hooks.

Recap of the previous chapter: Points in 3D Space

You learned about the Five SDK's event system and built a small app that reads the 3D position of a point from a click event.

In this chapter you will learn

How to place tags in 3D space.

Getting Ready

Create a new directory (src/5.tagging) along with its html file and jsx or tsx file. Carrying over the State code from the previous chapter would be too cumbersome, so we will build on the content of the Displaying a 3D Space chapter instead.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" href="data:;base64,iVBORw0KGgo=" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Tagging</title>
    <style>
      * {
        margin: 0;
        padding: 0;
      }
      html,
      body #app {
        width: 100%;
        height: 100%;
        overflow: hidden;
      }
    </style>
  </head>
  <body>
    <div id="app"></div>
    <script type="module" src="./index"></script>
  </body>
</html>
import { useState, useEffect } from "react";
import { parseWork } from "@realsee/five";

/**
 * React Hook: fetch the work object from a work.json URL
 * @param url the data URL of work.json
 * @returns the work object, or null while it is still loading
 */
function useFetchWork(url) {
  const [work, setWork] = useState(null);
  useEffect(() => {
    setWork(null);
    fetch(url)
      .then((response) => response.text())
      .then((text) => setWork(parseWork(text)));
  }, [url]);
  return work;
}

export { useFetchWork };
import { useState, useEffect } from "react";

/**
 * Get the current window dimensions
 */
function getWindowDimensions() {
  return { width: window.innerWidth, height: window.innerHeight };
}

/**
 * React Hook: get the current window dimensions
 */
function useWindowDimensions() {
  const [size, setSize] = useState(getWindowDimensions);
  useEffect(() => {
    const listener = () => setSize(getWindowDimensions());
    window.addEventListener("resize", listener, false);
    return () => window.removeEventListener("resize", listener, false);
  });
  return size;
}

export { useWindowDimensions };
import React from "react";
import { Five } from "@realsee/five";
import { useFiveCurrentState } from "@realsee/five/react";
import BottomNavigation from "@mui/material/BottomNavigation";
import BottomNavigationAction from "@mui/material/BottomNavigationAction";
import Paper from "@mui/material/Paper";
import DirectionsWalkIcon from "@mui/icons-material/DirectionsWalk";
import ViewInArIcon from "@mui/icons-material/ViewInAr";

/**
 * React Component: mode control
 */
const ModeController = () => {
  const [state, setState] = useFiveCurrentState();
  return (
    <Paper sx={{ position: "fixed", bottom: 0, left: 0, right: 0 }}>
      <BottomNavigation
        showLabels
        value={state.mode}
        onChange={(_, newValue) => {
          setState({ mode: newValue });
        }}
      >
        <BottomNavigationAction
          label="Panorama Roaming"
          icon={<DirectionsWalkIcon />}
          value={Five.Mode.Panorama}
        />
        <BottomNavigationAction
          label="Floorplan Overview"
          icon={<ViewInArIcon />}
          value={Five.Mode.Floorplan}
        />
      </BottomNavigation>
    </Paper>
  );
};

export { ModeController };
import React from "react";
import { createFiveProvider, FiveCanvas } from "@realsee/five/react";
import { useFetchWork } from "./useFetchWork";
import { useWindowDimensions } from "./useWindowDimensions";
import { ModeController } from "./ModeController";

/** the data URL of work.json */
const workURL =
  "https://vr-public.realsee-cdn.cn/release/static/image/release/five/work-sample/07bdc58f413bc5494f05c7cbb5cbdce4/work.json";

const FiveProvider = createFiveProvider();
const App = () => {
  const work = useFetchWork(workURL);
  const size = useWindowDimensions();
  return (
    work && (
      <FiveProvider initialWork={work}>
        <FiveCanvas {...size} />
        <ModeController />
      </FiveProvider>
    )
  );
};

export { App };
import React from "react";
import ReactDOM from "react-dom";
import { App } from "./App";

ReactDOM.render(<App />, document.querySelector("#app"));

export {};

Start the dev server with npm run dev, then navigate to the page at "http://localhost:3000/src/5.tagging/index.html".

Info

Check your console: the port number may change depending on your configuration and whichever ports are currently in use, so always trust the value printed in the console. If you use a different build tool, start the server according to that tool's requirements.

Building the Tagging Feature

Adding Tag Styles

Add the tag styles to the html file.

The styles are not strictly required; they just make the tags look nicer.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" href="data:;base64,iVBORw0KGgo=" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Tagging</title>
    <style>
      * {
        margin: 0;
        padding: 0;
      }
      html,
      body #app {
        width: 100%;
        height: 100%;
        overflow: hidden;
      }
      /* highlight-start */
      .tag {
        position: absolute;
        width: 0;
        height: 0;
        transform: translateZ(0);
      }
      .tag-pannel {
        position: absolute;
        width: 100px;
        min-height: 20px;
        transform: translate(-50%, 0);
        left: 50%;
        bottom: 10px;
        background: #333;
        color: #fff;
        border-radius: 2px;
        text-align: center;
        line-height: 20px;
        padding: 8px;
        font-size: 14px;
      }
      .tag-pannel:after {
        content: "";
        display: block;
        position: absolute;
        width: 10px;
        height: 10px;
        left: 50%;
        bottom: -5px;
        transform: translate(-50%, 0) rotate(45deg);
        background: #333;
        pointer-events: none;
      }
      /* highlight-end */
    </style>
  </head>
  <body>
    <div id="app"></div>
    <script type="module" src="./index"></script>
  </body>
</html>

About useFiveProject2d

This chapter uses the `useFiveProject2d` method. It maps a 3D coordinate to a 2D screen coordinate.

useFiveProject2d(vector: THREE.Vector3, testModel: boolean): THREE.Vector2 | null

  1. Pass in a 3D coordinate to get back a 2D screen coordinate, with the origin at the top-left and the unit in pixels. You can use it like { left: returnValue.x + "px", top: returnValue.y + "px" }.
  2. If the 3D coordinate cannot be projected onto the screen (for example, it is behind the camera or occluded), it returns null.
  3. The second parameter, testModel, controls whether model collision is taken into account, i.e. whether a coordinate occluded by the model returns null.

Writing the TaggingController

  1. Add a TaggingController file for the component.
  2. Store the tag positions and text in the tags React state.
  3. Store the tag currently being created in the newTag React state.
  4. Listen for Five SDK's intersectionOnModelUpdate event to keep the tag being created positioned under the cursor.
  5. Use the project2d method produced by useFiveProject2d (inside the tagElement method) to obtain the canvas screen coordinate of each tag, then render it by adjusting its style.
import React, { useState, useCallback } from "react";
import { useFiveEventCallback, useFiveProject2d } from "@realsee/five/react";
import Button from "@mui/material/Button";
import Paper from "@mui/material/Paper";

/**
 * React Component: tagging
 */
const TaggingController = () => {
  const project2d = useFiveProject2d();

  const [tags, setTags] = useState([]);
  const [newTag, setNewTag] = useState(null);

  const tagElement = useCallback((tag, key) => {
    const position = tag.position && project2d(tag.position, true);
    const style = position
      ? { left: position.x, top: position.y }
      : { display: "none" };
    return (
      <div className="tag" style={style} key={key}>
        <div className="tag-pannel">
          <span className="tag-content">{tag.label}</span>
        </div>
      </div>
    );
  }, []);

  const addTag = useCallback(() => {
    setNewTag({ label: window.prompt("Add a tag", "") || "Untitled" });
  }, []);

  useFiveEventCallback(
    "intersectionOnModelUpdate",
    (intersect) => {
      if (newTag) setNewTag({ position: intersect.point, label: newTag.label });
    },
    [newTag]
  );

  useFiveEventCallback(
    "wantsTapGesture",
    (raycaster) => {
      if (newTag && newTag.position) {
        setTags((tags) => tags.concat(newTag));
        setNewTag(null);
        return false;
      }
    },
    [newTag]
  );

  return (
    <React.Fragment>
      <Paper sx={{ position: "fixed", top: 10, left: 10 }}>
        <Button onClick={addTag}>Add Tag</Button>
      </Paper>
      {newTag && tagElement(newTag)}
      {tags.map((tag, index) => tagElement(tag, index))}
    </React.Fragment>
  );
};

export { TaggingController };

Using the Tagging Component

Insert it into the FiveProvider in the App file.

import React from "react";
import { createFiveProvider, FiveCanvas } from "@realsee/five/react";
import { useFetchWork } from "./useFetchWork";
import { useWindowDimensions } from "./useWindowDimensions";
import { ModeController } from "./ModeController";
// highlight-start
import { TaggingController } from "./TaggingController";
// highlight-end

/** the data URL of work.json */
const workURL =
  "https://vr-public.realsee-cdn.cn/release/static/image/release/five/work-sample/07bdc58f413bc5494f05c7cbb5cbdce4/work.json";

const FiveProvider = createFiveProvider();
const App = () => {
  const work = useFetchWork(workURL);
  const size = useWindowDimensions();
  return (
    work && (
      <FiveProvider initialWork={work}>
        <FiveCanvas {...size} />
        <ModeController />
        // highlight-start
        <TaggingController />
        // highlight-end
      </FiveProvider>
    )
  );
};

export { App };

Go back to your browser, and you will see an "Add Tag" button in the top-left corner of the page. Click it, enter a tag name, then move your mouse, click where you want the tag, and the tag is placed.

A handy little feature indeed 🥳.