Skip to main content
Realsee
Dashboard
Sign out
Documentation
Docs
/
Five SDK

Recording State

Record and replay Five SDK state changes in React function components and Hooks.

Recap of the previous chapter: Changing the Viewpoint

- You learned what State is, and how to read and modify it.

- You implemented an automatic look-around feature using State.

In this chapter you will learn

  • How to record user actions through State.
  • How to restore the user's view through State.

Preparation

As in the previous chapter, we create a new directory (src/3.recording-state) along with the corresponding html file and the jsx or tsx files.

You can start the jsx or tsx files by copying the contents from the previous chapter.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" href="data:;base64,iVBORw0KGgo=" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Recording state</title>
    <style>
      * {
        margin: 0;
        padding: 0;
      }
      html,
      body #app {
        width: 100%;
        height: 100%;
        overflow: hidden;
      }
    </style>
  </head>
  <body>
    <div id="app"></div>
    <script type="module" src="./index"></script>
  </body>
</html>
import { useState, useEffect } from "react";
import { parseWork } from "@realsee/five";

/**
 * React Hook: fetch a work object from the address of work.json
 * @param url the data address of work.json
 * @returns the work object, or null while it is still being fetched
 */
function useFetchWork(url) {
  const [work, setWork] = useState(null);
  useEffect(() => {
    setWork(null);
    fetch(url)
      .then((response) => response.text())
      .then((text) => setWork(parseWork(text)));
  }, [url]);
  return work;
}

export { useFetchWork };
import { useState, useEffect } from "react";

/**
 * Get the current window dimensions
 */
function getWindowDimensions() {
  return { width: window.innerWidth, height: window.innerHeight };
}

/**
 * React Hook: get the current window dimensions
 */
function useWindowDimensions() {
  const [size, setSize] = useState(getWindowDimensions);
  useEffect(() => {
    const listener = () => setSize(getWindowDimensions());
    window.addEventListener("resize", listener, false);
    return () => window.removeEventListener("resize", listener, false);
  });
  return size;
}

export { useWindowDimensions };
import React from "react";
import { Five } from "@realsee/five";
import { useFiveCurrentState } from "@realsee/five/react";
import BottomNavigation from "@mui/material/BottomNavigation";
import BottomNavigationAction from "@mui/material/BottomNavigationAction";
import Paper from "@mui/material/Paper";
import DirectionsWalkIcon from "@mui/icons-material/DirectionsWalk";
import ViewInArIcon from "@mui/icons-material/ViewInAr";

/**
 * React Component: mode controller
 */
const ModeController = () => {
  const [state, setState] = useFiveCurrentState();
  return (
    <Paper sx={{ position: "fixed", bottom: 0, left: 0, right: 0 }}>
      <BottomNavigation
        showLabels
        value={state.mode}
        onChange={(_, newValue) => {
          setState({ mode: newValue });
        }}
      >
        <BottomNavigationAction
          label="Panorama Walkthrough"
          icon={<DirectionsWalkIcon />}
          value={Five.Mode.Panorama}
        />
        <BottomNavigationAction
          label="Space Overview"
          icon={<ViewInArIcon />}
          value={Five.Mode.Floorplan}
        />
      </BottomNavigation>
    </Paper>
  );
};

export { ModeController };
import React, { useState, useEffect } from "react";
import { useFiveCurrentState } from "@realsee/five/react";
import IconButton from "@mui/material/IconButton";
import Paper from "@mui/material/Paper";
import FlipCameraAndroidIcon from "@mui/icons-material/FlipCameraAndroid";
import PauseIcon from "@mui/icons-material/Pause";

/**
 * ReactComponent: automatic look-around button
 */
const LookAroundController = () => {
  const [currentState, setState] = useFiveCurrentState();
  const [active, toggleActive] = useState(false);
  useEffect(() => {
    if (active) {
      const timer = window.setInterval(() => {
        setState((prevState) => {
          return { longitude: prevState.longitude + Math.PI / 360 };
        });
      }, 16);
      return () => window.clearInterval(timer);
    }
  }, [active]);
  return (
    <Paper sx={{ position: "fixed", top: 10, right: 10 }}>
      {active ? (
        <IconButton onClick={() => toggleActive(false)}>
          <PauseIcon />
        </IconButton>
      ) : (
        <IconButton onClick={() => toggleActive(true)}>
          <FlipCameraAndroidIcon />
        </IconButton>
      )}
    </Paper>
  );
};

export { LookAroundController };
import React from "react";
import { createFiveProvider, FiveCanvas } from "@realsee/five/react";
import { useFetchWork } from "./useFetchWork";
import { useWindowDimensions } from "./useWindowDimensions";
import { ModeController } from "./ModeController";
import { LookAroundController } from "./LookAroundController";

/** The data URL of work.json */
const workURL =
  "https://vr-public.realsee-cdn.cn/release/static/image/release/five/work-sample/07bdc58f413bc5494f05c7cbb5cbdce4/work.json";

const FiveProvider = createFiveProvider();
const App = () => {
  const work = useFetchWork(workURL);
  const size = useWindowDimensions();
  return (
    work && (
      <FiveProvider initialWork={work}>
        <FiveCanvas {...size} />
        <ModeController />
        <LookAroundController />
      </FiveProvider>
    )
  );
};

export { App };
import React from "react";
import ReactDOM from "react-dom";
import { App } from "./App";

ReactDOM.render(<App />, document.querySelector("#app"));

export {};

Start the dev server with npm run dev, then navigate to the current page at "http://localhost:3000/src/3.recording-state/index.html".

Info

Please check your console. The port number may change depending on your configuration and on which ports are currently in use, so rely on what the console prints. If you are using a different build tool, start the server according to that tool's requirements.

Recording / Playback

In this chapter we continue to build an interesting application using State.

Specifically, we will create an application that records the State changes that occur on the page and is able to play those actions back.

Writing the Recorder class

First, we need to write a Recorder class to support recording and playback. The Recorder class is not part of Five SDK; it is written purely to achieve the effect of this chapter.

  1. Implement the startRecording / endRecording methods, used to start and stop recording.
  2. Implement the record(state: State) method to capture the recorded content. It records everything that happens between startRecording and endRecording.
  3. Implement the play(callback) method for playback. After calling play, it will replay the recorded content by invoking the callback method for each entry in turn, playing back the State.
/**
 * Recorder class
 */
class Recorder {
  constructor() {
    this.startTime = 0;
    this.records = null;
  }
  /**
   * Whether anything has been recorded
   */
  hasRecords() {
    return this.records !== null;
  }
  /**
   * Record a keyframe
   * @param state the state of five
   * @returns
   */
  record(state) {
    if (this.records === null) return;
    this.records.push({
      state: Object.assign({}, state),
      time: Date.now() - this.startTime,
    });
  }
  /**
   * Start recording
   */
  startRecording() {
    this.startTime = Date.now();
    this.records = [];
  }
  /**
   * Stop recording
   */
  endRecording() {
    this.startTime = 0;
  }
  /**
   * Play back the recording
   * @param callback keyframe callback
   * @returns whether there is currently a recording
   */
  play(callback) {
    if (this.records === null || this.records.length === 0) return false;
    const records = this.records.slice();
    const keyframe = (keyIndex) => {
      const current = records[keyIndex];
      const next = records[keyIndex + 1];
      callback(current.state, next === undefined);
      if (next) {
        const delay = next.time - current.time;
        setTimeout(() => keyframe(keyIndex + 1), delay);
      }
    };
    keyframe(0);
    return true;
  }
}

export { Recorder };

Writing the recording component

We wrap the Recorder class into a React component.

  1. Add a RecorderController file to write the component.
  2. The component holds two React states, recording and playing, representing the "recording" and "playing back" statuses respectively.
  3. `useFiveEventCallback` lets you subscribe to Five SDK's built-in event callbacks.

Here we listen for the stateChange event, which fires whenever the state changes; we then record that State by calling recorder.record(state).

For more details on the events, see the Five SDK event list.

  1. When the playback button is pressed, calling recorder.play(callback) invokes the callback method for each previously recorded state one by one, replaying the records.
  2. Apply each record by calling Five SDK's setState method so that the recorded playback content takes effect and the view changes accordingly.
import React, { useState, useCallback } from "react";
import { useFiveEventCallback, useFiveState } from "@realsee/five/react";
import Button from "@mui/material/Button";
import IconButton from "@mui/material/IconButton";
import Paper from "@mui/material/Paper";
import FiberManualRecordIcon from "@mui/icons-material/FiberManualRecord";
import StopIcon from "@mui/icons-material/Stop";
import PlayArrowIcon from "@mui/icons-material/PlayArrow";
import { Recorder } from "./recorder";

/**
 * ReactComponent: state recording
 */
const RecorderController = () => {
  const [state, setState] = useFiveState();
  const [recording, toggleRecording] = useState(false);
  const [playing, togglePlaying] = useState(false);
  const [recorder] = useState(() => new Recorder());
  const startRecording = useCallback(() => {
    recorder.startRecording();
    toggleRecording(true);
  }, [recorder]);
  const endRecording = useCallback(() => {
    recorder.endRecording();
    toggleRecording(false);
  }, [recorder]);
  const play = useCallback(() => {
    const hasRecord = recorder.play((state, isFinal) => {
      setState(state);
      togglePlaying(!isFinal);
    });
    togglePlaying(hasRecord);
  }, []);
  useFiveEventCallback("stateChange", (state) => {
    recorder.record(state);
  });
  if (recording) {
    return (
      <Paper sx={{ position: "fixed", top: 10, left: 10 }}>
        <IconButton onClick={endRecording}>
          <StopIcon />
        </IconButton>
        <Button disabled>Recording</Button>
      </Paper>
    );
  }
  if (playing) {
    return (
      <Paper sx={{ position: "fixed", top: 10, left: 10 }}>
        <Button disabled>Playing back</Button>
      </Paper>
    );
  }
  return (
    <Paper sx={{ position: "fixed", top: 10, left: 10 }}>
      <IconButton onClick={startRecording}>
        <FiberManualRecordIcon />
      </IconButton>
      <IconButton onClick={play}>
        <PlayArrowIcon />
      </IconButton>
    </Paper>
  );
};

export { RecorderController };

Using the state recording component

Insert it into the FiveProvider in the App file.

import React from "react";
import { createFiveProvider, FiveCanvas } from "@realsee/five/react";
import { useFetchWork } from "./useFetchWork";
import { useWindowDimensions } from "./useWindowDimensions";
import { ModeController } from "./ModeController";
import { LookAroundController } from "./LookAroundController";
// highlight-start
import { RecorderController } from "./RecorderController";
// highlight-end

/** The data URL of work.json */
const workURL =
  "https://vr-public.realsee-cdn.cn/release/static/image/release/five/work-sample/07bdc58f413bc5494f05c7cbb5cbdce4/work.json";

const FiveProvider = createFiveProvider();
const App = () => {
  const work = useFetchWork(workURL);
  const size = useWindowDimensions();
  return (
    work && (
      <FiveProvider initialWork={work}>
        <FiveCanvas {...size} />
        <ModeController />
        <LookAroundController />
        // highlight-start
        <RecorderController />
        // highlight-end
      </FiveProvider>
    )
  );
};

export { App };

Go back to your browser and you will see a record button and a play button appear in the top-left corner of the page. Try it out and check whether it behaves as expected.

Nicely done. You can already write such a sophisticated program 🥳.

What you will learn in the next chapter

In the next chapter we will start working with models in 3D space.

  • Learn about the Five SDK's gesture interaction system.
  • Obtain the 3D position of a point.