Recording State — Vanilla JavaScript — Five SDK

Recap of the previous chapter: Changing the View
- You learned what State is, and how to read and modify it.
- You used State to build an auto look-around feature.

In this chapter you will learn

How to record user interactions through State.
How to replay user interactions through State.

Preparation

As in the previous chapter, we create a new directory (src/3.recording-state) along with its corresponding html file and jsx or tsx file.

You can start by copying the contents of the previous chapter's js or ts file.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <link rel="icon" href="data:;base64,iVBORw0KGgo=" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Changing the View | Knowing state</title>
    <link
      href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.1/dist/css/bootstrap.min.css"
      rel="stylesheet"
      crossorigin="anonymous"
    />
    <link
      href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.5.0/font/bootstrap-icons.css"
      rel="stylesheet"
    />
    <style>
      html,
      body,
      #app {
        width: 100%;
        height: 100%;
        overflow: hidden;
      }
    </style>
  </head>
  <body>
    <div id="app"></div>
    <!-- Mode switch -->
    <nav class="navbar fixed-bottom navbar-light bg-light">
      <div class="container-fluid justify-content-center">
        <div class="btn-group">
          <button class="btn btn-primary active js-Panorama">Panorama Roam</button>
          <button class="btn btn-primary js-Floorplan">Floorplan Overview</button>
        </div>
      </div>
    </nav>
    <!-- Look around -->
    <div class="card position-fixed m-2 top-0 end-0">
      <button class="btn btn-light js-lookAround-start">
        <i class="bi bi-arrow-repeat"></i>
      </button>
      <button class="btn btn-light js-lookAround-stop d-none">
        <i class="bi bi-pause"></i>
      </button>
    </div>
    <script type="module" src="./index"></script>
  </body>
</html>

import { Five, parseWork } from "@realsee/five";

const workURL =
  "https://vr-public.realsee-cdn.cn/release/static/image/release/five/work-sample/07bdc58f413bc5494f05c7cbb5cbdce4/work.json";

const five = new Five();

five.appendTo(document.querySelector("#app"));

fetch(workURL)
  .then((res) => res.json())
  .then((json) => {
    const work = parseWork(json);
    five.load(work);
  });

window.addEventListener("resize", () => five.refresh(), false);

{
  // === Mode switch ===
  const buttons = {
    Panorama: document.querySelector(".js-Panorama"),
    Floorplan: document.querySelector(".js-Floorplan"),
  };

  for (const [modeName, element] of Object.entries(buttons)) {
    element.addEventListener(
      "click",
      () => {
        five.setState({ mode: modeName });
      },
      false
    );
  }

  five.on("stateChange", (state) => {
    for (const [modeName, element] of Object.entries(buttons)) {
      if (modeName === state.mode) {
        element.classList.add("active");
      } else {
        element.classList.remove("active");
      }
    }
  });
}

{
  // === Look around ===
  let timer;
  const startButton = document.querySelector(".js-lookAround-start");
  const stopButton = document.querySelector(".js-lookAround-stop");
  startButton.addEventListener(
    "click",
    () => {
      window.clearInterval(timer);
      timer = window.setInterval(() => {
        five.setState({ longitude: five.state.longitude + Math.PI / 360 });
      }, 16);
      startButton.classList.add("d-none");
      stopButton.classList.remove("d-none");
    },
    false
  );
  stopButton.addEventListener(
    "click",
    () => {
      window.clearInterval(timer);
      startButton.classList.remove("d-none");
      stopButton.classList.add("d-none");
    },
    false
  );
}

export {};

import { Five, Mode, parseWork } from "@realsee/five";

const workURL =
  "https://vr-public.realsee-cdn.cn/release/static/image/release/five/work-sample/07bdc58f413bc5494f05c7cbb5cbdce4/work.json";

const five = new Five();

five.appendTo(document.querySelector("#app")!);

fetch(workURL)
  .then((res) => res.json())
  .then((json) => {
    const work = parseWork(json);
    five.load(work);
  });

window.addEventListener("resize", () => five.refresh(), false);

{
  // === Mode switch ===
  const buttons: Partial<Record<Mode, Element>> = {
    Panorama: document.querySelector(".js-Panorama")!,
    Floorplan: document.querySelector(".js-Floorplan")!,
  };

  for (const [modeName, element] of Object.entries(buttons)) {
    element.addEventListener(
      "click",
      () => {
        five.setState({ mode: modeName as Mode });
      },
      false
    );
  }

  five.on("stateChange", (state) => {
    for (const [modeName, element] of Object.entries(buttons)) {
      if (modeName === state.mode) {
        element.classList.add("active");
      } else {
        element.classList.remove("active");
      }
    }
  });
}

{
  // === Look around ===
  let timer: number | undefined;
  const startButton = document.querySelector(".js-lookAround-start")!;
  const stopButton = document.querySelector(".js-lookAround-stop")!;
  startButton.addEventListener(
    "click",
    () => {
      window.clearInterval(timer);
      timer = window.setInterval(() => {
        five.setState({ longitude: five.state.longitude + Math.PI / 360 });
      }, 16);
      startButton.classList.add("d-none");
      stopButton.classList.remove("d-none");
    },
    false
  );
  stopButton.addEventListener(
    "click",
    () => {
      window.clearInterval(timer);
      startButton.classList.remove("d-none");
      stopButton.classList.add("d-none");
    },
    false
  );
}

export {};

Start the dev server with npm run dev, then navigate to the current page at "http://localhost:3000/src/3.recording-state/index.html".

Tip

Check your console — the port number may change depending on your configuration and which ports are currently in use, so always rely on what the console prints. If you use a different development build tool, start the server according to that tool's requirements.

Recording / Playback

In this chapter we continue to build an interesting application using State.
We will create an app that records the State changes the user produces on the page and can replay those interactions.

Building the recording UI

We add UI buttons in the top-left corner of the page. We have designed:

A start-recording button
A stop-recording button
A play button

Along with two status indicators:

Recording
Playing

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <link rel="icon" href="data:;base64,iVBORw0KGgo=" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Recording State | Recording state</title>
    <link
      href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.1/dist/css/bootstrap.min.css"
      rel="stylesheet"
      crossorigin="anonymous"
    />
    <link
      href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.5.0/font/bootstrap-icons.css"
      rel="stylesheet"
    />
    <style>
      html,
      body,
      #app {
        width: 100%;
        height: 100%;
        overflow: hidden;
      }
    </style>
  </head>
  <body>
    <div id="app"></div>
    <!-- Mode switch -->
    <nav class="navbar fixed-bottom navbar-light bg-light">
      <div class="container-fluid justify-content-center">
        <div class="btn-group">
          <button class="btn btn-primary active js-Panorama">Panorama Roam</button>
          <button class="btn btn-primary js-Floorplan">Floorplan Overview</button>
        </div>
      </div>
    </nav>
    <!-- Look around -->
    <div class="card position-fixed m-2 top-0 end-0">
      <button class="btn btn-light js-lookAround-start">
        <i class="bi bi-arrow-repeat"></i>
      </button>
      <button class="btn btn-light js-lookAround-stop d-none">
        <i class="bi bi-pause"></i>
      </button>
    </div>
    <!-- highlight-start -->
    <!-- Recording -->
    <div class="card position-fixed m-2 top-0 start-0">
      <div class="btn-group align-items-center">
        <button class="btn btn-light js-recording-start">
          <i class="bi bi-record-fill"></i>
        </button>
        <button class="btn btn-light js-recording-stop d-none">
          <i class="bi bi-stop-fill"></i>
        </button>
        <button class="btn btn-light js-recording-play">
          <i class="bi bi-play-fill"></i>
        </button>
        <p class="badge bg-primary m-2 js-state-recording d-none">Recording</p>
        <p class="badge bg-primary m-2 js-state-playing d-none">Playing</p>
      </div>
    </div>
    <!-- highlight-end -->
    <script type="module" src="./index"></script>
  </body>
</html>

Writing the Recorder class

First, we need to write a Recorder class to support recording and playback. The Recorder class is not part of Five SDK; it is written solely to achieve the effect of this chapter.

Implement the startRecording / endRecording methods to start and stop recording.
Implement the record(state: State) method to record content. It captures everything that happens between startRecording and endRecording.
Implement the play(callback) method for playback. After play is called, it walks through the recorded content and invokes the callback in order to replay the State.

/**
 * Recorder class
 */
class Recorder {
  constructor() {
    this.startTime = 0;
    this.records = null;
  }
  /**
   * Whether anything has been recorded
   */
  hasRecords() {
    return this.records !== null;
  }
  /**
   * Record a keyframe
   * @param state five's state
   * @returns
   */
  record(state) {
    if (this.records === null) return;
    this.records.push({
      state: Object.assign({}, state),
      time: Date.now() - this.startTime,
    });
  }
  /**
   * Start recording
   */
  startRecording() {
    this.startTime = Date.now();
    this.records = [];
  }
  /**
   * Stop recording
   */
  endRecording() {
    this.startTime = 0;
  }
  /**
   * Replay the recording
   * @param callback keyframe callback
   * @returns whether there is currently a recording
   */
  play(callback) {
    if (this.records === null || this.records.length === 0) return false;
    const records = this.records.slice();
    const keyframe = (keyIndex) => {
      const current = records[keyIndex];
      const next = records[keyIndex + 1];
      callback(current.state, next === undefined);
      if (next) {
        const delay = next.time - current.time;
        setTimeout(() => keyframe(keyIndex + 1), delay);
      }
    };
    keyframe(0);
    return true;
  }
}

export { Recorder };

import { State } from "@realsee/five";

/**
 * Recorder class
 */
class Recorder {
  private records: { state: State; time: number }[] | null = null;
  private startTime: number;
  constructor() {
    this.startTime = 0;
    this.records = null;
  }
  /**
   * Whether anything has been recorded
   */
  hasRecords() {
    return this.records !== null;
  }
  /**
   * Record a keyframe
   * @param state five's state
   * @returns
   */
  record(state: State) {
    if (this.records === null) return;
    this.records.push({
      state: Object.assign({}, state),
      time: Date.now() - this.startTime,
    });
  }
  /**
   * Start recording
   */
  startRecording() {
    this.startTime = Date.now();
    this.records = [];
  }
  /**
   * Stop recording
   */
  endRecording() {
    this.startTime = 0;
  }
  /**
   * Replay the recording
   * @param callback keyframe callback
   * @returns whether there is currently a recording
   */
  play(callback: (state: State, isFinal: boolean) => void) {
    if (this.records === null || this.records.length === 0) return false;
    const records = this.records.slice();
    const keyframe = (keyIndex: number) => {
      const current = records[keyIndex];
      const next = records[keyIndex + 1];
      callback(current.state, next === undefined);
      if (next) {
        const delay = next.time - current.time;
        setTimeout(() => keyframe(keyIndex + 1), delay);
      }
    };
    keyframe(0);
    return true;
  }
}

export { Recorder };

Writing the recording logic

Append the following after the look around code from the previous chapter:

{
  //=== Recording ===
  const recorder = new Recorder();
  const startRecordingButton = document.querySelector(".js-recording-start");
  const stopRecordingButton = document.querySelector(".js-recording-stop");
  const playRecordingButton = document.querySelector(".js-recording-play");
  const recordingState = document.querySelector(".js-state-recording");
  const playingState = document.querySelector(".js-state-playing");

  five.on("stateChange", (state) => {
    if (recordingState.classList.contains("d-none")) return;
    recorder.record(state);
  });

  startRecordingButton.addEventListener(
    "click",
    () => {
      recorder.startRecording();
      startRecordingButton.classList.add("d-none");
      stopRecordingButton.classList.remove("d-none");
      playRecordingButton.classList.add("d-none");
      recordingState.classList.remove("d-none");
      playingState.classList.add("d-none");
    },
    false
  );

  stopRecordingButton.addEventListener(
    "click",
    () => {
      recorder.endRecording();
      startRecordingButton.classList.remove("d-none");
      stopRecordingButton.classList.add("d-none");
      playRecordingButton.classList.remove("d-none");
      recordingState.classList.add("d-none");
      playingState.classList.add("d-none");
    },
    false
  );

  playRecordingButton.addEventListener(
    "click",
    () => {
      const hasReocrd = recorder.play((state, isFinal) => {
        five.setState(state);
        if (isFinal) {
          startRecordingButton.classList.remove("d-none");
          stopRecordingButton.classList.add("d-none");
          playRecordingButton.classList.remove("d-none");
          recordingState.classList.add("d-none");
          playingState.classList.add("d-none");
        }
      });
      if (hasReocrd) {
        startRecordingButton.classList.add("d-none");
        stopRecordingButton.classList.add("d-none");
        playRecordingButton.classList.add("d-none");
        recordingState.classList.add("d-none");
        playingState.classList.remove("d-none");
      }
    },
    false
  );
}

{
  //=== Recording ===
  const recorder = new Recorder();
  const startRecordingButton = document.querySelector(".js-recording-start")!;
  const stopRecordingButton = document.querySelector(".js-recording-stop")!;
  const playRecordingButton = document.querySelector(".js-recording-play")!;
  const recordingState = document.querySelector(".js-state-recording")!;
  const playingState = document.querySelector(".js-state-playing")!;

  five.on("stateChange", (state) => {
    if (recordingState.classList.contains("d-none")) return;
    recorder.record(state);
  });

  startRecordingButton.addEventListener(
    "click",
    () => {
      recorder.startRecording();
      startRecordingButton.classList.add("d-none");
      stopRecordingButton.classList.remove("d-none");
      playRecordingButton.classList.add("d-none");
      recordingState.classList.remove("d-none");
      playingState.classList.add("d-none");
    },
    false
  );

  stopRecordingButton.addEventListener(
    "click",
    () => {
      recorder.endRecording();
      startRecordingButton.classList.remove("d-none");
      stopRecordingButton.classList.add("d-none");
      playRecordingButton.classList.remove("d-none");
      recordingState.classList.add("d-none");
      playingState.classList.add("d-none");
    },
    false
  );

  playRecordingButton.addEventListener(
    "click",
    () => {
      const hasReocrd = recorder.play((state, isFinal) => {
        five.setState(state);
        if (isFinal) {
          startRecordingButton.classList.remove("d-none");
          stopRecordingButton.classList.add("d-none");
          playRecordingButton.classList.remove("d-none");
          recordingState.classList.add("d-none");
          playingState.classList.add("d-none");
        }
      });
      if (hasReocrd) {
        startRecordingButton.classList.add("d-none");
        stopRecordingButton.classList.add("d-none");
        playRecordingButton.classList.add("d-none");
        recordingState.classList.add("d-none");
        playingState.classList.remove("d-none");
      }
    },
    false
  );
}

Switch back to your browser and you will see a record and a play button appear in the top-left corner of the page. Try them out and see whether they behave as expected.

Impressive — you can already write programs this sophisticated 🥳.

What you will learn in the next chapter

In the next chapter we start working with models in 3D space

Understand the Five SDK gesture interaction system.
Get the 3D position of a point.