Skip to content

Commit 36ff0dc

Browse files
townehtowneh
authored
docs: add controls, tracking, networking and feature guides (#123)
Address documentation gaps raised in open issues with new and expanded pages: - Settings and Controls: full default keybinding reference (desktop + VR) and where to rebind them - Getting Started: note that Git must be installed for package resolution on first open - Full Body Tracking: tracker calibration, pairing and manual role overrides - Hand Tracking: OpenXR articulated finger tracking and controller fallback - Players Menu: per-player audio, visibility, block, pin and direct connections - Chatbox: text chat, the typing indicator and the OSC bridge - Animation Recorder: recording the local avatar's motion from developer settings - Face Tracking: Basis-side setup overview alongside the existing external guide - Network Syncing: authoritative server model, replication, smoothing and client/server version compatibility Pages that need supporting screenshots carry image placeholders for follow-up contributions.
1 parent 3965939 commit 36ff0dc

12 files changed

Lines changed: 377 additions & 12 deletions

File tree

‎content/docs/avatar/animation-recorder.mdx‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
title: Animation Recorder
3+
description: Record your local avatar's motion to a file from the Basis client's developer settings.
4+
---
5+
6+
import { Callout } from 'fumadocs-ui/components/callout';
7+
8+
## Overview
9+
10+
The **Animation Recorder** captures your local avatar's motion frame by frame and writes it to a file. It records the avatar's root position and rotation, its full humanoid muscle pose, and its scale — everything needed to reproduce the body's movement.
11+
12+
<Callout type="info">
13+
Recording uses Unity's humanoid pose system, so it only works with a **humanoid** avatar.
14+
</Callout>
15+
16+
## Recording
17+
18+
The recorder lives in **Settings → Developer → Recorder**.
19+
20+
1. Open **Settings → Developer** and find the **Recorder** section.
21+
2. Set the **countdown** — how long the recorder waits before it starts capturing, giving you time to get into position. Set it to `0` to start immediately.
22+
3. Optionally enable **auto-stop** and set a **maximum duration** to cap the recording length automatically.
23+
4. Press **Start**. The status shows the countdown, then the elapsed time and frame count while recording.
24+
5. Press **Stop** when you're done (or let auto-stop end it).
25+
26+
{/* IMAGE PLACEHOLDER
27+
File: public/img/avatars/animation-recorder.png
28+
Show: the Settings → Developer → Recorder panel mid-recording, with the countdown/elapsed time and frame count visible.
29+
When ready, replace this comment with:
30+
![The animation recorder in the developer settings](/img/avatars/animation-recorder.png)
31+
*/}
32+
33+
## Where recordings are saved
34+
35+
Each recording is written to an `AvatarRecordings` folder inside your local Basis data folder, with a timestamped filename. The file is a compact binary capture of every frame — the interval since the previous frame, the root rotation and position, the humanoid muscle values and the avatar scale.

‎content/docs/avatar/face-tracking.mdx‎

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,38 @@
11
---
22
title: Face Tracking
3+
description: Setting up face and eye tracking on a Basis avatar — the components and avatar fields involved, and the full step-by-step guide.
34
---
45

5-
- Hai has created a comprehensive guide on how to add the Automatic Face Tracking component to an avatar in a Basis project
6-
- You can follow the link [here](https://docs.hai-vr.dev/docs/basis/avatar-customization/face-tracking) for instruction on how this works.
6+
import { Callout } from 'fumadocs-ui/components/callout';
7+
8+
## Overview
9+
10+
Basis can drive an avatar's facial expressions and eyes from a face tracker. It does this with **blendshapes** for expressions and visemes, and by rotating the avatar's **eye bones** for gaze. Much of the wiring is automatic: when an avatar loads, Basis detects the blendshape naming convention on the mesh (Unified Expressions or ARKit) and attaches the components that map incoming face-tracking values onto it.
11+
12+
<Callout type="info">
13+
Hai maintains the comprehensive step-by-step guide for adding the Automatic Face Tracking component to an avatar. Follow it for the full setup: [Face tracking on docs.hai-vr.dev](https://docs.hai-vr.dev/docs/basis/avatar-customization/face-tracking).
14+
</Callout>
15+
16+
## What gets set up on the avatar
17+
18+
The guide above walks through the process; this is what it's configuring under the hood, so you know what each piece is for.
19+
20+
- **Viseme / expression blendshapes** — your avatar's face mesh and the blendshape indices Basis drives for mouth shapes and expressions. These are picked up from the mesh's blendshape names.
21+
- **Blink** — the mesh and blendshape used to blink. Basis auto-blinks the avatar when no external face tracker is overriding the eyes.
22+
- **Eye bones** — the avatar's left and right eye bones. Basis calibrates each eye's local axes on load so gaze rotates correctly regardless of how the bones are oriented in the rig.
23+
24+
When face tracking is connected, incoming expression values are mapped onto the blendshapes and the eye angles drive the eye bones. When it isn't, the avatar falls back to automatic blinking and look-at-target gaze, so faces still feel alive without a tracker.
25+
26+
{/* IMAGE PLACEHOLDER
27+
File: public/img/avatars/face-tracking-setup.png
28+
Show: the Automatic Face Tracking component on an avatar in the Inspector, with the face/eye fields populated.
29+
When ready, replace this comment with:
30+
![Face tracking components set up on an avatar](/img/avatars/face-tracking-setup.png)
31+
*/}
32+
33+
{/* IMAGE PLACEHOLDER
34+
File: public/img/avatars/face-tracking-result.png
35+
Show: an avatar in-client mirroring the wearer's expression and eye gaze from a face tracker.
36+
When ready, replace this comment with:
37+
![An avatar's face driven by a face tracker](/img/avatars/face-tracking-result.png)
38+
*/}

‎content/docs/avatar/meta.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
22
"title": "Avatar",
33
"icon": "User",
4-
"pages": ["index", "setup", "building", "uploading", "jiggle", "authored-motion", "animation-rigging", "face-tracking", "shaders"]
4+
"pages": ["index", "setup", "building", "uploading", "jiggle", "authored-motion", "animation-rigging", "face-tracking", "animation-recorder", "shaders"]
55
}

‎content/docs/controls/chatbox.mdx‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
title: Chatbox
3+
description: Sending text chat in Basis, the typing indicator, and the OSC bridge for driving the chatbox from external apps.
4+
---
5+
6+
import { Callout } from 'fumadocs-ui/components/callout';
7+
8+
## Overview
9+
10+
The chatbox lets you send short text messages that appear above your avatar for everyone nearby to read — useful when voice isn't an option. Open it with the **Open Chat** key (`Y` by default — see [Settings and Controls](/en/docs/controls)), type a message and send.
11+
12+
- Messages can be up to **256 characters**.
13+
- A sent message shows above your avatar for about **10 seconds**, then clears.
14+
- While you're typing, others see a **typing indicator** over your avatar, so they know a message is coming.
15+
- An optional notification sound can play when a message arrives.
16+
17+
{/* IMAGE PLACEHOLDER
18+
File: public/img/controls/chatbox.png
19+
Show: a chat message displayed above an avatar in-client, ideally with another avatar's typing indicator visible.
20+
When ready, replace this comment with:
21+
![A chatbox message shown above an avatar](/img/controls/chatbox.png)
22+
*/}
23+
24+
<Callout type="info">
25+
You can hide an individual player's chat messages from their [player panel](/en/docs/controls/players-menu).
26+
</Callout>
27+
28+
## Driving the chatbox over OSC
29+
30+
The chatbox can also be controlled from an external application over **OSC**, on the address `/chatbox/input`. This lets tools type into the chatbox for you — for example a speech-to-text app or a status display.
31+
32+
The address accepts:
33+
34+
- A single **bool** — sets the typing indicator on or off without sending anything.
35+
- A **string** with two optional bools — `(text, openKeyboard, playNotificationSound)`. The text is the message; the first bool chooses whether to open the in-client composer or send straight away; the second chooses whether a notification sound plays.
36+
37+
Text coming in over OSC is sanitised the same way as text typed in-client. See [OSC](/en/docs/scripting/osc) for enabling and configuring the OSC connection.

‎content/docs/controls/full-body-tracking.mdx‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
title: Full Body Tracking
3+
description: How Basis assigns and calibrates SteamVR / OpenXR trackers to your body for full body tracking, plus pairing and manual role overrides.
4+
---
5+
6+
import { Callout } from 'fumadocs-ui/components/callout';
7+
8+
## Overview
9+
10+
Full body tracking (FBT) lets Basis drive your avatar's hips, feet and limbs from physical trackers rather than estimating them from your head and hands alone. Basis reads trackers through **OpenXR** (including the HTC Vive tracker profile), and works out which tracker belongs on which part of your body automatically from a single calibration pose.
11+
12+
The roles Basis can assign a tracker to are:
13+
14+
- **Hips** and **Chest**
15+
- **Left / Right Foot** and **Left / Right Toes**
16+
- **Left / Right Lower Leg** (shins)
17+
- **Left / Right Shoulder**
18+
- **Left / Right Lower Arm** (forearms)
19+
20+
## Calibrating
21+
22+
Basis uses **constellation calibration**: you stand in a T-pose and it classifies each free tracker by where it sits in space relative to your headset, then assigns it to the closest matching body role.
23+
24+
1. Put on your headset and trackers and make sure each tracker is tracking.
25+
2. Open the tracker calibration option in **Settings**.
26+
3. Stand in a **T-pose** — feet roughly shoulder-width apart, arms straight out to the sides.
27+
4. Trigger calibration and hold the pose until it completes.
28+
29+
Your avatar is put into a reference T-pose during calibration, the trackers are classified, and the FBT roles bind to your avatar's bones.
30+
31+
<Callout type="info">
32+
Calibration is geometry-based, so a clean T-pose matters. If a tracker ends up on the wrong limb, re-run calibration with a more deliberate pose, or pin that tracker to a role with an override (below).
33+
</Callout>
34+
35+
{/* IMAGE PLACEHOLDER
36+
File: public/img/controls/fbt-calibration.png
37+
Show: a user in a T-pose with the tracker calibration prompt visible in-client.
38+
When ready, replace this comment with:
39+
![Calibrating full body trackers in a T-pose](/img/controls/fbt-calibration.png)
40+
*/}
41+
42+
## Tracker pairing
43+
44+
Symmetrical tracker setups (for example a pair of ankle trackers) can be **paired** so Basis treats them as a linked set during classification. Pairings are saved to disk and reused, so you don't have to re-pair them every session.
45+
46+
## Manual role overrides
47+
48+
If the automatic classifier keeps misassigning a particular tracker — common with unusual mounting positions — you can **force a tracker to a specific role**. An override skips the scoring step for that tracker and assigns it directly, so the rest of the constellation still calibrates around it.
49+
50+
Like pairings, overrides persist between sessions, so a setup you've tuned once keeps working on the next launch with the same trackers.
51+
52+
{/* IMAGE PLACEHOLDER
53+
File: public/img/controls/fbt-role-override.png
54+
Show: the tracker role override UI with a tracker assigned to a specific body role.
55+
When ready, replace this comment with:
56+
![Assigning a tracker to a fixed role](/img/controls/fbt-role-override.png)
57+
*/}
58+
59+
<Callout type="info">
60+
Pairings and overrides are stored in your local Basis data folder, alongside the other per-user files documented in [Settings and Controls](/en/docs/controls). Deleting them resets you to fully automatic classification.
61+
</Callout>

‎content/docs/controls/hand-tracking.mdx‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
title: Hand Tracking
3+
description: Articulated hand and finger tracking in Basis through OpenXR, and how it falls back to controller input when a hand isn't tracked.
4+
---
5+
6+
import { Callout } from 'fumadocs-ui/components/callout';
7+
8+
## Overview
9+
10+
Basis supports **articulated hand tracking** through OpenXR, so your avatar's fingers follow your real hands on runtimes that expose hand tracking — such as Quest's camera-based tracking or a PC OpenXR runtime that provides it. No per-avatar setup is needed: hand tracking drives the same finger pose system that controllers do.
11+
12+
## What gets tracked
13+
14+
For each hand, Basis reads the wrist pose and the curl and spread of all five fingers:
15+
16+
- **Curl** — how far each finger is bent, from open to a full fist.
17+
- **Spread** — how far the fingers are splayed apart.
18+
19+
These feed your avatar's hand pose every frame, blended smoothly so finger motion looks natural rather than snapping between poses.
20+
21+
## Controllers and fallback
22+
23+
Hand tracking and controllers share one pose pipeline, so you can switch between them without any change in setup. When a hand isn't being optically tracked — you're holding controllers, or a hand leaves the tracking volume — Basis falls back to driving the fingers from the controller's **trigger and grip** instead, so your avatar's hands still close around grips and respond to input.
24+
25+
<Callout type="info">
26+
Hand tracking availability depends on your headset and OpenXR runtime. On Quest, enable hand tracking in the headset's system settings; Basis picks it up automatically when the runtime reports it.
27+
</Callout>
28+
29+
{/* IMAGE PLACEHOLDER
30+
File: public/img/controls/hand-tracking.png
31+
Show: an avatar's articulated hands following real finger movement in-client.
32+
When ready, replace this comment with:
33+
![Articulated hand tracking driving an avatar's fingers](/img/controls/hand-tracking.png)
34+
*/}

‎content/docs/controls/index.mdx‎

Lines changed: 71 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,83 @@
11
---
22
title: Settings and Controls
3+
description: Default keyboard, mouse and VR controller bindings for the Basis client, where to rebind them, and where to find logs and library data.
34
---
45

6+
import { Callout } from 'fumadocs-ui/components/callout';
7+
import { Tab, Tabs } from 'fumadocs-ui/components/tabs';
8+
59
## Controls
610

7-
Let's create a page to cover the available controls in BasisVR
11+
Basis has a fuller set of controls than most people realise on first launch. This page is a quick reference for the defaults across desktop and VR, and where to change them.
12+
13+
<Callout type="info">
14+
Every keyboard binding below is **rebindable** in-client under **Settings → Keyboard Bindings**. The button for each control shows its current key, and the default in brackets if you've changed it. **Escape** cancels a rebind in progress, and you can reset everything back to defaults from the same panel.
15+
</Callout>
16+
17+
{/* IMAGE PLACEHOLDER
18+
File: public/img/controls/keyboard-bindings.png
19+
Show: the in-client Settings → Keyboard Bindings panel with the grouped, rebindable controls visible.
20+
When ready, replace this comment with:
21+
![The Keyboard Bindings settings panel](/img/controls/keyboard-bindings.png)
22+
*/}
23+
24+
### Movement
25+
26+
| Action | Default key |
27+
| --- | --- |
28+
| Forward / Back / Left / Right | `W` `S` `A` `D` |
29+
| Movement (alternative) | Arrow keys |
30+
| Jump | `Space` |
31+
| Sprint | `Left Shift` |
32+
| Crouch | `C` |
33+
| Fly up / down | `R` / `F` |
34+
35+
### Camera (desktop)
36+
37+
| Action | Default |
38+
| --- | --- |
39+
| Look | Move the mouse |
40+
| Look (keyboard alternative) | `I` `K` `J` `L` (and `,` `.`) |
41+
| Toggle third-person view | `F` |
42+
| Zoom (in third-person) | Mouse scroll wheel |
43+
| Interact / select | Left mouse button |
44+
45+
### Communication and menus
46+
47+
| Action | Default key |
48+
| --- | --- |
49+
| Toggle microphone | `V` |
50+
| Open chat | `Y` |
51+
| Free cursor (unlock mouse for UI) | `Tab` |
52+
| Open / close menu | `Escape` |
53+
54+
<Callout type="info">
55+
**Toggle microphone** depends on your microphone mode. With push-to-talk, `V` transmits while held; in toggle mode it switches the mic on and off. Set the mode in the audio settings.
56+
</Callout>
57+
58+
### Switching input mode
59+
60+
You can move between desktop and the two XR runtimes at runtime:
861

62+
| Key | Mode |
63+
| --- | --- |
64+
| `F9` | Desktop |
65+
| `F10` | OpenXR |
66+
| `F11` | OpenVR |
967

10-
## Keyboard Shortcuts
68+
### VR controllers
1169

12-
- F9: switch to Desktop
70+
In VR the same actions map to the controllers rather than the keyboard:
1371

14-
- F10: switch to OpenXR
72+
- **Thumbstick** — movement (one controller) and turning (the other).
73+
- **Trigger** — interact / select, the controller equivalent of left mouse button.
74+
- **Grip** — used for descending in fly mode and for grabbing.
75+
- **Primary face button** — jump.
76+
- **Menu / system button** — opens the menu.
1577

16-
- F11: Switch to OpenVR
78+
<Callout type="info">
79+
VR controller bindings aren't part of the keyboard rebinding panel — they're driven by the active OpenXR / OpenVR interaction profile for your headset.
80+
</Callout>
1781

1882
## Help with Debugging
1983

@@ -42,5 +106,5 @@ Let's create a page to cover the available controls in BasisVR
42106
It stores a mapping of the avatars, worlds, and props currently contained in your library.
43107

44108
<Callout>
45-
Within the codebase, you can also find there is an `EmbeddedItems` class containing library items that are embedded into the application.
46-
</Callout>
109+
Within the codebase, you can also find there is an `EmbeddedItems` class containing library items that are embedded into the application.
110+
</Callout>

‎content/docs/controls/meta.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
22
"title": "Controls",
33
"icon": "Wrench",
4-
"pages": ["index"]
4+
"pages": ["index", "full-body-tracking", "hand-tracking", "players-menu", "chatbox"]
55
}

‎content/docs/controls/players-menu.mdx‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
---
2+
title: Players Menu
3+
description: The per-player panel in the Basis client — mute, volume, block, pin, highlight, direct connections and more.
4+
---
5+
6+
import { Callout } from 'fumadocs-ui/components/callout';
7+
8+
## Overview
9+
10+
Selecting another player in the menu opens their **player panel** — a set of per-player controls that only affect how *you* see and hear that person. Mutes, volume changes and blocks are local to your client; they don't change anything for anyone else in the room.
11+
12+
{/* IMAGE PLACEHOLDER
13+
File: public/img/controls/players-menu.png
14+
Show: the individual player panel open for a selected player, with the per-player controls visible.
15+
When ready, replace this comment with:
16+
![The per-player options panel](/img/controls/players-menu.png)
17+
*/}
18+
19+
## What you can do per player
20+
21+
### Audio
22+
23+
- **Mute / Unmute** — silences that player for you. Your volume setting is remembered, so unmuting restores it.
24+
- **Volume** — a per-player volume slider from silent up to a boost above normal, with a live level meter so you can see them speaking.
25+
26+
### Visibility and interaction
27+
28+
- **Show / Hide avatar** — hides that player's avatar from your view.
29+
- **Interactions** — enable or disable interacting with their avatar.
30+
- **Chat visibility** — show or hide their chat messages for you.
31+
32+
### Block
33+
34+
- **Block / Unblock** — a stronger action that applies both audio and avatar visibility at once, behind a confirmation prompt. Blocking is session-aware, so the other player is told you've blocked them for moderation purposes.
35+
36+
### Find and mark players
37+
38+
- **Highlight** — draws a beacon above the player so you can spot them in a crowd (it turns red if they're blocked).
39+
- **Pin** — marks a player as a favourite so they're easy to find again later. Pins are saved between sessions.
40+
41+
### Talk privately
42+
43+
- **Private chat** — add a player to a private voice group, or switch to talking to **only** that player.
44+
- **Direct connection** — request a peer-to-peer connection to a specific player. Once connected, the panel shows the round-trip ping, and you can set a per-player policy to ask, always accept or always decline future requests.
45+
46+
### Moderation
47+
48+
- **Teleport to** — jump to a player's location. This is an admin-only action and only appears if you have the permission for it.
49+
50+
<Callout type="info">
51+
Most of these settings persist per player, so someone you've muted, blocked or set a custom volume for stays that way the next time you meet them.
52+
</Callout>

‎content/docs/index.mdx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,10 @@ If you have the correct Unity version already installed but the "Add Modules" op
7070

7171
## Acquiring the Basis framework
7272

73+
<Callout type="warn">
74+
**Install Git first — even if you download the project as a ZIP.** Several of the framework's packages are pulled from Git repositories when Unity first opens the project (for example AudioLink, the Valve OpenXR utilities and RNNoise). If Git isn't installed and on your system `PATH`, those packages fail to resolve and the project opens with compile errors. Install [Git](https://git-scm.com/downloads), restart Unity Hub, then open the project. You can confirm Git is on your `PATH` by running `git --version` in a terminal.
75+
</Callout>
76+
7377
### Git
7478

7579
You can either directly [clone](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) the `developer` or `long-term-support` branches found at https://github.com/BasisVR/Basis or [fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) it to your own local repository, and then open the resulting Unity project on your local computer.

0 commit comments

Comments
 (0)