4DGSXBETA
This is the normative spec for the bundles this site plays. Every bundle also ships a zero-dependency single-file reference player — open the example match in it to see the whole spec implemented in ~800 lines.

4DGSX — the 4D Gaussian Splatting eXchange bundle (v0.2)

A match (or any simulated 4D scene) exported as one portable, self-contained directory that any platform — web, Quest/WebXR, native — can play back with a free camera. 4DGSX covers the 3D content (geometry + gaussian splat sets + a rigid transform track), the UI (semantic data + a layered, user-toggleable presentation system), and the audio (one or more user-toggleable sources — broadcast mix, crowd, commentary, spatial mics — each with an explicit clock map). RFL is the reference producer (gauntlet/volumetric.py) and index.html in each bundle is the reference player; the layout is deliberately sport-agnostic.

Format ids carried in the files: 4dgsx (scene manifest), 4dgsx-hud (data track), 4dgsx-ui (presentation track). Version stamps are semver-ish: additive fields bump minor, layout changes bump major. Must-ignore rule everywhere: consumers skip unknown fields, unknown event types, unknown UI component/anchor/content types. That rule — not any particular field — is what future-proofs the format.

Design rules (the part other projects should copy)

  1. Geometry once, transforms per frame. Rigid bodies get one static geometry/splat set each, in the body's local frame; playback is one 4×4 per body per frame. Never re-ship or re-train per-frame surfaces for rigid content.
  2. Text and UI are data, not pixels. Names, scores, clocks, chatter ship as a semantic track (hud.json); how they LOOK ships separately (ui.json + optional HTML panels) and the user can toggle every layer. Burned-in overlays are wrong under a free camera; 3D text meshes freeze typography; text in splats is both plus blurry.
  3. Audio is a sidecar of independent sources, each with an explicit clock map. Splat formats have no audio container, and a broadcast timeline legitimately diverges from sim time (inserted replays). Ship each track (mix, crowd, commentary, mics) as its own user-toggleable source with a piecewise-linear [match_t, audio_t] map; a source may anchor to a body for spatial playback (v0.2 — see the audio section).
  4. One clock. Track frames, hud events, UI bindings and the audio map are all keyed to match time in seconds. Presentation clocks (count-down, halves) are declared in hud.json and derived by the player.

Bundle layout

web/
  scene.json      manifest: version, buffers, prims, draws, bodies, camera,
                  pointers to hud/ui, audio + time map
  geometry.bin    mesh vertex + index data
  track.bin       per-frame rigid transforms (the "4D" in 4DGSX)
  hud.json        DATA: teams, players, anchors, clock, score, events, sfx
  ui.json         PRESENTATION: layers + components (see UI system)
  ui/*.html       designable sandboxed panels referenced by ui.json
  audio.m4a       v0.1 premix (optional); v0.2 stems live under audio/*
  splats/*.ply    per-body 3DGS gaussian sets (standard 3DGS PLY fields)
  points.bin      lightweight splat preview (optional viewer convenience)
  index.html      reference player (no dependencies, WebGL2)

scene.json

{
  "format": "4dgsx", "version": "0.2",
  "meta": { "hz": 25.0, "nframes": 2251, "camera": {...}, "grass": {...} },
  "hud": "hud.json",
  "ui": "ui.json",
  "audio": {"file": "audio.m4a", "map": [[0,0], [25.8,25.8], [25.8,30.1], ...],
            "sources": [...]},
           // or null. map: [match_t, audio_t] breakpoints, slope 1 between;
           // a duplicated match_t is inserted media (a goal replay).
           // file+map = v0.1 legacy premix; sources[] = v0.2, audio section.
  "bodies": ["corner_0", ..., "ball", "r0_pelvis", ...],   // track order
  "prims":  [{"vo","vc","io","ic"}, ...],   // ranges into geometry.bin
  "draws":  [{"p": primIdx, "b": bodyIdx, "rgba": [..], "checker": 0|1}, ...],
  "points": [{"b": bodyIdx, "ofs": pointOfs, "cnt": n}, ...],
  "buffers": {"vertexCount","vertexBytes","indexCount","indexBytes"},
  "times": [t_first, t_last]
}

geometry.bin

Little-endian, two consecutive sections:

  1. vertexCount × 24 B — interleaved float32 x,y,z, nx,ny,nz
  2. indexCount × 4 B — uint32 triangle indices into section 1

track.bin

float32 [nframes][nbodies][7] — x y z position then w x y z unit quaternion (MuJoCo convention), world frame, bodies order, at meta.hz. Interpolate: lerp position, nlerp shortest-path quaternion. 90 s of RFL ≈ 3.6 MB (≈ 40 kB/s); quantization is planned for a later version.

hud.json — the data track

{
  "format": "4dgsx-hud", "version": "0.1",
  "clock": {"mode": "down", "duration_s": 600, "halves": 2,
            "half_breaks": [305.2]},
  "teams":   [{"id": "A", "name", "code", "color": [r,g,b,a]}, ...],
  "players": [{"id": "r0", "team": "A", "number": 1, "name": "CR-7000",
               "anchor": {"body": "r0_pelvis", "offset": [0,0,0.62]}}, ...],
  "score":  [{"t": 0, "a": 0, "b": 0}, {"t": 25.8, "a": 1, "b": 0}, ...],
  "events": [{"t": 25.8, "type": "goal", "team": "A", "player": "r2",
              "replay_s": 4.3},
             {"t": 31.0, "type": "radio", "player": "r1",
              "text": "man on!", "dur_s": 3.5},
             {"t": 40.0, "type": "coach", "team": "B",
              "text": "Drop deep!", "dur_s": 5.0},
             {"t": 305.2, "type": "half", "n": 1},
             {"t": 88.0, "type": "drop"}],
  "sfx":    [{"t": 12.4, "kind": "kick", "mag": 0.8}, ...]
}

ui.json — the layered presentation system

The user-facing requirements this answers: labels tracked to bodies in 3D; floating panels; every piece of UI toggleable per user; designable by non-engine people; extensible to rich animated content and media without replacing anything.

{
  "format": "4dgsx-ui", "version": "0.1",
  "layers": [
    {"id": "players.names",  "title": "Player names",        "default": true},
    {"id": "players.radio",  "title": "Radio chatter",       "default": true},
    {"id": "match.scorebug", "title": "Score bug",           "default": true},
    {"id": "match.panel3d",  "title": "Stadium score panel", "default": false}
  ],
  "components": [
    {"id": "np_r0", "layer": "players.names",
     "anchor": {"type": "body", "body": "r0_pelvis",
                "offset": [0,0,0.62], "billboard": true},
     "content": {"type": "nameplate", "player": "r0"}},
    {"id": "scorebug", "layer": "match.scorebug",
     "anchor": {"type": "screen", "pos": [0.5, 0.03], "align": "top-center"},
     "content": {"type": "scoreboard", "style": "bug"}},
    {"id": "panel3d", "layer": "match.panel3d",
     "anchor": {"type": "world", "pos": [0, 0, 3.4], "billboard": "yaw",
                "size_m": [4.6, 1.55]},
     "content": {"type": "html", "src": "ui/score_panel.html",
                 "bind": ["teams", "score", "clock", "events"]}}
  ]
}

Layers are the unit of user control. Players render a toggle menu from this list, apply default, and persist the user's choices. User prefs always beat bundle defaults.

Components are {id, layer, anchor, content}:

The html panel contract (host ⇄ panel, JSON postMessage):

Panels are display-only in v0.1 (pointer-events off). Interactivity arrives via the reserved action channel without breaking old players.

audio — sources + the time map

scene.audio is null (silent bundle) or an object:

"audio": {
  // v0.1 legacy — the premixed broadcast track. Producers SHOULD keep
  // shipping it when a single premix exists; v0.1 players read only this.
  "file": "audio.m4a",
  "map":  [[0,0], [25.8,25.8], [25.8,30.1], ...],

  // v0.2 — independent, user-toggleable sources on the shared match clock
  "sources": [
    {"id": "crowd",      "title": "Crowd",        "file": "audio/crowd.m4a",
     "default": true, "gain": 1.0, "map": [...]},        // non-positional
    {"id": "pitch",      "title": "Pitch sounds", "file": "audio/pitch.m4a",
     "default": true, "gain": 1.0, "map": [...]},
    {"id": "commentary", "title": "Commentary",   "file": "audio/commentary.m4a",
     "default": true, "gain": 1.0, "map": [...]},
    {"id": "mic_r0", "title": "BLU 1 mic", "file": "audio/r0.m4a",
     "default": false,
     "anchor":  {"type": "body", "body": "r0_pelvis", "offset": [0, 0, 0.5]},
     "rolloff": {"ref": 2.0, "max": 45.0}}          // spatial, follows the body
  ]
}

The time map (per source): the broadcast timeline inserts replay_s seconds at each goal; map encodes exactly that. Reference behaviour: skip the inserted windows. A richer player may dwell at a jump and let the replay audio play over its own replay presentation. For fully synthesized spatial audio, use hud.sfx instead of (or on top of) the sources.

splats/*.ply

One file per dynamic body, standard 3DGS PLY fields (x y z nx ny nz f_dc_0..2 opacity scale_0..2 rot_0..3, binary little-endian), positions in the body's local frame, driven by the same track. v0.1 sets are surface-sampled isotropic gaussians; a trained-appearance upgrade replaces these files and nothing else. Budget note: the whole RFL scene is ~120k gaussians — inside the ~400k standalone-headset ceiling.

points.bin (optional preview)

pointCount × 20 B: float32 x,y,z, uint8 rgba, float32 radius, grouped per body by scene.json points[]. A cheap splat stand-in for renderers without a gaussian rasterizer.

Producing a bundle (RFL)

Changelog