dmf-studio-rnd/tg-post.md

# DMF Studio — dev update

**DMF Studio** is an open-architecture broadcast signal processor built around a node graph model. The idea is simple: every processing element — signal generator, capture input, compositor, gain stage, output — is an independent node. You wire them together visually in a browser, hit run, and the system routes live video and audio between them in real time.

Unlike traditional broadcast routers or vision mixers that bundle routing and processing into a closed appliance, DMF Studio runs on commodity hardware (a Linux server with DeckLink cards) and exposes the entire signal graph as software. Every node is a separate C++ process. Inter-node transport is MXL shared memory — a low-latency ring buffer system that timestamps every grain against a TAI clock. The browser frontend builds a JSON graph, sends it over WebSocket to a studio-manager daemon, which forks and execs the node processes and wires them together via UUID-addressed memory flows.

The long-term goal: a fully modular, vendor-neutral broadcast backbone — live production, ingest, playout — where the signal graph is code you can version-control and deploy like any other infrastructure.

---

Building a node-based broadcast signal processor where every node is an isolated C++ process exchanging video and audio through MXL shared memory over a TAI clock grid. Think of it as a modular patchbay you assemble visually and run on bare metal — no GPU, no frameworks, just shared memory and tight timing.

This stage we got a live SDI → PiP → SDI chain running from a browser UI. Here's what it took.

---

**What shipped**

- **PiP node** — composites two live video flows into one. Background drives the clock, inset overlays with V210 10-bit bilinear scaling. Supports any combination of sources: testpattern, NDI In, DeckLink In.

- **gaindb node** — audio gain/attenuation node. Reads a continuous MXL audio flow, applies dB gain via `pow(10, gain_db/20)` per sample, writes back out. Chain it anywhere between a source and a sink.

- **DeckLink In / NDI In** — both now write to the correct TAI grain index. Turns out there was a double-pacing bug: the capture loop was blocking on hardware frame delivery AND sleeping to the MXL clock — two pacers fighting each other, causing silent frame drops when they drifted apart. Fixed by resolving `mxlGetCurrentIndex()` after the hardware frame lands, not before.

- **Frontend** — Vue Flow canvas for building pipelines. Fixed a type coercion bug where select inputs (device port, sample rate) were storing strings, which made nlohmann/json throw on the C++ side. Also cleaned up auto-detected params: DeckLink and NDI sources don't expose resolution/fps in the UI anymore since they probe from the signal.

---

**The hard parts**

**MXL sync groups and TAI time.**
MXL uses TAI (not UTC, not wall clock — TAI, currently offset by 37 seconds from UTC). Grain indices are absolute TAI nanosecond timestamps divided by frame period. When we first wired up the PiP sync group, it was silently passing grain index (~44 billion) instead of TAI nanoseconds (~1.784 × 10¹⁸). The node exited in under a millisecond with status 0 and no log. We had to read MXL internals — `flow.cpp`, `Timing.hpp`, `FlowSynchronizationGroup.cpp` — to figure out that `mxlFlowSynchronizationGroupWaitForDataAt` expects nanoseconds since TAI epoch, not a frame counter.

**Sync groups don't work with hardware inputs.**
Even after fixing the timestamp, sync groups kept failing with TOO_LATE cascades when mixing testpattern with DeckLink. The reason: sync groups require both flows to have a grain at the *exact same TAI index*. A software source (testpattern) writes precisely on the TAI grid. A hardware source (DeckLink) writes to whatever index is current when the frame arrives from the SDI callback — which has hardware jitter of a few milliseconds. Even one miss causes a timeout, the node jumps forward, and you get a cascade.

The fix: drop sync groups for hardware-mixed inputs. Background is the master clock (blocking grain read). Inset is best-effort — try the exact index, fall back to `ri.headIndex` from the MXL ring buffer. One frame stale on the inset is visually invisible in a PiP.

Sync groups are still the right tool when both sources are locked to the same TAI reference via PTP — for example two genlocked DeckLink inputs with `ptp4l` + `phc2sys` on Linux syncing `CLOCK_TAI` to a PTP grandmaster (SMPTE ST 2059-2). On macOS dev machines we just live with best-effort.

**Bilinear scaling at 1278×720 was too slow.**
First test: testpattern background + DeckLink inset at configured 1280×720 PiP size → PiP running at 17fps instead of 25. The culprit was inside `scale_and_overlay`: for every output pixel on every row it was recomputing the source X coordinate — `sx = (dx + 0.5) * scale - 0.5`, floor, clamp, fractional weight. That's ~1.6 million float multiplies per frame that produce the same result on every row. Fixed by precomputing an X sample map (x0, x1, fx, 1-fx) once per call. Scale time dropped from ~32ms to ~8ms, PiP runs at 25fps with headroom.

---

**Architecture in one picture**

```
[testpattern]──video──┐
                      ├──[pip]──video──[decklinkout]
[decklinkin]──video───┘
     │
     └──audio──[gaindb]──audio──[decklinkout]
```

Every box is a separate process. Arrows are MXL shared memory flows — ring buffers in `/dev/shm`, addressed by UUID, timestamped in TAI. The studio-manager forks and execs nodes, injects `NODE_CONFIG` as a JSON env var, and monitors for crashes. The browser UI builds the graph and sends it over WebSocket.

---

**What's next**

- More processing nodes: audio mixer, video switcher/mixer (A/B cut, dissolve)
- PTP integration for production use — genlocked multi-source sync via `ptp4l` + `phc2sys` → `CLOCK_TAI`, enabling sync groups across hardware sources
- Graph persistence and live reconfiguration without full restart
- NDI discovery UI — pick sources by name, not by index number
- Proper crash recovery in studio-manager — restart crashed nodes and reconnect flows

**The bigger step: Kubernetes.**

Right now every DMF Studio node is a process on the same machine, exchanging video through `/dev/shm`. That works — and it's fast, zero-copy, nanosecond-timestamped. But it doesn't scale beyond one box.

The cluster layer for this is [`mxl-k8s`](https://github.com/qvest-digital/mxl-k8s) — a Kubernetes control plane for MXL built around the EBU Dynamic Media Facility Reference Architecture (V2.0, April 2026). The idea: each DMF Studio node becomes a pod. Flows that cross machine boundaries are handled transparently by a per-node gateway DaemonSet that owns the `libmxl-fabrics` handles (RDMA/RoCEv2/EFA/TCP), drives the cross-node grain transfer loop, and recovers on restarts. The media function itself never touches `libmxl-fabrics` — it still calls `mxlCreateFlowReader` against its local domain. An LD_PRELOAD shim intercepts the first access to a not-yet-materialised remote flow and blocks until the gateway has mirrored it locally.

The control plane: an agent DaemonSet watches each node's MXL domain via `fanotify` and publishes flows to the Kubernetes API. A cluster-scoped operator reconciles `MxlReceiver` intent ("this pod wants to consume that flow") into `MxlFlowMirror` objects, with ref-counted sharing when multiple consumers on the same node want the same flow.

What this means for DMF Studio: a testpattern pod on node A, a PiP pod on node B, a DeckLink Out pod on node C — wired together in the browser exactly the same way as today, with the cluster handling the fabric underneath. Horizontal scaling, failure isolation, and fabric rollout (from TCP in dev to RDMA in production) become Kubernetes operational concerns, not application code.

Demo video attached.