Initial commit: testpattern → fakesink pipeline over MXL shared memory
- NodeBase, Signal, FlowDef, V210 shared headers - testpattern node: SMPTE 75% color bars writer at 25fps - fakesink node: non-blocking MXL reader with per-second stats - studio-manager: FlowGraph data model, graph-driven fork/exec launcher - mxl pinned as submodule at 0ae1dc5 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
JohannesItten
+511
@@ -0,0 +1,511 @@
|
||||
# DMF Studio — Architecture & Core Concepts
|
||||
|
||||
A node-based media signal processing system built on the EBU Dynamic Media Facility (DMF) architecture and the open-source MXL SDK.
|
||||
|
||||
---
|
||||
|
||||
## 1. What This Project Is
|
||||
|
||||
A software-defined broadcast processing environment where media workflows are built as graphs of independent processing nodes. Think of it like a virtual patchbay and signal processor combined — but running entirely in software, on standard compute.
|
||||
|
||||
**Example pipelines:**
|
||||
```
|
||||
Blackmagic SDI → [ingest node] → MXL → [denoise node] → MXL → [NDI-out node]
|
||||
MXL video in → [PiP node] → MXL → [encoder node] → file
|
||||
```
|
||||
|
||||
**Three planned user-facing layers:**
|
||||
- **Nodes** — C++ programs that read/write media via MXL
|
||||
- **Studio Manager** — orchestrator that launches nodes and manages the graph
|
||||
- **Frontend** — Vue.js node editor (not yet built)
|
||||
|
||||
---
|
||||
|
||||
## 2. DMF Architecture Context
|
||||
|
||||
The EBU DMF Reference Architecture (White Paper v2.0, April 2026) defines a 6-layer stack:
|
||||
|
||||
```
|
||||
┌─────────────────────────────┐
|
||||
│ Application & UI │ ← Vue.js node editor (future)
|
||||
├─────────────────────────────┤
|
||||
│ Media Functions │ ← our nodes (testpattern, denoise, etc.)
|
||||
├─────────────────────────────┤
|
||||
│ Media Exchange (MXL) │ ← shared memory flows between nodes
|
||||
├─────────────────────────────┤
|
||||
│ Container Platform │ ← Docker / k8s (future)
|
||||
├─────────────────────────────┤
|
||||
│ Host Platform │ ← OS, drivers
|
||||
├─────────────────────────────┤
|
||||
│ Infrastructure │ ← physical servers, NICs, GPUs
|
||||
└─────────────────────────────┘
|
||||
```
|
||||
|
||||
Cross-cutting concerns (vertical columns): **Orchestration | Control | Monitoring | Security**
|
||||
|
||||
We are currently building the Media Functions layer and a simplified Orchestration stub (studio-manager). The Container Platform (k8s) comes later.
|
||||
|
||||
---
|
||||
|
||||
## 3. MXL — The Media Bus
|
||||
|
||||
**MXL (Media eXchange Layer)** is the open-source C++ SDK that implements the DMF Media Exchange layer.
|
||||
Source: `./mxl/` (Apache 2.0 license)
|
||||
Docs: `./mxl/docs/`
|
||||
|
||||
### Core Model
|
||||
|
||||
```
|
||||
Domain (a directory in tmpfs)
|
||||
└── <flow-uuid>.mxl-flow/
|
||||
├── data ← ring buffer metadata (mmap'd)
|
||||
├── flow_def.json ← NMOS IS-04 flow description
|
||||
└── grains/ ← one mmap'd file per ring slot (video)
|
||||
channels ← single mmap'd file (audio)
|
||||
```
|
||||
|
||||
| Term | Meaning |
|
||||
|------|---------|
|
||||
| **Domain** | A directory path (typically `/dev/shm/dmf-studio`). Multiple can coexist. |
|
||||
| **Flow** | A ring buffer of media data, identified by a UUID. |
|
||||
| **Grain** | One unit of a discrete flow — a single video frame. |
|
||||
| **Sample** | One unit of a continuous flow — one audio sample per channel. |
|
||||
| **Instance** | A connection from one process to a domain (`mxlCreateInstance`). |
|
||||
| **Writer** | One writer per flow (only one process can write). |
|
||||
| **Reader** | Multiple readers allowed per flow. |
|
||||
|
||||
### Flow Types
|
||||
|
||||
| Type | Media | API |
|
||||
|------|-------|-----|
|
||||
| Discrete | video/v210, ancillary data | `OpenGrain` / `CommitGrain` / `GetGrain` |
|
||||
| Continuous | audio/float32 | `OpenSamples` / `CommitSamples` / `GetSamples` |
|
||||
|
||||
### Inter-host (Fabrics API)
|
||||
|
||||
MXL also supports RDMA between hosts via libfabric (TCP/Verbs/EFA). Currently in beta. Addressing stays the same — `{host_id, domain_id, flow_id}` — the SDK handles transport transparently.
|
||||
|
||||
---
|
||||
|
||||
## 4. Node — Core Pattern
|
||||
|
||||
Each node is an **independent OS process**. This maps directly to a k8s Pod when containerized.
|
||||
|
||||
### What every node does
|
||||
|
||||
```
|
||||
startup:
|
||||
read NODE_CONFIG (JSON from env var)
|
||||
read MXL_DOMAIN (path from env var)
|
||||
mxlCreateInstance(domain)
|
||||
create FlowWriter(s) and/or FlowReader(s)
|
||||
|
||||
loop until SIGTERM:
|
||||
[read input grains]
|
||||
process
|
||||
[write output grains]
|
||||
sleep to next frame time
|
||||
|
||||
shutdown:
|
||||
mxlReleaseFlowWriter / mxlReleaseFlowReader
|
||||
mxlDestroyInstance
|
||||
```
|
||||
|
||||
### Threading per node
|
||||
|
||||
```
|
||||
main thread
|
||||
├── setup & teardown
|
||||
└── processing loop (MXL read → process → MXL write)
|
||||
|
||||
implicit:
|
||||
└── signal handler sets g_running = false → loop exits cleanly
|
||||
```
|
||||
|
||||
No separate control thread needed for v1. For future reconfiguration, add a thread that listens on a Unix socket or WebSocket.
|
||||
|
||||
### NODE_CONFIG format
|
||||
|
||||
Passed as a JSON string in the `NODE_CONFIG` environment variable.
|
||||
|
||||
```json
|
||||
{
|
||||
"node_id": "testpattern",
|
||||
"flow_id": "5fbec3b1-1b0f-417d-9059-8b94a47197ed",
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"fps_num": 25,
|
||||
"fps_den": 1
|
||||
}
|
||||
```
|
||||
|
||||
Nodes never care about who launched them or where other nodes are. They only care about flow UUIDs and domain path. This is what makes k8s migration trivial.
|
||||
|
||||
---
|
||||
|
||||
## 5. MXL API Patterns
|
||||
|
||||
### Writer (output node)
|
||||
|
||||
```cpp
|
||||
// 1. Create instance (one per process)
|
||||
mxlInstance inst = mxlCreateInstance("/dev/shm/dmf-studio", nullptr);
|
||||
|
||||
// 2. Create writer from NMOS IS-04 JSON
|
||||
mxlFlowWriter writer{};
|
||||
mxlFlowConfigInfo configInfo{};
|
||||
bool created = false;
|
||||
mxlCreateFlowWriter(inst, flow_def_json, nullptr, &writer, &configInfo, &created);
|
||||
|
||||
// MXL tells you the actual line stride — never hardcode it
|
||||
uint32_t stride = configInfo.discrete.sliceSizes[0]; // bytes per V210 line
|
||||
|
||||
// 3. Write loop
|
||||
mxlRational rate = {25, 1};
|
||||
uint64_t index = mxlGetCurrentIndex(&rate);
|
||||
|
||||
while (running) {
|
||||
mxlGrainInfo grain{};
|
||||
uint8_t* buf = nullptr;
|
||||
mxlFlowWriterOpenGrain(writer, index, &grain, &buf);
|
||||
|
||||
// buf is mmap'd memory — write directly into it
|
||||
fill_frame(buf, width, height, stride);
|
||||
|
||||
grain.flags = 0; // MXL_GRAIN_FLAG_INVALID if you couldn't fill
|
||||
mxlFlowWriterCommitGrain(writer, &grain);
|
||||
|
||||
// Sleep until next frame's start time
|
||||
uint64_t ns = mxlGetNsUntilIndex(index + 1, &rate);
|
||||
if (ns > 0 && ns < 2'000'000'000ULL)
|
||||
mxlSleepForNs(ns);
|
||||
|
||||
index++;
|
||||
}
|
||||
|
||||
// 4. Cleanup
|
||||
mxlReleaseFlowWriter(inst, writer);
|
||||
mxlDestroyInstance(inst);
|
||||
```
|
||||
|
||||
### Reader (input node)
|
||||
|
||||
```cpp
|
||||
mxlInstance inst = mxlCreateInstance("/dev/shm/dmf-studio", nullptr);
|
||||
|
||||
// Wait for the writer to start
|
||||
bool active = false;
|
||||
while (!active) {
|
||||
mxlIsFlowActive(inst, flow_id, &active);
|
||||
if (!active) mxlSleepForNs(100'000'000);
|
||||
}
|
||||
|
||||
mxlFlowReader reader{};
|
||||
mxlCreateFlowReader(inst, flow_id, nullptr, &reader);
|
||||
|
||||
mxlRational rate = {25, 1};
|
||||
uint64_t index = mxlGetCurrentIndex(&rate);
|
||||
uint64_t timeout_ns = 2'000'000'000ULL / 25; // 2 frame durations
|
||||
|
||||
while (running) {
|
||||
mxlGrainInfo grain{};
|
||||
uint8_t* buf = nullptr;
|
||||
|
||||
mxlStatus st = mxlFlowReaderGetGrain(reader, index, timeout_ns, &grain, &buf);
|
||||
|
||||
if (st == MXL_STATUS_OK) {
|
||||
process(buf, grain.grainSize);
|
||||
|
||||
} else if (st == MXL_ERR_OUT_OF_RANGE_TOO_LATE) {
|
||||
// Fell behind the ring buffer — jump to live head
|
||||
index = mxlGetCurrentIndex(&rate);
|
||||
continue;
|
||||
|
||||
} else if (st == MXL_ERR_TIMEOUT) {
|
||||
// Writer may have stalled — log and continue
|
||||
}
|
||||
|
||||
index++;
|
||||
}
|
||||
```
|
||||
|
||||
### Multi-input node (e.g. PiP — synchronize two flows)
|
||||
|
||||
```cpp
|
||||
mxlFlowSynchronizationGroup group{};
|
||||
mxlCreateFlowSynchronizationGroup(inst, &group);
|
||||
mxlFlowSynchronizationGroupAddReader(group, reader_a);
|
||||
mxlFlowSynchronizationGroupAddReader(group, reader_b);
|
||||
|
||||
mxlRational rate = {25, 1};
|
||||
uint64_t nextIndex = mxlGetCurrentIndex(&rate);
|
||||
|
||||
while (running) {
|
||||
uint64_t ts = mxlIndexToTimestamp(&rate, nextIndex);
|
||||
// Blocks until BOTH flows have data at this timestamp
|
||||
mxlFlowSynchronizationGroupWaitForDataAt(group, ts, timeout_ns);
|
||||
|
||||
// Now safe to read both
|
||||
mxlFlowReaderGetGrain(reader_a, nextIndex, 0, &grain_a, &buf_a);
|
||||
mxlFlowReaderGetGrain(reader_b, nextIndex, 0, &grain_b, &buf_b);
|
||||
|
||||
composite_pip(buf_a, buf_b, out_buf);
|
||||
nextIndex++;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. V210 Video Format
|
||||
|
||||
MXL's video format is `video/v210` — 10-bit 4:2:2 YCbCr.
|
||||
|
||||
### 10-bit limited range values
|
||||
|
||||
| Signal | Black | White | Neutral (chroma) |
|
||||
|--------|-------|-------|-----------------|
|
||||
| Y | 64 | 940 | — |
|
||||
| Cb/Cr | 64 | 960 | 512 |
|
||||
|
||||
### Packing: 6 pixels → 16 bytes (4 × 32-bit words)
|
||||
|
||||
In 4:2:2, pixels are paired — each pair shares one Cb and one Cr sample.
|
||||
|
||||
```
|
||||
Word 0 [bits 9:0][19:10][29:20]: Cb(pair0) | Y(px0) | Cr(pair0)
|
||||
Word 1: Y(px1) | Cb(pair1)| Y(px2)
|
||||
Word 2: Cr(pair1) | Y(px3) | Cb(pair2)
|
||||
Word 3: Y(px4) | Cr(pair2)| Y(px5)
|
||||
```
|
||||
|
||||
### Line size
|
||||
|
||||
For 1920 pixels: `1920/6 * 16 = 5120 bytes/line`
|
||||
For 1920×1080 frame: `5120 * 1080 = 5,529,600 bytes ≈ 5.3 MB/frame`
|
||||
|
||||
**Always get the stride from MXL** (`configInfo.discrete.sliceSizes[0]`), not hardcoded.
|
||||
|
||||
### SMPTE 75% color bar values (10-bit)
|
||||
|
||||
| Bar | Y | Cb | Cr |
|
||||
|---------|-----|-----|-----|
|
||||
| White | 721 | 512 | 512 |
|
||||
| Yellow | 674 | 176 | 543 |
|
||||
| Cyan | 581 | 589 | 176 |
|
||||
| Green | 534 | 253 | 207 |
|
||||
| Magenta | 251 | 771 | 817 |
|
||||
| Red | 204 | 435 | 848 |
|
||||
| Blue | 111 | 848 | 481 |
|
||||
|
||||
---
|
||||
|
||||
## 7. Timing System
|
||||
|
||||
MXL uses **TAI (International Atomic Time)**, which is ahead of UTC by ~37 seconds. All timing goes through MXL's time API — do not mix with `std::chrono::system_clock` for frame timing.
|
||||
|
||||
| Function | Purpose |
|
||||
|----------|---------|
|
||||
| `mxlGetTime()` | Current time in nanoseconds (TAI) |
|
||||
| `mxlGetCurrentIndex(&rate)` | Current frame index at system time |
|
||||
| `mxlGetNsUntilIndex(index, &rate)` | Nanoseconds until that index starts |
|
||||
| `mxlIndexToTimestamp(&rate, index)` | Index → TAI nanoseconds |
|
||||
| `mxlTimestampToIndex(&rate, ts)` | TAI nanoseconds → index |
|
||||
| `mxlSleepForNs(ns)` | Relative sleep in nanoseconds |
|
||||
| `mxlSleepUntil(ts)` | Sleep until a TAI timestamp |
|
||||
|
||||
The standard write-loop timing pattern:
|
||||
```cpp
|
||||
// Sleep until next frame time, then advance
|
||||
uint64_t ns = mxlGetNsUntilIndex(index + 1, &rate);
|
||||
if (ns > 0 && ns < 2'000'000'000ULL) mxlSleepForNs(ns);
|
||||
index++;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. NMOS IS-04 Flow Definition
|
||||
|
||||
MXL requires an NMOS IS-04-compatible JSON to define a flow when creating a writer. This JSON is also stored as `flow_def.json` in the flow directory, making flows self-describing.
|
||||
|
||||
Minimum required fields for `video/v210`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "<uuid>",
|
||||
"format": "urn:x-nmos:format:video",
|
||||
"label": "My Node Video Output",
|
||||
"description": "My Node MXL Video Flow",
|
||||
"media_type": "video/v210",
|
||||
"parents": [],
|
||||
"grain_rate": { "numerator": 25, "denominator": 1 },
|
||||
"frame_width": 1920,
|
||||
"frame_height": 1080,
|
||||
"interlace_mode": "progressive",
|
||||
"colorspace": "BT709",
|
||||
"tags": {
|
||||
"urn:x-nmos:tag:grouphint/v1.0": ["My Node:Video"]
|
||||
},
|
||||
"components": [
|
||||
{ "name": "Y", "width": 1920, "height": 1080, "bit_depth": 10 },
|
||||
{ "name": "Cb", "width": 960, "height": 1080, "bit_depth": 10 },
|
||||
{ "name": "Cr", "width": 960, "height": 1080, "bit_depth": 10 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Generated by `shared/FlowDef.hpp → dmf::make_video_flow_def(...)`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Studio Manager
|
||||
|
||||
The control plane. Owns:
|
||||
- Flow UUID generation
|
||||
- Node process lifecycle (fork/exec with env vars)
|
||||
- Graph state (v1: hardcoded; future: graph.json + WebSocket API)
|
||||
|
||||
Does **not** touch media. All media flows through MXL between node processes.
|
||||
|
||||
### Process model
|
||||
|
||||
```
|
||||
studio-manager (pid X)
|
||||
├── fork → dmf-node-testpattern (pid A) env: NODE_CONFIG, MXL_DOMAIN
|
||||
└── fork → dmf-node-fakesink (pid B) env: NODE_CONFIG, MXL_DOMAIN
|
||||
```
|
||||
|
||||
On SIGINT/SIGTERM: studio-manager sends SIGTERM to children, then waits.
|
||||
|
||||
### v1 hardcoded graph
|
||||
|
||||
```
|
||||
testpattern → [video flow UUID] → fakesink
|
||||
```
|
||||
|
||||
### Future graph.json
|
||||
|
||||
```json
|
||||
{
|
||||
"nodes": [
|
||||
{ "id": "tp1", "type": "testpattern", "config": { ... } },
|
||||
{ "id": "sink1", "type": "fakesink", "config": { ... } }
|
||||
],
|
||||
"flows": [
|
||||
{ "id": "<uuid>", "from": "tp1", "to": ["sink1"] }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Project File Structure
|
||||
|
||||
```
|
||||
dmf-studio-rnd/
|
||||
├── mxl/ ← MXL SDK (open-source, Apache 2.0)
|
||||
│ ├── lib/include/mxl/ ← public headers (mxl.h, flow.h, time.h, ...)
|
||||
│ ├── lib/src/ ← C++ implementation
|
||||
│ ├── docs/ ← Architecture.md, Fabrics.md, Usage.md, ...
|
||||
│ └── examples/ ← Docker + k8s examples
|
||||
│
|
||||
├── shared/ ← header-only utilities, all targets link here
|
||||
│ ├── Signal.hpp ← g_running atomic + SIGTERM/SIGINT handler
|
||||
│ ├── FlowDef.hpp ← NMOS IS-04 JSON generator for video/v210
|
||||
│ └── V210.hpp ← SMPTE 75% color bars V210 packer
|
||||
│
|
||||
├── nodes/
|
||||
│ ├── testpattern/main.cpp ← generates color bars, writes MXL video flow
|
||||
│ └── fakesink/main.cpp ← reads MXL video flow, logs frame statistics
|
||||
│
|
||||
├── studio-manager/main.cpp ← launches & monitors node processes
|
||||
│
|
||||
├── CMakeLists.txt ← root build: links mxl, fetches nlohmann/json
|
||||
└── ARCHITECTURE.md ← this file
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Build System
|
||||
|
||||
CMake 3.24+, C++20.
|
||||
|
||||
```bash
|
||||
# First time
|
||||
cmake -B build -DCMAKE_BUILD_TYPE=Debug
|
||||
cmake --build build -j$(nproc)
|
||||
|
||||
# After source changes
|
||||
cmake --build build -j$(nproc)
|
||||
```
|
||||
|
||||
MXL is included via `add_subdirectory(mxl)` with tools/tests/docs/utils disabled. The core MXL library has no external dependencies.
|
||||
|
||||
`nlohmann/json` is fetched automatically by CMake on first configure (requires internet access once).
|
||||
|
||||
### Binaries produced
|
||||
|
||||
| Binary | Path |
|
||||
|--------|------|
|
||||
| Studio manager | `build/studio-manager/dmf-studio-manager` |
|
||||
| TestPattern node | `build/nodes/testpattern/dmf-node-testpattern` |
|
||||
| FakeSink node | `build/nodes/fakesink/dmf-node-fakesink` |
|
||||
|
||||
### Running
|
||||
|
||||
```bash
|
||||
# Run everything (studio-manager launches node processes automatically)
|
||||
./build/studio-manager/dmf-studio-manager
|
||||
|
||||
# Or run a node manually for testing
|
||||
export MXL_DOMAIN=/dev/shm/dmf-studio
|
||||
export NODE_CONFIG='{"node_id":"testpattern","flow_id":"5fbec3b1-1b0f-417d-9059-8b94a47197ed","width":1920,"height":1080,"fps_num":25,"fps_den":1}'
|
||||
mkdir -p $MXL_DOMAIN
|
||||
./build/nodes/testpattern/dmf-node-testpattern
|
||||
```
|
||||
|
||||
On macOS, `/dev/shm` does not exist — use `/tmp/dmf-studio` instead (no RAM-backing, but works for development).
|
||||
|
||||
---
|
||||
|
||||
## 12. What Does Not Exist Yet
|
||||
|
||||
| Feature | Notes |
|
||||
|---------|-------|
|
||||
| Audio nodes | MXL continuous flow API — `OpenSamples/CommitSamples/GetSamples` |
|
||||
| Real processing nodes | denoise, PiP, format converter, encoder |
|
||||
| External device nodes | Blackmagic DeckLink ingest, NDI output, ST 2110 |
|
||||
| Vue.js frontend | Node graph editor; talks to studio-manager via WebSocket |
|
||||
| graph.json | Persist and load graph state from file |
|
||||
| WebSocket API | Studio-manager exposes REST+WebSocket for frontend |
|
||||
| Reconfiguration | Nodes currently only configured at startup |
|
||||
| Multi-host flows | MXL Fabrics API (libfabric, beta) for RDMA between hosts |
|
||||
| k8s deployment | Replace studio-manager fork/exec with Pod specs + ConfigMaps |
|
||||
| Security | MXL uses UNIX file permissions; future: TLS 1.3 + OAuth2 |
|
||||
|
||||
---
|
||||
|
||||
## 13. K8s Migration Path
|
||||
|
||||
The architecture is designed so **node code doesn't change** when moving to k8s.
|
||||
|
||||
| Now | With k8s |
|
||||
|-----|----------|
|
||||
| `fork/exec` in studio-manager | `kubectl apply` Pod spec |
|
||||
| `NODE_CONFIG` env var set by studio-manager | `NODE_CONFIG` in Pod spec `env:` from ConfigMap |
|
||||
| `MXL_DOMAIN=/dev/shm/dmf-studio` | `MXL_DOMAIN=/mnt/shm` as `hostPath` volume shared between Pods on same node |
|
||||
| Studio-manager monitors children | k8s restarts failed pods |
|
||||
| Studio-manager generates flow UUIDs | A lightweight k8s controller or init container |
|
||||
|
||||
The MXL domain on k8s requires all pods in the same pipeline to run on the same physical node (same shared memory). Use node affinity rules or a DaemonSet pattern.
|
||||
|
||||
---
|
||||
|
||||
## 14. Key Dependencies Reference
|
||||
|
||||
| Dependency | Version | Purpose | How included |
|
||||
|------------|---------|---------|--------------|
|
||||
| MXL SDK | 1.2.0-dev | Shared-memory media exchange | `add_subdirectory(mxl)` |
|
||||
| nlohmann/json | 3.11.3 | JSON config parsing, flow def generation | CMake FetchContent |
|
||||
| C++20 stdlib | — | Threading, filesystem, chrono | System |
|
||||
|
||||
MXL itself uses (only needed if building MXL tools/tests): `stduuid`, `spdlog`, `fmt`, `picojson`, `CLI11`, `catch2` — all via vcpkg. Not needed for our build.
|
||||
Reference in New Issue
Block a user