# DMF Studio — Architecture & Core Concepts

A node-based media signal processing system built on the EBU Dynamic Media Facility (DMF) architecture and the open-source MXL SDK.

---

## 1. What This Project Is

A software-defined broadcast processing environment where media workflows are built as graphs of independent processing nodes. Think of it like a virtual patchbay and signal processor combined — but running entirely in software, on standard compute.

**Example pipelines:**
```
Blackmagic SDI → [ingest node] → MXL → [denoise node] → MXL → [NDI-out node]
MXL video in → [PiP node] → MXL → [encoder node] → file
```

**Three planned user-facing layers:**
- **Nodes** — C++ programs that read/write media via MXL
- **Studio Manager** — orchestrator that launches nodes and manages the graph
- **Frontend** — Vue.js node editor (not yet built)

---

## 2. DMF Architecture Context

The EBU DMF Reference Architecture (White Paper v2.0, April 2026) defines a 6-layer stack:

```
┌─────────────────────────────┐
│   Application & UI          │  ← Vue.js node editor (future)
├─────────────────────────────┤
│   Media Functions           │  ← our nodes (testpattern, denoise, etc.)
├─────────────────────────────┤
│   Media Exchange (MXL)      │  ← shared memory flows between nodes
├─────────────────────────────┤
│   Container Platform        │  ← Docker / k8s (future)
├─────────────────────────────┤
│   Host Platform             │  ← OS, drivers
├─────────────────────────────┤
│   Infrastructure            │  ← physical servers, NICs, GPUs
└─────────────────────────────┘
```

Cross-cutting concerns (vertical columns): **Orchestration | Control | Monitoring | Security**

We are currently building the Media Functions layer and a simplified Orchestration stub (studio-manager). The Container Platform (k8s) comes later.

---

## 3. MXL — The Media Bus

**MXL (Media eXchange Layer)** is the open-source C++ SDK that implements the DMF Media Exchange layer.
Source: `./mxl/` (Apache 2.0 license)
Docs: `./mxl/docs/`

### Core Model

```
Domain (a directory in tmpfs)
└── <flow-uuid>.mxl-flow/
    ├── data              ← ring buffer metadata (mmap'd)
    ├── flow_def.json     ← NMOS IS-04 flow description
    └── grains/           ← one mmap'd file per ring slot (video)
        channels          ← single mmap'd file (audio)
```

| Term | Meaning |
|------|---------|
| **Domain** | A directory path (typically `/dev/shm/dmf-studio`). Multiple can coexist. |
| **Flow** | A ring buffer of media data, identified by a UUID. |
| **Grain** | One unit of a discrete flow — a single video frame. |
| **Sample** | One unit of a continuous flow — one audio sample per channel. |
| **Instance** | A connection from one process to a domain (`mxlCreateInstance`). |
| **Writer** | One writer per flow (only one process can write). |
| **Reader** | Multiple readers allowed per flow. |

### Flow Types

| Type | Media | API |
|------|-------|-----|
| Discrete | video/v210, ancillary data | `OpenGrain` / `CommitGrain` / `GetGrain` |
| Continuous | audio/float32 | `OpenSamples` / `CommitSamples` / `GetSamples` |

### Inter-host (Fabrics API)

MXL also supports RDMA between hosts via libfabric (TCP/Verbs/EFA). Currently in beta. Addressing stays the same — `{host_id, domain_id, flow_id}` — the SDK handles transport transparently.

---

## 4. Node — Core Pattern

Each node is an **independent OS process**. This maps directly to a k8s Pod when containerized.

### What every node does

```
startup:
  read NODE_CONFIG (JSON from env var)
  read MXL_DOMAIN (path from env var)
  mxlCreateInstance(domain)
  create FlowWriter(s) and/or FlowReader(s)

loop until SIGTERM:
  [read input grains]
  process
  [write output grains]
  sleep to next frame time

shutdown:
  mxlReleaseFlowWriter / mxlReleaseFlowReader
  mxlDestroyInstance
```

### Threading per node

```
main thread
├── setup & teardown
└── processing loop (MXL read → process → MXL write)

implicit:
└── signal handler sets g_running = false → loop exits cleanly
```

No separate control thread needed for v1. For future reconfiguration, add a thread that listens on a Unix socket or WebSocket.

### NODE_CONFIG format

Passed as a JSON string in the `NODE_CONFIG` environment variable.

```json
{
  "node_id":  "testpattern",
  "flow_id":  "5fbec3b1-1b0f-417d-9059-8b94a47197ed",
  "width":    1920,
  "height":   1080,
  "fps_num":  25,
  "fps_den":  1
}
```

Nodes never care about who launched them or where other nodes are. They only care about flow UUIDs and domain path. This is what makes k8s migration trivial.

---

## 5. MXL API Patterns

### Writer (output node)

```cpp
// 1. Create instance (one per process)
mxlInstance inst = mxlCreateInstance("/dev/shm/dmf-studio", nullptr);

// 2. Create writer from NMOS IS-04 JSON
mxlFlowWriter    writer{};
mxlFlowConfigInfo configInfo{};
bool created = false;
mxlCreateFlowWriter(inst, flow_def_json, nullptr, &writer, &configInfo, &created);

// MXL tells you the actual line stride — never hardcode it
uint32_t stride = configInfo.discrete.sliceSizes[0]; // bytes per V210 line

// 3. Write loop
mxlRational rate = {25, 1};
uint64_t index = mxlGetCurrentIndex(&rate);

while (running) {
    mxlGrainInfo grain{};
    uint8_t* buf = nullptr;
    mxlFlowWriterOpenGrain(writer, index, &grain, &buf);

    // buf is mmap'd memory — write directly into it
    fill_frame(buf, width, height, stride);

    grain.flags = 0; // MXL_GRAIN_FLAG_INVALID if you couldn't fill
    mxlFlowWriterCommitGrain(writer, &grain);

    // Sleep until next frame's start time
    uint64_t ns = mxlGetNsUntilIndex(index + 1, &rate);
    if (ns > 0 && ns < 2'000'000'000ULL)
        mxlSleepForNs(ns);

    index++;
}

// 4. Cleanup
mxlReleaseFlowWriter(inst, writer);
mxlDestroyInstance(inst);
```

### Reader (input node)

```cpp
mxlInstance inst = mxlCreateInstance("/dev/shm/dmf-studio", nullptr);

// Wait for the writer to start
bool active = false;
while (!active) {
    mxlIsFlowActive(inst, flow_id, &active);
    if (!active) mxlSleepForNs(100'000'000);
}

mxlFlowReader reader{};
mxlCreateFlowReader(inst, flow_id, nullptr, &reader);

mxlRational rate = {25, 1};
uint64_t index = mxlGetCurrentIndex(&rate);
uint64_t timeout_ns = 2'000'000'000ULL / 25; // 2 frame durations

while (running) {
    mxlGrainInfo grain{};
    uint8_t* buf = nullptr;

    mxlStatus st = mxlFlowReaderGetGrain(reader, index, timeout_ns, &grain, &buf);

    if (st == MXL_STATUS_OK) {
        process(buf, grain.grainSize);

    } else if (st == MXL_ERR_OUT_OF_RANGE_TOO_LATE) {
        // Fell behind the ring buffer — jump to live head
        index = mxlGetCurrentIndex(&rate);
        continue;

    } else if (st == MXL_ERR_TIMEOUT) {
        // Writer may have stalled — log and continue
    }

    index++;
}
```

### Multi-input node (e.g. PiP — synchronize two flows)

```cpp
mxlFlowSynchronizationGroup group{};
mxlCreateFlowSynchronizationGroup(inst, &group);
mxlFlowSynchronizationGroupAddReader(group, reader_a);
mxlFlowSynchronizationGroupAddReader(group, reader_b);

mxlRational rate = {25, 1};
uint64_t nextIndex = mxlGetCurrentIndex(&rate);

while (running) {
    uint64_t ts = mxlIndexToTimestamp(&rate, nextIndex);
    // Blocks until BOTH flows have data at this timestamp
    mxlFlowSynchronizationGroupWaitForDataAt(group, ts, timeout_ns);

    // Now safe to read both
    mxlFlowReaderGetGrain(reader_a, nextIndex, 0, &grain_a, &buf_a);
    mxlFlowReaderGetGrain(reader_b, nextIndex, 0, &grain_b, &buf_b);

    composite_pip(buf_a, buf_b, out_buf);
    nextIndex++;
}
```

---

## 6. V210 Video Format

MXL's video format is `video/v210` — 10-bit 4:2:2 YCbCr.

### 10-bit limited range values

| Signal | Black | White | Neutral (chroma) |
|--------|-------|-------|-----------------|
| Y      | 64    | 940   | —               |
| Cb/Cr  | 64    | 960   | 512             |

### Packing: 6 pixels → 16 bytes (4 × 32-bit words)

In 4:2:2, pixels are paired — each pair shares one Cb and one Cr sample.

```
Word 0 [bits 9:0][19:10][29:20]:  Cb(pair0) | Y(px0) | Cr(pair0)
Word 1:                            Y(px1)    | Cb(pair1)| Y(px2)
Word 2:                            Cr(pair1) | Y(px3)  | Cb(pair2)
Word 3:                            Y(px4)    | Cr(pair2)| Y(px5)
```

### Line size

For 1920 pixels: `1920/6 * 16 = 5120 bytes/line`
For 1920×1080 frame: `5120 * 1080 = 5,529,600 bytes ≈ 5.3 MB/frame`

**Always get the stride from MXL** (`configInfo.discrete.sliceSizes[0]`), not hardcoded.

### SMPTE 75% color bar values (10-bit)

| Bar     | Y   | Cb  | Cr  |
|---------|-----|-----|-----|
| White   | 721 | 512 | 512 |
| Yellow  | 674 | 176 | 543 |
| Cyan    | 581 | 589 | 176 |
| Green   | 534 | 253 | 207 |
| Magenta | 251 | 771 | 817 |
| Red     | 204 | 435 | 848 |
| Blue    | 111 | 848 | 481 |

---

## 7. Timing System

MXL uses **TAI (International Atomic Time)**, which is ahead of UTC by ~37 seconds. All timing goes through MXL's time API — do not mix with `std::chrono::system_clock` for frame timing.

| Function | Purpose |
|----------|---------|
| `mxlGetTime()` | Current time in nanoseconds (TAI) |
| `mxlGetCurrentIndex(&rate)` | Current frame index at system time |
| `mxlGetNsUntilIndex(index, &rate)` | Nanoseconds until that index starts |
| `mxlIndexToTimestamp(&rate, index)` | Index → TAI nanoseconds |
| `mxlTimestampToIndex(&rate, ts)` | TAI nanoseconds → index |
| `mxlSleepForNs(ns)` | Relative sleep in nanoseconds |
| `mxlSleepUntil(ts)` | Sleep until a TAI timestamp |

The standard write-loop timing pattern:
```cpp
// Sleep until next frame time, then advance
uint64_t ns = mxlGetNsUntilIndex(index + 1, &rate);
if (ns > 0 && ns < 2'000'000'000ULL) mxlSleepForNs(ns);
index++;
```

---

## 8. NMOS IS-04 Flow Definition

MXL requires an NMOS IS-04-compatible JSON to define a flow when creating a writer. This JSON is also stored as `flow_def.json` in the flow directory, making flows self-describing.

Minimum required fields for `video/v210`:

```json
{
  "id":             "<uuid>",
  "format":         "urn:x-nmos:format:video",
  "label":          "My Node Video Output",
  "description":    "My Node MXL Video Flow",
  "media_type":     "video/v210",
  "parents":        [],
  "grain_rate":     { "numerator": 25, "denominator": 1 },
  "frame_width":    1920,
  "frame_height":   1080,
  "interlace_mode": "progressive",
  "colorspace":     "BT709",
  "tags": {
    "urn:x-nmos:tag:grouphint/v1.0": ["My Node:Video"]
  },
  "components": [
    { "name": "Y",  "width": 1920, "height": 1080, "bit_depth": 10 },
    { "name": "Cb", "width": 960,  "height": 1080, "bit_depth": 10 },
    { "name": "Cr", "width": 960,  "height": 1080, "bit_depth": 10 }
  ]
}
```

Generated by `shared/FlowDef.hpp → dmf::make_video_flow_def(...)`.

---

## 9. Studio Manager

The control plane. Owns:
- Flow UUID generation
- Node process lifecycle (fork/exec with env vars)
- Graph state (v1: hardcoded; future: graph.json + WebSocket API)

Does **not** touch media. All media flows through MXL between node processes.

### Process model

```
studio-manager (pid X)
├── fork → dmf-node-testpattern (pid A)  env: NODE_CONFIG, MXL_DOMAIN
└── fork → dmf-node-fakesink    (pid B)  env: NODE_CONFIG, MXL_DOMAIN
```

On SIGINT/SIGTERM: studio-manager sends SIGTERM to children, then waits.

### v1 hardcoded graph

```
testpattern → [video flow UUID] → fakesink
```

### Future graph.json

```json
{
  "nodes": [
    { "id": "tp1", "type": "testpattern", "config": { ... } },
    { "id": "sink1", "type": "fakesink", "config": { ... } }
  ],
  "flows": [
    { "id": "<uuid>", "from": "tp1", "to": ["sink1"] }
  ]
}
```

---

## 10. Project File Structure

```
dmf-studio-rnd/
├── mxl/                        ← MXL SDK (open-source, Apache 2.0)
│   ├── lib/include/mxl/        ← public headers (mxl.h, flow.h, time.h, ...)
│   ├── lib/src/                ← C++ implementation
│   ├── docs/                   ← Architecture.md, Fabrics.md, Usage.md, ...
│   └── examples/               ← Docker + k8s examples
│
├── shared/                     ← header-only utilities, all targets link here
│   ├── Signal.hpp              ← g_running atomic + SIGTERM/SIGINT handler
│   ├── FlowDef.hpp             ← NMOS IS-04 JSON generator for video/v210
│   └── V210.hpp                ← SMPTE 75% color bars V210 packer
│
├── nodes/
│   ├── testpattern/main.cpp    ← generates color bars, writes MXL video flow
│   └── fakesink/main.cpp       ← reads MXL video flow, logs frame statistics
│
├── studio-manager/main.cpp     ← launches & monitors node processes
│
├── CMakeLists.txt              ← root build: links mxl, fetches nlohmann/json
└── ARCHITECTURE.md             ← this file
```

---

## 11. Build System

CMake 3.24+, C++20.

```bash
# First time
cmake -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j$(nproc)

# After source changes
cmake --build build -j$(nproc)
```

MXL is included via `add_subdirectory(mxl)` with tools/tests/docs/utils disabled. The core MXL library has no external dependencies.

`nlohmann/json` is fetched automatically by CMake on first configure (requires internet access once).

### Binaries produced

| Binary | Path |
|--------|------|
| Studio manager | `build/studio-manager/dmf-studio-manager` |
| TestPattern node | `build/nodes/testpattern/dmf-node-testpattern` |
| FakeSink node | `build/nodes/fakesink/dmf-node-fakesink` |

### Running

```bash
# Run everything (studio-manager launches node processes automatically)
./build/studio-manager/dmf-studio-manager

# Or run a node manually for testing
export MXL_DOMAIN=/dev/shm/dmf-studio
export NODE_CONFIG='{"node_id":"testpattern","flow_id":"5fbec3b1-1b0f-417d-9059-8b94a47197ed","width":1920,"height":1080,"fps_num":25,"fps_den":1}'
mkdir -p $MXL_DOMAIN
./build/nodes/testpattern/dmf-node-testpattern
```

On macOS, `/dev/shm` does not exist — use `/tmp/dmf-studio` instead (no RAM-backing, but works for development).

---

## 12. What Does Not Exist Yet

| Feature | Notes |
|---------|-------|
| Audio nodes | MXL continuous flow API — `OpenSamples/CommitSamples/GetSamples` |
| Real processing nodes | denoise, PiP, format converter, encoder |
| External device nodes | Blackmagic DeckLink ingest, NDI output, ST 2110 |
| Vue.js frontend | Node graph editor; talks to studio-manager via WebSocket |
| graph.json | Persist and load graph state from file |
| WebSocket API | Studio-manager exposes REST+WebSocket for frontend |
| Reconfiguration | Nodes currently only configured at startup |
| Multi-host flows | MXL Fabrics API (libfabric, beta) for RDMA between hosts |
| k8s deployment | Replace studio-manager fork/exec with Pod specs + ConfigMaps |
| Security | MXL uses UNIX file permissions; future: TLS 1.3 + OAuth2 |

---

## 13. K8s Migration Path

The architecture is designed so **node code doesn't change** when moving to k8s.

| Now | With k8s |
|-----|----------|
| `fork/exec` in studio-manager | `kubectl apply` Pod spec |
| `NODE_CONFIG` env var set by studio-manager | `NODE_CONFIG` in Pod spec `env:` from ConfigMap |
| `MXL_DOMAIN=/dev/shm/dmf-studio` | `MXL_DOMAIN=/mnt/shm` as `hostPath` volume shared between Pods on same node |
| Studio-manager monitors children | k8s restarts failed pods |
| Studio-manager generates flow UUIDs | A lightweight k8s controller or init container |

The MXL domain on k8s requires all pods in the same pipeline to run on the same physical node (same shared memory). Use node affinity rules or a DaemonSet pattern.

---

## 14. Key Dependencies Reference

| Dependency | Version | Purpose | How included |
|------------|---------|---------|--------------|
| MXL SDK | 1.2.0-dev | Shared-memory media exchange | `add_subdirectory(mxl)` |
| nlohmann/json | 3.11.3 | JSON config parsing, flow def generation | CMake FetchContent |
| C++20 stdlib | — | Threading, filesystem, chrono | System |

MXL itself uses (only needed if building MXL tools/tests): `stduuid`, `spdlog`, `fmt`, `picojson`, `CLI11`, `catch2` — all via vcpkg. Not needed for our build.