combiner.md

2026-07-09 15:49:35 +03:00
parent d2095d14d3
commit 3820d0eeb7
3 changed files with 167 additions and 7 deletions
@@ -0,0 +1,138 @@
+# AV Combiner Node — Implementation Steps
+
+Takes video from one upstream node and audio from another, outputs both as new MXL flows.
+Does NOT need a sync group — audio and video run at different rates and are handled independently.
+
+---
+
+## Step 1 — Frontend (`dmf-studio-ui/src/nodeTypes.ts`)
+
+Add the node definition. Port IDs use `video_in-in` / `video_out-out` pattern so that
+`handleToPort()` produces distinct keys (`video_in_flow_id` vs `video_out_flow_id`).
+
+```typescript
+avcombiner: {
+  type: 'avcombiner',
+  label: 'AV Combiner',
+  ports: [
+    { id: 'video_in-in',   kind: 'video', direction: 'in'  },
+    { id: 'audio_in-in',   kind: 'audio', direction: 'in'  },
+    { id: 'video_out-out', kind: 'video', direction: 'out' },
+    { id: 'audio_out-out', kind: 'audio', direction: 'out' },
+  ],
+  params: [],
+},
+```
+
+Config keys the node receives:
+- `video_in_flow_id.id`  — input video flow UUID
+- `audio_in_flow_id.id`  — input audio flow UUID
+- `video_out_flow_id.id` — output video flow UUID
+- `audio_out_flow_id.id` — output audio flow UUID
+
+---
+
+## Step 2 — CMake
+
+Create `nodes/avcombiner/CMakeLists.txt`:
+
+```cmake
+add_executable(dmf-node-avcombiner main.cpp)
+target_compile_features(dmf-node-avcombiner PRIVATE cxx_std_20)
+target_link_libraries(dmf-node-avcombiner PRIVATE dmf-shared)
+install(TARGETS dmf-node-avcombiner RUNTIME DESTINATION bin)
+```
+
+In the root `CMakeLists.txt`, add alongside the other nodes:
+
+```cmake
+add_subdirectory(nodes/avcombiner)
+```
+
+---
+
+## Step 3 — `nodes/avcombiner/main.cpp`
+
+Structure (follow the same patterns as `ndiout`):
+
+```
+1. Check video_in_flow_id and video_out_flow_id both present — exit if not
+2. Check audio_in_flow_id and audio_out_flow_id (optional — audio is optional)
+
+3. Wait for video input flow active  (mxlIsFlowActive loop, 100ms sleep)
+4. Create video input reader         (mxlCreateFlowReader)
+5. Get video config info             (mxlFlowReaderGetConfigInfo → video_stride)
+6. Read video format from flow_def   (dmf::read_video_flow_info(domain(), flow_id))
+   → width, height, fps_num, fps_den
+
+7. If has_audio:
+   Wait for audio input flow active
+   Create audio input reader
+   Read audio format from flow_def   (dmf::read_audio_flow_info(domain(), flow_id))
+   → sample_rate, channels, samples_per_grain
+
+8. Create video output writer        (mxlCreateFlowWriter with make_video_flow_def)
+   → video_out_stride from configInfo.discrete.sliceSizes[0]
+
+9. If has_audio:
+   Create audio output writer        (mxlCreateFlowWriter with make_audio_flow_def)
+
+10. Init clocks:
+    video_index = mxlGetCurrentIndex(&video_rate)
+    audio_index = mxlGetCurrentIndex(&audio_rate)   // if has_audio
+
+11. Main loop (same pattern as ndiout):
+
+    // Audio — non-blocking
+    if (has_audio) {
+        mxlFlowReaderGetSamplesNonBlocking(audio_in_reader, audio_index, samples_per_grain, &in_slice)
+        if OK:
+            mxlFlowWriterOpenSamples(audio_out_writer, audio_index, samples_per_grain, &out_slice)
+            memcpy each channel fragment (frag0, frag1 wrap pattern)
+            mxlFlowWriterCommitSamples
+            audio_index += samples_per_grain
+        if TOO_LATE: jump to headIndex
+    }
+
+    // Video — non-blocking
+    mxlFlowReaderGetGrainNonBlocking(video_in_reader, video_index, &grain, &in_buf)
+    if OK:
+        mxlFlowWriterOpenGrain(video_out_writer, video_index, &out_grain, &out_buf)
+        memcpy(out_buf, in_buf, video_out_stride * height)
+        mxlFlowWriterCommitGrain
+        video_index++
+    if TOO_EARLY: mxlSleepForNs(1ms)
+    if TOO_LATE:  jump to headIndex
+```
+
+### Audio memcpy pattern (wrapped ring buffer)
+
+MXL audio slices can wrap around the ring buffer — always copy both fragments:
+
+```cpp
+const size_t frag0 = in_slice.base.fragments[0].size / sizeof(float);
+const size_t frag1 = in_slice.base.fragments[1].size / sizeof(float);
+for (int c = 0; c < channels; ++c) {
+    const auto* src0 = reinterpret_cast<const float*>(
+        static_cast<const uint8_t*>(in_slice.base.fragments[0].pointer) + c * in_slice.stride);
+    auto* dst0 = reinterpret_cast<float*>(
+        static_cast<uint8_t*>(out_slice.base.fragments[0].pointer) + c * out_slice.stride);
+    std::memcpy(dst0, src0, frag0 * sizeof(float));
+    if (frag1 > 0) {
+        const auto* src1 = reinterpret_cast<const float*>(
+            static_cast<const uint8_t*>(in_slice.base.fragments[1].pointer) + c * in_slice.stride);
+        auto* dst1 = reinterpret_cast<float*>(
+            static_cast<uint8_t*>(out_slice.base.fragments[1].pointer) + c * out_slice.stride);
+        std::memcpy(dst1, src1, frag1 * sizeof(float));
+    }
+}
+```
+
+---
+
+## Reference nodes
+
+- Input flow setup (wait + reader + flow_def read): `nodes/ndiout/main.cpp`, `nodes/decklinkout/main.cpp`
+- Output flow setup (writer creation): `nodes/ndiin/main.cpp`, `nodes/decklinkin/main.cpp`
+- Audio memcpy pattern: `nodes/ndiout/main.cpp` lines 172–190
+- `make_video_flow_def` / `make_audio_flow_def` / `read_video_flow_info` / `read_audio_flow_info`: `shared/FlowDef.hpp`