dmf-studio-rnd/combiner.md

AV Combiner Node — Implementation Steps

Takes video from one upstream node and audio from another, outputs both as new MXL flows. Does NOT need a sync group — audio and video run at different rates and are handled independently.

---

Step 1 — Frontend (dmf-studio-ui/src/nodeTypes.ts)

Add the node definition. Port IDs use video_in-in / video_out-out pattern so that handleToPort() produces distinct keys (video_in_flow_id vs video_out_flow_id).

avcombiner: {
  type: 'avcombiner',
  label: 'AV Combiner',
  ports: [
    { id: 'video_in-in',   kind: 'video', direction: 'in'  },
    { id: 'audio_in-in',   kind: 'audio', direction: 'in'  },
    { id: 'video_out-out', kind: 'video', direction: 'out' },
    { id: 'audio_out-out', kind: 'audio', direction: 'out' },
  ],
  params: [],
},

Config keys the node receives:

• video_in_flow_id.id — input video flow UUID
• audio_in_flow_id.id — input audio flow UUID
• video_out_flow_id.id — output video flow UUID
• audio_out_flow_id.id — output audio flow UUID

---

Step 2 — CMake

Create nodes/avcombiner/CMakeLists.txt:

add_executable(dmf-node-avcombiner main.cpp)
target_compile_features(dmf-node-avcombiner PRIVATE cxx_std_20)
target_link_libraries(dmf-node-avcombiner PRIVATE dmf-shared)
install(TARGETS dmf-node-avcombiner RUNTIME DESTINATION bin)

In the root CMakeLists.txt, add alongside the other nodes:

add_subdirectory(nodes/avcombiner)

---

Step 3 — nodes/avcombiner/main.cpp

Structure (follow the same patterns as ndiout):

1. Check video_in_flow_id and video_out_flow_id both present — exit if not
2. Check audio_in_flow_id and audio_out_flow_id (optional — audio is optional)

3. Wait for video input flow active  (mxlIsFlowActive loop, 100ms sleep)
4. Create video input reader         (mxlCreateFlowReader)
5. Get video config info             (mxlFlowReaderGetConfigInfo → video_stride)
6. Read video format from flow_def   (dmf::read_video_flow_info(domain(), flow_id))
   → width, height, fps_num, fps_den

7. If has_audio:
   Wait for audio input flow active
   Create audio input reader
   Read audio format from flow_def   (dmf::read_audio_flow_info(domain(), flow_id))
   → sample_rate, channels, samples_per_grain

8. Create video output writer        (mxlCreateFlowWriter with make_video_flow_def)
   → video_out_stride from configInfo.discrete.sliceSizes[0]

9. If has_audio:
   Create audio output writer        (mxlCreateFlowWriter with make_audio_flow_def)

10. Init clocks:
    video_index = mxlGetCurrentIndex(&video_rate)
    audio_index = mxlGetCurrentIndex(&audio_rate)   // if has_audio

11. Main loop (same pattern as ndiout):

    // Audio — non-blocking
    if (has_audio) {
        mxlFlowReaderGetSamplesNonBlocking(audio_in_reader, audio_index, samples_per_grain, &in_slice)
        if OK:
            mxlFlowWriterOpenSamples(audio_out_writer, audio_index, samples_per_grain, &out_slice)
            memcpy each channel fragment (frag0, frag1 wrap pattern)
            mxlFlowWriterCommitSamples
            audio_index += samples_per_grain
        if TOO_LATE: jump to headIndex
    }

    // Video — non-blocking
    mxlFlowReaderGetGrainNonBlocking(video_in_reader, video_index, &grain, &in_buf)
    if OK:
        mxlFlowWriterOpenGrain(video_out_writer, video_index, &out_grain, &out_buf)
        memcpy(out_buf, in_buf, video_out_stride * height)
        mxlFlowWriterCommitGrain
        video_index++
    if TOO_EARLY: mxlSleepForNs(1ms)
    if TOO_LATE:  jump to headIndex

Audio memcpy pattern (wrapped ring buffer)

MXL audio slices can wrap around the ring buffer — always copy both fragments:

const size_t frag0 = in_slice.base.fragments[0].size / sizeof(float);
const size_t frag1 = in_slice.base.fragments[1].size / sizeof(float);
for (int c = 0; c < channels; ++c) {
    const auto* src0 = reinterpret_cast<const float*>(
        static_cast<const uint8_t*>(in_slice.base.fragments[0].pointer) + c * in_slice.stride);
    auto* dst0 = reinterpret_cast<float*>(
        static_cast<uint8_t*>(out_slice.base.fragments[0].pointer) + c * out_slice.stride);
    std::memcpy(dst0, src0, frag0 * sizeof(float));
    if (frag1 > 0) {
        const auto* src1 = reinterpret_cast<const float*>(
            static_cast<const uint8_t*>(in_slice.base.fragments[1].pointer) + c * in_slice.stride);
        auto* dst1 = reinterpret_cast<float*>(
            static_cast<uint8_t*>(out_slice.base.fragments[1].pointer) + c * out_slice.stride);
        std::memcpy(dst1, src1, frag1 * sizeof(float));
    }
}

---

Reference nodes

• Input flow setup (wait + reader + flow_def read): nodes/ndiout/main.cpp, nodes/decklinkout/main.cpp
• Output flow setup (writer creation): nodes/ndiin/main.cpp, nodes/decklinkin/main.cpp
• Audio memcpy pattern: nodes/ndiout/main.cpp lines 172–190
• make_video_flow_def / make_audio_flow_def / read_video_flow_info / read_audio_flow_info: shared/FlowDef.hpp