combiner.md

2026-07-09 15:49:35 +03:00
parent d2095d14d3
commit 3820d0eeb7
3 changed files with 167 additions and 7 deletions
@@ -0,0 +1,138 @@
+# AV Combiner Node — Implementation Steps
+
+Takes video from one upstream node and audio from another, outputs both as new MXL flows.
+Does NOT need a sync group — audio and video run at different rates and are handled independently.
+
+---
+
+## Step 1 — Frontend (`dmf-studio-ui/src/nodeTypes.ts`)
+
+Add the node definition. Port IDs use `video_in-in` / `video_out-out` pattern so that
+`handleToPort()` produces distinct keys (`video_in_flow_id` vs `video_out_flow_id`).
+
+```typescript
+avcombiner: {
+  type: 'avcombiner',
+  label: 'AV Combiner',
+  ports: [
+    { id: 'video_in-in',   kind: 'video', direction: 'in'  },
+    { id: 'audio_in-in',   kind: 'audio', direction: 'in'  },
+    { id: 'video_out-out', kind: 'video', direction: 'out' },
+    { id: 'audio_out-out', kind: 'audio', direction: 'out' },
+  ],
+  params: [],
+},
+```
+
+Config keys the node receives:
+- `video_in_flow_id.id`  — input video flow UUID
+- `audio_in_flow_id.id`  — input audio flow UUID
+- `video_out_flow_id.id` — output video flow UUID
+- `audio_out_flow_id.id` — output audio flow UUID
+
+---
+
+## Step 2 — CMake
+
+Create `nodes/avcombiner/CMakeLists.txt`:
+
+```cmake
+add_executable(dmf-node-avcombiner main.cpp)
+target_compile_features(dmf-node-avcombiner PRIVATE cxx_std_20)
+target_link_libraries(dmf-node-avcombiner PRIVATE dmf-shared)
+install(TARGETS dmf-node-avcombiner RUNTIME DESTINATION bin)
+```
+
+In the root `CMakeLists.txt`, add alongside the other nodes:
+
+```cmake
+add_subdirectory(nodes/avcombiner)
+```
+
+---
+
+## Step 3 — `nodes/avcombiner/main.cpp`
+
+Structure (follow the same patterns as `ndiout`):
+
+```
+1. Check video_in_flow_id and video_out_flow_id both present — exit if not
+2. Check audio_in_flow_id and audio_out_flow_id (optional — audio is optional)
+
+3. Wait for video input flow active  (mxlIsFlowActive loop, 100ms sleep)
+4. Create video input reader         (mxlCreateFlowReader)
+5. Get video config info             (mxlFlowReaderGetConfigInfo → video_stride)
+6. Read video format from flow_def   (dmf::read_video_flow_info(domain(), flow_id))
+   → width, height, fps_num, fps_den
+
+7. If has_audio:
+   Wait for audio input flow active
+   Create audio input reader
+   Read audio format from flow_def   (dmf::read_audio_flow_info(domain(), flow_id))
+   → sample_rate, channels, samples_per_grain
+
+8. Create video output writer        (mxlCreateFlowWriter with make_video_flow_def)
+   → video_out_stride from configInfo.discrete.sliceSizes[0]
+
+9. If has_audio:
+   Create audio output writer        (mxlCreateFlowWriter with make_audio_flow_def)
+
+10. Init clocks:
+    video_index = mxlGetCurrentIndex(&video_rate)
+    audio_index = mxlGetCurrentIndex(&audio_rate)   // if has_audio
+
+11. Main loop (same pattern as ndiout):
+
+    // Audio — non-blocking
+    if (has_audio) {
+        mxlFlowReaderGetSamplesNonBlocking(audio_in_reader, audio_index, samples_per_grain, &in_slice)
+        if OK:
+            mxlFlowWriterOpenSamples(audio_out_writer, audio_index, samples_per_grain, &out_slice)
+            memcpy each channel fragment (frag0, frag1 wrap pattern)
+            mxlFlowWriterCommitSamples
+            audio_index += samples_per_grain
+        if TOO_LATE: jump to headIndex
+    }
+
+    // Video — non-blocking
+    mxlFlowReaderGetGrainNonBlocking(video_in_reader, video_index, &grain, &in_buf)
+    if OK:
+        mxlFlowWriterOpenGrain(video_out_writer, video_index, &out_grain, &out_buf)
+        memcpy(out_buf, in_buf, video_out_stride * height)
+        mxlFlowWriterCommitGrain
+        video_index++
+    if TOO_EARLY: mxlSleepForNs(1ms)
+    if TOO_LATE:  jump to headIndex
+```
+
+### Audio memcpy pattern (wrapped ring buffer)
+
+MXL audio slices can wrap around the ring buffer — always copy both fragments:
+
+```cpp
+const size_t frag0 = in_slice.base.fragments[0].size / sizeof(float);
+const size_t frag1 = in_slice.base.fragments[1].size / sizeof(float);
+for (int c = 0; c < channels; ++c) {
+    const auto* src0 = reinterpret_cast<const float*>(
+        static_cast<const uint8_t*>(in_slice.base.fragments[0].pointer) + c * in_slice.stride);
+    auto* dst0 = reinterpret_cast<float*>(
+        static_cast<uint8_t*>(out_slice.base.fragments[0].pointer) + c * out_slice.stride);
+    std::memcpy(dst0, src0, frag0 * sizeof(float));
+    if (frag1 > 0) {
+        const auto* src1 = reinterpret_cast<const float*>(
+            static_cast<const uint8_t*>(in_slice.base.fragments[1].pointer) + c * in_slice.stride);
+        auto* dst1 = reinterpret_cast<float*>(
+            static_cast<uint8_t*>(out_slice.base.fragments[1].pointer) + c * out_slice.stride);
+        std::memcpy(dst1, src1, frag1 * sizeof(float));
+    }
+}
+```
+
+---
+
+## Reference nodes
+
+- Input flow setup (wait + reader + flow_def read): `nodes/ndiout/main.cpp`, `nodes/decklinkout/main.cpp`
+- Output flow setup (writer creation): `nodes/ndiin/main.cpp`, `nodes/decklinkin/main.cpp`
+- Audio memcpy pattern: `nodes/ndiout/main.cpp` lines 172–190
+- `make_video_flow_def` / `make_audio_flow_def` / `read_video_flow_info` / `read_audio_flow_info`: `shared/FlowDef.hpp`
@@ -38,7 +38,9 @@ struct NodeProcess {
 class StudioManager {
 public:
    StudioManager(fs::path bin_dir, std::string domain)
-        : bin_dir_(std::move(bin_dir)), domain_(std::move(domain)) {}
+        : bin_dir_(std::move(bin_dir))
+        , domain_(std::move(domain))
+        , save_path_(bin_dir_ / "last_graph.json") {}

    ~StudioManager() { shutdown(); }

@@ -53,6 +55,7 @@ public:
            garbage_collect_locked();
            try {
                graph_ = parse_graph(graph_json);
+                save_graph_locked(graph_json);
                start_all_locked();
            } catch (const std::exception& e) {
                err = e.what();
@@ -140,6 +143,7 @@ public:
 private:
    fs::path    bin_dir_;
    std::string domain_;
+    fs::path    save_path_;
    FlowGraph   graph_;
    std::vector<NodeProcess>            processes_;
    std::mutex                          mutex_;
@@ -235,6 +239,13 @@ private:
        if (changed) notify(s);
    }

+    void save_graph_locked(const nlohmann::json& j) {
+        std::ofstream f(save_path_);
+        if (f) f << j.dump(2);
+        else fprintf(stderr, "[studio-manager] warning: could not save graph to %s\n",
+                     save_path_.c_str());
+    }
+
    // ── static helpers ───────────────────────────────────────────────────────

    static FlowGraph parse_graph(const nlohmann::json& j) {
@@ -28,12 +28,23 @@ int main(int argc, char* argv[]) {

    dmf::StudioManager manager(bin_dir, domain);

-    if (argc > 1) {
-        auto r = manager.load_graph_file(argv[1]);
+    {
+        bool clean = false;
+        const char* explicit_path = nullptr;
+        for (int i = 1; i < argc; ++i) {
+            if (std::string(argv[i]) == "--clean") clean = true;
+            else explicit_path = argv[i];
+        }
+
+        fs::path load_path = explicit_path ? fs::path(explicit_path) : bin_dir / "last_graph.json";
+        if (!clean && fs::exists(load_path)) {
+            fprintf(stderr, "[studio-manager] restoring graph from %s\n", load_path.c_str());
+            auto r = manager.load_graph_file(load_path.string());
            if (r.value("type", "") == "error") {
                fprintf(stderr, "[studio-manager] %s\n",
                    r.value("message", "load failed").c_str());
-            return 1;
+                if (explicit_path) return 1;  // explicit path failure is fatal; auto-restore is not
+            }
        }
    }