go-mxl-player/MXLFABRICS_PLAN.md

MXL Fabrics ingress plan

Goal

Receive remote MXL audio and video through go-mxl/fabrics, place the received data into local MXL storage, and feed it into the existing player without changing playback, retry, session, synchronization, playlist, renderer, or audio-output semantics.

Start with the SHM provider. TCP, verbs, and EFA follow after the lifecycle and media paths are stable.

Correct architecture

go-mxl/fabrics is not a second reader backend. A Fabrics target receives remote transfers into memory owned by a local mxl.Writer:

remote fabrics.Initiator
          |
          | remote transfer
          v
    fabrics.Target
          |
          | writes into a local mxl.Writer ring
          v
     local MXL flow
          |
          v
 VideoFrame / AudioSamples -> playback workers -> output

Consequences:

• the local MXL media path remains the reference implementation;
• Fabrics is an ingress/transport concern, not a playback backend;
• playback and playlist code must not know about providers or endpoints;
• each remotely transferred flow requires its own target;
• target completions must be matched to the exact local grain/sample range;
• do not add a blind polling loop that could return stale or overwritten media.

Confirmed API facts

The project pins github.com/qvest-digital/go-mxl v1.1.0-rc.1. It includes github.com/qvest-digital/go-mxl/fabrics and providers shm, tcp, verbs, efa, and any.

The receive-side flow is:

1. create a local mxl.Instance;
2. create an mxl.Writer from the flow definition;
3. create a fabrics.Instance from the local MXL instance;
4. create a fabrics.Target;
5. call Target.Setup with provider, node/service, writer, and options;
6. publish the returned TargetInfo to the remote initiator;
7. consume completions with ReadGrain or ReadSamples.

TargetInfo is control-plane data. Initially serialize it to a file for the known sender; automatic discovery is out of scope.

fabrics.ErrNotReady means no completion is ready before the timeout. fabrics.ErrInterrupted can be caused by Go runtime signals and means retry the read operation. Neither is a feed failure by itself.

Current native-library state

The installed pkg-config metadata reports libmxl 1.2.0.0 and libmxl-fabrics 1.2.0.0. The pinned Go module records native MXL v1.1.0-rc1 as its corresponding version. Resolving libmxl-fabrics currently fails because pkg-config cannot find libfabric.

This does not prove 1.2 is incompatible. Compare the stack with the user's known-working C++ sender/receiver and choose one tested version set. Do not conceal ABI/API mismatches with casts or copied declarations.

Configuration boundary

Fabrics code should live in internal/adapter/mxlfabrics. A provisional configuration is:

type IngressConfig struct {
    Provider       fabrics.Provider
    Node           string
    Service        string
    Options        json.RawMessage
    TargetInfoPath string
    Flow           mxl.FlowDefinition
}

Use the exact flow-definition type exposed by the selected library version. Video and audio have independent configurations and targets. They may use different remote endpoints/providers while landing in the same local MXL domain.

Do not add --backend local|fabrics. The useful distinction is whether a feed is already local or needs Fabrics ingress. Choose final per-feed CLI names after the configuration spike.

Implementation stages

Stage 0 — Align and prove the native stack

• Record exact versions/commits of libfabric, libmxl, libmxl-fabrics, and go-mxl used by the working C++ pair.
• Decide whether to retain Go/native 1.1 RC or move the complete set together.
• Make all pkg-config and dynamic-linker checks pass.
• Run the known C++ pair, then the matching upstream Go SHM target/initiator examples.
• Enumerate interfaces/providers through the Go API.
• Preserve the current player test baseline.

pkg-config --modversion libfabric libmxl libmxl-fabrics
pkg-config --cflags --libs libfabric libmxl libmxl-fabrics
ldd /usr/local/lib/libmxl-fabrics.so
go test ./...

Acceptance: one documented compatible version set; repeatable Go SHM transfer; clean cancellation/shutdown; no player production changes.

Stage 1 — Minimal SHM ingress spike

• Create a local MXL instance and writer from a supplied flow definition.
• Set up one SHM target and serialize its TargetInfo to an explicit file.
• Receive video grains and audio sample ranges from the known sender in isolated experiments.
• Verify completion indices against the corresponding local ring contents.
• Document buffer lifetime, overwrite behavior, and sender-absent behavior.

Acceptance: video and audio transfers work independently; no busy loop; shutdown promptly releases resources.

Stage 2 — Define the ingress lifecycle

Create a narrow adapter owned by the composition root. It must:

• validate before allocating native resources;
• create the writer before target setup and publish info only after setup;
• classify timeouts, interruptions, cancellation, and terminal errors;
• remain context-cancellable although native reads use timeouts;
• allow only one completion reader per target;
• close every resource exactly once and unwind partial setup safely.

Confirm the precise close order against the chosen version. Expected order:

1. cancel and join the completion loop;
2. close target;
3. close returned TargetInfo;
4. close the Fabrics instance after all targets;
5. close local readers/writers while respecting the target's writer reference;
6. close the local MXL instance last.

Acceptance: allocation-failure tests, prompt idle/active cancellation, race-safe repeated start/stop, and no resource leaks.

Stage 3 — Video ingress

• Receive completion indices and obtain the exact completed local grain.
• Map metadata/payload into playback.VideoFrame.
• Preserve borrowed-buffer lifetime, or make one deliberate bounded copy if required for safety.
• Reject unsupported formats explicitly.
• Track received, delivered, overwritten, and malformed grains.

Never independently poll for the newest local frame after a completion; it may be a different index.

Acceptance: correct V210 playback; monotonic indices except documented restart; stable 25/50/59.94/high-rate playback; clean disappearance/restoration.

Stage 4 — Audio ingress

• Use ReadSamples completion ranges and read exactly the reported head/count.
• Map rate, channels, format, label, UUID, and index into the audio contract.
• Preserve SDL backpressure and queue policy.
• Track samples, discontinuities, overruns, and queue depth.

Acceptance: correct mono/stereo/multichannel and common rates; no corruption/drift; no duplicated/skipped ranges; no stale queued audio after stop/reconnect.

Stage 5 — Independent and synchronized A/V

Independent A/V uses two independent targets/lifecycles; one failure must not stop the other.

For synchronized A/V:

• land both flows in a compatible local MXL domain;
• retain native local-MXL synchronization where supported;
• coordinate completion availability so sync never consumes data not yet received;
• verify different endpoints/providers and fractional frame rates such as 59.94.

Fabrics transports media; it is not a reason to invent manual synchronization. If native synchronization cannot safely use received rings, write a separate sync design first.

Acceptance: failure isolation when independent; one retry lifecycle when synchronized; runtime sync toggle retains current semantics; long playback stays aligned.

Stage 6 — Player integration

• Add per-feed local/Fabrics transport configuration.
• Construct ingress outside internal/playback and connect it to existing workers/slots.
• Preserve stop, resume, replace, remove, sync toggle, retry, and playlist behavior.
• Keep playlist entries provider-neutral where possible.
• Add provider, endpoint, ingress state, completion age, and counters to stats.

Acceptance: local behavior is unchanged; Fabrics supports video-only, audio-only, independent A/V, and sync A/V; bad config fails before native goroutines; feeds remain independently controllable.

Stage 7 — Provider rollout

Enable and validate in order:

1. SHM;
2. TCP;
3. verbs;
4. EFA if deployment requires it.

Provider selection should affect only Fabrics interface configuration and deployment prerequisites. Playback controllers must not branch on provider names.

Stage 8 — Playlist and resilience validation

Test empty startup; audio/video only; independent and synchronized A/V; either sender disappearing/restoring; finite/infinite retry; stop/resume/remove/replace; sync toggling during playback/backoff; wait/next playlists; timed looping/manual navigation; 59.94/high-rate video; and repeated long-running startup/shutdown.

Record CPU, memory, received/displayed FPS, completion delta, dropped/overwritten media, audio queue, reconnect latency, and native resource counts.

Test layers

1. Unit tests for configuration, error classification, and lifecycle transitions.
2. Adapter tests with injected wrappers around native allocations and reads.
3. Real SHM initiator/target integration tests.
4. Player smoke tests for all playback combinations.
5. Race and repeated lifecycle tests.
6. Provider-specific TCP/verbs/EFA live tests.

Native/live tests must be explicit integration tests and skip with a useful reason when prerequisites are absent.

Out of scope for the first pass

• player as Fabrics initiator/sender;
• automatic endpoint or TargetInfo discovery;
• changing provider on an active target;
• manual A/V sync without a reviewed design;
• new media formats bundled into transport work;
• provider-specific logic in playback, playlist, renderer, or GUI;
• unsafe workarounds for native version mismatches.

First action tomorrow

1. Build/install the exact known-working native stack.
2. Record every native version/commit and the C++ reference commands.
3. Make all pkg-config checks pass.
4. Run the C++ sender/receiver baseline.
5. Run the matching upstream Go SHM target/initiator example.
6. Capture its flow definition and TargetInfo exchange.
7. Start Stage 1 only when transfer and clean shutdown are repeatable.