go-mxl-player/gui.md

# M9 — GUI (Dear ImGui)

## Context

The player already works (M0–M8): SDL3 window (purego), Vulkan renderer
(`internal/renderer`, `christerso/vulkan-go/vk` wrapper), MXL source
(`internal/source`). All rendering happens inside one render pass, one
command buffer, single-flight (fence-gated).

Dear ImGui is a header-only C++ library. The Go binding `cimgui-go`
compiles its core at build time via cgo. cimgui-go ships SDL2 + Vulkan
backends in C++, but we use SDL3 via purego, so we write custom Go
backends that bridge to our existing SDL3 and Vulkan code. ImGui's core
(no backend) has no SDL/Vulkan dependency — it just produces draw data
(vertices, indices, commands). Our backends feed it input and render its
draw data.

Architecture:

```
cmd/mxl-player/main.go
    |
    +-- internal/imgui/        new package
    |       imgui.go           context, NewFrame/EndFrame, draw data
    |       input_sdl3.go      SDL3 events -> ImGui IO
    |       backend_vulkan.go  Vulkan pipeline + font atlas + draw
    |
    +-- internal/renderer/     existing, unchanged
    +-- internal/sdl/          existing, add a few input helpers
    +-- internal/source/       existing, unchanged
```

The ImGui Vulkan backend renders *inside* the existing render pass,
after the video `Draw(3)`, before `EndRenderPass`. It has its own
pipeline, vertex/index buffers, font texture, and descriptor set —
all owned by `internal/imgui`, not `internal/renderer`.

## Steps

### M9.1 — ImGui core init + SDL3 input backend

Goal: ImGui context exists, receives SDL3 input, produces draw data
(not yet rendered).

1. `go get github.com/AllenDang/cimgui-go@v1.5.0`
2. Create `internal/imgui/imgui.go`:
   - `type Context struct { ctx *imgui.Context; io *imgui.IO }`
   - `func New() *Context` — creates context + IO, sets display size
     from `sdl.GetWindowSizeInPixels`, sets delta time each frame.
   - `func (c *Context) BeginFrame(dt time.Duration, winW, winH int32)`
     — updates IO (display size, delta time), calls `imgui.NewFrame()`.
   - `func (c *Context) EndFrame() *imgui.DrawData` — calls
     `imgui.EndFrame()` / `imgui.Render()`, returns draw data for the
     Vulkan backend to consume.
   - `func (c *Context) Destroy()`
3. Create `internal/imgui/input_sdl3.go`:
   - `func (c *Context) ProcessEvent(event *[128]byte)` — called from
     the main loop for every SDL3 event, mutates `c.io`:
     - `SDL_EVENT_KEY_DOWN` / `SDL_EVENT_KEY_UP` → set key in
       `io.AddInputCharactersUTF8` for text, set key index.
     - `SDL_EVENT_MOUSE_BUTTON_DOWN` / `UP` → set mouse button.
     - `SDL_EVENT_MOUSE_MOTION` → set mouse position.
     - `SDL_EVENT_MOUSE_WHEEL` → set wheel delta.
     - `SDL_EVENT_WINDOW_RESIZED` → set display size.
   - Key mapping: SDL3 scancode → ImGui key enum (a static lookup table
     or switch).
   - Mouse: ImGui wants float coordinates; SDL3 gives int32.
4. Wire into `main.go`:
   - After window creation: `imguiCtx := imgui.New()`
   - `defer imguiCtx.Destroy()`
   - In the event poll loop: `imguiCtx.ProcessEvent(&event)`
   - Before any widget code: `imguiCtx.BeginFrame(dt, winW, winH)`
5. SDL3 text input: call `SDL_StartTextInput` on window creation so
   `SDL_EVENT_TEXT_INPUT` events fire (add to `internal/sdl`).
   - SDL3 text input event: `SDL_EVENT_TEXT_INPUT = 0x303`, data is a
     UTF-8 string at a fixed offset in the event struct.

Verify: app runs, no crash, window shows video as before. ImGui is
initialized but invisible (no widgets yet, no render backend).

### M9.2 — ImGui Vulkan render backend

Goal: ImGui draw data appears on screen inside the existing render
pass.

1. Create `internal/imgui/backend_vulkan.go`:
   - `type VulkanBackend struct { ... }`
   - Holds: pipeline, pipeline layout, descriptor set layout/pool/set,
     font image + view + sampler, vertex buffer, index buffer (all
     `vk.*` types from the wrapper).
2. `func NewVulkanBackend(pd vk.PhysicalDevice, dev vk.Device, queue
   vk.Queue, cmdPool vk.CommandPool, rp vk.RenderPass, format
   vk.Format) (*VulkanBackend, error)`:
   - **Font atlas**: `imgui.GetIO().Fonts.Build()` then
     `GetTexDataAsRGBA32(&pixels, &w, &h)` → create
     `vk.CreateImage2D(w, h, Sampled|TransferDst)` →
     `vk.CreateImageView` → upload pixels via staging buffer +
     `CopyBufferToImage` → `vk.CreateSampler`.
   - **Descriptor set**: layout with one
     `DescriptorCombinedImageSampler` binding (binding 0, fragment
     stage) → pool → set →
     `vk.UpdateImageDescriptor(set, 0, fontView, fontSampler)`.
   - **Pipeline layout**: one set layout, push constants
     (ImGui's `ImDrawVert`-based push constant, 16 bytes for the
     scale/translate vec2s, vertex+fragment stages).
   - **Graphics pipeline**: `vk.CreateGraphicsPipeline` with
     `Blend: true`, vertex shader + fragment shader (embed ImGui's
     SPIR-V or compile from `imgui_impl_vulkan`'s GLSL). Vertex
     attributes: position (vec2), UV (vec2), color (vec4) — matches
     `ImDrawVert` layout.
   - **Vertex/index buffers**: created with a max size (e.g. 1 MB
     vertex, 256 KB index); recreated (larger) if draw data exceeds
     capacity.
3. Embed ImGui shaders:
   - Compile `imgui/shaders/imgui.vert` and `imgui/shaders/imgui.frag`
     (from Dear ImGui's repo) to SPIR-V with `glslc`.
   - `//go:embed` in `backend_vulkan.go`.
4. `func (b *VulkanBackend) RecordDraw(cmd vk.CommandBuffer, drawData
   *imgui.DrawData, frameIndex uint32)`:
   - Called inside the existing render pass (between video `Draw(3)`
     and `EndRenderPass`).
   - Map/`CopyToMapped` vertex + index data from `drawData` into the
     staging buffers (or use `vk.Map`/`Unmap` on a host-visible
     buffer).
   - `cmd.BindPipeline(b.pipeline)`
   - `cmd.BindDescriptorSet(b.layout, 0, b.set)`
   - `cmd.BindVertexBuffers(...)`
   - `cmd.BindIndexBuffer(...)`
   - `cmd.SetViewport(...)` (full window)
   - Push constants (scale/translate from drawData).
   - For each draw list, for each command: `cmd.DrawIndexed(...)`.
5. Wire into `main.go` (or `renderer.DrawFrame`):
   - After `cmd.Draw(3, 1, 0, 0)` (video), before `EndRenderPass`:
     `imguiBackend.RecordDraw(cmd, drawData, imageIndex)`.

Verify: add a test widget `imgui.Text("hello")` in `BeginFrame`/
`EndFrame`. Run — you should see "hello" overlaid on the video.

### M9.3 — The GUI: config panel + stats overlay

Goal: usable GUI for entering connection params and showing stats.

1. In `main.go`, between `imguiCtx.BeginFrame(...)` and
   `imguiCtx.EndFrame()`, build the GUI:
   - **Config panel** (window, shown by default):
     ```go
     imgui.Begin("Connection")
     imgui.InputText("Domain", &domainBuf, 256)
     imgui.InputText("Video UUID", &videoBuf, 256)
     imgui.InputText("Audio UUID", &audioBuf, 256)
     if imgui.Button("Connect") {
         // trigger source.Open / OpenSync with the entered values
     }
     imgui.End()
     ```
   - **Stats overlay** (window, toggled by F1, no title bar, no
     background, top-left):
     ```go
     if showStats {
         imgui.SetNextWindowPos(...)
         imgui.Begin("Stats", &showStats, imgui.WindowFlagsNoTitleBar | imgui.WindowFlagsNoBackground)
         imgui.Text(fmt.Sprintf("FPS: %.1f", fps))
         imgui.Text(fmt.Sprintf("Dropped: %d", dropped))
         imgui.Text(fmt.Sprintf("Index: %d", shownIndex))
         imgui.Text(fmt.Sprintf("Frame: %dx%d", videoW, videoH))
         imgui.Text(fmt.Sprintf("Format: v210 10-bit"))
         imgui.End()
     }
     ```
   - **Dummy controls** for future M10 flow discovery:
     ```go
     imgui.Button("List Flows")  // no-op yet
     imgui.Button("Refresh")      // no-op yet
     ```
2. State: `domainBuf`, `videoBuf`, `audioBuf` are `[256]byte` buffers
   (ImGui's `InputText` needs a fixed buffer + capacity). Convert to
   Go string on "Connect".
3. "Connect" button:
   - Close existing source if any.
   - Call `source.Open` / `source.OpenSync` with the buffer values.
   - On error, display `imgui.Text` in red below the button.
4. F1 toggle: handle in the keydown event switch, flip `showStats`.

Verify: type a domain + UUID, click Connect, video appears. Toggle
stats with F1. Resize window — GUI stays usable.

### M9.4 — Wire GUI to engine + polish

Goal: GUI controls the engine, not just displays.

1. **Connection lifecycle**: "Connect" button triggers source open +
   renderer buffer creation (if resolution changed). "Disconnect"
   button closes source, video freezes on last frame (or clears to
   black).
2. **Stats read from engine**: expose `Stats` struct from the loop
   (fps, dropped, index, frameTime, resolution, format). GUI reads it
   each frame.
3. **Freeze control** (placeholder for M10): `imgui.Checkbox("Freeze",
   &frozen)`. When frozen, stop calling `DrawFrame` (keep last image
   on screen, keep polling events + GUI).
4. **Input focus**: when ImGui wants keyboard input (text field
   focused), don't pass key events to the app (e.g., don't toggle
   fullscreen on 'F' while typing). Check
   `imgui.GetIO().WantCaptureKeyboard`.
5. **Mouse capture**: when ImGui wants mouse, don't let the app
   process mouse events. Check `imgui.GetIO().WantCaptureMouse`.
6. **DPI awareness**: scale ImGui font + style by the window's
   `SDL_GetDisplayContentScale` (or hardcode 1.0 for now; polish
   later).

Verify: full workflow — launch app, enter params, connect, see video +
stats, toggle stats, disconnect, reconnect. All via GUI, no CLI flags
needed (though flags still work for headless/automation).

## Notes

- ImGui's vertex layout (`ImDrawVert`): `pos [2]float32, uv [2]float32,
  col uint32` = 20 bytes. The vertex shader applies a scale/translate
  push constant to convert from ImGui's screen coordinates to clip
  space.
- ImGui's fragment shader samples the font texture (and any user
  textures) using the UV from the vertex. Color is the vertex color,
  multiplied by the texture sample.
- The backend pipeline uses alpha blending:
  `src=SRC_ALPHA, dst=ONE_MINUS_SRC_ALPHA, op=ADD`.
- ImGui produces draw data *after* `EndFrame`/`Render`. The flow is:
  `BeginFrame` → build widgets → `EndFrame`/`Render` → get draw data →
  record Vulkan commands from draw data → submit.
- Font atlas upload is one-time, during `NewVulkanBackend`. Vertex/index
  buffers are updated every frame from ImGui's draw data (host-visible,
  persistently mapped).
- The ImGui pipeline is separate from the decode pipeline. Both render
  into the same render pass / framebuffer / color attachment.