M9 — GUI (Dear ImGui)

Context

The player already works (M0–M8): SDL3 window (purego), Vulkan renderer (internal/renderer, christerso/vulkan-go/vk wrapper), MXL source (internal/source). All rendering happens inside one render pass, one command buffer, single-flight (fence-gated).

Dear ImGui is a header-only C++ library. The Go binding cimgui-go compiles its core at build time via cgo. cimgui-go ships SDL2 + Vulkan backends in C++, but we use SDL3 via purego, so we write custom Go backends that bridge to our existing SDL3 and Vulkan code. ImGui's core (no backend) has no SDL/Vulkan dependency — it just produces draw data (vertices, indices, commands). Our backends feed it input and render its draw data.

Architecture:

cmd/mxl-player/main.go
    |
    +-- internal/imgui/        new package
    |       imgui.go           context, NewFrame/EndFrame, draw data
    |       input_sdl3.go      SDL3 events -> ImGui IO
    |       backend_vulkan.go  Vulkan pipeline + font atlas + draw
    |
    +-- internal/renderer/     existing, unchanged
    +-- internal/sdl/          existing, add a few input helpers
    +-- internal/source/       existing, unchanged

The ImGui Vulkan backend renders inside the existing render pass, after the video Draw(3), before EndRenderPass. It has its own pipeline, vertex/index buffers, font texture, and descriptor set — all owned by internal/imgui, not internal/renderer.

Steps

M9.1 — ImGui core init + SDL3 input backend

Goal: ImGui context exists, receives SDL3 input, produces draw data (not yet rendered).

go get github.com/AllenDang/cimgui-go@v1.5.0
Create internal/imgui/imgui.go:
- type Context struct { ctx *imgui.Context; io *imgui.IO }
- func New() *Context — creates context + IO, sets display size from sdl.GetWindowSizeInPixels, sets delta time each frame.
- func (c *Context) BeginFrame(dt time.Duration, winW, winH int32) — updates IO (display size, delta time), calls imgui.NewFrame().
- func (c *Context) EndFrame() *imgui.DrawData — calls imgui.EndFrame() / imgui.Render(), returns draw data for the Vulkan backend to consume.
- func (c *Context) Destroy()
Create internal/imgui/input_sdl3.go:
- func (c *Context) ProcessEvent(event *[128]byte) — called from the main loop for every SDL3 event, mutates c.io:
  - SDL_EVENT_KEY_DOWN / SDL_EVENT_KEY_UP → set key in io.AddInputCharactersUTF8 for text, set key index.
  - SDL_EVENT_MOUSE_BUTTON_DOWN / UP → set mouse button.
  - SDL_EVENT_MOUSE_MOTION → set mouse position.
  - SDL_EVENT_MOUSE_WHEEL → set wheel delta.
  - SDL_EVENT_WINDOW_RESIZED → set display size.
- Key mapping: SDL3 scancode → ImGui key enum (a static lookup table or switch).
- Mouse: ImGui wants float coordinates; SDL3 gives int32.
Wire into main.go:
- After window creation: imguiCtx := imgui.New()
- defer imguiCtx.Destroy()
- In the event poll loop: imguiCtx.ProcessEvent(&event)
- Before any widget code: imguiCtx.BeginFrame(dt, winW, winH)
SDL3 text input: call SDL_StartTextInput on window creation so SDL_EVENT_TEXT_INPUT events fire (add to internal/sdl).
- SDL3 text input event: SDL_EVENT_TEXT_INPUT = 0x303, data is a UTF-8 string at a fixed offset in the event struct.

Verify: app runs, no crash, window shows video as before. ImGui is initialized but invisible (no widgets yet, no render backend).

M9.2 — ImGui Vulkan render backend

Goal: ImGui draw data appears on screen inside the existing render pass.

Create internal/imgui/backend_vulkan.go:
- type VulkanBackend struct { ... }
- Holds: pipeline, pipeline layout, descriptor set layout/pool/set, font image + view + sampler, vertex buffer, index buffer (all vk.* types from the wrapper).
func NewVulkanBackend(pd vk.PhysicalDevice, dev vk.Device, queue vk.Queue, cmdPool vk.CommandPool, rp vk.RenderPass, format vk.Format) (*VulkanBackend, error):
- Font atlas: imgui.GetIO().Fonts.Build() then GetTexDataAsRGBA32(&pixels, &w, &h) → create vk.CreateImage2D(w, h, Sampled|TransferDst) → vk.CreateImageView → upload pixels via staging buffer + CopyBufferToImage → vk.CreateSampler.
- Descriptor set: layout with one DescriptorCombinedImageSampler binding (binding 0, fragment stage) → pool → set → vk.UpdateImageDescriptor(set, 0, fontView, fontSampler).
- Pipeline layout: one set layout, push constants (ImGui's ImDrawVert-based push constant, 16 bytes for the scale/translate vec2s, vertex+fragment stages).
- Graphics pipeline: vk.CreateGraphicsPipeline with Blend: true, vertex shader + fragment shader (embed ImGui's SPIR-V or compile from imgui_impl_vulkan's GLSL). Vertex attributes: position (vec2), UV (vec2), color (vec4) — matches ImDrawVert layout.
- Vertex/index buffers: created with a max size (e.g. 1 MB vertex, 256 KB index); recreated (larger) if draw data exceeds capacity.
Embed ImGui shaders:
- Compile imgui/shaders/imgui.vert and imgui/shaders/imgui.frag (from Dear ImGui's repo) to SPIR-V with glslc.
- //go:embed in backend_vulkan.go.
func (b *VulkanBackend) RecordDraw(cmd vk.CommandBuffer, drawData *imgui.DrawData, frameIndex uint32):
- Called inside the existing render pass (between video Draw(3) and EndRenderPass).
- Map/CopyToMapped vertex + index data from drawData into the staging buffers (or use vk.Map/Unmap on a host-visible buffer).
- cmd.BindPipeline(b.pipeline)
- cmd.BindDescriptorSet(b.layout, 0, b.set)
- cmd.BindVertexBuffers(...)
- cmd.BindIndexBuffer(...)
- cmd.SetViewport(...) (full window)
- Push constants (scale/translate from drawData).
- For each draw list, for each command: cmd.DrawIndexed(...).
Wire into main.go (or renderer.DrawFrame):
- After cmd.Draw(3, 1, 0, 0) (video), before EndRenderPass: imguiBackend.RecordDraw(cmd, drawData, imageIndex).

Verify: add a test widget imgui.Text("hello") in BeginFrame/ EndFrame. Run — you should see "hello" overlaid on the video.

M9.3 — The GUI: config panel + stats overlay

Goal: usable GUI for entering connection params and showing stats.

In main.go, between imguiCtx.BeginFrame(...) and imguiCtx.EndFrame(), build the GUI:

Config panel (window, shown by default):

imgui.Begin("Connection")
imgui.InputText("Domain", &domainBuf, 256)
imgui.InputText("Video UUID", &videoBuf, 256)
imgui.InputText("Audio UUID", &audioBuf, 256)
if imgui.Button("Connect") {
    // trigger source.Open / OpenSync with the entered values
}
imgui.End()

Stats overlay (window, toggled by F1, no title bar, no background, top-left):

if showStats {
    imgui.SetNextWindowPos(...)
    imgui.Begin("Stats", &showStats, imgui.WindowFlagsNoTitleBar | imgui.WindowFlagsNoBackground)
    imgui.Text(fmt.Sprintf("FPS: %.1f", fps))
    imgui.Text(fmt.Sprintf("Dropped: %d", dropped))
    imgui.Text(fmt.Sprintf("Index: %d", shownIndex))
    imgui.Text(fmt.Sprintf("Frame: %dx%d", videoW, videoH))
    imgui.Text(fmt.Sprintf("Format: v210 10-bit"))
    imgui.End()
}

Dummy controls for future M10 flow discovery:

imgui.Button("List Flows")  // no-op yet
imgui.Button("Refresh")      // no-op yet

State: domainBuf, videoBuf, audioBuf are [256]byte buffers (ImGui's InputText needs a fixed buffer + capacity). Convert to Go string on "Connect".
"Connect" button:
- Close existing source if any.
- Call source.Open / source.OpenSync with the buffer values.
- On error, display imgui.Text in red below the button.
F1 toggle: handle in the keydown event switch, flip showStats.

Verify: type a domain + UUID, click Connect, video appears. Toggle stats with F1. Resize window — GUI stays usable.

M9.4 — Wire GUI to engine + polish

Goal: GUI controls the engine, not just displays.

Connection lifecycle: "Connect" button triggers source open + renderer buffer creation (if resolution changed). "Disconnect" button closes source, video freezes on last frame (or clears to black).
Stats read from engine: expose Stats struct from the loop (fps, dropped, index, frameTime, resolution, format). GUI reads it each frame.
Freeze control (placeholder for M10): imgui.Checkbox("Freeze", &frozen). When frozen, stop calling DrawFrame (keep last image on screen, keep polling events + GUI).
Input focus: when ImGui wants keyboard input (text field focused), don't pass key events to the app (e.g., don't toggle fullscreen on 'F' while typing). Check imgui.GetIO().WantCaptureKeyboard.
Mouse capture: when ImGui wants mouse, don't let the app process mouse events. Check imgui.GetIO().WantCaptureMouse.
DPI awareness: scale ImGui font + style by the window's SDL_GetDisplayContentScale (or hardcode 1.0 for now; polish later).

Verify: full workflow — launch app, enter params, connect, see video + stats, toggle stats, disconnect, reconnect. All via GUI, no CLI flags needed (though flags still work for headless/automation).

Notes

ImGui's vertex layout (ImDrawVert): pos [2]float32, uv [2]float32, col uint32 = 20 bytes. The vertex shader applies a scale/translate push constant to convert from ImGui's screen coordinates to clip space.
ImGui's fragment shader samples the font texture (and any user textures) using the UV from the vertex. Color is the vertex color, multiplied by the texture sample.
The backend pipeline uses alpha blending: src=SRC_ALPHA, dst=ONE_MINUS_SRC_ALPHA, op=ADD.
ImGui produces draw data after EndFrame/Render. The flow is: BeginFrame → build widgets → EndFrame/Render → get draw data → record Vulkan commands from draw data → submit.
Font atlas upload is one-time, during NewVulkanBackend. Vertex/index buffers are updated every frame from ImGui's draw data (host-visible, persistently mapped).
The ImGui pipeline is separate from the decode pipeline. Both render into the same render pass / framebuffer / color attachment.

10 KiB Raw Blame History Unescape Escape