Public release README.md fix
This commit is contained in:
Dmitry Sergeev
@@ -0,0 +1,233 @@
|
||||
# M9 — GUI (Dear ImGui)
|
||||
|
||||
## Context
|
||||
|
||||
The player already works (M0–M8): SDL3 window (purego), Vulkan renderer
|
||||
(`internal/renderer`, `christerso/vulkan-go/vk` wrapper), MXL source
|
||||
(`internal/source`). All rendering happens inside one render pass, one
|
||||
command buffer, single-flight (fence-gated).
|
||||
|
||||
Dear ImGui is a header-only C++ library. The Go binding `cimgui-go`
|
||||
compiles its core at build time via cgo. cimgui-go ships SDL2 + Vulkan
|
||||
backends in C++, but we use SDL3 via purego, so we write custom Go
|
||||
backends that bridge to our existing SDL3 and Vulkan code. ImGui's core
|
||||
(no backend) has no SDL/Vulkan dependency — it just produces draw data
|
||||
(vertices, indices, commands). Our backends feed it input and render its
|
||||
draw data.
|
||||
|
||||
Architecture:
|
||||
|
||||
```
|
||||
cmd/mxl-player/main.go
|
||||
|
|
||||
+-- internal/imgui/ new package
|
||||
| imgui.go context, NewFrame/EndFrame, draw data
|
||||
| input_sdl3.go SDL3 events -> ImGui IO
|
||||
| backend_vulkan.go Vulkan pipeline + font atlas + draw
|
||||
|
|
||||
+-- internal/renderer/ existing, unchanged
|
||||
+-- internal/sdl/ existing, add a few input helpers
|
||||
+-- internal/source/ existing, unchanged
|
||||
```
|
||||
|
||||
The ImGui Vulkan backend renders *inside* the existing render pass,
|
||||
after the video `Draw(3)`, before `EndRenderPass`. It has its own
|
||||
pipeline, vertex/index buffers, font texture, and descriptor set —
|
||||
all owned by `internal/imgui`, not `internal/renderer`.
|
||||
|
||||
## Steps
|
||||
|
||||
### M9.1 — ImGui core init + SDL3 input backend
|
||||
|
||||
Goal: ImGui context exists, receives SDL3 input, produces draw data
|
||||
(not yet rendered).
|
||||
|
||||
1. `go get github.com/AllenDang/cimgui-go@v1.5.0`
|
||||
2. Create `internal/imgui/imgui.go`:
|
||||
- `type Context struct { ctx *imgui.Context; io *imgui.IO }`
|
||||
- `func New() *Context` — creates context + IO, sets display size
|
||||
from `sdl.GetWindowSizeInPixels`, sets delta time each frame.
|
||||
- `func (c *Context) BeginFrame(dt time.Duration, winW, winH int32)`
|
||||
— updates IO (display size, delta time), calls `imgui.NewFrame()`.
|
||||
- `func (c *Context) EndFrame() *imgui.DrawData` — calls
|
||||
`imgui.EndFrame()` / `imgui.Render()`, returns draw data for the
|
||||
Vulkan backend to consume.
|
||||
- `func (c *Context) Destroy()`
|
||||
3. Create `internal/imgui/input_sdl3.go`:
|
||||
- `func (c *Context) ProcessEvent(event *[128]byte)` — called from
|
||||
the main loop for every SDL3 event, mutates `c.io`:
|
||||
- `SDL_EVENT_KEY_DOWN` / `SDL_EVENT_KEY_UP` → set key in
|
||||
`io.AddInputCharactersUTF8` for text, set key index.
|
||||
- `SDL_EVENT_MOUSE_BUTTON_DOWN` / `UP` → set mouse button.
|
||||
- `SDL_EVENT_MOUSE_MOTION` → set mouse position.
|
||||
- `SDL_EVENT_MOUSE_WHEEL` → set wheel delta.
|
||||
- `SDL_EVENT_WINDOW_RESIZED` → set display size.
|
||||
- Key mapping: SDL3 scancode → ImGui key enum (a static lookup table
|
||||
or switch).
|
||||
- Mouse: ImGui wants float coordinates; SDL3 gives int32.
|
||||
4. Wire into `main.go`:
|
||||
- After window creation: `imguiCtx := imgui.New()`
|
||||
- `defer imguiCtx.Destroy()`
|
||||
- In the event poll loop: `imguiCtx.ProcessEvent(&event)`
|
||||
- Before any widget code: `imguiCtx.BeginFrame(dt, winW, winH)`
|
||||
5. SDL3 text input: call `SDL_StartTextInput` on window creation so
|
||||
`SDL_EVENT_TEXT_INPUT` events fire (add to `internal/sdl`).
|
||||
- SDL3 text input event: `SDL_EVENT_TEXT_INPUT = 0x303`, data is a
|
||||
UTF-8 string at a fixed offset in the event struct.
|
||||
|
||||
Verify: app runs, no crash, window shows video as before. ImGui is
|
||||
initialized but invisible (no widgets yet, no render backend).
|
||||
|
||||
### M9.2 — ImGui Vulkan render backend
|
||||
|
||||
Goal: ImGui draw data appears on screen inside the existing render
|
||||
pass.
|
||||
|
||||
1. Create `internal/imgui/backend_vulkan.go`:
|
||||
- `type VulkanBackend struct { ... }`
|
||||
- Holds: pipeline, pipeline layout, descriptor set layout/pool/set,
|
||||
font image + view + sampler, vertex buffer, index buffer (all
|
||||
`vk.*` types from the wrapper).
|
||||
2. `func NewVulkanBackend(pd vk.PhysicalDevice, dev vk.Device, queue
|
||||
vk.Queue, cmdPool vk.CommandPool, rp vk.RenderPass, format
|
||||
vk.Format) (*VulkanBackend, error)`:
|
||||
- **Font atlas**: `imgui.GetIO().Fonts.Build()` then
|
||||
`GetTexDataAsRGBA32(&pixels, &w, &h)` → create
|
||||
`vk.CreateImage2D(w, h, Sampled|TransferDst)` →
|
||||
`vk.CreateImageView` → upload pixels via staging buffer +
|
||||
`CopyBufferToImage` → `vk.CreateSampler`.
|
||||
- **Descriptor set**: layout with one
|
||||
`DescriptorCombinedImageSampler` binding (binding 0, fragment
|
||||
stage) → pool → set →
|
||||
`vk.UpdateImageDescriptor(set, 0, fontView, fontSampler)`.
|
||||
- **Pipeline layout**: one set layout, push constants
|
||||
(ImGui's `ImDrawVert`-based push constant, 16 bytes for the
|
||||
scale/translate vec2s, vertex+fragment stages).
|
||||
- **Graphics pipeline**: `vk.CreateGraphicsPipeline` with
|
||||
`Blend: true`, vertex shader + fragment shader (embed ImGui's
|
||||
SPIR-V or compile from `imgui_impl_vulkan`'s GLSL). Vertex
|
||||
attributes: position (vec2), UV (vec2), color (vec4) — matches
|
||||
`ImDrawVert` layout.
|
||||
- **Vertex/index buffers**: created with a max size (e.g. 1 MB
|
||||
vertex, 256 KB index); recreated (larger) if draw data exceeds
|
||||
capacity.
|
||||
3. Embed ImGui shaders:
|
||||
- Compile `imgui/shaders/imgui.vert` and `imgui/shaders/imgui.frag`
|
||||
(from Dear ImGui's repo) to SPIR-V with `glslc`.
|
||||
- `//go:embed` in `backend_vulkan.go`.
|
||||
4. `func (b *VulkanBackend) RecordDraw(cmd vk.CommandBuffer, drawData
|
||||
*imgui.DrawData, frameIndex uint32)`:
|
||||
- Called inside the existing render pass (between video `Draw(3)`
|
||||
and `EndRenderPass`).
|
||||
- Map/`CopyToMapped` vertex + index data from `drawData` into the
|
||||
staging buffers (or use `vk.Map`/`Unmap` on a host-visible
|
||||
buffer).
|
||||
- `cmd.BindPipeline(b.pipeline)`
|
||||
- `cmd.BindDescriptorSet(b.layout, 0, b.set)`
|
||||
- `cmd.BindVertexBuffers(...)`
|
||||
- `cmd.BindIndexBuffer(...)`
|
||||
- `cmd.SetViewport(...)` (full window)
|
||||
- Push constants (scale/translate from drawData).
|
||||
- For each draw list, for each command: `cmd.DrawIndexed(...)`.
|
||||
5. Wire into `main.go` (or `renderer.DrawFrame`):
|
||||
- After `cmd.Draw(3, 1, 0, 0)` (video), before `EndRenderPass`:
|
||||
`imguiBackend.RecordDraw(cmd, drawData, imageIndex)`.
|
||||
|
||||
Verify: add a test widget `imgui.Text("hello")` in `BeginFrame`/
|
||||
`EndFrame`. Run — you should see "hello" overlaid on the video.
|
||||
|
||||
### M9.3 — The GUI: config panel + stats overlay
|
||||
|
||||
Goal: usable GUI for entering connection params and showing stats.
|
||||
|
||||
1. In `main.go`, between `imguiCtx.BeginFrame(...)` and
|
||||
`imguiCtx.EndFrame()`, build the GUI:
|
||||
- **Config panel** (window, shown by default):
|
||||
```go
|
||||
imgui.Begin("Connection")
|
||||
imgui.InputText("Domain", &domainBuf, 256)
|
||||
imgui.InputText("Video UUID", &videoBuf, 256)
|
||||
imgui.InputText("Audio UUID", &audioBuf, 256)
|
||||
if imgui.Button("Connect") {
|
||||
// trigger source.Open / OpenSync with the entered values
|
||||
}
|
||||
imgui.End()
|
||||
```
|
||||
- **Stats overlay** (window, toggled by F1, no title bar, no
|
||||
background, top-left):
|
||||
```go
|
||||
if showStats {
|
||||
imgui.SetNextWindowPos(...)
|
||||
imgui.Begin("Stats", &showStats, imgui.WindowFlagsNoTitleBar | imgui.WindowFlagsNoBackground)
|
||||
imgui.Text(fmt.Sprintf("FPS: %.1f", fps))
|
||||
imgui.Text(fmt.Sprintf("Dropped: %d", dropped))
|
||||
imgui.Text(fmt.Sprintf("Index: %d", shownIndex))
|
||||
imgui.Text(fmt.Sprintf("Frame: %dx%d", videoW, videoH))
|
||||
imgui.Text(fmt.Sprintf("Format: v210 10-bit"))
|
||||
imgui.End()
|
||||
}
|
||||
```
|
||||
- **Dummy controls** for future M10 flow discovery:
|
||||
```go
|
||||
imgui.Button("List Flows") // no-op yet
|
||||
imgui.Button("Refresh") // no-op yet
|
||||
```
|
||||
2. State: `domainBuf`, `videoBuf`, `audioBuf` are `[256]byte` buffers
|
||||
(ImGui's `InputText` needs a fixed buffer + capacity). Convert to
|
||||
Go string on "Connect".
|
||||
3. "Connect" button:
|
||||
- Close existing source if any.
|
||||
- Call `source.Open` / `source.OpenSync` with the buffer values.
|
||||
- On error, display `imgui.Text` in red below the button.
|
||||
4. F1 toggle: handle in the keydown event switch, flip `showStats`.
|
||||
|
||||
Verify: type a domain + UUID, click Connect, video appears. Toggle
|
||||
stats with F1. Resize window — GUI stays usable.
|
||||
|
||||
### M9.4 — Wire GUI to engine + polish
|
||||
|
||||
Goal: GUI controls the engine, not just displays.
|
||||
|
||||
1. **Connection lifecycle**: "Connect" button triggers source open +
|
||||
renderer buffer creation (if resolution changed). "Disconnect"
|
||||
button closes source, video freezes on last frame (or clears to
|
||||
black).
|
||||
2. **Stats read from engine**: expose `Stats` struct from the loop
|
||||
(fps, dropped, index, frameTime, resolution, format). GUI reads it
|
||||
each frame.
|
||||
3. **Freeze control** (placeholder for M10): `imgui.Checkbox("Freeze",
|
||||
&frozen)`. When frozen, stop calling `DrawFrame` (keep last image
|
||||
on screen, keep polling events + GUI).
|
||||
4. **Input focus**: when ImGui wants keyboard input (text field
|
||||
focused), don't pass key events to the app (e.g., don't toggle
|
||||
fullscreen on 'F' while typing). Check
|
||||
`imgui.GetIO().WantCaptureKeyboard`.
|
||||
5. **Mouse capture**: when ImGui wants mouse, don't let the app
|
||||
process mouse events. Check `imgui.GetIO().WantCaptureMouse`.
|
||||
6. **DPI awareness**: scale ImGui font + style by the window's
|
||||
`SDL_GetDisplayContentScale` (or hardcode 1.0 for now; polish
|
||||
later).
|
||||
|
||||
Verify: full workflow — launch app, enter params, connect, see video +
|
||||
stats, toggle stats, disconnect, reconnect. All via GUI, no CLI flags
|
||||
needed (though flags still work for headless/automation).
|
||||
|
||||
## Notes
|
||||
|
||||
- ImGui's vertex layout (`ImDrawVert`): `pos [2]float32, uv [2]float32,
|
||||
col uint32` = 20 bytes. The vertex shader applies a scale/translate
|
||||
push constant to convert from ImGui's screen coordinates to clip
|
||||
space.
|
||||
- ImGui's fragment shader samples the font texture (and any user
|
||||
textures) using the UV from the vertex. Color is the vertex color,
|
||||
multiplied by the texture sample.
|
||||
- The backend pipeline uses alpha blending:
|
||||
`src=SRC_ALPHA, dst=ONE_MINUS_SRC_ALPHA, op=ADD`.
|
||||
- ImGui produces draw data *after* `EndFrame`/`Render`. The flow is:
|
||||
`BeginFrame` → build widgets → `EndFrame`/`Render` → get draw data →
|
||||
record Vulkan commands from draw data → submit.
|
||||
- Font atlas upload is one-time, during `NewVulkanBackend`. Vertex/index
|
||||
buffers are updated every frame from ImGui's draw data (host-visible,
|
||||
persistently mapped).
|
||||
- The ImGui pipeline is separate from the decode pipeline. Both render
|
||||
into the same render pass / framebuffer / color attachment.
|
||||
Reference in New Issue
Block a user