Skip to content

Add named markers to replay recordings as MP4 chapters#289

Draft
rgarcia wants to merge 2 commits into
mainfrom
hypeship/replay-markers
Draft

Add named markers to replay recordings as MP4 chapters#289
rgarcia wants to merge 2 commits into
mainfrom
hypeship/replay-markers

Conversation

@rgarcia

@rgarcia rgarcia commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

What

Adds named, timestamped markers to screen recordings. A marker records a point in time during an active recording; at finalize the markers are injected into the output MP4 as chapter markers, so users can scrub/crop to them in post (ffprobe -show_chapters, QuickTime, VLC, and most NLEs all read MP4 chapters).

The feature is purely additive and backwards-compatible.

Why

Recordings are currently opaque blobs — there's no way to flag "the interesting thing happened here" while a session runs. MP4 chapters are the standard, tool-agnostic way to annotate timeline positions, so a marker dropped during recording survives into the downloaded file with no extra sidecar.

API

New endpoint POST /recording/mark:

  • Body: { "name": string (required), "id"?: string }
  • 201 { "name": string, "offsetMs": integer } on success
  • 409 when no recording is in progress
  • 400 for an empty/oversized name or missing body

offsetMs is documented as provisional (measured against the recording start time at mark time); the authoritative offset is the chapter start computed at finalize.

How

  • Mark(name) on the recorder appends a marker under the existing mutex and is safe under concurrent calls.
  • At finalize, when markers exist, an ffmetadata file is built (integer-millisecond TIMEBASE=1/1000 chapters, special characters escaped) and passed as a second input with -map 0 -map_chapters 1. A sentinel chapter is prepended at START=0 because ffmpeg forces the first chapter to start at 0 — without it the first real marker's timestamp would be clamped.
  • With no markers, the remux command is byte-identical to before. Any marker/metadata failure falls back to the plain remux, so a recording is never failed or corrupted by this path.
  • The OpenAPI spec is the source of truth; lib/oapi/oapi.go was regenerated via make oapi-generate (openapi-down-convert + oapi-codegen), not hand-edited.

Tests

  • Unit tests for the chapter-metadata builder: sentinel at START=0, integer START/END tiling, negative-offset markers dropped, ascending ordering, special-character escaping, and a nonzero origin-shift parameter.
  • Unit tests for Mark(): 409 path when not recording, trim/validation, and concurrency (run under -race).
  • A backwards-compat test locking the no-marker remux args to the pre-feature command and asserting -map_chapters is injected only when markers are present.
  • An integration test (real ffmpeg, lavfi testsrc, no display needed) that records a synthetic clip, marks at known offsets, finalizes, and asserts via ffprobe -show_chapters that chapters land within ~1.5 frame intervals.

go vet ./..., go build ./..., and the full non-e2e suite (go test -race) pass locally.

Add a Mark(name) API to the recorder and a POST /recording/mark endpoint
that records named, timestamped markers during an active recording. At
finalize the markers are written into the output MP4 as chapter markers via
an ffmetadata input, so they can be scrubbed to in any tool that reads MP4
chapters (ffprobe, QuickTime, VLC, NLEs).

The feature is additive and backwards-compatible: with no markers the remux
command is byte-identical to before, and any marker/metadata failure falls
back to the plain remux so a recording is never corrupted.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@rgarcia

rgarcia commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Independent review — replay markers as MP4 chapters

Reviewed the full diff, read all changed files, rebuilt, ran unit + integration tests, and verified a few correctness concerns with real ffmpeg/ffprobe. Not blocking — the core mechanism is sound and well-tested. A few minor items below.

Verified correct

  • Offset math (markers.go:41): int64 milliseconds end-to-end via Duration.Milliseconds(), no lossy float-seconds hop. originShiftMs is a real int64 parameter, negatives dropped (:42).
  • Sentinel START=0: prepended only when the first real marker is > 0 (:56-58), correctly omitted when a marker is already at 0. END/START tiling is correct (END[i]=START[i+1], last END=durationMs).
  • Escaping (:83-92): covers \ = ; # \n; strings.NewReplacer is single-pass so backslash-first ordering does not double-escape.
  • Backwards-compat: chapterInputArgs("") returns nil, so the no-marker remux args are byte-identical to before. Locked by TestChapterInputArgs + TestFinalizeRemuxArgs_BackwardsCompat.
  • Stream preservation: confirmed with a real two-stream (video+audio) fragmented MP4 that -map 0 -map_chapters 1 -c copy keeps both streams and only pulls chapters from input 1. Arg ordering valid.
  • Failure handling: a buildChapterMetadata error logs and falls back to the plain remux (metaPath stays "") — never fails/corrupts the recording.
  • Concurrency: markers is mutex-guarded; Mark correctly returns ErrNotRecording when cmd == nil or already exited.
  • Generated code: regenerated oapi.go from the spec (down-convert → oapi-codegen → SSE patch → gofmt) and it is byte-identical to the committed file — not hand-edited. Handler mirrors existing endpoints; 409/400/500 mapping is correct.

Minor / nits

  1. markers.go:26-27 references a "downstream fork" in a doc comment. That is leaked context about a non-public consumer, and it is the sole justification for an effectively-dead parameter. Suggest dropping the fork reference (and reconsidering the param, see Remove dependency #2).
  2. originShiftMs is dead in this repo — every call site passes 0. The parameter, the shift branch, and TestBuildChapterMetadata_OriginShift exist only for an out-of-tree caller. Borderline YAGNI/out-of-scope; consider dropping it until this repo needs it.
  3. Backwards/zero-length chapter edge case: a marker timestamped at/after endTime yields START >= END (verified: produces START=20000 END=8000). Not reachable in normal flow (Mark requires an active recording and endTime is captured at process exit) and ffmpeg tolerates it, but there is no clamp and no test. Low-priority robustness nit.
  4. No real-ffmpeg no-markers finalize testTestFinalize_InjectsChaptersWithRealFFmpeg only exercises the with-chapters path; the no-marker path is locked at the arg level only. Optional.

Build green, go vet clean, all recorder + api unit tests and the ffmpeg integration test pass locally.

buildChapterMetadata always received 0 for the origin shift, so remove
the parameter, its drop-when-shifted branch, and the corresponding test.
Chapter starts are now computed directly from each marker's offset.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant