Add ExecuWhisper macOS dictation example (Parakeet + LFM2.5 formatter)

[seyeong-han]

Fully On-Device Free Dictation App

clean-reformatting.mp4

What you hear in the clip:

"Uh can we, can we move the meeting uh to Friday, actually no, let's let's make it Monday at 10 in the morning."

What gets pasted:

Can we move the meeting to Friday? Actually, no, let's make it Monday at 10 AM in the morning.

execuwhisper-overlay-dictation.mp4

Summary

ExecuWhisper is a native macOS dictation app that runs fully on-device using ExecuTorch. ASR via NVIDIA Parakeet-TDT (Metal backend); a fine-tuned LiquidAI LFM2.5-350M cleans disfluencies, casing, and punctuation (MLX delegate). No cloud, no API keys, no telemetry.

This PR adds the source under execuwhisper/macos/, mirroring the existing voxtral_realtime/macos/ layout.

What's in this diff

• New execuwhisper/macos/ directory:
  • ExecuWhisper/ — Swift app source.
  • ExecuWhisperTests/ — XCTest target.
  • docs/ — demo script, support runbook, release-QA checklist.
  • scripts/ — build.sh, create_dmg.sh, sign_release.sh, verify_*, probe_formatter.py, benchmark_helper.py.
  • project.yml — xcodegen spec (no DEVELOPMENT_TEAM hard-coded; user supplies via env var).
  • README.md, CHANGELOG.md, THIRD_PARTY_NOTICES.md, .gitignore.

Upstream dependencies (in review)

The helper binaries this app embeds live in three PRs against pytorch/executorch that are still in review. Reviewers/users have two paths:

• A. Wait for them to land on pytorch/executorch:main and build everything from a single source checkout.
• B. Use the prebuilt arm64 helpers attached to this PR's GitHub Release and only build the Swift app locally.

PR	What it adds
pytorch/executorch#18861	parakeet_helper (ASR runtime, Metal) + make parakeet-metal
pytorch/executorch#19195	LFM2.5 MLX export pipeline + lfm2_5_350m model class + lfm_2_5-mlx Makefile target
pytorch/executorch#19562	lfm25_formatter_helper (formatter runtime, MLX) + make lfm_2_5_formatter-mlx

Models

• ASR: younghan-meta/Parakeet-TDT-ExecuTorch-Metal
• Formatter: younghan-meta/LFM2.5-350M-ExecuWhisper-Formatter (new dedicated repo for the ExecuWhisper-specific fine-tune; includes fp32 weights, MLX-exported .pte, tokenizer, and full eval reports)

Prebuilt binaries

GitHub Release execuwhisper-v0.1.0 on the fork (until this PR merges) ships:

• lfm25_formatter_helper-arm64-darwin.tar.gz
• parakeet_helper-arm64-darwin.tar.gz
• mlx.metallib
• helpers-arm64-darwin.tar.gz (one-shot bundle of the above)

Helpers are pre-signed with the hardened runtime + disable-library-validation + allow-dyld-environment-variables entitlements. Not redistributing libomp.dylib (third-party LLVM OpenMP) — users obtain via brew install libomp. Not redistributing the .app / .dmg (legal review pending).

Validation

Release-gate eval for the formatter (AMI corpus subset):

Metric	Value	Gate	Status
Forbidden	0.030	≤ 0.10	✅
Coverage	0.874	≥ 0.85	✅
Verdict	RELEASE-READY	—	✅

In-app smoke tests: dictation overlay, hotkey, replacements, snippets, session export, long-input chunking, helper warm reuse — all passing on macOS 14.x and 15.x, MacBook Pro M3 / Mac Studio M2.

Known limitations

• macOS-only; Apple Silicon required (M1+); macOS 14+.
• Three upstream ExecuTorch PRs still in review (linked above).
• Formatter occasionally over-summarizes self-corrections ("actually no — make it tomorrow"); see model card for full eval breakdown.
• 30-word chunker is naive on word boundaries; smarter chunking deferred.

What changed since the previous push of this branch

• Moved from ExecuWhisper/ (top-level) to execuwhisper/macos/ (mirrors voxtral_realtime/macos/ convention).
• Dropped the committed .xcodeproj/; added .gitignore and an xcodegen generate build step.
• Removed hard-coded DEVELOPMENT_TEAM; users supply via env var.
• Sanitized requirements_et-mlx.txt / requirements_et-metal.txt (removed @ file:///Users/younghan/... editable installs; documented pip install -e <executorch_path> step in README).
• Updated HF repo references from younghan-meta/LFM2.5-ExecuTorch-MLX to the new dedicated younghan-meta/LFM2.5-350M-ExecuWhisper-Formatter.
• Removed internal handover docs (FINETUNING_HANDOVER.md, FINETUNING_DATASET_HANDOFF.md).
• Added THIRD_PARTY_NOTICES.md, ASCII architecture diagram, acknowledgements.
• Replaced personal-name test fixture in TextPipelineTests.swift with a generic example.
• Reworded "internal DMG" language across CHANGELOG.md, RELEASE_QA_CHECKLIST.md, and create_dmg.sh.
• Added BSD copyright headers to probe_formatter.py, sign_release.sh, verify_project_settings.sh, verify_release.sh.
• Eval results published to the HF model card under eval/.

Reviewer guidance

• Focus on: the Swift app (ExecuWhisper/Services/, ExecuWhisper/Views/), the build wiring (project.yml, scripts/build.sh), and the README's prebuilt-vs-source story.
• Auto-tested: xcodebuild -scheme ExecuWhisper test exercises the formatter prompt builder, replacement pipeline, session compatibility, and helper bridge reuse.
• Out of scope for this PR: Windows/Linux ports, DMG distribution (legal review pending), CI for native macOS apps in this repo (no precedent).

[psiddh]

cc @metascroy @digantdesai

[metascroy]

@claude review this code

[psiddh]

@claude review the code

[metascroy]

Review

This PR adds a ~12k-line macOS Swift app (57 source files + tests + scripts). Overall it's well-structured — actors for the model bridges, proper locking in the accumulators, atomic file writes, checksum-verified downloads, and graceful fallbacks in the formatter path. No critical/blocking bugs. Below are the issues worth addressing, roughly by severity.

Notable issues

1. Misaligned raw load can crash WAV parsing — Medium
RunnerBridge.swift:548-552 (readUInt32) uses bytes.load(fromByteOffset:as:UInt32.self). In the chunk-walk loop (:566-579), offset is only kept even (chunkSize % 2 padding), not 4-byte aligned. UnsafeRawBufferPointer.load(fromByteOffset:as:) requires natural alignment for T; a data/fact/extensible-fmt chunk whose size isn't a multiple of 4 pushes the next header read to a 2-mod-4 offset and traps ("load from misaligned raw pointer"). Canonical recorder/decoder output happens to stay aligned, so it's latent, but it's fragile for imported/edge WAVs. Fix: use loadUnaligned(fromByteOffset:as:) (Swift 5.7+) for both the UInt16 and UInt32 reads.

2. Out-of-order helper messages under actor reentrancy — Medium
RunnerBridge.swift:335-337 and FormatterBridge.swift:278-280 dispatch each stdout line as a separate Task { await self.handleHelperLine(...) } from a background reader. Actor reentrancy does not guarantee these run in submission order, so .status/.result/.ready can be processed out of sequence. Impact is usually cosmetic (status text), but a reordered .result/.error vs .ready is possible. Prefer funneling lines through a single serial AsyncStream the actor consumes in order, rather than one Task per line.

3. Large PCM written synchronously to the helper's stdin — Medium/Low
RunnerBridge.sendTranscriptionRequest (:441-442) does stdinHandle.write(contentsOf: pcmData) on the actor. For long recordings this blocks the actor thread, and if the OS pipe buffer fills before the helper drains its payloadByteCount, you risk a stdin/stdout deadlock. Works today because the helper reads the payload immediately, but it couples correctness to helper behavior. Consider chunked/off-actor writes.

4. Duplicate side-effecting construction at launch — Medium
ExecuWhisperApp.swift:14-16 give @State inline initializers (= Preferences(), = ModelDownloader(), = ReplacementStore()) and init() reassigns them via _preferences = State(initialValue:) (:42-46). SwiftUI still evaluates the inline defaults and discards them, so each is constructed twice. Since these constructors create directories and ReplacementStore.init writes replacements.json, those side effects run against throwaway instances. Drop the inline initializers: @State private var preferences: Preferences.

Minor (flagging, not blocking)

• Version constant mismatch: PasteController.helperVersion = "3" (PasteController.swift:27) vs helperVersion = "2" in Support/PasteHelper/main.swift:20. Harmless today (the install check compares a self-written .version file, never the binary's --version), but confusing and a latent footgun if you ever start comparing them.
• warmupTask capture: TranscriptStore.swift:809 references maxRecordingDuration unqualified inside a [weak self] closure — compiles now with a warning, but is an error in Swift 6 language mode. Use self?./guard let self first.
• Force-unwrap: PersistencePaths.swift:13 force-unwraps FileManager…urls(...).first! — practically never nil, but a hard crash if it is.
• Stale reference: DictationManager.targetApp is never reset to nil after a dictation, holding an NSRunningApplication until the next run. Harmless.
• TextPipeline.validateFormatterOutput (:263-265) hardcodes specific sentences ("okay, so the plan is finish the build", etc.) as rejection triggers — looks like test-data leakage into production logic; brittle but not harmful.

Verified OK

Download integrity (all 5 assets covered by model_manifest.json with SHA-256 + size + HTML-sniff checks), staging→activate atomic install, paste helper (unprivileged, copied from signed bundle), AudioRecorder actor isolation, DictationManager/GlobalHotKeyManager state balancing, ImportedAudioDecoder bounds/memory guards, ReplacementStore bad-decode preservation, and session persistence.

Net: no major blockers. I'd fix #1 (loadUnaligned) and #4 before merge, and consider #2/#3 for robustness.

Style nits intentionally omitted.

[metascroy]

Inline, line-anchored version of my earlier review comment. Severities and reasoning are in each thread below; style nits intentionally omitted.