Architecture

Reelbird is a native macOS app (Swift 6, SwiftUI + AppKit, Metal) with a command line tool that shares the same engine. It has no server and no third-party Swift dependencies.

ReelbirdCore     project format, edit document, events, timeline + motion maths (pure Swift)
ReelbirdCapture  ScreenCaptureKit, camera, microphone, input events, segmented writers, health checks
ReelbirdRender   Metal compositor, preview player, exporter
ReelbirdShare    S3-compatible uploads (SigV4), HLS playlists, static viewer page, Keychain config
ReelbirdAI       on-device captions (SpeechAnalyzer), titles/chapters (Foundation Models), silence detection, Find Sensitive Info (Vision)
ReelbirdApp      the Mac app
ReelbirdCLI      `reelbird` command line tool + MCP server

Dependencies flow inward: every module depends on ReelbirdCore and nothing else, except the app and the CLI, which compose all of them.

Recording

One host clock (CMClockGetHostTimeClock) drives everything. Timeline zero is the host time of the first screen frame. Media, input events and edits all use seconds on that timeline.

• Screen: SCStream in 420v (NV12) at native pixel size, with Reelbird’s own windows excluded. The system cursor is hidden and re-drawn later from the event log. Hardware HEVC (H.264 for the compact preset, unless the frame is too large for H.264).
• Windows: the picker lists every app window, not just the current Space’s: on-screen ones front to back, then windows on other desktops, in full-screen Spaces or minimized, grouped by app (CaptureSources.arrange). macOS gives no public signal that tells those off-screen cases apart. ScreenCaptureKit can’t capture an off-screen window, so Start first asks macOS to bring it on screen (WindowActivation: unminimize through Accessibility when allowed, activate the app so macOS switches Space, then a Dock-style reopen) and waits up to 2 s; if the window doesn’t come, the picker asks you to bring it on screen. A window that leaves the screen mid-take (another Space, minimized) doesn’t stop the recording: ScreenCaptureKit sends idle or no frames, the take holds its last frame and the “screen stopped updating” warning shows until the window is back.
• Audio: microphone and system audio captured by ScreenCaptureKit on the same clock (macOS 15+), as separate AAC tracks. AAC in fragmented MP4 can’t hold a gap (the writer would close it up and shift everything after), so each track stays continuous: gaps become silence, overlaps are trimmed, a sample stamped on another clock is put on arrival time, and chunks a busy encoder can’t take yet wait in a fixed queue. Audio lost past that queue becomes silence of the same length and raises the drift warning.
• Camera: its own 30 fps track, so the bubble’s size, shape and place can change after recording. The live bubble is a floating panel kept out of the capture; its frame goes to events.jsonl as camera events when recording starts and whenever a drag or resize ends (RecordingSession.updateCameraBubble, on the recording clock, moved inside the captured area). The first edit places the camera from them: the start place, then a 300 ms glide for each move (Project format). The layout shortcut (⌥⇧L, registered only while a take with the camera runs) switches the live layout (bubble, camera only, screen only); each switch is a camera event with a layout, and the first edit turns them into layout segments (Project format).
• Events: pointer position (sampled at 120 Hz), clicks, scrolls, cursor image changes and, when enabled, keystrokes (a listen-only event tap, which needs Input Monitoring; for a window recording only while that window’s app is in front) go to events.jsonl, as do markers (RecordingSession.addMarker, from the marker shortcut, the control bar’s flag or MCP add_marker). The format also has a focus event for focused-element frames; nothing records it yet.
• Segments: every track is written by AVAssetWriter as fragmented MP4 with 2-second segments delivered through the writer delegate and written to disk as they arrive. A crash loses at most the segment in progress. Stopping concatenates the segments into one file per track (no re-encoding), verifies it, writes the finished project.json with a full flush, then deletes the segments (the share stream’s after its upload finishes). The recorder holds the bundle’s lock for the whole take, so recovery from another process waits instead of rebuilding a take that is still being finished. Details: Project format.
• Pause: samples are dropped while paused; the paused ranges go into project.json and become cuts in the edit.
• Health: the recorder warns while recording if the microphone delivers silence, frames stall, audio drifts, the disk fills up, or uploads fall behind. A camera, microphone or system audio track that fails (the camera can’t start, an encoder stops, the file doesn’t open at stop) is reported while recording and after stop (RecordingSession.trackIssues; the CLI and MCP add warnings to the result), never dropped silently.

Sharing

Sharing is optional and serverless: the app uploads to an S3-compatible bucket you own (Cloudflare R2 recommended, free egress).

1. When recording starts, the app reserves a short ID and uploads a placeholder viewer page, so the link exists before the first frame.
2. While recording, a second writer produces an H.264 stream (screen scaled to at most 1920 px wide, the camera bubble where the live bubble is, mixed audio, 30 fps) in 2-second CMAF segments, kept in the bundle’s share/ folder and uploaded as they are written, plus an HLS playlist that grows (EVENT). If the app quits mid-upload, a small journal lets the next launch finish it.
3. On stop, the link is copied immediately; the last segment and the end marker follow within about a second.
4. After editing, Update link uploads the polished MP4 under a new revision name and rewrites meta.json and the viewer page, which switches to it. The link never changes.

The viewer page is static HTML that reads meta.json. Safari plays HLS natively; other browsers load a pinned, integrity-checked hls.js only while there is no polished MP4. The full bucket layout is in Sharing setup.

Editing and rendering

edit.json holds every decision; media is never modified. A render plan is computed once from the manifest, edit and events: the zoom camera path (log-scale zoom, critically damped spring, anchored on the focus point), the smoothed cursor path (resampled, spring-damped) and click effects. The plan is deterministic, so preview and export always match. One Metal compositor draws both: background, padded rounded screen with shadow, zoom, cursor, click effects, camera bubble, annotations (highlights, then privacy blurs anchored to screen-video cells, then arrows, boxes and labels), captions. The plan also holds the camera layouts over time (LayoutTimeline) and what the bubble does during zooms (CameraZoomFraming: it shrinks on the zoom’s curve and steps aside from the zoomed cursor), so the camera’s tile at any moment (RenderPlan.cameraTile) is the same in preview and export. Trim, cuts and speed segments become one time map (EditTimeline): the composition that preview and export read places each kept piece at its output time and scales it to its speed (scaleTimeRange, sound time-stretched with its pitch kept), and the compositor maps each output frame back to recording time for the cursor, zooms and overlays.

The editor’s preview decodes with AVPlayer and renders on its own thread with a CAMetalDisplayLink. Export reads tracks with AVAssetReader, composites on the GPU and encodes with VideoToolbox through AVAssetWriter (H.264, HEVC, ProRes) or ImageIO (GIF).

Audio

Raw is sacred: the microphone and system audio stay as recorded, and all processing is non-destructive. After a recording opens (and whenever “Enhance voice” settings change) the microphone is rendered offline into a voice stem (ReelbirdRender/VoiceStem.swift): high-pass → AUSoundIsolation in a manual-rendering AVAudioEngine (about 95× real time; its 92.5 ms latency removed exactly) → expander → compressor → de-esser → presence EQ → blend with the raw track. The denoiser sits behind a VoiceDenoiser protocol so ML models can replace it. Preview and export play the stem in place of the microphone; the preview also carries the raw track at zero gain for “Hear original”. Both run the same per-track tap (AudioTrackProcessor): gain (auto level to −16 LUFS + boost), ducking of system audio from the voice’s speech ranges, and a look-ahead limiter; export limits the sum too and cuts the sound at the picture’s last frame to the sample (the audio mix reader hands out whole buffers past its time range, so its own end is never trusted). Captions transcribe the raw microphone. Details and file layout: Project format.

Captions and AI

Captions use SpeechAnalyzer on macOS 26 and later, and on-device SFSpeechRecognizer on macOS 15. Both produce word timings, saved to transcript.json next to captions.vtt. Titles and chapters use the Foundation Models framework on Macs with Apple Intelligence. Find Sensitive Info samples the screen video, runs Vision text recognition only on frames that changed, matches emails, keys, card and phone numbers (ReelbirdCore/SensitiveInfo.swift) and links sightings across frames into blurs that follow moving text (SensitiveTracker). Silence detection reads the audio tracks directly. Nothing here uses the network.

Threading

Blocking AVFoundation calls never run on Swift concurrency’s cooperative thread pool. That pool has one thread per core and never grows, and AVFoundation finishes part of every AVAssetReader read on it: once every pool thread is blocked in copyNextSampleBuffer, nothing can finish and the process deadlocks (measured on a 14-core Mac: 14 concurrent decode loops in tasks hang forever, 13 finish in 0.2 s). A starved pool also stalls DispatchQueue.global() at the same QoS, so a global queue is no way out either.

• Decode, encode and GPU-wait loops run through BlockingWork.run (ReelbirdCore/BlockingWork.swift): a serial dispatch queue with a thread of its own, bridged with a continuation. Cancelling the calling task sets the CancellationFlag the loop checks per buffer. Audio analysis, the voice stem, thumbnails, silence detection, the PCM decode for captions, the sensitive-info scan, the timeline waveform and closing a take use it; export and the preview’s render loop have their own threads.
• Use AVFoundation’s async APIs where they exist (load(_:), loadTracks, AVAssetImageGenerator.image(at:) / images(for:), finishWriting()), never a semaphore around a callback. A semaphore or .wait() never runs on a pool thread, and neither does Task.detached with blocking work (it runs on the same pool).
• Tests follow the same rule: Swift Testing runs tests in parallel on the pool, so test helpers that decode go through BlockingWork too. PoolStarvationTests runs more decodes at once than there are cores and fails loudly (a watchdog thread aborts) if the pool starves again.

Command line and MCP

reelbird drives the same modules as the app. Every command can print JSON, and reelbird mcp exposes the same operations as MCP tools over stdio (JSON-RPC, one message per line).

Project format

See Project format.