OpenCode Hex: Local Voice Input and Screen Annotations to Speed Up Working with AI AgentsNote · digest for November 9
A strict top-down view of a desk with a vintage OpenCode Hex microphone and a notebook of voice commands.
Note3 sections≈ 4 min

OpenCode Hex: Local Voice Input and Screen Annotations to Speed Up Working with AI Agents

When you need to explain a tricky layout bug or refactoring logic to an AI agent in detail, you usually have to pick between two awkward options. Either you spend forever typing a multi-line wall of text on the keyboard, burning energy on typos, or you turn on cloud voice dictation—and immediately run into corporate security rules forbidding you from sending descriptions of proprietary architecture to third-party servers.

I looked for a way around that fork until I examined OpenCode Hex—a free, open-source Rust utility (MIT license, stable release on October 31) from the Anomaly team. How do you stop wasting time typing long prompts by hand and pass task context to AI agents by voice and graphics straight from your own machine with zero cloud?

Keyboard Friction: Why Typing Speed Slows Down Agent Workflows

The bottleneck lies in the gap between how fast neural networks think and how slowly we feed them thoughts through keys. When working with terminal and editor agents like Claude Code, Cursor, or OpenCode, you have to spell out which module threw an error and which files are off-limits. An average person types at 40–50 words per minute, whereas out loud we speak at 130–150 words per minute. If every assignment takes minutes of manual typing, the keyboard eats up to half your working time with AI—and if fatigue pushes you to write shorter prompts, the agent loses context and outputs half-baked code.

In the end, the fastest AI assistant moves only as fast as your fingers.

Voice and Marker Instead of a Thousand Words: How Dictation and Annotate Mode Work

To remove that brake, the project's creators (we previously covered their console agent OpenCode for coding and automation) built input around holding a key (Option on macOS, Ctrl + Space or Alt on Windows and Linux). Hold the button, speak the task, and release: a local speech-recognition model (Whisper or Parakeet) transcribes your voice into text and pastes it wherever the cursor blinks—terminal, browser, or chat window. You're probably wondering: "What about library names that standard dictation always mangles?" Hex has a Custom Vocabulary for that: add terms like k8s, fastapi, or useCallback to the config to push transcription accuracy up to 99%, and via a Model Context Protocol (MCP) server, agents read fresh notes directly.

The biggest payoff, though, comes where explaining in words takes too long and it's easier to point at the screen.

For those moments, the utility includes Annotate mode. Say an engineer needs to fix a broken dashboard and adjust the backend without listing CSS classes. The developer opens the report in a browser next to the agent window, presses Option + Shift + S on Mac (or F9 on Linux), circles the crooked chart with the cursor without clicking, draws an arrow to the date filter, and explains out loud why grouping shifts when changing time zones. The marker stroke fades after a second, yet stays in the recording: once stopped, Hex bundles the annotated frames and speech transcript into a single package and sends it straight into the agent dialog (or via the hex annotate command).

The result is felt right away: instead of five minutes painfully describing "the second chart from the left," framing the task takes fifteen seconds of speech and one hand gesture. The engineer speeds up writing instructions by 3–4x, and the agent sees both the visual bug and the exact code task on the first try.

Cloud-Free Privacy and Setup on Mac, Windows, and Linux

When working with a mic and screen capture, protecting a closed security perimeter comes first. While cloud services ship audio to external servers, OpenCode Hex runs recognition 100% locally on your device: audio never leaves RAM and isn't saved to disk, and text dictation logs stay local for 7 days with a Clear All button. Thanks to Rust, the background process uses under 80 MB of memory. The stable build for macOS 15+ (Apple Silicon and Intel) installs via an installer or Homebrew, while beta builds are available for Windows 10/11 (x64 with AVX2 and ARM64; Annotate mode is coming later) and Linux (Debian, Ubuntu, Fedora, and openSUSE under X11/i3 and Wayland/wlroots).

Details like this restore tools to their original purpose. Ultimately, we tame AI agents not to tap keys in front of a monitor until midnight, but for free time, a clear head by evening, and unhurried real-world time with the people we care about.

Want a deep diveVotes help us choose what to break down in detail. This material has 0 votes.
Sources · 2