Input Injection

How injection works

Once a Vision Pro is paired, every spatial keystroke and every pointer motion from vmux turns into a small JSON envelope on the wire. MacKeyHost decodes the envelope and synthesizes a CGEvent — the same event type that real USB and Bluetooth keyboards generate when macOS reads from them. The synthetic event is posted at the HID-level event tap (cghidEventTap), which means it enters the Mac's event queue at the lowest layer, before any per-app input handling.

Practically, this is indistinguishable from typing on a hardware keyboard plugged into the Mac. macOS routes the event to the frontmost responder, key repeat is handled by the OS, and Input Sources / dead keys / IME composition all behave the way they would for any external keyboard.

For text — letters, numbers, punctuation, anything you type with the spatial keyboard — MacKeyHost uses CGEvent.keyboardSetUnicodeString so the actual character codepoint travels with the event. This is layout-independent: typing é on the Vision Pro produces é on the Mac regardless of whether the Mac is set to U.S. English or French. For special keys (arrows, escape, F-keys) MacKeyHost uses fixed virtual key codes that match Apple's hardware keyboard layout.

Modifier behavior

Modifiers travel as four independent boolean flags inside the key event: shift, control, option, command. MacKeyHost applies them as CGEventFlags on both the key-down and key-up so macOS sees the modifier held for the duration of the keypress, exactly as it would on hardware.

The mental model matches a real Mac keyboard:

The Vision Pro spatial keyboard exposes Cmd, Control, Option, and Shift as on-screen modifier keys. Tap to latch, tap again to release. Latched modifiers are sent on the next keypress; you can chain Cmd+Shift+P by latching both then tapping the letter.

There is no Caps Lock and no Fn key. Caps Lock has no direct equivalent in the spatial keyboard — use Shift latching for the few keys you need. Fn-modified keys (display brightness, volume) are not transmitted; see "What does not pass through" below.

Special keys reference

These are the named keys MacKeyHost recognizes on the wire and the macOS virtual key codes it posts:

All of these can be sent with any combination of modifiers. Cmd+Tab opens the Mac app switcher; Option+Right jumps a word forward; Shift+End selects to end of line.

Mouse and pointer

MacKeyHost handles five mouse event types from vmux:

The Mac maintains its own internal cursor position. Each Move event updates the position by the delta and clamps it to the bounds of the main display — the cursor cannot slide off the main display into a secondary monitor or run past the menu bar. (If you have multiple displays, the cursor is restricted to the primary; this is a current limitation.)

Tap-to-click on the Vision Pro touchpad ornament generates a click + release pair in quick succession, producing a normal single click on the Mac. To drag, tap-and-hold to send the click without an immediate release, drag, then lift to send the release. Scroll uses two-finger movement on the ornament; both axes are supported.

There is no double-click latency tuning — macOS uses the system double-click interval, so two quick taps register as a double-click if delivered within that window.

Rate limit

MacKeyHost rate-limits incoming events to 100 events per second. Anything over that in a one-second window is silently dropped. In normal use you will never hit this — typing tops out around 10 events per second, and pointer motion is already throttled by the trackpad ornament. The cap exists as a safety guard against a runaway sender.

If you need to paste long text, paste from the Mac's clipboard rather than typing it character-by-character on the spatial keyboard. The clipboard transit is instantaneous and avoids the rate limit entirely.

What does not pass through

A few categories of input cannot be injected by any third-party Mac tool, MacKeyHost included. These are deliberate macOS protections.

You can confirm a Mac app has Secure Input active by checking Input Sources in the menu bar — when Secure Input is on, a small lock icon appears. If you find a Mac app blocks all keystrokes from MacKeyHost while normal apps work, Secure Input is the cause.

Latency and what to expect

End-to-end latency from spatial-keyboard release to Mac character render is dominated by the Multipeer Connectivity transport and the macOS event queue. Typical numbers:

If typing feels sluggish, you are likely on the BLE fallback. Toggle Wi-Fi off and back on at both ends to nudge AWDL back into the picture.

Related

• Pairing with Vision Pro — how the link gets established before injection can happen.
• Security and permissions — Accessibility, encrypted transport, what an attacker on the same Wi-Fi can and can't do.
• Troubleshooting — modifier weirdness, missing keys, lost permissions.