Five minutes to five seconds

Our field technicians install wearable devices in hospitals. Before this tool existed, bringing one online meant a technician standing in a hallway driving a sequence of manual steps by hand — roughly five minutes per device. There are five or six technicians and a deployment is hundreds of devices, so the time on site was the whole cost, and the site is a working hospital.

One constraint shaped every decision. The laptop is on hospital guest wifi, in a basement, in a stairwell. The network is not reliable and never will be. Whatever I built had to survive the link dropping mid-operation without inventing a state that never happened on the hardware.

What it is

A headless client that runs on the technician's MacBook. It holds an authenticated WebSocket to our admin application, scans continuously for nearby devices and ranks them, and executes connect-and-run jobs against the ones it is told to. The technician drives it from a page in the admin app they already use: the laptop does the radio work, the browser is the interface.

It ships as a standalone macOS installer. That mattered more than I expected. Field technicians are not developers and should not have to install a toolchain to do their job.

The parts that took the thinking

Two delivery classes, chosen per message type. Events that record something that happened to hardware go into a durable outbox and drain in order, with exponential backoff capped at a minute. The drain breaks on the first failure rather than skipping ahead, because the ordering is the point — a later event that implies an earlier one is a lie if the earlier one never arrived. Telemetry that is cheap to lose, like scan summaries and the current top-ranked devices, is marked droppable and never queued at all. Paying persistence cost for data that is stale in a second is just a slower way to be wrong.

Idempotency, because retries are the normal case, not the exception. Every command carries a key and is written to a journal. If a command arrives whose status is already terminal, the stored result is replayed with a flag saying so and the radio is never touched. A technician who taps twice, or a reconnect that replays a queued command, cannot write to a device twice.

Write ordering, which is really a trust boundary. The device is written first, and only once the hardware confirms does the client ask the admin application to record it. A failed Bluetooth write sends nothing to production. The client holds no production database credentials at all — it asks the server to write and the server decides. A laptop in a hospital basement is never a thing that can corrupt the fleet record.

A watchdog on a monotonic clock. Any packet resets it. Thirty seconds of total silence emits a restart event, drops the socket, and exits non-zero so launchd brings the process back with a backoff. It detects silence, not radio state, which means a genuinely empty room looks identical to a wedged adapter. That was deliberate: the recovery is cheap and the false positive costs nothing.

What changed

On-site device maintenance went from roughly five minutes to about five seconds per device, for the entire field technician team. I was the sole author.