Three architectures for therapy AI scribes

Cloud-only, hybrid, and on-device. A taxonomy with examples from the field — Mentalyc, Upheal, Heidi, Blueprint, Bryl — and why the choice between them is a privacy decision more than a feature decision.

There are three plausible places to run an AI therapy scribe. The press releases tend to obscure which one a vendor is doing. The choice changes everything downstream — what gets stored, who gets subpoenaed, what fails in Airplane Mode, what your patients have to consent to.

This post is the taxonomy I wish I had when I was evaluating tools.

1. Cloud-only

The audio goes up. The transcript is generated on a server the vendor controls. The note is drafted by an LLM the vendor pays for tokens on. Everything comes back as a row in a dashboard. Examples: Mentalyc, Upheal, Blueprint AI, Abridge (general medical), Suki (general medical).

This is the dominant pattern in 2026, partly because it's the easiest to ship — you can iterate the model server-side without forcing app updates, you can A/B different prompts, you can centralize moderation and abuse handling. It's also the model that supports tight EHR integrations: the cloud-side note already has a structured format the EHR API expects.

The cost is the surface area. The vendor receives PHI. The vendor needs a BAA with you. The vendor needs sub-processor agreements with their LLM provider, their cloud provider, and possibly their diarization vendor. Every one of those is a place a breach can happen. The mid-tier health-tech breach pattern (an unrotated key in a public bucket; a dependency's zero-day; an acquirer changing storage policy) is common enough to be predictable. We wrote a longer essay on the pattern.

2. Hybrid — local capture, cloud inference

The phone records. Audio is held locally for privacy theater. But to draft the note, the audio (or its derived transcript) is sent to the vendor's server — sometimes briefly, sometimes durably. The marketing line is “your audio stays on your device”; the engineering reality is that the transcript, which contains the same clinical content, takes the same trip the audio would have. Examples: certain configurations of Heidi Health and a handful of newer entrants pitching “phone-first” positioning.

Hybrid is the most marketing-friendly architecture and the easiest to misread. Read the data-flow diagram, not the marketing line. If the LLM is on a server, your patients' words are on a server, even if the raw audio isn't. The legal exposure is similar to cloud-only; the sub-processor count is sometimes smaller.

3. On-device

Everything runs on the phone in your hand. An on-device ASR model — Parakeet TDT v3 or Apple's built-in Speech framework — transcribes locally. A small language model — Apple Foundation Models on a recent iPhone, or a quantized Phi-4 / Llama / Gemma class model otherwise — drafts the note locally. The vendor receives nothing. There is no BAA because there is no PHI to be a Business Associate to. Example: Bryl (the project I work on).

On-device is the hardest to ship. The model bundle is large (1–6 GB). Performance varies by device. You can't A/B prompt changes without a TestFlight cycle. You can't do tight EHR integrations because the EHR API expects a server-side caller. The set of supported phones is small — the on-device model needs the Neural Engine and the unified memory of A17 Pro and later.

What you buy with that engineering tax is the architecture: the vendor cannot leak data they never received.

Why the choice is a privacy decision, not a feature decision

The three architectures look feature-equivalent in a sales demo. They diverge on three axes that aren't always visible in marketing:

1. Subpoena exposure. If a court orders the vendor to produce session audio, what does the vendor have? Cloud-only: audio & transcripts. Hybrid: transcripts. On-device: nothing.
2. Acquisition risk. If the vendor is acquired, the new owner inherits the data store and may quietly change retention policies. The protection in cloud and hybrid models is contractual. In on-device, the protection is architectural — there is no data store to inherit.
3. Failure mode. Cloud and hybrid scribes fail entirely without a network. On-device scribes fail entirely without a charged phone. You can verify on-device by toggling Airplane Mode mid-session.

If you're comparing two scribes and both pitch “HIPAA-compliant, end-to-end encrypted,” ask: what happens to my audio between the moment the session ends and the moment the note appears? The answer separates the architectures.

Where each architecture earns its keep

The honest version, with no axe-grinding:

• Cloud-only wins when you need rapid model iteration, deep EHR auto-paste, and team-admin tooling, and your compliance threshold is a signed BAA from a serious vendor.
• Hybrid wins when you want some local privacy story but also need server-side features. Read the data-flow carefully — “phone-first” is sometimes a metaphor.
• On-device wins when your compliance threshold is architectural — when you'd rather a vendor literally cannot have the data than rely on contractual protection. It also wins when your patient base is sensitive enough that “the session stays between us” needs to be a literal claim, not a contractual one.

The technical asterisk

On-device inference in 2026 is good but not perfect. A 3.8B-parameter quantized model is qualitatively different from GPT-4-class cloud models. For some tasks (long-form summarization across 20+ sessions, complex differential drafting) the cloud models are still better. We try to be specific about this — Bryl benchmarks at 6.4% Word Error Rate on a 120-session corpus, and the note quality is comparable to mid-tier cloud scribes for solo psychotherapy. For couples and family work, multi-speaker accuracy is improving but not yet at parity with the better-resourced cloud diarizers.

If on-device hits the right side of that trade-off for your practice, you get architectural privacy. If it doesn't, the cloud or hybrid options are real and serious. The point of this post is to make the trade-off legible, not to argue there's only one right answer.

A short reading list

• Privacy by architecture vs HIPAA — long version of the contractual-vs-architectural distinction.
• What on-device AI actually means — the engineering side of running Parakeet TDT v3 + a 3.8B model on a phone.
• Bryl vs Mentalyc vs Upheal — the side-by-side from the vendor angle.
• Architectural attestation — what a compliance file would say about Bryl specifically.