What 'on-device AI' actually means for therapy notes

A clinician's guide to the technical anatomy of an on-device scribe — Apple Foundation Models, Parakeet TDT v3, the Secure Enclave, and what 'verifiable' looks like with a packet inspector.

You don't need to be a developer to use a tool you trust. But you do, if you're a clinician carrying client confidentiality on your back, deserve to know what the tool is actually doing. The phrase on-device AI has been used loosely enough by enough vendors that it's worth being precise about what it means inside Bryl.

This is a long post. It is structured the way a diligent colleague might think through it: what runs where, why it matters, and how to verify the claims yourself.

The anatomy of a session

A 50-minute talk-therapy session inside Bryl passes through five distinct technical stages. None of them touch the network for content.

1. Capture

You tap Record. The iPhone's built-in microphone — or an external mic if you've plugged one in via USB-C — feeds an AAC-encoded audio stream into the app's sandboxed Documents directory. The file lives under iOS Data Protection (NSFileProtectionComplete) from the moment of capture, and is sealed in place with AES-256-GCM the instant recording stops, keyed to a value held in the iOS Keychain. While your iPhone is locked, the file is unreadable to any process, including iOS itself.

There is no upload. There is no “auto-save to cloud.” The Record CTA never gates on a model download — capture-first is a design invariant; the audio is the source of truth, even if you're on a plane.

2. Transcription

When you tap Transcribe, Bryl runs Parakeet TDT v3 — NVIDIA's open-source ASR model, ported to Apple Silicon by the FluidAudio team — against the audio. Parakeet hits roughly 210x real-time on an iPhone 15 Pro with a Word Error Rate of about 6.4% on a 120-session talk-therapy corpus. The model file is downloaded from a public bucket on first use (you can skip this step) and then runs offline.

Apple's SFSpeechRecognizer is the always-available fallback — it ships with iOS, requires no download, and produces serviceable transcripts for shorter clips. Both run entirely on-device.

3. PII redaction

Before the transcript is shown to the language model that will draft the note, names are replaced with stable placeholders (CLIENT_1, CLIENT_2), and numeric identifiers (phone numbers, SSNs, credit card numbers, dates, emails) are masked to category-prefixed tokens (PHONE_1, EMAIL_1, DATE_1). This step is intentionally deterministic — a regex-and-NER pipeline, not an LLM — so it's auditable and consistent. After the model writes the draft, the placeholders are restored so the rendered note reads naturally.

The redaction runs on the input path to the model. The transcript shown to you in the app stays in its original form by default; an eye-toggle on the transcript pane swaps to the redacted view so you can see exactly what the model sees.

4. Drafting

The transcript goes to a local language model. On iPhone 15 Pro and newer running iOS 26, that's Apple Foundation Models — the on-device LLM Apple ships with the OS, accessed via the Foundation Models framework. On older devices, Bryl falls back to a bundled 3.8B-parameter Phi-4 model, quantized to 4-bit (Q4_K_M) and run via llama.cpp's Metal backend.

The model receives a system prompt instructing it to write in the chosen template (SOAP, DAP, BIRP, EMDR, Intake, or Progress) and the redacted transcript. It does not have internet access. It cannot have internet access. The model weights and the prompt and the response all live inside the app sandbox on the iPhone.

5. Discard

The audio survives long enough for you to review the draft, edit it, and seal it. The instant you sign — touch-and-hold for 1.5 seconds — the audio file is removed from the device. The transcript and the signed note remain (encrypted at rest); the bytes that captured the actual session are gone.

You can change this default if you want to keep the audio (some clinicians do, for forensic supervision purposes). The toggle lives in Settings → Privacy & Security and applies to every future seal. The strict default is on because the audio is the liability, not the asset.

What “encrypted in transit” actually means

A common claim from cloud scribes is “your audio is encrypted in transit and at rest.” Both halves of that claim are true and neither addresses the question that matters.

In the cloud-scribe model, your audio is briefly decrypted in memory on the vendor's server when their transcription model needs to read it. That window is short, but it exists, and during that window, the data is subject to the vendor's breach risk, the vendor's subprocessors, the vendor's legal exposure to subpoena, and the vendor's key-management hygiene.

In the Bryl model, the data is never on a server you don't own. The encryption keys are in your iPhone's Keychain. The decryption happens in the same Secure Enclave that protects your FaceID. The only entity that can read the audio is the licensed clinician who recorded it.

Those are different threat models, not different shades of the same one.

How to verify these claims yourself

We could publish a thousand whitepapers. The verification you should run yourself takes ninety seconds.

Test 1: Airplane Mode

1. Turn on Airplane Mode on your iPhone.
2. Open Bryl.
3. Record a session, transcribe it, generate a note, edit it, sign it, export it.
4. Everything works.

If anything required network access for content, it would fail in Airplane Mode. It does not. That is the proof.

Test 2: Packet inspector

For the engineering-curious clinician (or your IT person):

1. Set up a network packet inspector — Charles Proxy, Proxyman, or tcpdump on a Mac sharing internet to your iPhone.
2. Configure the iOS Charles SSL profile so the proxy can decrypt TLS.
3. Run a complete record → transcribe → generate → sign cycle in Bryl.
4. Inspect the captured traffic.

You should see exactly three categories of outbound requests:

• Apple StoreKit calls to validate your subscription. Apple-controlled, no clinical content.
• (If you opted in) Sentry crash reports, with no clinical content in the payload.
• (If you opted in) PostHog event names, with no clinical content in the payload.

You should see zero requests carrying audio, transcript, or note text. If you do see one, that's a bug, and we want to hear about it at security@bryl.app.

What we cannot do

Architectural privacy is strong precisely because it constrains what we're capable of doing. There are several features that we will not build, because they would require violating the architecture.

We cannot build a multi-device sync (e.g., your iPhone session shows up on your iPad) without the audio touching a server. iCloud Backup can be encrypted with your passcode and used to restore a replacement device — but Apple cannot decrypt that backup. Cross-device sync at the application level is a roadmap question we're still wrestling with.

We cannot build an AI tutor that watches your sessions and gives you feedback on your therapeutic style. The audio leaving your iPhone, even to a vendor you trust, is the line.

We cannot build an EHR integration that auto-pastes the signed note into SimplePractice. The note has to leave Bryl somehow — clipboard or share sheet — and the act of copying-and-pasting into your EHR is a deliberate, observable handoff that you control. An API integration that uploaded the note for you would mean Bryl had a backend that processed PHI, which is the property we are most committed to not having.

These are tradeoffs we are choosing. We think they're the right ones for the population we serve.

What this means for your compliance file

Your professional liability insurer, your licensing board's ethics committee, and (in some jurisdictions) state regulators will eventually ask you for documentation of how an AI tool you use handles client data. The architectural attestation on this site is written for that file.

If you need a signed PDF version on Roushan, Inc. letterhead — or jurisdiction-specific language for a state where the regulatory framework is unusually specific — write to legal@bryl.app and we will provide it.

The work you do in the room belongs in the room. The technical apparatus that supports the work should respect that. That is what on-device AI means here.