Two paths for the same task: the photo leaves the device, or it never does.
For most of the last decade, building anything that analysed a human face followed one shape. The user uploads a photo. The photo travels to your infrastructure. A GPU somewhere runs inference. A JSON result comes back. Somewhere in that chain, a copy of someone’s face is sitting on a disk you are now responsible for.
That architecture was never chosen because it was good. It was chosen because it was the only thing that worked. Models were too large, browsers too slow, and WebAssembly too immature to do anything useful with a video stream at thirty frames per second.
That constraint is gone. And as of this month, the cost of ignoring it went up.
The regulatory floor moved on 2 August 2026
On 2 August 2026, the EU AI Act’s transparency obligations came into effect. Among them: providers of emotion recognition and biometric categorisation systems must now inform the people exposed to them. Systems already on the market before that date have until 2 December 2026 to comply.
The Digital Omnibus on AI, published in the Official Journal in July 2026, pushed several high-risk deadlines out to December 2027. A lot of teams have read that as blanket relief. It is not. The transparency obligations were explicitly left on the original schedule, and they are the ones that touch consumer-facing facial analysis.
This sits on top of an existing patchwork. Under GDPR Article 9, biometric data processed for the purpose of uniquely identifying a person is a special category requiring an explicit lawful basis. In the United States, Illinois’ Biometric Information Privacy Act attaches statutory damages to a scan of face geometry collected without prior written consent, and unlike its Texas and Washington counterparts it carries a private right of action, which is why it has generated most of the litigation in this space.
None of this makes facial analysis illegal. It makes storing it expensive. Which raises the obvious engineering question: why are you storing it?
What the browser can actually do now
Google’s MediaPipe Face Landmarker, running in the browser via WebAssembly, outputs 478 three-dimensional face landmarks per detected face, alongside 52 blendshape coefficients representing facial expression and a transformation matrix for effects rendering. It runs against a static image or a live camera stream. It ships as a task bundle loaded from a CDN, with a GPU delegate available where WebGL support allows.
Four hundred and seventy-eight points is a dense mesh. It is more than enough geometric signal to compute facial ratios, symmetry deviations, proportional relationships, and landmark-anchored overlays for virtual try-on. For a meaningful category of consumer applications, the ceiling is no longer the model. It is what you choose to do with the coordinates.
The critical property here is architectural rather than technical: the pixels never leave the device. There is no upload endpoint. There is no object storage bucket. There is no retention policy to write, no breach surface to defend, and no subject access request to service, because there is no subject data. Your compliance posture becomes a statement of fact about the code rather than a promise about your operational discipline.
I want to be precise about a nuance that gets glossed over. Running inference client-side does not automatically place you outside data protection law. If your application initiates the processing, you may still be a controller in the legal sense even when the computation happens on the user’s hardware, and transparency obligations attach to what your system does, not only to what it keeps. On-device processing dramatically shrinks the risk surface. It does not vaporise it. Anyone treating “it’s client-side” as a complete legal answer should get that reviewed by counsel rather than by a blog post.
Why a face shape detector is a good test case
I run a browser-based face shape detector that performs all of its landmark extraction and geometric scoring on the client, with no server-side image handling anywhere in the pipeline. It is a useful case study precisely because it is unglamorous. The task is bounded, the output is deterministic given the same landmarks, and the failure modes are visible to the user immediately.
It is also a category where the privacy argument is not abstract. People submitting a photo of their own face to a styling tool have a reasonable and specific concern about where that image ends up, and most consumer tools in this space either upload to a server or are vague enough that a user cannot tell. A face shape detector that runs entirely in the browser can make a claim its competitors cannot, and it can make it verifiably.
That verifiability matters more than the claim. “We take your privacy seriously” is not a technical statement and nobody should accept it as one. A better one: if the analysis genuinely runs in the browser, the user can disconnect their network after page load and watch it still work. I publish a breakdown of the three models — client-side, server-side with deletion, server-side with retention — because the distinction is the one users are least equipped to evaluate and most affected by.
What it actually costs to build
The architecture is superior. It is not free. Four things cost more than the server version, and I would rather state them than pretend the trade is one-sided.
Cold start is your real latency budget
The model bundle has to download and initialise before the first inference. On a fast connection this is unremarkable. On a mid-tier Android device on a congested mobile network — which, depending on your market, may be the modal user — it is the single largest contributor to perceived slowness. Server inference amortises this cost across every user; on-device inference charges it to each one individually. Caching, preloading on intent signals rather than on page load, and a genuinely informative loading state are not polish. They are the product.
On-device inference charges the model download to every user individually.
The main thread is the enemy
Naive implementations run landmark detection on the main thread and destroy interaction responsiveness. The symptom surfaces in Core Web Vitals as a poor Interaction to Next Paint score, and INP has been a ranking input since it replaced First Input Delay. If your on-device AI feature tanks INP, you have traded a privacy win for a distribution loss. Move inference into a Web Worker, throttle frame processing deliberately rather than running as fast as the hardware allows, and measure on real devices rather than on your laptop.
Device variance is unbounded
With server inference you control the hardware. With on-device inference your execution environment is every phone your users own, including devices without WebGL2, browsers with restrictive WASM policies, and iOS Safari’s particular memory ceilings. You need a graceful degradation path, and you need to decide in advance whether that path is a reduced-fidelity mode or an honest “not supported on this device” message. Silently producing worse results is the worst of the three options.
Your execution environment is every phone your users own.
You lose your telemetry
This is the trade discussed least. When images never reach you, you cannot inspect failure cases. You cannot build a dataset of inputs where the model underperformed. You cannot audit for demographic skew in your own traffic, because you have no traffic to audit. Landmark models inherit the composition of their training data, and face geometry models have documented accuracy variation across ethnic groups and with occlusion, extreme pose, and non-standard lighting. Choosing on-device processing means choosing to give up the instrumentation that would let you measure that. The mitigation is anonymous, aggregated, opt-in outcome signals — a confidence score, a completion event, a user correction — never the image itself. It is a weaker feedback loop, and pretending otherwise is dishonest.
When the cloud is still the right answer
On-device is not universally correct, and its advocates usually overstate the case. Three situations argue the other way.
If your model is large — a full generative pipeline rather than a landmark extractor — the download cost dominates and the browser is the wrong venue. If your task genuinely requires identity matching against a stored gallery, the gallery has to live somewhere, and that somewhere is a server with every obligation attached. And if you operate in a regulated context that requires auditable records of what was processed and when, deliberately destroying that record creates a different compliance problem than the one you solved.
The rule of thumb I would offer: on-device wins when the task is measurement and loses when the task is matching. Extracting geometry, computing proportions, scoring left-right balance the way a face symmetry test does — all measurement, all derived from coordinates the client already holds. Determining whether this face is the same as that face — matching, and it needs a stored gallery. The first belongs on the client. The second does not.
Measurement belongs on the client. Matching does not.
A checklist for teams evaluating the switch
- Classify the task. Measurement or matching? If measurement, on-device is likely viable today.
- Estimate the bundle cost against your actual traffic profile. Model the p75 device and connection in your largest market, not your own.
- Instrument INP before and after. If it regresses, you have a distribution problem regardless of the privacy win.
- Design the degradation path explicitly. Decide what happens on unsupported hardware before you ship, not after.
- Replace image telemetry with aggregate outcome telemetry. Accept the weaker signal; do not pretend you still have the strong one.
- Write the transparency disclosure anyway. Post-August 2026, if your system performs biometric categorisation, users must be told — client-side or not.
The interesting shift is not that browsers got fast enough. It is that for a growing class of applications, the architecture that is easiest to defend legally has become the architecture that is also cheapest to operate and fastest for the user. Those three things pointing in the same direction is rare. It is worth noticing when it happens.
