2025 – present · Capture flow and generation quality gate
Sortak
AI passport photos that actually pass
The problem
Passport and ID photos get rejected over millimetres. Head height, eye line, background, expression, shadows — every authority publishes its own spec, and a photo that passes in one country fails in the next. The usual fix is to walk into a studio and pay for it.
Sortak lets someone take the photo on their phone and get back something print-ready. That moves the hard problem from photography into software: the app now has to guarantee a result it cannot see, because generative models do not produce the same output twice.
The failure mode that matters is not an ugly photo. It is a photo that looks fine but is no longer the person who uploaded it — a generative model quietly drifting the face until identity is lost. That gets an application rejected, and the user has no way to tell by looking.
How I approached it
Fix the capture before fixing the output
The quality of everything downstream is set at capture. I built the camera flow with a face guide overlay that positions the subject correctly before the shutter is available, so the model receives a usable frame instead of compensating for a bad one. A lot of that work was unglamorous geometry — one commit exists purely because the guide oval was anchored with a fixed padding-top and sat too high on short viewports.
Score every generated photo, deterministically
I built face-scorer, a Python FastAPI service that compares each generated photo against the user's original. Identity is an ArcFace embedding cosine similarity from InsightFace's buffalo_l model, calibrated to a 0–100 score. Geometry comes from MediaPipe FaceMesh — inter-eye distance, jaw width, lip width — normalised into ratios so they survive changes in scale and crop.
Turn a score into a correction, not just a verdict
A number that says 'this is wrong' is not useful on its own. Deviations beyond tolerance become critique strings carrying corrective prompt fragments, which feed a retry. The gate does not merely reject a bad generation — it tells the next attempt what to fix. Skin tone only critiques the warm/cool chroma axis; lightness is recorded but never critiqued, because it is dominated by ambient lighting and relighting a dim capture is intended behaviour, not a defect.
Make it deterministic and fail open
The same two images always produce the same score, which means a disputed result can be reproduced instead of argued about. The service is authenticated with a shared secret and returns a structured 422 when it cannot find a face — and the calling edge function fails open rather than blocking a user behind a service that might be down.
What it's built with
- Frontend
- React, TypeScript, Vite, Tailwind CSS, shadcn-ui, TanStack Query, React Hook Form + Zod
- Backend
- Supabase — Postgres, auth, storage, and 25 edge functions covering OTP login, signed uploads, a batch pipeline and operator tooling
- Quality gate
- Python, FastAPI, InsightFace (ArcFace), MediaPipe FaceMesh, ONNX Runtime, OpenCV — containerised and deployed separately
- Payments & infra
- Paymob card payments with webhook verification, device fingerprinting for abuse prevention, Docker, Railway
What I'd do differently
- The scoring thresholds were tuned by hand against a small set of examples. That was fast to get moving and is the part I trust least — a labelled evaluation set would have told me whether the calibration generalises, rather than whether it satisfies the faces I happened to test with.
- The quality gate arrived after generation was already working. Building it alongside would have made the retry loop part of the design instead of something fitted around an existing pipeline.
- Loading ArcFace and FaceMesh makes the service heavy to cold-start. For a request path a user waits on, I would look at keeping it warm or splitting identity scoring from geometry critique so the cheaper check can answer first.