Running ML models in the browser with ONNX Runtime Web: a practical guide
DEV Community

Running ML models in the browser with ONNX Runtime Web: a practical guide

Running ML models in the browser with ONNX Runtime Web: a practical guide

FaceVision does face detection, recognition and liveness checks without sending a single frame to a server, because every model runs in the browser through ONNX Runtime Web. That architecture is the reason its privacy story is simple. It is also the part students find hardest to get working, so here is the path that worked.

Export, then verify

Export the model to ONNX from its training framework and immediately run the same input through both the original and the exported graph. Compare outputs numerically. Every model in FaceVision was verified against its actual ONNX graph and, where possible, the reference implementation - not assumed from documentation. Half the "the model is broken in the browser" reports I have seen were pre-processing mismatches that a five-minute comparison would have caught.

Choose the execution provider deliberately

  • WebGPU - fastest where available; check support and fall back gracefully.
  • WASM with SIMD and threads - the reliable default; needs the right headers for multi-threading.
  • WebGL - legacy; avoid for new work.

Pre- and post-processing is where bugs live

Models expect a specific channel order, normalisation and input size. Browser image data arrives as RGBA bytes in a different layout. Write the conversion once, test it against a known image, and keep the output tensors' shapes in a comment next to the code. Post-processing - decoding anchors for a detector, normalising an embedding before cosine similarity - deserves the same care.

Performance that feels live

  • Warm up the session with one dummy inference on page load; the first run is always slow.
  • Run inference in a Web Worker so the camera preview never stutters.
  • Downscale frames before detection; run recognition only on detected crops.
  • Skip frames when the queue is full rather than letting latency grow.

What the backend does when the model does not

In FaceVision the FastAPI and PostgreSQL backend stores only vector embeddings and matches them at enrolment and verification. Because no images are ever persisted, the privacy claim is easy to explain and audit. I wrote about the design reasoning in face recognition that never uploads a face.

Browser-side inference is not a gimmick

For anything involving faces, documents or health data, it is often the architecture that makes the product acceptable to the people using it.

About Pranjul Rathour

At an Integral Startup Foundation hackathon. In a packed college auditorium. Pranjul Rathour Presenting Annapurna on stage.

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG, fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at student hackathons across India.

Updated 2026-09-06. Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pra*************@gmail.com · Invite me to your campus Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

FaceVision does face detection, recognition and liveness checks without sending a single frame to a server, because every model runs in the browser through ONNX Runtime Web. That architecture is the reason its privacy story is simple. It is also the part students find hardest to get working, so here is the path that worked. Export, then verify Export the model to ONNX from its training framework and immediately run the same input through both the original and the exported graph. Compare outputs numerically. Every model in FaceVision was verified against its actual ONNX graph and, where possible, the reference implementation - not assumed from documentation. Half the "the model is broken in the browser" reports I have seen were pre-processing mismatches that a five-minute comparison would have caught. Choose the execution provider deliberately WebGPU - fastest where available; check support and fall back gracefully. WASM with SIMD and threads - the reliable default; needs the right headers for multi-threading. WebGL - legacy; avoid for new work. Pre- and post-processing is where bugs live Models expect a specific channel order, normalisation and input size. Browser image data arrives as RGBA bytes in a different layout. Write the conversion once, test it against a known image, and keep the output tensors' shapes in a comment next to the code. Post-processing - decoding anchors for a detector, normalising an embedding before cosine similarity - deserves the same care. Performance that feels live Warm up the session with one dummy inference on page load; the first run is always slow. Run inference in a Web Worker so the camera preview never stutters. Downscale frames before detection; run recognition only on detected crops. Skip frames when the queue is full rather than letting latency grow. What the backend does when the model does not In FaceVision the FastAPI and PostgreSQL backend stores only vector embeddings and matches them at enrolment and verification. Because no images are ever persisted, the privacy claim is easy to explain and audit. I wrote about the design reasoning in face recognition that never uploads a face . Browser-side inference is not a gimmick. For anything involving faces, documents or health data, it is often the architecture that makes the product acceptable to the people using it. About Pranjul Rathour At an Integral Startup Foundation hackathon In a packed college auditorium Pranjul Rathour Presenting Annapurna on stage Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG, fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at student hackathons across India. Updated 2026-09-06. Reach out if you want to talk GenAI, book a campus session, or invite him to judge: Email: pra*************@gmail.com Invite / talk menu: https://pranjulrathour.scult.in/invite Portfolio & blog: https://pranjulrathour.scult.in LinkedIn: https://www.linkedin.com/in/pranjul-rathour/ X: https://x.com/PranjulRathourx Instagram: https://www.instagram.com/pranjulrathour.in/ Bluesky: https://bsky.app/profile/pranjulrathour.bsky.social GitHub: https://github.com/Pranjulrathour Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pra*************@gmail.com · Invite me to your campus Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.