Project Overview

flutter_virtual_tryon is a general-purpose face-tracking and overlay engine for Flutter — not a “glasses package,” but a reusable computer-vision core that any overlay (glasses, sunglasses, contact lenses, or a fully custom Canvas drawing) sits on top of. It hides platform-specific computer vision behind one stable, frozen API: VisionBackend.auto() picks Google ML Kit on Android/iOS or MediaPipe Face Landmarker on web, and neither native dependency leaks into the platform that doesn’t need it.

It started as the hand-rolled camera/ML Kit/MediaPipe code inside Kalo Chasma, a production eyewear marketplace’s “Vision Mirror” try-on feature. Rather than let that logic stay app-locked, I extracted, generalized, tested, and published it as a standalone open-source package — then migrated Kalo Chasma itself onto it, so the package is validated by a real app in production, not just its own test suite.

Features Built

My Responsibilities

• API design & freeze: designed the entire public surface first — VirtualTryOn/VirtualTryOnImage widgets, VisionBackend, the overlay hierarchy, TrackingData — then froze it before implementation, so every later milestone was additive rather than breaking.
• Platform-adaptive engine: conditional Dart exports (dart.library.io vs. dart.library.js_interop) keep Google ML Kit out of web builds and MediaPipe out of native builds entirely, with unsupported platforms (macOS) degrading honestly to manual-only mode instead of crashing.
• Coordinate-space unification: normalized ML Kit’s raw-sensor coordinates and MediaPipe’s landmark space into one consistent, unmirrored TrackingData convention — including diagnosing and fixing a real iOS front-camera bug (landmarks rotated 90°, then rendering 180° upside-down) with a platform-specific relabeling fix, not a coordinate hack.
• Renderer: a shared CustomPainter pipeline (OverlaysPainter) used identically by both the live-camera widget and the still-photo widget, with pixel-space eye-anchored placement, rotation, and scale — verified with genuine pixel-capture render tests rather than golden-image comparisons.
• Still-image try-on (0.2.0): extended the engine to detect faces in a single supplied photo (detectStill) and exposed it as VirtualTryOnImage, reusing the same paint pipeline via an aspect-ratio-locked box instead of reimplementing letterbox math.
• Calibration tooling: a plain ChangeNotifier-based CalibrationController plus an OverlayCalibrator widget that lets a developer drag/pinch/twist an overlay live against the camera and copy a ready-to-paste Dart snippet — no external design tool required.
• Testing & release engineering: grew the suite from 0 to 145 unit/widget tests (83%+ line coverage, 100% on every file that doesn’t touch real camera/ML Kit hardware), then owned the full pub.dev release loop — versioning, CHANGELOG, dry-run validation, and five published releases (0.1.0 → 0.3.0).

Tech Stack

Architecture Highlights

• One frozen public API, three backends: VisionBackend.auto() resolves to whichever real engine the current platform supports; app code never branches on platform itself.
• Honest degradation, not a fake success: platforms without a detection backend (macOS) report backendUnavailable through the normal error channel instead of silently doing nothing.
• Shared paint pipeline: the live-camera widget and the still-photo widget both render through the same OverlaysPainter, so a fix or a new overlay type only has to be written once.
• Relabel, don’t reflow: the mirrored-source fixes (both the live iOS front camera and 0.3.0’s still-photo mirroredSource) correct orientation by relabeling which landmark is “left” vs. “right,” never by mutating coordinates — so a fix for rotation can never accidentally shift an overlay off the face.
• Test philosophy: unit tests never depend on a real camera or ML model; hand-built fixtures and MockVisionBackend keep the suite fast and deterministic, while real-device verification is tracked explicitly as a separate, documented step.

Release History

Outcome / Highlights

• Production-validated, not just tested: powers Kalo Chasma’s live Vision Mirror feature today — real users, real devices, not a demo-only package.
• Three platforms, one API: Android and iOS get live ML Kit tracking, web gets MediaPipe, behind a single VisionBackend.auto() call app code never has to branch on.
• Privacy by construction: all detection runs on-device or in-browser; the package never uploads a frame anywhere.
• 145 tests, analyzer-clean: shipped with a real regression suite, including pixel-level render assertions and platform-specific coordinate-math fixtures.
• Real bugs, real fixes, documented: two on-device iOS bugs (rotation, then mirrored orientation) were found, root-caused, and fixed with regression tests and a public CHANGELOG entry explaining exactly what was wrong and why the fix works.

Skills Demonstrated

• Extracting and generalizing app-specific code into a clean, reusable, published package without breaking the app it came from
• Designing a public API that stays stable across five releases by freezing it before implementation
• Unifying three different computer-vision coordinate spaces (ML Kit Android, ML Kit iOS, MediaPipe web) into one consistent convention
• Diagnosing and fixing real on-device orientation bugs with root-cause understanding, not trial-and-error patches
• Running a disciplined release loop: versioning, changelog, dry-run validation, and pub.dev publishing

Work With Me

I build and publish production-grade Flutter packages, cross-platform on-device ML features, and polished Material 3 UX.