GlimtGlimt turns one second of every day into a film. The interesting part is that you don't have to film that second: Glimt finds it in the videos you already took, and picks the second where your child is in the picture, even in a crowded birthday video with five other kids. All of it runs on the iPhone: no server, no account, no upload. This is how it works, with the models, the maths and real timings.
The pipeline
Why not just use Apple's Vision?
Vision does a lot here: it finds faces (VNDetectFaceCaptureQualityRequest, which also rates how sharp
and frontal each face is), their landmarks (VNDetectFaceLandmarksRequest) and, on iOS 18, how good a frame
looks (CalculateImageAestheticsScoresRequest, which also flags receipts and screenshots). What it doesn't
offer is who a face is. The People album in Photos knows, but that index isn't open to apps.
The first version of Glimt used VNGenerateImageFeaturePrintRequest on a crop of the face. It's a
general image fingerprint: it compares hair, light, colours and background as much as the face itself. In practice it
happily matched siblings, and any child photographed in the same room at the same time of day. It was the wrong tool.
The face-recognition model
Glimt now carries its own: SFace (Zhong et al., 2021, "Sigmoid-Constrained Hypersphere Loss for Robust Face Recognition"), a MobileFaceNet backbone, published in the OpenCV Zoo under the Apache 2.0 licence. It reports 99.40 % on LFW. It takes a 112 × 112 face and returns 128 numbers, an embedding; two photos of the same person give embeddings that point the same way.
Conversion was ONNX → PyTorch (onnx2torch) → Core ML (coremltools, ML Program, float16),
then 8-bit linear weight quantization. The result is 9.8 MB, loads in about 0.5 s the first time, and its output
matches the original ONNX model at cosine 0.998 on a test input. The input normalisation is inside the graph, so the
app hands it plain 0–255 RGB pixels.
Alignment matters more than you'd think
SFace was trained on faces aligned to a fixed template: eyes, nose tip and mouth corners at known spots in the 112 × 112 square. A face that is tilted or off-centre gives a noticeably worse embedding. Vision's landmarks give the five points (pupils, the nose region's centre, the outermost lip points); Glimt then finds the rotation, uniform scale and shift that best moves them onto the template. In two dimensions that least-squares problem has a closed form if you treat points as complex numbers:
a = Σ conj(zᵢ − z̄) · (wᵢ − w̄) / Σ |zᵢ − z̄|² // rotation + scale, one complex number
w = a · z + (w̄ − a · z̄) // the transform for any point zCore Image applies it (one affine transform, software renderer, a 112-pixel output), and the crop goes to the model. Reference photos are embedded twice, as-is and mirrored, and the two are averaged: a slightly steadier reference for free.
Is it the same child? Calibrating the threshold
The app compares each face's embedding with the reference photos by cosine similarity and keeps the best match. Where to draw the line was measured, not guessed, on public-domain White House portraits (official portraits years apart, and a family portrait with both parents and two daughters):
The same man in portraits taken four years apart scores 0.83; the same people picked out of the family photo 0.70–0.85. Strangers stay under 0.25. The hard case is family: two sisters in the same shot score 0.43 against each other, mother and daughters up to 0.41 (same light and pose make relatives look more alike), while mother against her daughters in a different photo drops to 0.26–0.30. So the line sits at 0.45: above the sisters, well below the same person. Five photos is a sanity check, not an evaluation, and the line will be tuned on real family videos.
Two more rules keep look-alikes out: in a video, one lucky frame isn't enough (it takes two frames at 0.45, or one clearly above at 0.57), and when several faces are in a frame, only the best-matching one is considered.
Several reference photos
One reference photo is one angle, one light and one age. Children change fast. Glimt lets you add up to five photos, and the face in a frame only has to resemble one of them, so a profile shot or last winter's face can still be recognised. When a photo has several people, you tap the right face; Glimt never guesses "the biggest one".
Picking the second, and framing it
Each frame gets a score: 35 % how good it looks, 20 % sharpness (variance of a Laplacian on a 96 × 96 grey copy: blurry frames change slowly from pixel to pixel), 45 % whether there's a clear face. With a reference set, that whole score counts 40 % and "is it the child" 60 %: an ordinary frame of your child beats a lovely frame of someone else. The chosen second is the window with the best average, so one good frame next to a blur doesn't win.
Framing for a portrait film had its own bug worth sharing. Each frame named the clearest face, and the app averaged the positions across the second. With two kids, the clearest face alternated between them, and the average landed on the wall in the middle. Now each frame names the child (or the most prominent face: size × quality, which is steadier), and the app takes the biggest group of positions that lie close together, not the mean. When the face is small, the crop zooms in until it is about a fifth of the frame's height, at most 1.6× for video.
How fast is it?
The surprise: recognising a face is the cheap part, about 2 ms on the Neural Engine. Decoding the video frame costs fifteen times more. In practice Glimt goes through a busy day of videos in a few seconds, and a month in a few minutes.
Running in the background
Filling a year of missing days takes a while, so it should keep going when you leave the app. iOS 26's
BGContinuedProcessingTask is made for that: iOS shows the progress in the Dynamic Island and on the Lock
Screen. Three things we learned the hard way, all from a small on-device log:
- No GPU in the background. Vision and Core ML pick the GPU by themselves, fail quietly when the app isn't in front, and every frame looks empty. GPU access needs a special entitlement and supported hardware; the iPhone 15 Pro reported none.
- But not everything has a CPU fallback. Forcing Vision onto the CPU broke face detection: its face-quality network ships without CPU weights on the iPhone ("Missing weights path … nonane"). Glimt now asks for the Neural Engine first, then the CPU, then falls back to plain face boxes.
- Progress has to keep moving. One day with a long video from iCloud made no progress for half a minute, and iOS ended the task. Glimt now reports progress after every analysed frame and every download step, and asks Photos for a medium-quality version of each video for analysis (films still use the original).
Privacy
Glimt has no network code. Faces, embeddings and choices stay in the app's own storage on the iPhone. The only thing that can touch the network is Photos itself, fetching your own videos from your own iCloud Photos when they aren't on the phone.
Credits: SFace by Yaoyao Zhong and colleagues; ONNX conversion by Chengrui Wang; OpenCV Zoo (Apache 2.0). Questions or ideas: glimt@theill.com.