Server-Side Face Attendance: What InsightFace and Anti-Spoofing Taught Me

The anti-spoofing model downloaded cleanly. The checksum matched. The tensor shapes were right, inference ran without a single error, and it was completely blind. Fed a real face, a black square or random noise, it answered with almost exactly the same probabilities. Wired to the obvious rule, "reject the punch if it isn't live", it would have refused every employee in the company.

That was one of several moments building face attendance for BoltHR, the multi-tenant HR platform I work on, where the documentation said one thing and the model did another. This is what I learned putting face recognition, framing checks and passive anti-spoofing on a Django server, and why almost every threshold in that code was measured rather than chosen.

The shape of the system

Employees clock in with the mobile app (React Native + Expo) or a shared kiosk. The phone captures the face and the server does all the deciding: detection, recognition, framing, liveness and spoof checks. Nothing that judges who you are runs on the client. That costs upload bandwidth and model memory on the API, and I accept it: the result decides who gets paid for which hours.

The recognition core is InsightFace's buffalo_l pack running on ONNX Runtime, CPU only:

FaceAnalysis(
    name="buffalo_l",
    providers=["CPUExecutionProvider"],
    allowed_modules=["detection", "recognition"],
).prepare(ctx_id=0, det_size=(320, 320))

Every punch produces an embedding, which is compared by cosine similarity against the employee's enrolled reference faces. The match threshold sits at 0.4. Every attempt, successful or not, goes into an attendance log, because a disputed punch is a payroll argument and you want the evidence.

Lesson 1: the model pack you loaded is not the model pack you think

Notice allowed_modules above. Detection and recognition only. That keeps memory and startup down, and it also means face.pose is always None. Any check written as if abs(face.pose[1]) > 30 doesn't fail. It quietly never runs.

So orientation comes from the five RetinaFace keypoints instead (two eyes, nose, two mouth corners), expressed as ratios rather than degrees. Yaw is the nose's offset projected onto the eye-to-eye axis. It has a real zero, because when you face the camera your nose sits between your eyes. I measured about 0.05 looking straight ahead and about −0.47 turned away. Those two are far enough apart to use.

Pitch was the opposite. Three people holding their heads level measured 0.714, 1.004 and 1.319, an 85% spread, while one person's deliberate tilts moved the ratio by ×0.39 and ×2.71. A long-nosed face with its chin up reads lower than another face held level. No constant separates those cases. So pitch is only ever compared against a baseline from the same person in the same session, which is why the enrolment flow captured the head turns first: yaw can be judged cold and anchors the session, and pitch can't.

Lesson 2: some liveness ideas die on contact with the data

My first idea for passive liveness was landmark motion. Over a short burst, a real face should move in ways a photo can't. In practice it doesn't work, and I measured why. A printed photo moved and rescaled in front of the camera drifted 0.000–0.020. A real face changing expression drifted 0.003–0.021. The ranges overlap completely, because a 10% rescale alone produces more apparent drift than the signal you're looking for.

I took it out. I also left a test behind called test_a_moved_photo_is_NOT_caught, so anyone who reinvents the idea has to delete an explanation first.

What did work was a challenge: ask for a head turn and check that the yaw actually moved. That gap is two orders of magnitude, not a rounding error. But it costs the user effort, and UAT showed that staff found a multi-pose enrolment too fiddly. So the mobile app now sends one high-quality enrolment photo and skips the challenge. That trade-off is written down where people will see it: what's left at enrolment is the approving HR person and the supervisor who took the photo. The server still runs the challenge on any multi-frame enrolment, so a burst of identical stills is refused.

Lesson 3: the model card can be wrong, so test the model

Passive anti-spoofing uses MiniFASNet (V2 plus V1SE, softmax summed the way upstream does it) over the same ONNX Runtime. The published card says to scale input to /255. Here's what I measured, as the difference in output between a real face and noise:

BGR / 255          0.0024   <- what the card says; degenerate
BGR raw 0-255      0.6331   <- correct
BGR (x-127.5)/128  0.0115
RGB / 255          0.0027

With /255 the network saturates and answers the same for everything. I wrote off three separate model conversions as "broken exports" before I found that they were all being fed the same wrong preprocessing. The class index was wrong on the card too: class 1 is real, which matches the original upstream repo and not the hosted model card.

Two things came out of that:

• •A vet_face_spoof management command that proves the model's answer changes between a genuine face and a spoof-like one. The negative sample has to be face-shaped. Random noise is out of distribution, and a perfectly working model scores it 0.95 "real".
• •Pinned SHA-256 checksums in an install_face_spoof command that the deploy runs. It's idempotent, so it only downloads on a new host, and non-fatal, so a hiccup at the download source can't break a deploy.

Lesson 4: the deploy can delete your model without telling you

The deploy runs git clean -fd. That deletes untracked files and spares ignored ones. At first the model weights were only untracked, so every deploy wiped them. The only symptom was that anti-spoofing switched itself off, because I'd made missing models mean "check is off" rather than "refuse everyone". Nothing errored. Punches just stopped being checked.

Putting models/ in .gitignore fixed it. A comment next to it says it's there on purpose, so nobody tidies it away.

The other operational trap was loading. InsightFace and an ONNX embedding model used to load at import time in every process. A manage.py command that exited while the native load was still running killed the thread inside ONNX Runtime's C++: "terminate called without an active exception", exit code 134, and every scheduled runner logged a failure. Now the heavy models warm up on a background thread only in web processes, gated by an environment flag set in the WSGI entry point. Everywhere else they load on first use.

Lesson 5: the reference face is a safety-critical record

Recognition is only as good as what you compare against, so enrolment ended up with more rules than punching:

• •An enrolment photo isn't stored where punches match until it's approved. The live face table is matched against every punch, so a row there is already deciding who is at work. There is no "pending" state to review. Requests live in their own model and become live rows only on approval. I refused to add a status column to the live table: one missed filter would put an unreviewed face into production matching.
• •The duplicate check matters most. If employee A's photo is filed under employee B, nothing fails loudly. The kiosk just records B's attendance whenever A punches, and both people's pay is wrong. Enrolment reuses the same threshold the kiosk uses, and if the face already belongs to a different person the server returns a 409.
• •Enrolment quality is stricter than a punch. A bad punch photo fails once and you retry. A bad reference degrades every future match for that person. Blur (variance of the Laplacian) and exposure are measured on the face crop, not the whole frame, because a sharp background behind a blurred face passes a whole-image check.
• •The approver sees the face, rendered as an image in the approval task, not a "View attachment" link. You can't verify a face from a filename.

Rolling it out without breaking old builds

Liveness enforcement is a per-organisation flag. It defaults to off and fails closed when it's on. The reason is the app store: old builds send a single frame. When an OTA release turned out to drop frames from its burst, refusing short bursts as "not enough frames" would have blocked every punch from that build. So with the flag off, a burst too short to judge is treated as a single frame. A burst punch produces a byte-identical embedding to the old path (cosine 1.000000), so existing enrolments were never touched.

The oval guide drawn on the phone and the oval the server checks are the same numbers in two codebases. I treat them as a contract: change one side and the app draws a guide that the server correctly rejects.

Takeaways

• •Measure every threshold on your real detector. Several of my numbers came out opposite to what the documentation claimed.
• •Write a vetting command for every model you ship. "Inference runs" doesn't prove the model can tell anything apart.
• •Decide what "model missing" means, then make sure you'd notice it. Fail-open was right for a deploy hiccup and wrong for silently losing the weights.
• •Keep the reference data out of the live path until a human has looked at it. The costly mistakes in face attendance happen at enrolment, not at the punch.
• •Remove an idea that doesn't work, and leave a note saying why. That note saves the next person from trying it again.