Software · accessibility · computer vision in the browser
Hands-free control
A pointer you steer with your head and eyes, through the webcam already on your laptop. Rest it on something to choose it, or blink, smile or raise your eyebrows. The first thing it does is play chess, for people who can't use their hands.
Part of Work · Apps you can open
Most computers assume two hands on a mouse and a keyboard. Eye trackers and head mice exist, but they cost money and need setting up. This one needs only a webcam and a browser tab: it follows where your face points and where your eyes look, moves a ring on the screen, and treats a short rest, or a facial gesture, as a click.
Chess is the first use, because it is a game of a few big, clear choices: which piece, and which square. The same pointer can drive any page built for it.
Terms used here
- Face landmarks
- 478 points on the face (eye corners, eyelids, irises, nose, lips, outline) found in each video frame, with an estimate of how far each is from the camera.
- Blendshapes
- 52 scores from 0 to 1 for what the face is doing: how closed each eye is, how open the jaw, how raised the brows, how wide the smile.
- Head pose
- Which way the face points: turned left or right, tilted up or down.
- Dwell
- Choosing something by resting the pointer on it for a moment, here 0.8 seconds to start with.
- Calibration
- A few dots shown across the screen; you face each one, and the pointer learns how far you move to reach it.
- Hysteresis
- A gesture starts above one threshold and ends below a lower one, so a score hovering near the line counts once, not many times.
01 · How it works
- Face points. Google's MediaPipe Face Landmarker, served from this site, finds 478 points on your face in every frame, in 3D, plus the 52 blendshape scores.
- Which way the face points. Three directions are built from the rigid upper face: along the eyes, down the forehead, and straight out of the face. The pointer follows that last one. It ignores how far you sit and where you are in the picture, and the mouth and cheeks are not used, so talking or smiling doesn't push it.
- The eyes, if you want them. Where each iris sits between the corners and lids of its eye adds a correction after a 9-dot calibration, for people who move their eyes more than their head.
- Calibration. Without it, the ring starts in the middle and moves as your head turns from where it rested, with about 15 degrees to reach an edge. With 5 or 9 dots, a straight-line fit learns your own range, and the page reports how far off each dot would have been if it had been left out of the fit.
- Steady, not slow. A small dead zone keeps the ring still while your head only sways, and the One Euro filter smooths hard when you are still and lightly when you move, so it neither shakes nor lags.
- Big targets pull. Only the things worth choosing can be chosen: on the chess board, your pieces that can move, then the squares the chosen piece can reach. The ring jumps to the nearest of them and holds on until you are clearly past its edge.
- Clicking without hands. Rest on a target for 0.8 seconds (adjustable). Or choose a gesture: a long blink (held past 0.4 seconds, so ordinary blinks are ignored), opening your mouth, raising your eyebrows, or a smile. Each starts above one threshold and ends below a lower one, must be held for a moment, and can be tuned to your face in a few seconds. While a gesture is under way, the ring holds still. A different gesture, held for most of a second, takes a move back.
02 · Privacy
Nothing leaves your computer
The camera starts only when you press the button, after the page has said what it will do. The video goes from the camera to the face tracker inside the browser tab and nowhere else: no picture, face point or score is sent to this site or anyone else. The tracker's files come from this site, not from another server. Stop, press Esc or leave the tab and the camera turns off. Your settings and calibration stay in your own browser, and one button forgets them.
03 · How well it works
Tested on simulated faces; a real camera comes next
The logic is tested without a camera: a small 3D head (eye corners, lids, irises, forehead, nose) is turned, nodded, tilted, moved and shaken, and its points are fed in frame by frame, as the camera would give them. In those tests, a 5-dot calibration lands within 0.8% of the screen between the dots, and a 9-dot one within 0.6%. Turning to a chess square and resting there chooses it in about 1.3 seconds, dwell included. Normal blinks are ignored and a long one counts once.
Those numbers come from simulated faces, not people. A real webcam adds noise, uneven light, glasses and the tracker's own errors, so the accuracy on a real camera is not yet measured. I will measure it with people and put the numbers here. Until then, calibrate, use the big targets, and raise the dwell time if things get chosen too soon.
Known limits
- The calibration is a straight-line fit: near the corners of a large screen, a wide turn of the head may land a little off.
- If you move your chair, choose Re-centre or calibrate again.
- Phones work less well: the camera sits off to one side and the screen is small.
JavaScript · MediaPipe Face Landmarker (Apache 2.0) · One Euro filter (Casiez, Roussel and Vogel, 2012) · chess.js · Stockfish 19