How Gesture-Driven Effects Work

Every Playground effect is a small function that runs once per video frame. Here is what happens behind the scenes. 1. Detect: MediaPipe finds up to two hands (21 landmarks each) and names the gesture, such as Open Palm, Closed Fist, Thumb Up or Victory. Face landmarks and the background mask load only when an effect needs them. 2. Measure: we turn landmarks into simple signals, such as which fingers are up, how fast the palm is moving, or how far apart your thumb and index finger are. 3. Decide: each effect looks for a trigger. A pinch below a distance threshold, a palm held open, a fast flick, a smile or a blink. 4. Draw: the effect paints particles, glows and shapes on a canvas laid exactly over the video. A few tricks keep effects feeling good: - Hold-to-confirm: Spell Mirror waits for a gesture to be held for half a second, so poses you pass through on the way do not trigger it - Hysteresis: Finger Gun needs the thumb to drop clearly before it fires, so small wobbles do not misfire - Cooldowns: a short pause after each shot or clap stops one motion from firing twice All of this happens locally in your browser at webcam frame rate.