Charlie Gerard’s 2019 prototype let a player trigger attacks in a browser-based Street Fighter-style game by moving an Arduino-mounted motion sensor. An MPU6050 accelerometer/gyroscope captured six channels of movement data; TensorFlow.js classified the resulting pattern as punch, hadoken, or uppercut; Node.js, Johnny-Five and WebSockets then turned that label into a game-control command. It is best understood as an educational IMU gesture-recognition project, not a plug-and-play controller for Street Fighter II, Street Fighter 6, arcade hardware or consoles.
The original tutorial was published on September 1, 2019. Its architecture remains useful, but its board libraries, browser APIs and package versions should be revalidated before attempting a 2026 build.
The complete control loop
The system separates sensing, machine learning and game input:
Body gesture ↓ Accelerometer + gyroscope ↓ Arduino or phone ↓ Node.js/browser data stream ↓ TensorFlow.js classifier ↓ Gesture label ↓ Keyboard or WebSocket game command
The game itself does not learn to fight. This is supervised classification: examples are recorded with labels, and a model learns to distinguish their sensor patterns. The game remains a conventional browser game receiving input events. The technical source is Charlie Gerard’s original tutorial, with additional description in Hackster’s coverage.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
#1 Best Overall
- Compatible with Switch and Windows.
- Supports ultimate software - customize button mapping and create macros
- Dynamic button layout - button mapping changes with mode functionally and visually
- Ultra-moddable arcade stick - Universal mounting plate supports Sanwa arcade sticks
- Three ways to connect: Bluetooth, 2.4g and wired USB-C. The included 2.4g receiver has a special hidden compartment to store it safely.
What “body movements” means here
The prototype does not use a webcam or skeletal pose estimation. It detects movement through an MPU6050 attached to, or held by, the player. The sensor measures linear acceleration and rotation, so “body movement” is an accessible description of wearable or handheld IMU gesture control rather than full-body tracking.
The original 2019 hardware and software
Hardware
- Arduino MKR1000
- MPU6050 accelerometer/gyroscope
- Push button
- Jumper wires
- Battery
- Breadboard or protoboard
The MPU6050 provides six numerical channels: acceleration X, Y and Z, plus gyroscope X, Y and Z. The button marks the recording window: hold it while performing a gesture and release it when the example ends. The MKR1000’s network connectivity enabled wireless communication; the tutorial notes that an Arduino Uno could be used with a tethered connection.
Software
- Vanilla JavaScript
- TensorFlow.js
- Node.js
- Johnny-Five
- WebSockets
- A browser game able to receive control commands
The original code and its dependencies date from 2019. Do not assume that its exact installation steps or sensor APIs work unchanged with current Arduino cores, Johnny-Five releases or TensorFlow.js packages.
How gesture data becomes a model
1. Record labeled examples
Several examples of each demonstrated class are collected and saved with labels such as punch, hadoken and uppercut. Every row contains six sensor readings. The classifier can only learn the variation represented in those examples: a model trained on one person’s grip, orientation and speed may fail for another person.
2. Build a fixed feature vector
The tutorial keeps a fixed-length window. Its example retains 50 readings; with six channels per reading, each input contains 300 numbers:
Rank #2
- FIGHT LIKE A PRO: Nacon Daija arcade stick officially licensed for PlayStation PS4|PS5 and PC was developed in collaboration with professional gamers and designed for top fighting games like Marvel Tokon Fighting Souls, Street Fighter, Tekken, Guilty Gear Strive, and Mortal Kombat.
- SANWA JOYSTICK & ACTION BUTTONS: A leading brand among professional fighting game players and a guarantee of premium quality and durability
- FULL BUTTON MAPPING: Buttons are fully configurable using the Nacon Daija Arcade Stick software on PC and Mac; you can easily create game profiles with the intuitive and user-friendly software
- COMFORT & PRECISION: Textured palm rest provides sturdy and comfortable support so you can remain at the top of your game for as long as possible; switch between two different joystick heads to choose the best solution for perfect control and precision
- LOCK FUNCTION: Enable the button lock feature to prevent accidental presses and avoid interrupting intense fights
50 readings × 6 channels = 300 features
Text files are converted into JavaScript objects, then arrays of features and labels. This design is easy to implement but assumes gestures fit the same number of samples. A movement that is much faster or slower can be truncated, padded or misclassified.
3. Encode labels
String labels become integer class IDs and then one-hot vectors. The numeric assignment depends on the class order in the code—for example, one run might map punch to 0, hadoken to 1 and uppercut to 2.
4. Split the dataset
The tutorial uses approximately 80% of examples for training and 20% for validation/testing. That is a useful teaching split, not a cross-player accuracy benchmark. If nearly identical windows from one physical gesture appear in both portions, validation can look better than live performance.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteThe sample TensorFlow.js network
The example is a small dense classifier:
const params = {
learningRate: 0.1,
epochs: 40
};
const model = tf.sequential();
model.add(tf.layers.dense({
units: 10,
activation: 'sigmoid',
inputShape: [trainingFeatures.shape[1]]
}));
model.add(tf.layers.dense({
units: 3,
activation: 'softmax'
}));
const optimizer = tf.train.adam(params.learningRate);
model.compile({
optimizer,
loss: 'categoricalCrossentropy',
metrics: ['accuracy']
});The hidden layer has 10 sigmoid units and the output layer has three softmax units. Adam, a learning rate of 0.1 and 40 epochs are experimental settings from the tutorial, not modern universal defaults. The model is trained with validation data and saved for later prediction. No dependable accuracy percentage is established by the source.
Live prediction and game input
- Read accelerometer and gyroscope values.
- Buffer a new gesture window.
- Detect the end of the window, originally by button release.
- Convert the six-channel sequence to the expected tensor shape.
- Run the saved model.
- Map the winning class index to a gesture name.
- Send the corresponding keyboard or WebSocket command to the browser game.
The prediction code uses a class list such as hadoken, punch and uppercut. A highest softmax value is still only a relative winner; a robust controller should require a probability threshold or margin before dispatching an attack.
Rank #3
- Heavy-duty steel joystick with microswitch contact buttons: precise and robust
- 8 action buttons with microswitch leaf contacts: smoother, quieter and highly sensitive
- AUTO function: trigger very fast repetitive shots
- COMBO function: program and launch shots and special movements at the touch of a button
- Soft Touch" coating, rubberized wrist rest and 3-meter cable
What the prototype can—and cannot—do
- Demonstrated: recognition of a small set of prerecorded gesture classes and control of a web game.
- Not demonstrated: every fighting-game command, complete commercial Street Fighter compatibility, autonomous game strategy or reinforcement learning.
- Hadoken qualification: the label
hadokenidentifies a learned gesture class. It does not prove that Capcom’s original special-move input logic is being emulated.
Think of the project as a sensor-to-command bridge. Adding more attacks is possible in principle, but each new class needs representative labeled data and a reliable mapping to an input event.
Recreating it in 2026
Choose a target before buying parts
Start with a browser game or a local test page that accepts keyboard events. This isolates sensing and classification from game-specific integration.
Current integrated-IMU option
The Arduino Nano 33 BLE Sense Rev2 has an nRF52840 microcontroller, operates at 3.3 V and includes a BMI270 accelerometer/gyroscope plus a BMM150 magnetometer. Arduino lists a U.S. store price of $39.70 with headers when checked in August 2026. It is a development board, not a finished controller. Its IMU differs from the MPU6050, so the original sensor code is not drop-in compatible; consult the official datasheet.
Lower-cost integrated-IMU option
The Arduino Nano 33 BLE Rev2 is listed at $23.10 on the U.S. store (August 2026). It includes an IMU and Bluetooth Low Energy but lacks the Sense version’s broader microphone, environmental and other sensors. It can be sufficient for inertial gesture recognition.
Historical reproduction
For fidelity to the original, use the MKR1000 and external MPU6050 described in the primary tutorial. Current availability, pricing and compatibility are not established, so this is a preservation route rather than the default 2026 recommendation.
Rank #4
- STREET FIGHTER 6 JURI EDITION: Officially licensed by Sony, this Qanba Sapphire S1 leverless arcade controller features exclusive Street Fighter 6 Juri artwork for a bold, tournament-ready look.
- 18 ADJUSTABLE THROW DISTANCE BUTTONS: Equipped with 18 Qanba adjustable throw distance buttons compatible with Cherry MX switch series, allowing buttons and micro-switches to be installed and replaced to match your personal preferences.
- CUSTOMIZABLE BUTTON MAPPING: Features a button map profile switch that lets you remap buttons and save multiple configuration files, giving you a competitive edge tailored to your playstyle.
- COMFORT AND PORTABILITY: Includes a detachable memory foam wrist rest for extended play sessions, a detachable shoulder strap for easy transport, and a built-in storage compartment for accessories.
- BROAD COMPATIBILITY AND CONNECTIVITY: Compatible with PlayStation 5, PlayStation 4, and PC, and includes a 3.5mm headphone jack for audio convenience during intense gaming sessions.
Phone route
The author also describes using a phone’s accelerometer and gyroscope through the Generic Sensor API. This removes dedicated hardware, but browser sensor permissions, HTTPS, device support and orientation handling must be tested. It is inexpensive for an existing compatible phone, not necessarily reproducible or rugged.
A practical build sequence
- Define orientation: document which physical directions correspond to positive and negative X, Y and Z.
- Make a recorder: save timestamps, six channels and a gesture label.
- Collect varied samples: include slow, fast, weak, strong, left-handed and right-handed performances.
- Normalize values: account for scale and mounting orientation where appropriate.
- Add a neutral class: ordinary movement should have a “no gesture” outcome.
- Evaluate by session: reserve a complete later session, or another performer, for testing.
- Gate confidence: dispatch only above a chosen probability or margin.
- Debounce: add a cooldown or state machine so one gesture cannot fire repeatedly.
- Separate mapping: keep gesture labels independent from game-specific keys.
- Measure latency: time the interval from gesture completion to command dispatch.
- Keep recovery controls: retain a keyboard or physical stop button.
Common failure modes
False positives
Walking, turning, adjusting the mount or returning to neutral can resemble an attack. Neutral data, confidence gating and cooldown logic reduce accidental commands.
Orientation changes
Rotating the sensor changes every channel pattern. Use a consistent mount and document it during recording and play.
Timing mismatch
A fixed 50-reading window is sensitive to gesture duration. Consider resampling, sliding windows or a temporal model when timing variation is large.
Class imbalance and leakage
Unequal examples can bias the dominant class. Randomly splitting near-duplicate windows can also inflate validation results; hold out sessions or performers instead.
Recommended Free Tools
Best Value
- Compatible with Switch/Switch 2, PS4, PS3, Xbox Series S/X, Xbox One, Xbox 360, PC Windows, macOS, Steam Deck, Android.
- Compatible with Sanwa Denshi buttons and joystick that is perfect for customization
- Supports the XINPUT and DINPUT when being used on PC
- Supports the Turbo functions with different speed settings.
- F300 is compatible with PS5 by MAYFLASH Magic-S Ultimate adapter or MagicBoots-S5 Converter.
Repeated attacks
Continuous prediction may emit multiple events from one movement. Cooldowns, onset/offset detection and explicit state transitions are essential.
Connectivity and browser issues
A working classifier can still fail at the final step because of WebSocket disconnects, blocked keyboard events, browser security rules or stale Node.js packages. Test the game-control channel independently before debugging the model.
IMU, webcam and model choices
| Choice | Strengths | Costs and limits |
|---|---|---|
| IMU | Works in low light, uses little data, can be worn and directly measures motion | Needs mounting, is orientation-sensitive and does not observe the whole body |
| Webcam pose tracking | Can observe body posture without a wearable sensor | Needs suitable lighting, camera placement and a separate pose-estimation pipeline |
| Fixed-window dense model | Simple tensors and straightforward TensorFlow.js implementation | Assumes similar gesture duration and can be brittle at window boundaries |
| Temporal model or dynamic time warping | Better tolerance for timing variation | More data, tuning and implementation complexity |
The original performs inference in the Node.js/TensorFlow.js workflow; it does not establish that the MKR1000 runs the neural network. A modern project can keep inference on the computer or investigate TinyML deployment on a suitable board, but that is a separate engineering effort.
Accessibility and safety
Gesture input can offer an alternative to a conventional controller, but large or forceful punches are not automatically accessible. Support adjustable gesture sizes, remapping, low-effort movements and a non-motion input. Use controlled movements, a clear play area, a secure mount and an immediate pause/reset control to reduce wrist, shoulder and surrounding-object risks.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesFinal assessment
This project is an excellent demonstration of supervised gesture classification: collect IMU examples, train a small TensorFlow.js model and convert its labels into browser-game events. Its educational value is still strong in 2026, especially when rebuilt with session-based testing, neutral detection, confidence gating and modern hardware. It should not be sold as a supported commercial Street Fighter controller or assumed to run unchanged from the 2019 code.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
