Guide · TinyML · AI
TinyML: a neural net that never phones home
You don’t need the cloud to put a bit of intelligence in a device. TinyML runs a small neural network on the ESP32 itself — spotting a wake word, a gesture, or the sound of a leak — with the microphone data never leaving the chip. Here’s what it is, why the ESP32-S3 is the one to buy, and the pipeline from recording to running.
What TinyML is — and why keep it local
TinyML is machine-learning inference shrunk to fit a microcontroller: a model trained on a big machine, then quantised and compiled down to run in kilobytes with TensorFlow Lite Micro. Keeping it on-device buys three real things — privacy (audio/video never leaves), instant latency (no network round-trip), and it works offline on a battery. It’s the exact opposite trade-off to a cloud LLM agent: tiny and local instead of huge and remote.
The hardware reality: buy the S3
A model is multiply-accumulate all the way down, and that’s where the chip matters. The ESP32-S3 adds vector (SIMD) instructions that accelerate exactly those ops, plus the PSRAM you need to hold a model and an audio/image buffer — so it runs inference several times faster than a classic ESP32. A plain ESP32 can do simple models; anything with audio or vision wants the S3. See the chip guide, and mind where the memory goes — a model plus its buffers is your RAM budget.
The pipeline: record → train → flash
You don’t hand-write a network. The usual flow, with Edge Impulse (or the ESP-DL toolchain):
- Collect a dataset — record samples of each class (say “on”, “off”, and background) right from the device.
- Train in the browser — it picks features and fits a small model.
- Optimise — quantise to int8 so it’s small and fast, and check it still fits the RAM/latency budget.
- Flash — export an Arduino/ESP-IDF library and call it from your sketch; the output is just a class + confidence.
What to build first
Start with something small and audibly obvious — the win is motivating, and the model stays tiny:
| Project | Sensor | Difficulty |
|---|---|---|
| Wake word / keyword | microphone (I2S) | the classic first project |
| Sound event (leak, glass, alarm) | microphone | easy, high value |
| Gesture / motion | accelerometer (IMU) | easy, cheap sensor |
| Visual (person / object) | camera | harder — S3 + PSRAM only |
Rule of thumb: fewer classes and a clean signal win. A two-word detector on a microphone is a weekend; live object detection is a project. Either way, when the model fires you have a plain result to act on — flash an LED, publish over MQTT, or show it on a served dashboard.
Show what the model sees — a live dashboard from the same chip, in a few lines of C++.