TensorFlow Lite Micro on Arduino: Filling the Gap Until There Is an Official Package Again

If you want to run a TensorFlow Lite model on an Arduino today, you quickly find that there is no obvious answer. There used to be one: Google’s official Arduino package, Arduino_TensorFlowLite, built from tflite-micro-arduino-examples. That package has not been updated for a long time, and I think it is worth looking at how we got here and what you can use in the meantime. What happened to the official package The official package was Read more

By pschatzmann, ago

TinyModem: Sending Data Through Sound with an ESP32 or a Raspberry Pi Pico

Remember the screeching of the dial-up modem? Turning data into sound is still one of the simplest ways to connect devices: you can use an audio cable, send it from a speaker to a microphone, put it on a telephone line or a radio, or just store it in a WAV file. No Wi-Fi, no pairing, no extra hardware. TinyModem is a header-only C++ library of software audio modems for Arduino and the desktop. It Read more

By pschatzmann, ago

TinySTT: Speech-to-Text on a Microcontroller – a Proof of Concept

If you look for speech recognition examples for microcontrollers, you almost always end up with wake word detection or keyword spotting: the micro_speech example of TensorFlow Lite Micro, which recognizes “yes” and “no”; Espressif’s ESP-SR with WakeNet for the wake word and MultiNet for a list of commands; keyword spotting models trained with Edge Impulse; or Picovoice’s Porcupine and Rhino. They work well for what they do, but they all classify a small, fixed set Read more

By pschatzmann, ago

Arduino Audio Tools: Streaming Video and Audio via HTTP

If you have ever streamed video from an ESP32, you have most likely used the CameraWebServer example: the camera delivers JPEG images, which the web server pushes one after the other to the browser. This works, but there is no sound, the bandwidth is high, and the result is not a real video stream that you could record or play in VLC. A real stream needs a video codec such as H.264, an audio codec Read more

By pschatzmann, ago

The Cheapest Way to Run a Neural TTS

Classic embedded text-to-speech (formant synthesizers like SAM or eSpeak) is tiny and fast, but it sounds like a robot from the 80s. Neural TTS sounds natural, but it is usually something you run on a PC or in the cloud. I wanted to know: what is the cheapest hardware that can run a neural text-to-speech model locally, faster than real time? To find out I ported tiny-tts to C++ as TinyTTS. It is a VITS-style, Read more

By pschatzmann, ago

Neural TTS on a Microcontroller ?

To produce the most natural-sounding speech, modern Text-To-Speach (TTS) systems rely on neural networks. TinyTTS is a proof of concept: It asks whether a VITS-style neural TTS model can run on a microcontroller at all. It is a header-only C++ port of tiny-tts. The model has about 1.6M parameters. It has four stages: text encoder, flow, duration predictor and a HiFi-GAN-style vocoder. There is no inference runtime such as TFLite Micro. Everything is hand-written C++ Read more

By pschatzmann, ago

AudioTools: The VideoPlayer

A follow-up to AudioTools video support, which covers container/codec selection (AVI vs. MP4 vs. MPG, MJPEG vs. H.264). This post is the other half: the pipeline that actually plays a file, a minimal sketch, and every class involved. Why this exists On a desktop, “play this video” hands the problem to an OS: a media framework demuxes the container, decodes on a scheduler-managed thread, and a compositor times frames to the display’s refresh. An ESP32 Read more

By pschatzmann, ago

AudioTools: Supporting Videos

I added video support to my AudioTools library: Different container formats can be used to store video on audio together in the same file. Here is a quick overivew of the provided functionality that is relevant for Microcontrollers: Container Formats The simplest and therefore most recommended container format is AVI (Audio Video Interleave). It is simple to both encode and decode and you can easily mix audio and video together. It supports both different audio Read more

By pschatzmann, ago

Arduino LCD TFT Boards and Material Design

Material Design is an open-source design system created by Google in 2014 to help developers and designers build consistent, high-quality digital interfaces across Android, iOS, the web, and other platforms. I did not find any existing Arduino Library which would allow me to easily implement some Material Design Applications on Arduino, so I decided to implement my own TinyMaterialDesign library. I also added a comprehensive tutorial which explains the library in detail. Further information can Read more

By pschatzmann, ago