ElatoAI -- Secure websockets with bidirectional audio and Opus audio compression for AI speech-to-speech projects

[akdeb]

Hi everybody, I launched a repo that helps you run Realtime AI speech models on an ESP32S3 on Arduino and Edge functions.

👾 www.github.com/akdeb/ElatoAI

This time last year I didn't know what an ESP32 was. I had played with popular microcontrollers like Arduino and Raspberry Pi but the ESP32 was a new learning experience for me. After spending many hours on this amazing repo by @pschatzmann and links2004's websockets repo and several iterations, I created a version that works well for production grade scenarios.

It uses the following libraries:

    bblanchon/ArduinoJson@^7.1.0 
    links2004/WebSockets@^2.4.1 
    ESP32Async/ESPAsyncWebServer@^3.7.6 
    https://github.com/esp-arduino-libs/ESP32_Button.git#v0.0.1 
    https://github.com/pschatzmann/arduino-audio-tools.git#v1.0.1
    https://github.com/pschatzmann/arduino-libopus.git#a1.1.0

Repos linked:

• https://github.com/bblanchon/ArduinoJson,
• https://github.com/Links2004/arduinoWebSockets,
• https://github.com/ESP32Async/ESPAsyncWebServer,
• https://github.com/esp-arduino-libs/ESP32_Button,
• https://github.com/pschatzmann/arduino-libopus

The really cool thing is the circuit setup is fairly simple

Hardware options

And if you are looking for a circuit board that provides all of this in one PCB, I also started selling a device and a devkit here:
Device: https://www.elatoai.com/products
AI Devkit: https://www.elatoai.com/products/ai-devkit

I would love for you to check out how we are using arduino-audio-tools in our repo to bring AI speech models closer to edge devices like the ESP32. Good luck on your projects and please let me know if you have any questions / thoughts.