Building a Custom AI Tamagotchi Using Meta's Open-Source Muse Framework
A DIY approach to creating a programmable AI pendant using affordable development hardware.
1DIY Alternative to Muse Charm
Instead of purchasing the proprietary Muse Charm, a $50 Waveshare ESP32-S3 1.75C touchscreen pendant serves as a functional alternative. This hardware is officially sanctioned by Meta for building Muse Gadgets. By leveraging open-source code and tokens, users can connect directly to the Muse AI cloud without needing the commercial device.
2Initial Hardware Setup Challenges
The Waveshare device arrived as a blank slate, requiring manual configuration via a PC. Initial integration with Meta’s GitHub repository resulted in poor performance, where the AI failed to respond audibly and struggled to interpret voice commands. The device lacked essential features like smart home discovery and functional remote camera shutter controls.
3Enhancing Functionality via Claude Code
Using Claude Code, the device was transformed through iterative reprogramming. A conversation history interface was added to address display limitations, and an ElevenLabs British AI voice was integrated to provide audible responses. These modifications effectively bridged the gap between the raw development kit and a more polished, interactive user experience.
4Customizing the AI Interaction Layer
The project expanded to include a Tamagotchi-style minigame featuring hunger, boredom, and cleanliness metrics. Claude generated multiple avatars and programmed specific interactions, such as treats causing messes or excessive play inducing fatigue. These custom features demonstrate the potential for highly personalized AI behavior beyond the standard Meta-provided Jolly avatar.
5Optimizing Remote Camera Capabilities
The developer successfully implemented a remote shutter button and a live preview feature over Bluetooth. This allows the pendant to trigger the phone camera from across a room. By refining the audio input logic to ignore short, accidental button presses, the system became significantly more responsive and reliable for daily use.
6Expanding Utility Beyond AI
The pendant was further modified to function as a practical pocket watch with a three-hand clock face, programmed in fifteen minutes. Additionally, a video player was installed to stream clips directly on the small screen. These additions prove that the hardware can serve multiple purposes beyond its primary AI assistant role.
7Security and Privacy Precautions
Due to concerns regarding data privacy and potential leaks, the device was isolated on a guest Wi-Fi network. The developer maintains a cautious stance toward Meta’s Muse agent, particularly regarding access to personal email or home networks. This highlights the necessity of sandboxing experimental AI hardware to protect sensitive user information.
8The Value of Programmable Hardware
The experiment suggests that the true value of a dedicated AI pendant lies in its programmability rather than its out-of-the-box features. While the commercial Muse Charm offers convenience, development kits provide the freedom to customize functionality. The ultimate utility of such devices depends on the user's willingness to engage with code.