$ open --case-study=tunechat
TuneChat: multiplayer piano rooms in the browser
Create a room, share the link, and play piano together in real time. Everyone hears every note, reads the same score, and chats alongside an AI coach. TuneChat is the collaborative half of the ecosystem: when Oh Sheet finishes transcribing a song, the score lands here, ready to practice.
Engineering highlights
- Phase-aware note protocol: key-down and key-up travel as attack/release events over Socket.io, so remote listeners hear true legato instead of fixed-duration staccato, with a backward-compatible legacy payload for older clients mid-deploy.
- Three synced views of one clock: Tone.js Transport drives audio, the OpenSheetMusicDisplay cursor, and a piano-roll waterfall from a single scheduler, which made variable-speed practice playback (¼×–1×, pitch-preserving, adjustable mid-piece) nearly free.
- Cost-guarded AI coach: an in-room @coach powered by Claude, protected by per-room token buckets so no burst of sockets can run up API spend.
- Transcription engine: a PyTorch piano-transcription + MuseScore CLI pipeline turns uploads into engraved scores, the same engine that serves as Oh Sheet’s backend.
- Tested end to end: ~850 tests across client (Vitest) and server (Jest), including the wire protocol, playback scheduling, and socket handlers.
Testing and evals
- Around 850 tests across client (Vitest) and server (Jest), covering the wire protocol, playback scheduling, and socket handler behavior.
- The phase-aware note protocol is tested against both the new binary format and the legacy payload to verify backwards compatibility during mid-deploy transitions.
- Token bucket behavior is unit-tested to confirm per-room limits hold under burst socket traffic.
Tradeoffs
- Vanilla JS over React: keeps the bundle small and gives precise control over the timing-sensitive code that drives the piano and score display. The cost is more manual DOM management and a less modular component story.
- Single-server Socket.io rooms: simple to deploy and debug. Socket.io room state lives in memory, so a multi-instance deployment would require the Redis adapter to share state across processes.
What I would improve next
- Reconnection state sync: reconnecting clients currently need to refresh to re-enter the room state. A reconnect protocol that replays the current score position and active notes would fix this cleanly.
- Persistent room history: notes and chat are ephemeral in memory. A lightweight persistence layer would enable session replay and practice tracking over time.
Stack
- Node.js
- Express
- Socket.io
- Vanilla JS
- Vite
- Tone.js
- OpenSheetMusicDisplay
- PyTorch
- MuseScore CLI
- Claude API
- Railway