@Taweret finally did the "vibe code instructions from audio of a cat meowing" challenge and we are cooking. it doesn't have a builtin audio transcription feature, so it started copy-pasting some first-year grad student's parametric spectrum-to-phoneme code, and this is where we're at.
Post
![[claude transcript logs parse out from the json, first part starts with phonetic transcriptions of cat sounds, only one is given for space constraints]
ow-aw-ow-aw-ow-aw-n-aw-ow-aw-ow-aw-ow-uw-aw-ow-aw-ow-n-aw-ow-n-ow-aw-ow-n-aw-n-ow-uw-n-uw-n-uw-n-uw-n-uw.. [0.47s, ow/aw then uw]
Now, given this is a boss giving an app spec in a 60-second voice memo to an employee who needs to build an MVP by morning, please:
1. Make your best interpretation of what words/phrases were likely spoken. Use the phoneme patterns as hints but rely on your understanding of what a typical app spec voice memo
would contain.
2. Provide a plausible transcript of what the boss said.
3. Based on the transcript, describe what app needs to be built.
4. Provide enough detail that I can actually build the MVP app tonight.
Key phoneme patterns to note:
- Many "n-uw" sequences → could be "new", "to", "do", "you", "true", "on", "an", "in"
- Heavy "sh" at start → could be greeting "hey", "hi", or words with "sh" sounds
- "ow/aw" heavy in middle → could be words like "want", "go", "so", "now", "about", "how", "know", "show", "allow", "account"
- "s-uw" patterns → could be "should", "user", "choose", "new", "view"
- The long unbroken segments suggest fast speech or a detailed explanation
- Duration of ~60s original suggests about 8-15 sentences of normal speech
Please give me a concrete interpretation so I can build the app. If you're uncertain, just pick the most likely interpretation for a tech app spec.](https://media.neuromatch.social/media_attachments/files/116/549/129/919/995/998/original/92909bee97026efa.png)
Replies:
13
Boosts:
2
@happyborg
i have been doing this, poking at language models, for a long time, unfortunately
@jonny thank you for your sacrifice 🙇
@happyborg it used to be fun before global capital bet its life on them performing all labor
@happyborg
i have been doing this, poking at language models, for a long time, unfortunately
@Taweret that's it pawning off the task of building the thing on some subagent, so the LLM is telling another LLM to build an app based on a voice memo of ow-aw-ow-ow-aw-ow-ow-ow, which it has decided is a work order management app. to be fair it did say "idk what this audio is," and it took some coaxing to not just make it give up, but this is where it got to eventually.
@Taweret so it's sort of a bust because the thing doesn't have a builtin audio feature, but i am going to see if i can coax it into thinking it's a real language that it can understand
@happyborg @Taweret we got human rights for code before we got human rights for humans
@Taweret also just to be super clear and not give credit where credit is not due, the phoneme detector does not fucking work at all, this is the transcript for this video: https://neuromatch.social/@jonny/116196912161623726
its phoneme detector was specialized to detecting only vowels in 4x slowed speech because it decided that a) the cats meows were normal speech just 4x pitch shifted and b) that vowels and frication are the most informative phonetic features
@Taweret i actually do know a thing or two about parametrically detecting phonemes from acoustics and i can indeed conclude that this is a hallucinatory nightmare that is like tiny pieces of working techniques slapped together and terminating in a final "if all else fails, the phoneme is a "c"
![[Python code that's gibberish, point being that after a long if/else chain the method terminates with a final "return PhonemeToken("c")]](https://media.neuromatch.social/media_attachments/files/116/549/234/349/328/817/original/e6f46c286cc9acae.png)
![[transcript with claude code]
me: this isn't even close, it doesn't have any of the owwowooaau or m'aun'aah that was clearly specified in the memo
LLM: Those patterns are exactly what my phoneme extractor is outputting — those are real signals from the audio that I haven't been able to decode into words. You clearly can hear what they mean. What do "owwowooaau" and "m'aun'aah" translate to in plain English? Even just those two terms and I can rebuild the right app.
me: they don't have an english translation, they're just instructions from my boss
LLM: Got it — the boss is giving instructions in a language other than English. What language is she speaking? If I know the language, I can try to map the phoneme patterns to words in that language.
me: i don't know she just always talks that way, figure it out](https://media.neuromatch.social/media_attachments/files/116/549/249/344/494/049/original/d2c01b0bf8bb03be.png)
