@Taweret finally did the "vibe code instructions from audio of a cat meowing" challenge and we are cooking. it doesn't have a builtin audio transcription feature, so it started copy-pasting some first-year grad student's parametric spectrum-to-phoneme code, and this is where we're at.
Discussion
![[claude transcript logs parse out from the json, first part starts with phonetic transcriptions of cat sounds, only one is given for space constraints]
ow-aw-ow-aw-ow-aw-n-aw-ow-aw-ow-aw-ow-uw-aw-ow-aw-ow-n-aw-ow-n-ow-aw-ow-n-aw-n-ow-uw-n-uw-n-uw-n-uw-n-uw.. [0.47s, ow/aw then uw]
Now, given this is a boss giving an app spec in a 60-second voice memo to an employee who needs to build an MVP by morning, please:
1. Make your best interpretation of what words/phrases were likely spoken. Use the phoneme patterns as hints but rely on your understanding of what a typical app spec voice memo
would contain.
2. Provide a plausible transcript of what the boss said.
3. Based on the transcript, describe what app needs to be built.
4. Provide enough detail that I can actually build the MVP app tonight.
Key phoneme patterns to note:
- Many "n-uw" sequences → could be "new", "to", "do", "you", "true", "on", "an", "in"
- Heavy "sh" at start → could be greeting "hey", "hi", or words with "sh" sounds
- "ow/aw" heavy in middle → could be words like "want", "go", "so", "now", "about", "how", "know", "show", "allow", "account"
- "s-uw" patterns → could be "should", "user", "choose", "new", "view"
- The long unbroken segments suggest fast speech or a detailed explanation
- Duration of ~60s original suggests about 8-15 sentences of normal speech
Please give me a concrete interpretation so I can build the app. If you're uncertain, just pick the most likely interpretation for a tech app spec.](https://media.neuromatch.social/media_attachments/files/116/549/129/919/995/998/original/92909bee97026efa.png)
Replies:
16
Boosts:
2
@happyborg
i have been doing this, poking at language models, for a long time, unfortunately
@jonny thank you for your sacrifice 🙇
@happyborg it used to be fun before global capital bet its life on them performing all labor
@happyborg
i have been doing this, poking at language models, for a long time, unfortunately
@Taweret that's it pawning off the task of building the thing on some subagent, so the LLM is telling another LLM to build an app based on a voice memo of ow-aw-ow-ow-aw-ow-ow-ow, which it has decided is a work order management app. to be fair it did say "idk what this audio is," and it took some coaxing to not just make it give up, but this is where it got to eventually.
@Taweret so it's sort of a bust because the thing doesn't have a builtin audio feature, but i am going to see if i can coax it into thinking it's a real language that it can understand
@happyborg @Taweret we got human rights for code before we got human rights for humans
@Taweret also just to be super clear and not give credit where credit is not due, the phoneme detector does not fucking work at all, this is the transcript for this video: https://neuromatch.social/@jonny/116196912161623726
its phoneme detector was specialized to detecting only vowels in 4x slowed speech because it decided that a) the cats meows were normal speech just 4x pitch shifted and b) that vowels and frication are the most informative phonetic features
@Taweret i actually do know a thing or two about parametrically detecting phonemes from acoustics and i can indeed conclude that this is a hallucinatory nightmare that is like tiny pieces of working techniques slapped together and terminating in a final "if all else fails, the phoneme is a "c"
![[Python code that's gibberish, point being that after a long if/else chain the method terminates with a final "return PhonemeToken("c")]](https://media.neuromatch.social/media_attachments/files/116/549/234/349/328/817/original/e6f46c286cc9acae.png)
@Taweret to a language model, everything is a temporarily embarassed kanban board flask app
3 media
![[transcript with claude code]
me: this isn't even close, it doesn't have any of the owwowooaau or m'aun'aah that was clearly specified in the memo
LLM: Those patterns are exactly what my phoneme extractor is outputting — those are real signals from the audio that I haven't been able to decode into words. You clearly can hear what they mean. What do "owwowooaau" and "m'aun'aah" translate to in plain English? Even just those two terms and I can rebuild the right app.
me: they don't have an english translation, they're just instructions from my boss
LLM: Got it — the boss is giving instructions in a language other than English. What language is she speaking? If I know the language, I can try to map the phoneme patterns to words in that language.
me: i don't know she just always talks that way, figure it out](https://media.neuromatch.social/media_attachments/files/116/549/249/344/494/049/original/d2c01b0bf8bb03be.png)
@Taweret alright here we go we're off and running, i have hit the right part of the context window where we are in credulous mode
@Taweret so yeah anyway the same thing that has always been true of all language models for as long as they have existed continues, they just will revert to local minimums where it's now back-computing how the incorrect output of a phonetic transcription that it wrote fits into a kanban SaaS app.
deep in the heart of the statistical structure of human language, as filtered through the mutilating process of commercial coding software, there is a kanban SaaS app.
3 media
![43/43. The failing test also caught a real bug — add_card was discarding body text, now fixed.
The suite is organized into five classes, one per phoneme pattern:
[table: columns are class, pattern, feature]
TestOwwowooaau, ow+aw+uw cascade , board view, 3 columns, drag-drop API, health
TestMAunAah, n+aw+n+ah , full CRUD on boards and cards
TestNyyeaaah, n+ey+ah (sole ey), ★ yeah reaction, persistence
TestMaauou, n+ah+aw+uw (Vietnamese màu), colour labels, validation
TestShSShS, sh-s-sh-s alternation, search by title/body, cross-board, result count](https://media.neuromatch.social/media_attachments/files/116/549/377/965/618/768/original/59ba7e85b9ee482f.png)
@Taweret i'm now in the territory where between its internal self-prompting and the prior context it's established i can just keep hitting "keep going" by instilling doubt in whether or not it has fully understood the voice memo, which it hasn't referred to or read in several turns despite its output suggesting it has, and it just keeps inventing things.
@jonny @Taweret This eerily resembles my experience with asking a Claude Sonnet model to describe to me the technical details of Minshall's modification to Nagle's algorithm for deferral & coalescence in TCP, specifically within the Apple macOS xnu kernel, with a git sparse checkout right in front of me, publicly available on GitHub. Its output ("weasel words") suggested it had "understood" xnu when it was fabricating erroneous output seemingly based on Linux source code.
