#Tag · bonfire.cafe

A story about never ever giving up...❤️‍🔥

After several weeks, questioning my life choices, I've finally figured out why my #Whisper #SpeechToText system had been so slow on #Windows:

It was because apparently the #Rust-FFI wrapped #CPlusPlus code (Whisper.cpp) didn't compile with AVX and AVX2 enabled ( #SIMD!). I've tried it on two Windows machines (both AVX-capable). On one of the machines, with #Linux, it has successfully detected AVX/AVX2, though and has run fast.

1/?

Jan :rust: :ferris:

@janriemer@floss.social · 2 months ago

Hmm... 🤔

My suspicion why it's "not working" is:

Even though I do `cargo run --release` I've seen, during my investigation of the above compiling-fail-nightmare, that it puts artifacts into `Debug` folder.

So it might be that the program (Whisper.cpp to be precise) runs as a debug build and is just _terribly_ slow. 🐌

Oh boy, the struggle continues... 🤸

This might be related:
https://codeberg.org/tazz4843/whisper-rs/issues/226

Codeberg.org

x10 slower in release build

Running Whisper for transcribing in a release build seems to take approximately x10 as long as a debug build. I tested this with the `audio_transcription.rs` example using the `jfk.wav` file and the `ggml-medium.en-q5_0.bin` model. In Debug ``` PS C:\Repositories\wh...

Jan :rust: :ferris:

@janriemer@floss.social replied · 2 days ago

A story about never ever giving up...❤️‍🔥

After several weeks, questioning my life choices, I've finally figured out why my #Whisper #SpeechToText system had been so slow on #Windows:

1/?

alcinnz boosted

Jan :rust: :ferris:

@janriemer@floss.social · 2 months ago

Progress on my little speech2text/transcription project:

1. You press some hotkeys.
2. You speak into your microphone.
3. You wait for approx. 10 secs. (depending on your hardware)
4. Text starts to magically appear on your screen!

It feels like True Magic™! 🪄 ✨

This is why I love software development! ❤️

#Speech2Text #AI #Whisper #Rust #RustLang #Audio #AudioTranscription

Jan :rust: :ferris:

@janriemer@floss.social · 2 months ago

Ok, I have to correct myself:

Compiling any C/C++ project on Windows is an absolute clusterfuck!

I've now almost spent more time trying to compile my program for Windows than writing the actual code for it - let that sink in!

#Windows #CPlusPlus #Compile #Build #BuildSystem #CMake

Jan :rust: :ferris:

@janriemer@floss.social replied · 2 months ago

Whoop! It compiles now on Windows!

You'll never guess what the #error was...

...on my Windows machine I had a file sync program running in the background, which apparently tripped up the compilation process (the program to compile was in a folder that had been under sync)!

Once I moved the program out of this folder, it all compiled fine!

Holy cow! 🤯

Unfortunately, my program doesn't seem to work on #Windows yet. It just gets stuck after passing audio to #Whisper. 😢

#Fail

Jan :rust: :ferris:

@janriemer@floss.social · 2 months ago

Progress on my little speech2text/transcription project:

1. You press some hotkeys.
2. You speak into your microphone.
3. You wait for approx. 10 secs. (depending on your hardware)
4. Text starts to magically appear on your screen!

It feels like True Magic™! 🪄 ✨

This is why I love software development! ❤️

#Speech2Text #AI #Whisper #Rust #RustLang #Audio #AudioTranscription

Doug Webb

@douginamug@mastodon.xyz · 3 months ago

This post was written by voice input through the #Futo keyboard which uses #whisper. Free, open source and local - no internet connection! Seems to work very well for English and quite good for German too. I only had to add hashtags and correct two words manually.

Important addition: I was motivated to look for a solution for a friend who has multiple sclerosis, and it makes typing really hard for him. His messages are already longer and more detailed :)