Discussion

So how have I found out about AVX not being detected? Well, it was actually quite easy (once you know where to look!):

During compile of Whisper.cpp, it shows detection of AVX/AVX2 in the log output.
You can see it here in their CI:
https://github.com/ggml-org/whisper.cpp/actions/runs/18441378587/job/52542245446#step:5:52

They have different CI jobs for compiling for Windows (MSYS2, MSVC) - interestingly, only the MSVC one shows the AVX log.

But I was using MSVC on my local Windows machines, where it failed to detect AVX.

So what to do now?

2/?

GitHub

release : v1.8.1 · ggml-org/whisper.cpp@a91dd3b

Port of OpenAI's Whisper model in C/C++. Contribute to ggml-org/whisper.cpp development by creating an account on GitHub.

Jan :rust: :ferris:

@janriemer@floss.social replied · 4 months ago

Whoop! It compiles now on Windows!

You'll never guess what the #error was...

...on my Windows machine I had a file sync program running in the background, which apparently tripped up the compilation process (the program to compile was in a folder that had been under sync)!

Once I moved the program out of this folder, it all compiled fine!

Holy cow! 🤯

Unfortunately, my program doesn't seem to work on #Windows yet. It just gets stuck after passing audio to #Whisper. 😢

#Fail

Jan :rust: :ferris:

@janriemer@floss.social replied · 4 months ago

Hmm... 🤔

My suspicion why it's "not working" is:

Even though I do `cargo run --release` I've seen, during my investigation of the above compiling-fail-nightmare, that it puts artifacts into `Debug` folder.

So it might be that the program (Whisper.cpp to be precise) runs as a debug build and is just _terribly_ slow. 🐌

Oh boy, the struggle continues... 🤸

This might be related:
https://codeberg.org/tazz4843/whisper-rs/issues/226

Codeberg.org

x10 slower in release build

Running Whisper for transcribing in a release build seems to take approximately x10 as long as a debug build. I tested this with the `audio_transcription.rs` example using the `jfk.wav` file and the `ggml-medium.en-q5_0.bin` model. In Debug ``` PS C:\Repositories\wh...

Jan :rust: :ferris:

@janriemer@floss.social replied · 3 months ago

A story about never ever giving up...❤️‍🔥

After several weeks, questioning my life choices, I've finally figured out why my #Whisper #SpeechToText system had been so slow on #Windows:

It was because apparently the #Rust-FFI wrapped #CPlusPlus code (Whisper.cpp) didn't compile with AVX and AVX2 enabled ( #SIMD!). I've tried it on two Windows machines (both AVX-capable). On one of the machines, with #Linux, it has successfully detected AVX/AVX2, though and has run fast.

1/?

Jan :rust: :ferris:

@janriemer@floss.social replied · 3 months ago

So how have I found out about AVX not being detected? Well, it was actually quite easy (once you know where to look!):

They have different CI jobs for compiling for Windows (MSYS2, MSVC) - interestingly, only the MSVC one shows the AVX log.

But I was using MSVC on my local Windows machines, where it failed to detect AVX.

So what to do now?

2/?

GitHub

release : v1.8.1 · ggml-org/whisper.cpp@a91dd3b

Port of OpenAI's Whisper model in C/C++. Contribute to ggml-org/whisper.cpp development by creating an account on GitHub.

Jan :rust: :ferris:

@janriemer@floss.social replied · 3 months ago

Well, sometimes you just need to literally _try stuff out_ to increase the probability of success!

Let's recap:
- two Windows machines: AVX not detected ❌
- one of the machines also a Linux machine (dual boot): AVX detected ✅
- CI runner on GitHub Actions with MSVC toolchain (Whisper.cpp): AVX detected ✅

#GitLab, where my project is hosted (soon will be open source 🤞), offers Windows CI runners (beta), so let's try that!

https://docs.gitlab.com/ci/runners/hosted_runners/windows/

🥁

3/?

Jan :rust: :ferris:

@janriemer@floss.social replied · 3 months ago

And this finally worked!! :awesome:

The compilation has produced an .exe file with AVX/AVX2 enabled and it is finally usable on Windows (transcription speed _with_ AVX for two sentences: <7 secs; _without_ AVX: > 60 secs).

But after all this you might ask: we have solved the problem...but we still don't know why it failed to detect AVX in the other instances?

Unfortunately, I still don't know!

All I know is that "Reality has a surprising amount of detail":

http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail

4/4

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances

Bonfire social · 1.0.1-beta.35 no JS en

Automatic federation enabled