I've found out the reason for the error:
error: could not find native static library `ggml`, perhaps an -L flag is missing?
Apparently, when cross-compiling, it can sometimes happen (especially with Windows target) that the compiler searches for "lib*" as prefix in the file name.
So when renaming compiled `ggml` to `libggml`, the error disappeared.
Unfortunately, I still get a linker error, but I don't bother anymore (direct Windows compile in CI works now)