guess i know what i'm testing today.
update: llama.cpp isn't ready. some compatibility bugs still left in the tracker. guess i'm going to wait a few days.
guess i know what i'm testing today.
update: llama.cpp isn't ready. some compatibility bugs still left in the tracker. guess i'm going to wait a few days.
hmm, i dont know if it helps anything, but i recently came across https://github.com/JustVugg/colibri which seems to use techniques to run enormous models on machines with constrained resources
You need to log in to see that page.
@lritter hah, let me know how it goes pls 🙂