Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU
https://github.com/xaskasdf/ntransformer
#HackerNews #Llama3.1 #RTX3090 #NVMe #GPU #bypass #CPU #AItechnology
#Tag
Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU
https://github.com/xaskasdf/ntransformer
#HackerNews #Llama3.1 #RTX3090 #NVMe #GPU #bypass #CPU #AItechnology