@scottjenson
It has been a while, but I remember there was a device you could mount while instantiating your docker container that allows various Python scripts to access the GPU. I did it while playing with ollama for a hackday project 3 or 4 years ago.
Discussion
Loading...
Discussion
@scottjenson
It has been a while, but I remember there was a device you could mount while instantiating your docker container that allows various Python scripts to access the GPU. I did it while playing with ollama for a hackday project 3 or 4 years ago.
@nikatjef Yeah, I'm SURE there a way to make it work. I'm very new to this and the general (maybe unfair) rule I'm learning is "things don't run well in docker". (I mean of course some things run great but....)
I had the same issue with Home Assistant, I wanted to run that in CasaOS but my #HomeAssistant friends were "NOOOOOOOOOOOO!" (which I now thank them profusely)
@scottjenson
So the thing to keep in mind about Docker, and containerization in general, is that it is really meant as a way to isolate a process from the underlying system.
Everything the process needs, including libraries, needs to be in your container or mounted when you instantiate it.
Next, anything that talks to the process from outside needs to be pointed at the right end point on your Docker host.
90% of Docker issues are in those two areas.
@nikatjef thank you, that makes total sense. It's just that services like casaOS make it particularly easy for novice people like me to dabble with things like this and it's a good example of playing with fire.
I didn't mean to throw shade on docker, I was more throwing shade on my poor understanding of how it works.
My #ux suggestion is that #casaOS should provide some type of guidance for images that would not do well such as ollama or home assistant.
@scottjenson
And I should point out there are a couple home assistants that are written almost exclusively to be run inside of containers due to the potential for misuse...
Gonna have to look into CasaOS though, have not heard of it, but I suspect there might be more to your misery there than there should be.
@scottjenson
Heheh, I didn't think you were throwin shade, I thought you had merely gotten in a bit over your head. 😊 Sorry, if my reply made it seem otherwise.
I would almost recommend looking at installing #Proxmox. Yes, it would be massive overkill for what I think you are trying to do, but there are a bunch of helper a scripts for Proxmox that can kind of walk you through a lot of this and ease you into some of the more advanced concepts
bonfire.cafe
A space for Bonfire maintainers and contributors to communicate
Automatic federation enabled