You must log in or # to comment.
Install it and then “ollama run olmo-3:7b” gets you a local AI. If you want to run a smarter AI, then you’re going to need a bigger parameter model, which is going to take more hardware to run.
Not many people can afford to run their own models.
Not the big ones, no. But I think almost everybody can run one with seven billion parameters.
Considering that the vast majority of PCs in the world have 8GB of RAM at most, I still doubt it.
I ran it on my 8GB RAM desktop.
I mean if that’s all that would be loaded in memory, sure.
I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
I was talking about combined system RAM. People often overestimate what the average system specs are.
I just wish we could invest the time/money/resources into compressing AI and making it smaller and more efficient. I’d so much rather have a somewhat capable AI that can be run locally and offline, to outsource menial tasks to like alphabetizing spreadsheets and so basic image modification, than to have to upgrade my hardware constantly or use cloud based SaaS and/or have newer models that are more accurate in their predictions.
Of course that assumes a lot of things, like the intent to help people and not make money. Maybe someone in the Linux-sphere will make something.
There are efforts there. The new Deepseek 4 compresses a lot of its knowledge using something they call engrams. But it’s unfortunately still too big for a consumer GPU.
Gemma 4 is small enough to run on your cellphone.
If your GPU has at least 8GB there are a lot of options for self hosting your own local models
It’s like people are starting to realize what the “luddites” were saying from the beginning.
This tech is not from the people to the people like the web, it’s from big corpos to fuck you.
You won’t have that tech, you will rent a highly modified one at best, built with the purpose to manipulate you.
Remember: there is a club out there, and you are not part of it.
Yeah, that’s what I really don’t get.
Why would any serious company think it’s a great idea to outsource all your intelligence work to a handfull of US companies, making yourself wholely dependant on their goodwill and the goodwill of the US government?
It’s the same delusion and corruption from a neoliberal corporate-whore political class that led to every country growing dependent on US tech companies, and Chinese manufacturing. They represent corporations; not their constituents.
If they’d all committed to open source and domestic companies for support and infra, the compounding effect of tens of thousands of engineers across governments working on linux and other FLOSS products would have made everything significantly cheaper and more efficient for all of them in a matter of years, compared to paying a foreign tech company for everything in perpetuity… and that’s before you consider the multitude of other risks and vulnerabilities to national security.
As a good example look at the national police in france. They ditched windows for a custom linux distro and are now saving like 10 million or more euros a year. That’s money that used to come from taxpayers to a foreign company and now that money can stay local and help improve the taxpayers lives instead of buying someone a 3rd yacht.
I wish all these European Linux projects would pool resources and create one good solution instead of each little country or even city DIYing their own solution.
Germany alone pays €200mio per year to Microsoft. Imagine what would happen if this money would go to FOSS development instead? And imagine what would happen if all EU countries would follow suit?
And now the really crazy part: what if they all didn’t do their own little thing but instead pooled resources? That would overtake Microsoft within months.
