Saying it’ll never happen and giving up won’t make that 2% get any bigger
Make the tools easy enough that the irritation of the original platform gives them enough of a push to try something easy to use.
If you want to help beta test, I’m working on an appliance-level automated installer for provisioning home cloud servers.
Got friends who need help? Help them or send them my way (or to your nearest techie)
Decouple people from big tech, allow people to run, fix, and maintain their own hardware instead of replacing it on an industrial scale every 6-9 months and centralizing ownership in data centers.
My entire stack runs off a 14kw solar array with a battery. Net negative. It’s totally doable.
Edit: also I tested my stack against a stimulated 1k active users and couldn’t get it to break a sweat, so I’m not saying everyone needs to do things this way, but your techiest uncle could host the entire family in a privacy cooperative, for instance
LLM and VLM are generally the type of model that needs a ton of resources to run. Birdnet uses an audio classification model, but there are dozens of types of models.
I’m building specialized task-focused pipelines which keep the memory footprint small for self-hosted users, and I’m working on a unified framework too make building other pipelines and applications much easier: https://docs.circuitforge.tech/
If you give me some details on what you’re after I might be able to find you some projects or at least clarify keywords
Edit: an example of how efficient these can get, I have a little ESP32-driven waveshare display running an ONNX wake-word detection model which is absolutely nuts. 4mb psram and a dual-core 240mhz cpu and its only job is handling “hey Mycroft” and establishing a proper speech-to-text connection with my server downstairs
Specialized models can be made incredibly efficient. The big tech guys are trying to replace humans which is why their crap takes such huge servers to run. Building local-first/only just means you need to specialize the model and the pipeline to match what you’re doing
That’s my specialty! My goal with my projects has always been to lower the barrier to entry while compromising as little as possible on things that matter. I remember everything Google and Apple built before they started dumbing it down and removing features. I remember how much effort went into onboarding users and making sure people could adopt things easily. It didn’t always work, but I’ve been very successful in making my entire de-google’d stack useful to my own family.
The hardest part is done, now I just need to make it replicable for others to spin up their own. If you’re interested, I’ve got a repo scaffolded but I haven’t published the actual stack installer yet: https://git.circuitforge.tech/Circuit-Forge/turaco/issues