Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • Domi@lemmy.secnd.meEnglish
    1·
    1 month ago

    Yes, I got a Strix Halo machine before the RAM price hike and use it to run all my ML stuff on it.

    Currently using llama-swap with llama.cpp/ComfyUI and opencode/Open WebUI as frontend.

    I’m running Qwen3.6-27b, Voxtral Mini 4b, Piper and Qwen Image. Also, some embedding and reranking models.

    I use them for:

    • Tagging and classification of my documents in Paperless
    • Home Assistant (voice assistant)
    • Translations (both text and image)
    • Transcriptions
    • Some light coding and debugging
    • Avatar/Backdrop generation for DnD sessions