Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • PetteriPano@lemmy.world
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    2
    ·
    1 天前

    Running qwen3.6 27b through llama.cpp.

    It’s about as capable as sonnet 3.5.

    I use it for light scripting, but real coding is done by cloud models.

    I’m also using it as the brain for my Hermes agent. It sends me digests of news, subreddits, chats that I’d like to read but don’t have time for. It does a great job researching things on the web for me, too.

    • SuspiciousCarrot78@aussie.zoneOP
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      1 天前

      Do you mean Sonnet 4.5?

      I don’t have the rig to run it at real speeds but I’ve played with it over API. Seems pretty good.

      • PetteriPano@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        20 小时前

        No, it needs a lot more babysitting than 4.5 does. 3.5 was on the same level of mistakes, at least on the quants I have to use.