Both Ubuntu and Fedora have made it official: support is coming soon for running local generative AI instances.
An epic and still-growing thread in the Fedora forums states one of the goals for the next version: the Fedora AI Developer Desktop Objective. It is causing some discontent, and at least one Fedora contributor, SUSE’s Fernando Mancera, has resigned.
Mind: I’m not the person running the local model.
I did say that the efficiency would be a different question neither of us can answer in this case, but I fully agree with you. I merely pointed out that a local model wouldn’t be a permanent waste of electricity.
That’s relative to how much space you have. I also have games on my disk that I haven’t played in a while, so they’re more or less wasted space. But they’re not particularly large, so I can spare a few GB for them, and if I do want to play them, I can jump in spontaneously.