Both Ubuntu and Fedora have made it official: support is coming soon for running local generative AI instances.
An epic and still-growing thread in the Fedora forums states one of the goals for the next version: the Fedora AI Developer Desktop Objective. It is causing some discontent, and at least one Fedora contributor, SUSE’s Fernando Mancera, has resigned.
Always pretty funny to see the FOSS community willingly follow the lead of corporate investment trends, but for fractions of pennies on the dollar compared to big tech.
If you’re gonna act like Microsoft you might as well do it for the money.
-The system image will not be pre-configured with applications that inspect or monitor how users interact with the system or otherwise place user privacy at risk.
-Tools and applications included in the AI Desktop will not be pre-configured to connect to remote AI services.
-AI tools will not be added to Fedora’s existing system images, Editions, etc, by the AI Desktop initiative.
- It’s a new system image, your Fedora and Ubuntu installs will not be modified.
- The applications that would have privacy implications will be opt-in by default.
- It will not use remote AI unless you configure it to.
I don’t see the problem.
What’s wrong with just installing ollama?
FOMO is a thing. Who’s gonna want to run the old version packaged in the distro anyway. Those things go stale pretty quickly, especially at the rate we’re seeing updates in local inference.
I think it is good to have optional support for local models that lets people use them in an offline and private and easy way. There is a lot of non technical folks using linux nowadays and many chose it for privacy and greater control over their data.
Depending on the implementation it could hook into certain os contexts and events to actually be helpful.
Either way I don’t see the cat going back in the bag with regards to LLMs. That being said I run Debian everywhere except my work machine which is ubuntu.
In this case, the “bag” is a sucking black hole, and venture capitalists are throwing physics-defying amounts of money in it to drag the LLMs out. As soon as they stop that, the “cat” goes back in the “bag”.
Local LLM models are an exception, but they are also atrocious by comparison. Most users will get some limited utility from an LLM if they had one, maybe, but it is being accommodated and foisted everywhere like its the invention of the mouse. It is nowhere near as paradigm shifting, but is being hyped, advertised, and marketed more aggressively than any product in history. So, the roaring hype makes everyone think that if they don’t get on board too, they’ll be left in the dust, so now well-meaning projects are getting bloated up for it too.
Many of us just want this technology to get the fuck away from us until it is worth using or dies already. Is that so very much to ask?
Using an llm mondel that isn’t super advanced is actually quite freeing in my opinion, the generated output is always mediocre at best, but it’s usually good enough for boilerplate and can be decent if you need to unstuck yourself. It also isn’t good enough to lull you into just letting the llm do all the work for you since it makes obvious mistakes.
“It’s just good enough for some things once in a while, but is too bad to rely on in any serious way,” doesn’t sound like a great use of my electricity, but I guess I’ve wasted electricity on less. Still, doing it on purpose seems worse.
I hope it will be that way. in the end it could be the incentive to improve accessibility and UI automation tools for wayland.
I mean if it’s completely optional and opt-in, sure go for it. Knock yourself out. I won’t be using it (my computer isn’t powerful enough to run local LLMs)
They’re discussing a new spin specifically for AI developers, not changing existing distros to include new stuff.
CTFD
Oh good. We can all just ignore the AI version then?
It’s Linux, you always could ignore any software that you don’t want
No we can’t, we must be mad, because “AI”
This guy social medias
Yes
Well, we get to slam it and heckle its users, because, well, we need SOME joy in The world…
lol they already support running local models. wtf is the distro gonna do…? pre-install llama.cpp? this is so silly to me that people are resigning over this, too.
Open claw by default would be cool
Is that the one that really whips the llama.cpp’s ass?
Now I have to install winamp and get a bunch of sick skins for it.
SUSE contributes to Fedora? WTF?
not anymore. IBM decided a few years ago that they don’t want grubby fossy fingers in their pie.
kind of makes me question why anyone who supports the foss part of Linux would continue to use fedora or any IBM product.
I say this as an ex fedora fanboy.
fuck IBM.
Someone from SUSE used to contribute to Fedora
Shame
Shaame!
I don’t really understand the point? You can already do a lot on Linux using AI via CLI with bash.
I’m not a fan, but obviously, AI on the desktop is not about the CLI and bash, which are not the desktop.
I don’t really understand the point?
Correct: you don’t. I’m basing this only off what you wrote, but I’m reasonably confident of my answer. Glad I could answer the question as asked.
I miss the good ole days on the Internet when random strangers wouldn’t try to white knight people who casually assault the English language.
If you don’t want people to correct you then be correct in the first place or accept the ribbing as a sign that you messed up.
And to the downvoters: there is enough free range ignorance on the Internet, it isn’t an endangered species that you need to protect. literally
Soooo, which distro to switch to next? Or are they all gonna go down this route eventually? Maybe I’ll try a *BSD for once.
Debian or anything based directly on it
I just installed Debian and it asked me for my Real Name, pass.
/s
Debian or anything based directly on it
Debian can also run AI models. Pretty dumb reason to get mad about.
Every linux distro can run AI models, there is 0 reason to support distros that ship with AI bullshit by default
its interesting when people seem to intentionally misunderstand statements.
pretty much anything that is “you build it youself” so Arch, NixOS, Gentoo, etc
but yeah I’m also tempted to finally give BSD a go.
nixos comes with systemd
SystemD does something everyone uses, though. Many may prefer other methods, but systemD still does cover the need for init, logging, and system control, which everyone with a PC does need.
LLMs are useful to only a subset of users, so including it in general distribution is waste, because many people will end up with this software that was downloaded, installed, and maintained by someone, but they never use at all. It may not be actively using their RAM, but it is still using their disk space and has dependencies that may load whether or not it is run.
In a time of grievous constraint on hardware availability, that kind of waste is literally impossible to tolerate for many, and will force users to fewer distro choices, so they do have skin in the game and grounds to make their voices heard.
LLMs are useful to only a subset of users, so including it in general distribution is waste, because many people will end up with this software that was downloaded, installed, and maintained by someone, but they never use at all.
That’s absolutely untrue in this case.
If you read the article (the headline text is a link to an article) you would see that they specifically address this:
-AI tools will not be added to Fedora’s existing system images, Editions, etc, by the AI Desktop initiative.
It won’t be included in the general distribution. They’re talking about a new system image that someone would have to choose to install.
There seems an epidemic around here of this phenomenon where, when discussing an article or other set of information, additional context, even when already brought up or requested, is regarded as not germane to the discussion.
I did read the article, as well as most of the other comments in this thread, some of which were referring to the “creep” effect of undesirable tools into other places once they get into some of the main places. That’s what I am referring to. If you don’t want to participate in that branch of the discussion, that’s an option, as is even a drive-by down vote.
Scolding me for not rigidly sticking within the scope of the topic that started the conversation, even after the topic has drifted or broadened, seems not only counterproductive, but a waste of your time.
correct
I switched from Kubuntu to CachyOS last week, after 10 years or so. CachyOS is based on Arch, and did not disappoint so far, extremely fast, makes Ubuntu look old and sluggish. It’s really impressive. The basic installation was easy. The GUI package manager isn’t as polished but works. A little bit of terminal tweaking was required to install some packages (VMM and KRDC gave me some trouble) but the documentation was ok. Absolutely can recommend.
Cachy is a nice mix of latest and stable. Still recommend Debian for work, but Cachy is great if you want something that just works
I’ve used PikaOS which is Debian based and IMHO it’s right up there with CachyOS. super fast install, fast distro, great package manager, and repos with the latest stuff. very stable too.
