Frustrations with Arch Linux
Oh boy, this may cause some drama.
Since mid-2021, I use the distribution that its known for its annoying “I use Arch btw” meme. “Annoying”, as you often can tell that they don't actually use it in an “ironic” way, despite claiming so. I installed Arch via Archcraft, slowly stripped it off its Archcraft-specific repos and packages (only keeping a few for testing purposes), and largely called it a day. While Archcraft shipped with some defaults that go against what “vanilla Arch” is preaching, Arch itself appears to be guilty of the very same sins it denounces.
My main issue largely focuses on the people behind Arch due to being the root of my recent hiccups with the OS, makepkg.conf now being automatically overwritten to make the work of a single, non-Arch dev “easier”, the stark contrast between Arch Wiki pages and its very few contributors and huge gaps in editing histories, the Arch forum pretty much being dominated by less than a handful of users, and the rather-old GRUB debacle that affected children of Arch and any Arch user using GRUB, which initially was treated as a non-issue because one out of the two Arch testers claimed “works for me” and thus resulted in the issue being addressed four days AFTER the outcry.
How Arch treats its own configuration files
Usually, updates involving Arch-specific configuration files automatically prompt the creation of .pacnew files that need to be renamed in order to overwrite existing configurations. This allows users to check its diffs before making possibly breaking changes to their systems and it works well in practice. Arch Linux is weird in the sense that its mirrorlists never really are being cleared off dead mirrors and thus always generates mirrorlists including links that have been dead for years (like “gutscheindrache.de”). While it's often being recommended to use reflector to update mirrorlists, it's a weird and almost unnecessary option, as Arch hasn't seen the addition of new, possibly better mirrors n quite a while and thus the list stays the same with each mirrorlist update. I haven't had to update my mirrorlist since manually wiping all the dead mirrors but I did give Reflector a shot, only to be prompted with every single mirror generating “timeout” errors.
At the end of the day, overwriting such files still is optional, despite the “recommendation” to always overwrite the current local defaults. As long as you know what you're doing (and I do), it's often better to straight up ignore what the Arch Wiki and the forum likes to recommend.
Unfortunately, one particular file does not receive the convenient .pacnew treatment and recently was automatically overwritten with defaults proposed by a single non-Arch developer that submitted the same commit first to Fedora. Although this commit is supposed to address perceived shortcomings of debugging programs compiled with GCC, this commit also enabled the automatic pulling of debug packages, link time optimization and some... odd changes to the compression algorithm zstd, which now is set to --ultra -20, which the initial commit doesn't even address in its summary.
So, the first issue is that this commit is supposed to affect compiled packages for debugging purposes, yet Arch blindly copied this configuration, which still is being tested in the latest Fedora, into their own makepkg package of Pacman and casually left out the generation of a .pacnew file. This is highly problematic as I noticed the very same issue three other users submitted to the forum just a few weeks ago, with makepkg stalling for minutes during package compression of relatively big binary packages (in my case LibreWolf but also the lightweight Makepkg wrapper yay-bin) and the sudden pulling of debug packages. Although the additional debug packages can be deleted and its respective function disabled in makepkg.conf, the changes to the compression put an insane amount of strain to all of my PC's CPU's, pushing it to 75% for minutes and causing my ancient Acer Aspire and my crappy budget Asus to overheat and become unusable until both have properly cooled down again (which doesn't happen right after compression has finished).
The worst part is that this is an Arch-exclusive change even its own few active forum members refuse to address properly. Affected users are just being told to remove --ultra -20 without ever explaining why this option is being set automatically in the first place. One linked thread even includes links to the latest commits but redirects to irrelevant lines (like the compression algorithm two lines above zstd, which doesn't include the ultra option, and a comment briefly describing the lto flag).
Arch doesn't use its own configuration system consistently and and now pushes completely untested for-devs-only configs on an opt-out basis without addressing the significant regression affecting end users. And its few remaining forum users put the blame on end users for using AUR wrappers, even though this change was introduced quietly to said end users because... the few remaining devs expect every Arch user to always read every single commit of every single packages on Gitlab whilst also telling them to always update every package to even receive help on said forum and because “that's how Arch is doing it”?!
Seriously, nearly every Arch user didn't expect any significant changes to makepkg.conf in the first place (the only mention of this change was submitted to Hacker News two months ago and I had to search manually for it AFTER the compression regression began to affect me) and there isn't even an explanation given to the change of the zstd algorithm, which is the only algorithm that was changed. You're expected to read the code at all times now and do the guess work on something that shouldn't affect non-devops users at all. This push towards a “better developer experience” now is so common among operating systems that it makes Arch look like yet-another dev-focused OS for the sake of it but requiring much more manual intervention than Fedora.
Who is Arch Linux even attempting to target at this point, if it even refuses to adhere to its own way of consistency?! Why even offer something like .pacnew when it no longer is being used across Pacman-specific configs?!
Is the Arch Wiki even active anymore?
The vast majority of commentary addressing the Arch Wiki is extremely positive. The perceived lack of even minor issues poses a huge red flag, especially when taking into account that the Arch Wiki is much smaller and less used than the English Wikipedia and thus is largely stuck in a phase equivalent to Wikipedia's infancy decades ago. The German Arch Wiki in particular is prone to violating NOPV and many articles include custom hacks with self-written scripts that only partly get shared with the mention that they “will release the full script in the future” (which never happens) or include a TODO in the article itself. Other articles have been stubs for at least a decade or remain very sparse (and the “WiFi” article hasn't been updated since 2018, thus not even addressing iwd at all).
That's the German Arch Wiki, which may not interest most Arch users because... the majority of Arch users may not be German. But the English Arch Wiki has its own fair share of problems that never really are being addressed.
At the time of writing this, the Arch Wiki's only run by three maintainers and one guy with no official status whatsoever, who, for some reason, still is being listed as a high-ranking Arch Wiki member. Another guy with a red username (no profile, just an username) also recently begun to edit a bunch of articles, exclusively sticking to articles covering the most popular packages like systemd and iwd. And even those are rather “meh”, especially when comparing systemd to the more-general init article, and iwd... well, iwd is listed as supporting WPA2, yet the article covering wireless networking denies it – in practice, iwd fails to even discover my home's WPA2-secured WiFi.
And my main issue, which every. fucking. guide. and forum post failed to address properly, was an inherited misconfig of Archcraft that also exists on EndeavourOS, namely systemd-resolved.service and avahi_daemon.service clashing with each other. It's an easy-to-miss not on their respective articles, yet strangely enough, Arch Linux does not really want users to use Avahi to manage mDNS. Disabling systemd-resolved.service and configuring Avahi caused my devices to disconnect from WiFi and generate quite a lot of error messages in systemd's journal, forcing me to permanently switch to resolved.
Both articles so have seen rather little activity but there's something odd about the edit histories of many articles. A lot of Arch Wiki entries show a near-identical edit history in terms of edit dates; a lot of activity in 2014, followed by a noticeable decline; a sudden uptick in 2018, only to decline again in 2019; a huge increase in edits right around the onset of the COVID-19 pandemic and a significant decrease towards the end of 2022, with little activity from 2023 onward. Ironically, this means that the Steam Deck, which runs on Arch Linux, did not result in more people wanting to contribute to Arch, in fact it appears to be declining further, which is particularly noticeable among the non-English Wikis.
And let's not get into Talk pages now often being used as a secondary support forum while the Arch Wiki deletes old discussions on Talk pages randomly, rather than moving them to an archive. The same applies to specific user profiles and some edit histories.
Speaking of “talking”: The Arch Linux Forum
Oh boy, this will piss off a lot of fanboys but the forum demonstrates a very unevenly distributed userbase. Only four to five long-term users at best respond to support requests by dozens of largely newbies on a daily basis. If you're familiar with the forum, you'll immediately recognize the handful of users I'm referring to.
From the outside, it really makes Arch Linux look like a very small hobby project suffering massively from Eternal September. But once having looked deeper into this, it's the tone of the remaining users that just makes the contribution to the forum inherently uninviting. There's an utter distaste towards “help vampires”, which I do understand, however there's a difference between a help vampire even needing help when moving the mouse cursor and a new user merely suffering from “information overload” (which happened to me after reading the horrendous manpage for iwd, which I still can't tell whether it was simply broken when I read it or Intel is being Intel). A lot of users do apologize when having missed a certain line on the Wiki or a manpage, in fact I haven't seen a single user fitting the description of a help vampire – and those that may resemble a help vampire at first often also demonstrate poorer English skills (and, unsurprisingly, those people are from countries like Turkey/Türkiye).
Actually, some of the remaining forum users really do fit the stereotype of the “average Linux user that cannot take any criticism for shit”.
https://bbs.archlinux.org/viewtopic.php?id=287289
This thread alone tells me that many Arch Linux users... really only want to flex for the sake of looking like some sort of hot, smooort stuff. None of those guys are special or even geniuses for being able to install Arch and being familiar with OS installations via a CLI. As much as I like Pacman and some manual ways of doing things but Arch users that get offended like some of those guys in this thread really should go outside and touch some grass for once. They blame the inconsistencies of Arch's own philosophy on end users because “they have to be the idiots for not understanding Arch”. It's a plain logical fallacy to deflect any sort of responsibility and the very fact that Arch Linux is a public project (seriously if criticism triggers you so hard, then make Arch a private project or stop contributing; don't host it online for anyone willing to contribute if even the tiniest bit of valid criticism is being interpreted like a personal attack on your very life).
Arch pulls untested developer builds for GRUB, then blamed end users and Arch children for its bugs
Roughly two years ago, Arch Linux pushed an update for the popular boot manager GRUB. Just shortly afterwards, Arch children such as EndeavourOS reported massive issues and immediately stopped the distribution of this faulty package whilst properly troubleshooting the issue. As it turned out, Arch Linux quietly switched to a testing branch of GRUB hosting an untested developer build of the boot manager, one which hasn't even been tested by the guys at GNU themselves. One of the two package testers that was responsible for the introduction of the untested dev build to end users initially did not see the issue with this, despite Arch Linux providing a separate testing branch where such dev builds can be tested before pushing them to the main repos. He later began to took the matter personally and blamed Arch children in particular for not checking their updates properly, highlighting the disapproval of Arch maintainers towards the bare existence of distributions based on Arch. This resulted in EndeavourOS announcing its migration to systemd-boot. Four days after the GRUB issue was first reported, Arch Linux issued a warning on their homepage, stating that users will have to do some manual intervention to fix the problem.
To this day, Arch Linux pulls GRUB builds from all branches and pushes dev builds and release candidates to its own stable branch, begging the question why Arch even provides testing repos in the first place. The responsible tester never apologized for having misjudged the situation, considering GRUB's popularity even among the majority of “vanilla Arch” users. He just lashed out and went quiet, nothing else.
TL;DR (finally)
This is a trait among Arch users in particular that I've come across an awfully lot throughout the years. They just can't admit their own faults and say “hey, you're right, I'm wrong, sorry”. It's that stubborn attitude that has driven a lot of competent contributors away over the years, which makes Talk pages on the Arch Wiki either look like an untouched wasteland or a forum replacement for that kind of newbie that doesn't know the difference between a Talk page on a Wiki and a support forum.
What personally pisses me off the most is their own interpretation of “KISS” and how they do not fully adhere to it themselves. What is Arch supposed to “keep simple”, if the majority of technical decisions never really see a proper discussion and contributors act according to their individual preferences alone and interpret the “Arch philosophy” differently?! If it weren't for Archcraft and EndeavourOS that provided me with a solid starting point just to get away from Windows without ending up with similar issues now affecting Ubuntu (my first choice that ultimately ran just as inefficiently as Windows 10), I wouldn't have give Arch Linux a chance at all and instead switched to a minimal Debian. With Gentoo – a move that surprised everyone within the Linux ecosystem – now providing a growing binary repository alongside its source-based repos, is there even a point in sticking with the more-restrictive, binary-focused Arch that aims to fully switch to the systemd ecosystem and pushes configurations only benefiting a certain set of devs?!
I still run Arch Linux and I still love Pacman and the AUR. The recent Makepkg issue even kept me distracted while my dentist did his job. But it's just another Linux distribution, nothing special. You can easily do anything you can do on Arch on and with any other distribution and may even receive better support.
I try to avoid the rest of its ecosystem, especially its community, and now even consider to finally give Gentoo a proper test, now that my main issue with the source-based-only approach no longer is a problem.
Note #1 (18 March): Fixed some typos and clarified the “help vampire” section a little. While I was mitigating a Fluxbox bug that generates broken config files (most notably keys, which resulted in Vim's syntax highlighting failing and all letter and number keys to behave randomly), I learned that each part of the Arch Linux ecosystem pretty much operates on its own. Anything regarding the Arch Wiki happens on the Arch Wiki (despite the forum offering “Forum & Wiki discussions”; Wiki admins and maintainers pretty much never leave their Wiki); the Forum remains entirely separate from the sparsely-used mailing lists and IRC channels; the devs hardly ever use anything outside Gitlab since the migration (even their dev-focused mailing lists see merely one largely-ignored email every once in a blue moon, while you're somewhat expected to keep track of often-cryptic package commits); and even Pacman is considered to be entirely separate from Arch Linux by one of its devs:
Speaking of Gitlab: Has anyone else noticed that the devs still claim “spam issues” on their Gitlab project page? It's been nearly a year since Arch Linux migrated to Gitlab and more than three months since the bugtracker also was moved to Gitlab, yet the notice still says that this is supposed to be a “temporary solution”. At this point, I highly doubt that they suffered from a significant amount of “spam” right after completing their migration and just want to make it harder to contribute (similarly to how Arch Wiki requires any first-time contributor to copy/paste and execute some command in a terminal to prevent “spam”). This also is indicated by the news not mentioning anything regarding “temporary spam mitigation” (a mere pointer to the notice on said Gitlab page doesn't count).