The Linux way; never ever break user experience

Through the years it has become more and more obvious to me that there’s two camps in open source development, and one camp is not even aware of how the other camp works (or succeeds, rather), often to their own detriment. This was blatantly obvious in Miguel de Icaza’s blog post What Killed the Linux Desktop, in which he accused Linus Torvalds for setting the attitude of breaking API’s to the developer’s heart content without even realizing that they (Linux kernel developers) do the exact opposite of what he claimed; Linux never ever breaks user-space API. This triggered a classic example of many thorned discussions between the two camps, which illustrates how one side doesn’t have a clue of how the other side operates, or even; that there’s entirely different way of doing things. I will name the camps “The Linux guys” (even though they don’t work strictly on the Linux kernel), and “The user-space guys”, which is people that work on user-space, GNOME being one of the peak examples.

This is not an attempt to put people in two black-and-white categories, there’s a spectrum of behaviors, and surely you can find people in a group with the mentality of the other camp. Ultimately it’s you the one that decides if there’s a divide in attitudes or not.

The point of this post is to explain the number one rule of Linux kernel development; never ever break user experience, why that is important, and how far off the user-space camp is.

When I say “Linux” I’m referring to the Linux kernel, because that’s the name of the kernel project.

The Linux way

There are many exhaustive details of what makes the Linux kernel development work on a day-to-day basis, and many reasons for why it is the way it is, and why it’s good and desirable to work in that way. But for now I will simply concentrate on what it is.

Never ever break user experience. This point cannot be stressed enough. Forget about development practices, forget about netiquette, forget about technical competence… what is important is the users, and to never break their expectations. Let me quote Linus Torvalds:

The biggest thing any program can do is not the technical details of the program itself; it’s how useful the program is to users.

So any time any program (like the kernel or any other project), breaks the user experience, to me, that’s the absolute worst failure that a software project can make.

This is a point that is so obvious, and yet many projects (especially big ones), often forget; that they are nothing without their user-base. If you start a small project all by yourself, you are painfully aware that in order for your project to succeed, you need users. Without users you cannot get more developers, and your project could very well disappear from the face of the Earth, and nobody would notice. But once your project is big enough, one or two users complaining about something start being less of an issue, in fact, hundreds of them might be insignificant, and at some point, you loose any measure of what percentage of users are complaining, this problem might grow to the point that developers say “users don’t know what they want”, in order to ignore the importance of users, and their needs.

But that’s not the Linux way; it doesn’t matter if you have one user, or ten, or millions, your project still succeeds (or fails) because of the same reason; it’s useful to some people (or not). And if you break user experience, you risk that usefulness, and you risk your project being irrelevant for your users. That is not good.

Of course, there are compromises, sometimes you can do a bit of risk analysis: OK; this change might affect 1% of our current users, and the change would be kind of annoying, but it would make the code so much more maintainable; let’s go for it. And it’s all about to where you draw the line. Sometimes it might be OK to break user experience, if you have good reasons for it, but you should really try to avoid it, and if you go forward, provide an adjustment period, a configuration for the old behavior, and even involve your users in the whole process to make sure the change is indeed needed, and their discomfort is minimized.

At the end of the day it’s all about trust. I use project X not only because it works for me, but because I trust that it will keep working for me in the years to come. If for some reason I expected it to break next year, I might be better off looking for something else right now that I trust I could keep relying on indefinitely, than having project X break on me while I’m on the middle of a deadline, and I don’t have time for their shenanigans.

Obvious stuff, yet many project don’t realize that. One example is when the udidks2 project felt they should change the address of the mount directories from `/media/foo`, to `/run/media/$user/foo`. What?! I’m in the middle of something important, and all of a sudden I can’t find my disks’ content in /media? I had to spend a considerable amount of time until I found the reason; no, udisks2 didn’t had a bug; they introduced this change willingly and knowingly. They didn’t give any deprecation warning while they moved to the new location, they didn’t have an option to keep the old behavior, they just moved it, with no explanation, in one single commit (here), from one version to the next. Am I going to keep using their project? No. Why would I? Who knows when would be the next time they decide to break some user experience unilaterally without deprecation warnings or anything? The trust is broken, and many others agree.

How about the Linux kernel? When was the last time your Linux kernel failed you in some way that it was not a bug, but that the developers knowingly and willingly broke things for you? Can’t think of any? Me neither. In fact, people often forget about the Linux kernel, because it just works. The external drivers (like NVIDIA or AMD) is not a problem of the kernel, but the drivers themselves, and I will explain later on. You have people bitching about all kinds of projects, and threatening forks, and complaining about the leadership, and whatnot. None of that happens with the Linux kernel. Why? Because it just works. Not for me, not for 90% of the users, for everybody (or 99.99% of everybody).

Because they never ever break user experience. Ever. Period.

The deniers

Miguel de Icaza, after accusing Linus not maintaining a stable ABI for drivers, went on arguing that it was kernel developers’ fault of spreading attitudes like:

We deprecated APIs, because there was a better way. We removed functionality because “that approach is broken”, for degrees of broken from “it is a security hole” all the way to “it does not conform to the new style we are using”.

What part of “never ever break user experience” didn’t Icaza understand? It seems he only mentions the internal API, which does change all the time in the Linux kernel, and which has never had any resemblance of a promise that it wouldn’t (thus the “internal” part), and ignoring the public user-space API, which does indeed never break, which is why you, as a user, don’t have to worry about your user-space not working on Linux v3.0, or Linux v4.0. How can he not see that? Is Icaza blind?

Torvalds:

The gnome people claiming that I set the “attitude” that causes them problems is laughable.

One of the core kernel rules has always been that we never ever break any external interfaces. That rule has been there since day one, although it’s gotten much more explicit only in the last few years. The fact that we break internal interfaces that are not visible to userland is totally irrelevant, and a total red herring.

I wish the gnome people had understood the real rules inside the kernel. Like “you never break external interfaces” – and “we need to do that to improve things” is not an excuse.

Even after Linus Torvalds and Alan Cox explained to him how the Linux kernel actually works in a Google+ thread, he didn’t accept anything.

Lennart Poettering being face to face with both (Torvalds and Cox), argued that this mantra (never break user experience) wasn’t actually followed (video here). Yet at the same time his software (the systemd+udev beast) recently was criticized for knowingly and willingly breaking user experience by making the boot hang for 30s per device that needed firmware. Linus’ reply was priceless (link):

Kay, you are so full of sh*t that it’s not funny. You’re refusing to
acknowledge your bugs, you refuse to fix them even when a patch is
sent to you, and then you make excuses for the fact that we have to
work around *your* bugs, and say that we should have done so from the
very beginning.

Yes, doing it in the kernel is “more robust”. But don’t play games,
and stop the lying. It’s more robust because we have maintainers that
care, and because we know that regressions are not something we can
play fast and loose with. If something breaks, and we don’t know what
the right fix for that breakage is, we *revert* the thing that broke.

So yes, we’re clearly better off doing it in the kernel.

Not because firmware loading cannot be done in user space. But simply
because udev maintenance since Greg gave it up has gone downhill.

So you see, it’s not that GNOME developers understand the Linux way and simply disagree that’s the way they want to go, it’s that they don’t even understand it, even when it’s explained to them directly, clearly, face to face. This behavior is not exclusive to GNOME developers, udisks2 is another example, and there’s many more, but probably not as extreme.

More examples

Linus Torvalds gave Kay a pretty hard time for knowingly and willingly introducing regressions, but does Linux fares better? As an example I can think of a regression I found with Wine, after realizing the problem was in the kernel, I bisected the commit that introduced the problem and notified Linux developers. If this was udev, or GNOME, or any other crappy user-space software, I know what their answer would be: Wine is doing something wrong, Wine needs to be fixed, it’s Wine’s problem, not ours. But that’s not Linux, Linux has a contract with user-space and they never break user experience, so what they did is to revert the change, even though it made things less-than-ideal on the kernel side, that’s what was required so you, the user, doesn’t experience any breakage. The LKML thread is here.

Another example is what happened when Linux moved to 3.0; some programs expected a 2.x version, or even 2.6.x, these programs were clearly buggy, as they should check that the version is greater than 2.x, however, the bugs were already there, and people didn’t want to recompile their binaries, and they might not even be able to do that. It would be stupid for Linux to report 2.6.x, when in fact it’s 3.x, but that’s exactly what they did. They added an option so the kernel would report a 2.6.x version, so the users would have the option to keep running these old buggy binaries. Link here.

Now compare the switch to Linux 3.0 which was transparent and as painless as possible, to the move to GNOME 3. There couldn’t be a more perfect example of a blatant disregard to current user experience. If your workflow doesn’t work correctly in GNOME 3… you have to change your workflow. If GNOME 3 behaves almost as you would expect, but only need a tiny configuration… too bad. If you want to use GNOME 3 technology, but you would like a grace period while you are able to use the old interface, while you adjust to the new one… sucks to be you. In fact, it’s really hard to think of any way in which they could have increased the pain of moving to GNOME 3. And when users reported their user experience broken, the talking points were not surprising: “users don’t know what they want”, “users hate change”, “they will stop whining in a couple of months”. Boy, they sure value their users. And now they are going after the middle-click copy.

If you have more examples of projects breaking user experience, or keeping it. Feel free to mention them in the comments.

No, seriously, no regressions

Sometimes even Linux maintainers don’t realize how important this rule is, and in such cases, Linus doesn’t shy away from explaining it to them (link):

Mauro, SHUT THE FUCK UP!

It’s a bug alright – in the kernel. How long have you been a
maintainer? And you *still* haven’t learnt the first rule of kernel
maintenance?

If a change results in user programs breaking, it’s a bug in the
kernel. We never EVER blame the user programs. How hard can this be to
understand?

> So, on a first glance, this doesn’t sound like a regression,
> but, instead, it looks tha pulseaudio/tumbleweed has some serious
> bugs and/or regressions.

Shut up, Mauro. And I don’t _ever_ want to hear that kind of obvious
garbage and idiocy from a kernel maintainer again. Seriously.

I’d wait for Rafael’s patch to go through you, but I have another
error report in my mailbox of all KDE media applications being broken
by v3.8-rc1, and I bet it’s the same kernel bug. And you’ve shown
yourself to not be competent in this issue, so I’ll apply it directly
and immediately myself.

WE DO NOT BREAK USERSPACE!

The fact that you then try to make *excuses* for breaking user space,
and blaming some external program that *used* to work, is just
shameful. It’s not how we work.

Fix your f*cking “compliance tool”, because it is obviously broken.
And fix your approach to kernel programming.

And if you think that was an isolated incident (link):

Rafael, please don’t *ever* write that crap again.

We revert stuff whether it “fixed” something else or not. The rule is
“NO REGRESSIONS”. It doesn’t matter one whit if something “fixes”
something else or not – if it breaks an old case, it gets reverted.

Seriously. Why do I even have to mention this? Why do I have to
explain this to somebody pretty much *every* f*cking merge window?

This is not a new rule.

There is no excuse for regressions, and “it is a fix” is actually the
_least_ valid of all reasons.

A commit that causes a regression is – by definition – not a “fix”. So
please don’t *ever* say something that stupid again.

Things that used to work are simply a million times more important
than things that historically didn’t work.

So this had better get fixed asap, and I need to feel like people are
working on it. Otherwise we start reverting.

And no amount “but it’s a fix” matters one whit. In fact, it just
makes me feel like I need to start reverting early, because the
maintainer doesn’t seem to understand how serious a regression is.

Compare and contrast

Now that we have a good dose of examples it should be clear how the difference in attitudes from the two camps couldn’t be more different.

In the GNOME/PulseAudio/udev/etc. camp, if a change in API causes a regression on the receiving end of that API, the problem is in the client, and the “fix” is not reverted, it stays, and the application needs to change, if the user suffers as a result of this, too bad, the client application is to blame.

In the Linux camp, if a change in API causes a regression, Linux has a problem, the change is not a “fix”, it’s a regression and it must be reverted (or otherwise fixed), so the client application doesn’t need to change (even though it probably should), and the user never suffers as a result. To even hint otherwise is cause for harsh public shaming.

Do you see the difference? Which of the two approaches do you think is better?

What about the external API?

Linux doesn’t support external modules, if you use an external module, you are own your own. They have good reasons for this; all modules can and should be part of the kernel, this makes maintenance easy for everybody.

Each time an internal API needs to be changed, the person that does the change can do it for all the modules that are using that API. So if you are a company, let’s say Texas Instruments, and you manage to get your module into the Linux mainline, you don’t have to worry about API changes, because they (Linux developers), would do the updates for you. This allows the internal API to always be clean, consistent, relevant, and useful. As an example of a recent change, Russell King (the ARM maintainer), introduced a new API to set the DMA mask, and in the process updated all users of dma_set_mask() to use the new function dma_set_mask_and_coherent(), and by doing that found potential bugs in many instances. So companies like Intel, NVIDIA, and Texas Instruments, benefit from cleaner and more robust code without moving a finger, Rusell did it all in his 51 patch series.

In addition, by having all the modules on the same source tree, when a generic API is to be added, it’s easy to consider all possible use-cases, because the code is readily available. An example of this is the preliminary Common Display Framework, which takes into consideration drivers from Renesas, NVIDIA, Samsung, Texas Instruments, and other Linaro companies. After this framework is done, all existing display drivers would benefit, but things would be specially easier for future generations of drivers. It’s only because of this refactoring that the amount of drivers supported by Linux can grow without the amount of code exploding uncontrollably, which is one of the advantages Linux has over Windows, OSX, and other operating systems’ kernels.

If companies don’t play along in this collaborate effort, like is the case with NVIDIA’s and AMD’s proprietary drivers, is to their own detriment, and there’s nobody to blame but those companies. Whenever you load one of these drivers, Linux goes immediately into a tainted mode, which means that if you find problems with the Linux kernel, Linux developers cannot help you. It’s not that they don’t want to help, but it’s that they might be physically incapable. If a closed-source module has a bug and corrupts memory on the kernel side, there is no way to find that out, and might show as some other module, or even the core itself crashing. So if a Linux developer sees a crash say, on a wireless driver, but the kernel is tainted, there is only so much he can do before deciding it’s not worth his time to investigate this issue which has a good chance of being caused by a proprietary driver.

Thus if a Linux update broke your NVIDIA driver, blame NVIDIA. Or even better, don’t use the proprietary driver, use noveau.

Conclusion

Hopefully after reading this article it would be clear to you what is the number one rule of Linux kernel development, why it is a good rule, and why other projects should follow it.

Unfortunately it should also be clear that other projects, particularly those related to GNOME, don’t follow it, and why that causes such backlash, controversy, and forks.

In my opinion there’s no hope in GNOME, or any other user-space project, being nearly as successful as Linux if they don’t follow the simplest most important rule. Linux will always keep growing in importance and development power, and these others are forever doomed to forks, nearly identical alternatives, and their developers jumping ship after their trust gets broken. If only they would follow this simple rule, or at least understand it.

Bonus video

Tux

Advertisements

The problem with GNOME 3

Since I started using Linux I used GNOME, v1.2 in those times. It has always done what I needed, maybe not perfectly, and not fully, but for the most part. GNOME 3 changed all that.

I complained about GNOME 3 since day one, and I discussed with GNOME 3 developers many problems with their rationale about why what they were doing made sense, and I foresaw many of the problems they are now facing.

I even started the GNOME user survey, in an effort to make GNOME developers see the light.

I blogged once before about GNOME 3, but only superficially, and even then GNOME developers didn’t take it well. Some GNOME developers might be tired of my arguments in Google+ and other media, but it’s time to dive into the issues with GNOME 3 in this blog.

Two years after the first release of GNOME 3, once GNOME developers have had the time to polish the rough edges, it’s time to make the call; GNOME 3 sucks. Here’s why.

The purpose of GNOME

Before we start, we need to clarify the purpose of GNOME, or rather; the purpose of any software project in general, which is to be useful to the users. Your software might the most efficient at doing certain task, it might have the simpler code, and the best development practices, but it’s all for naught if it’s not useful to anybody.

Software has to be useful for other people, otherwise nobody will use your software, and nobody will contribute to your software. Quite likely there will always be other software that does something similar to yours, so you need to convince users that your software is better than the alternatives, usually by providing a good user experience.

Once the user has spent time analyzing the alternatives and has chosen your software, the user expects your software to keep working, and in the same way. If the software keeps changing the way it behaves from one version to the next, it’s not achieving it’s main goal; to be useful to the users. If you do this, your users will move on to another project that might not have as many features, but at least they can rely on it and they don’t have to spend any more of their valuable time learning what broke now in the new release.

More important than providing a good user experience, is to not break existing user expectations.

You would think this is as obvious as a leopard’s spots, yet many software projects, including GNOME, don’t seem to understand this fact.

No project is more important than the users of the project. — Linus Torvalds

What users want

The problems I’ll talk about are not my invention, but conformed by GNOME user surveys, which show the most important issues according to the users:

  1. Lack of configuration options
  2. Developers don’t listen to users
  3. Bring back traditional interface

I knew these were important issues myself before the survey, but a survey was needed in order to know for sure. Of course, even after the survey, GNOME developers reject these issues are important to their users.

Developer attitude

In my opinion the main problem with GNOME is not just the code, code can be fixed, but the attitude from the developers (which is reflected in the code), and as users made it clear in the survey.

Many open source projects would kill to have the user-base GNOME had, and welcome their input with open arms, but GNOME neglected their users, they thought they were irrelevant, and they tried to dismiss their complaints with typical defenses, which of course don’t make sense.

no-feedback

People hate new things

By far the most widely used defense for the changes in GNOME 3 is that some people are backwards, and always hate new things, so it’s no surprise that they hate GNOME 3.

Any rational person would see the problem with this claim, but unfortunately I must spell it, because I had to explain it at great length to GNOME developers, and based on the responses, one would think they are incapable of seeing hasty generalization fallacies.

Say 50% of users hate GNOME 3, how many of them have legitimate reasons, and how many hate it just because they always hate new things? It’s impossible to know. So to dismiss every complaint saying “people hate new things” is totally stupid; you cannot know beforehand if they belong to this group or not, you first need to hear their complaints, and then decide if the complaints are invalid or not.

Saying “Jon doesn’t have legitimate complaints” makes sense (after analyzing his complaints), saying “people hate new things” does not, and GNOME developers do the later when people complain about GNOME 3.

Users don’t know what they want

Really? People don’t know if they want a punch in the face or not? Some people might not know what they want, but they definitely know what they don’t want.

Moreover, that’s a very rudimentary notion of user choice. Malcolm Gladwell explains rather well how it’s true that people don’t know what they want, or rather; they can’t tell you what they want, but when they get a taste of it, they most definitely can tell you: this is what I want.

If it wasn’t clear by now; GNOME 3 is not what a lot of people wanted.

gnome-hate

People complained about GNOME 2 too

Did this actually happen? If you google GNOME 2 for results between 2002 and 2004 you would find barely (if any) results that show negative comments, so if it did happen, the backlash probably wasn’t that bad, and it probably died rather quickly. Check the thread in Slashdot, and try to find one comment suggesting to go back to GNOME 1; I couldn’t find any.

Did anybody create forks after GNOME 2 was released the same way people forked after GNOME 3? No.

Yes, people complained about GNOME 2, but not nearly as much as they do about GNOME 3. There’s even an entire wikipedia article dedicated to the controversy over GNOME 3.

 

Sorry GNOME developers, none of your excuses for not listening to your users make sense. You should always listen to your users. Without them your project is nothing.

Configuration

The biggest complaint users had was the lack of configuration options, which could have been solved very easily, if the developers had more than two neurons in their brain, but instead, there’s only sub-par alternatives.

Advanced mode

GNOME 3 got rid of a bunch of useful configurations users relied on, and to defend themselves they said you could use gsettings manually, or install (or write) an extension, and so on, but I argued that there should be an “advanced mode” that the user can optionally turn on in order to further configure the system the way they wanted.

GNOME developers argued that this was a bad idea, because more code would need to be maintained which is absolutely not true for gsettings; the configurations are already there in the code, the only thing missing is the UI. Then they argued that it would mess up the configuration UI, which they wanted to keep clean, but adding a single check box can’t possibly mess with anything. In the end, I never received a single satisfactory answer of why an advanced mode didn’t make sense.

Tweak tool

The tweak tool is the closest thing to a sane configuration interface, however, it should be cleaned up, distributed by default, and integrated in the GNOME control center by adding an “Advanced mode” checkbox that when switched on enables these settings which should be distributed among the different “capplets”.

Even then, the amount of configurations available is not enough.

Shell extensions

Finally, there’s shell extensions which sound like a great idea at first, but if you think about it for thirty seconds, you realize the problem; there’s no APIs for extensions. This means extensions bring many problems.

First, there’s no guarantee that an extension will work the next version of GNOME, because it’s modifying the JavaScript code directly, and there’s no guarantee that the code will remain the same. There’s no extension API like with Chrome, or Firefox. The result is not unexpected; extensions break very easily from one version of GNOME to the next.

The second issue is that extensions can easily break GNOME, because they can modify absolutely anything in the shell.

gsettings

Another option is to manually change the configurations through the command line. This is obviously not user-friendly, and not ideal either.

 

GNOME developers made the mistake of not allowing configurability directly into GNOME, and as a result they have made the much needed configurations second class citizens, than no matter what you do, they don’t work as expected. The problem is very easy to solve; the configurations should be integrated into GNOME, and activated by a single switch, so their normal configurations are not disturbed. But talking sense to these guys is next to impossible.

Classic mode

Finally, after so much backlash from the users for years, GNOME developers had to implement the classic mode, so users could be able to run their beloved GNOME 2 interface with GNOME 3 technology.

Unfortunately, GNOME classic is more of a hack than anything else; it’s full of bugs and inconsistencies. Sure, this mode is still in its infancy, and will probably improve in the next releases, but it’s safe to bet that it would never be as polished and usable as GNOME 2 was.

Sorry GNOME devs; too little, too late.

Suckage

I am not going to list every single reason why GNOME 3 sucks, I would never finish this post, and there’s plenty of people that have already added their share (I’ll list some of them).

Personally for me the most obvious way to see that GNOME 3 defaults are brain-dead is the way alt+tab works. The whole purpose of work-spaces is to separate work, instead of having all your browser windows, terminals, editors, email client, and so on, on the same work-space, you split them among different ones. Even their own design references for GNOME shell say so. So, to make alt-tab cycle through the windows in all works-paces instead of only the current one is idiotic; it undermines the whole purpose of work-spaces.

Instead of accepting this obvious fact (which many users complained about), they disregard any feedback, and say “oh, you can use an extension for that”. Never mind the fact that a huge percentage of their users need to install the extension just to have a sane behavior, and that the extension can break at any moment.

gnome-3-is

Another quite obviously stupid behavior is to switch to an already running “application” instead of opening a new one. For example; if I press the Windows key, and type chromium, I expect a new browser window to open, but no, instead I’m unexpectedly dragged to another work-space where there’s already a browser window. This makes sense on single-window applications, but not on other ones.

Linus Torvalds and others complained about this, and I proposed a solution; add a menu item when right clicking on the application that says “Always open new window”; problem solved. Now each time the user tries to open this application, it’s done in a new window. But no, users don’t know what they want; GNOME developers know better what’s best for us.

The list of stupidities in GNOME 3 is never ending:

gnome-3

The Linux way

What GNOME should have done is simple; don’t ever, ever break user experience. This is how the Linux project has managed to become the most successful software project in history. There are no forks of the Linux kernel, simply because there’s no need; user experience is never broken; what works in v2.0, works in v3.0, and will work in v4.0. New features are added carefully, making sure nothing breaks, and users always have a way to make the system work the way they expect it to work.

GNOME shell should have been developed alongside GNOME 2 and users should have given the option to turn this mode on and off. Slowly, GNOME shell would get more features from GNOME 2, until it was a full replacement, and at the same time GNOME 2 would evolve to integrate GNOME 3 technologies (like GTK+ 3.0). Eventually GNOME shell becomes the default, but always with the option to use the classic interface. After some cycles, the GNOME shell interface is bound to be a full replacement for the classic one.

Unfortunately, they chose not to do this.

Going this way requires more effort, of course, but it’s the only way to move forward without loosing developers and users along the way. GNOME developers argued that they didn’t have the resources to do such careful move, but that is clearly false, as the developers working on MATE, Cinnamon and GNOME classic show; there’s developers interested in making things work for the traditional mode.

They chose not to listen to the warnings, and they became impatient and went forward with a move that had the potential of angering a lot of users. Well, this potential was fully realized, and now they are paying the consequences.

They were wrong

They argued this was like GNOME 2; the people complaining about it will eventually learn the new ways, and most will be happy. They were wrong.

Even to this day people keep complaining about GNOME 3, how the interface doesn’t make sense, how the developers don’t listen, and how the design is brain-dead.

There’s no other way to put it; GNOME 3 was a mistake.

msn-pecan 0.1.4 released

Hi,

Long time no see! I’ve been busy with many other things and haven’t had time to work on msn-pecan, but I’m back.

This a minor bugfix release, mostly to fix issues while reconnecting. These will be useful for HTTP non-persistent connections which is a work in progress. There’s plenty many other patches pending in the queue, some you can try on the master branch.

Felipe Contreras (6):
      node: avoid writes when connection is closed
      ssl-conn: cleanup properly on errors
      io: properly cancel connect operations
      build: fix platform detection
      win32: update instructions for Arch Linux
      win32: version bump

Download from the (usual place).

Top 10 suggestions from the 2011 GNOME user survey

I’ve taken the time to read one by one the suggestions from the 2011 GNOME user survey, I’ve only managed to read 20% of them so far, but I don’t think they will shift that much (I might update this post if they do).

The next version of the survey will have these and other suggestions already listed, so it would be easier for users’ voice to be heard, and summarize the results.

Note: I already posted most of these on my previous post, but apparently people didn’t like my critical tone, so these are the suggestions only, not my opinion.

1. Better customization

This is by far the most requested, people don’t want to manually fiddle with gconf/dconf, extensions, or gnome-tweak, they want a whole lot more options integrated. One suggestion was to have an advanced mode.

Not only do people want more options, but they think some of the current ones are useless, and that defaults are all wrong. They also want more options for power management, like deciding what happens when you close the lid. Also more options to change the appearance: font, icons, keyboard bindings, screensaver, etc. Also, disable the accessibility stuff.

2. GNOME 2

People love their GNOME 2. A lot of them suggested to have two interfaces: the traditional one, and the shell, others requested to improve the fallback mode, but most of them demanded to get rid of GNOME shell completely. A lot of people also asked specifically to bring back the GNOME 2 panel, and taskbar.

3. Improve performance and footprint

A lot people think GNOME is too bloated and asked for better performance, less CPU usage, less memory usage, smoother animations, faster start-up times, etc. Specially the ones that have used GNOME shell, which apparently is a resource hog, or at least that’s what users say.

4. Nautilus

People are definitely not happy with nautilus. They complain it’s too slow, takes too much time to start, and lacks a lot functionality. Many suggested to get rid of it and use some of the already existing alternatives.

5. Notifications

The new notifications are annoying; one shouldn’t need to move the mouse cursor the corner to see them; that makes it easy to miss them, which defeats the purpose of a notification.

6. Shutdown / Restart / Suspend

This is a no-brainer; just add this option. Yes, you can see them by pressing “alt” but people don’t want that, there’s no excuse to remove basic functionality, and users are complaining hard for this particular feature.

7. Improve theming

Users want better themes and icons, they don’t like the default theme, also want a dark theme, and think the amount of customization a theme can achieve is not enough.

8. Better multi-monitor support

Another very requested feature was to improve multi-monitor support, which apparently works very badly in GNOME shell.

9. Improve Evolution

A considerable amount of people suggested to get rid of Evolution completely, because it has so many problems, and because there are better alternatives (i.e. Thunderbird). Some people only want to use the calendar part of it, because it’s integrated to other parts of GNOME’s UI, so they suggested to split this component. The rest suggested many ways to improve it, like stop the constant crashing.

10. Listen to users

Lastly, a lot of people think that GNOME developers are ignoring users, and their suggestions. Also, they don’t like the developers’ attitude that users are idiots, and developers/designers know best.

The problem

As #10 shows, the problem is that GNOME developers don’t listen to the users, or at least that’s the perception from a lot of users, and since the community rejected the idea of running and/or blessing this survey, it would be safe to assume that they would ignore the results. The story of this survey is rather interesting, and explains a lot, but you can read about it through Bruce Byfield’s mouth instead of mine in his article ‘The Survey That GNOME Would Rather Ignore‘.

Even before the survey was run, GNOME developers said the results would be worthless, however, the only valid criticism was the possibility of non-response bias, which fortunately didn’t happen, and I explained why in my analysis of the survey results. I still haven’t received any comments from GNOME developers after I publicized the analysis of the results, so if there’s any valid criticism remaining, I still haven’t heard of it.

I wish they would look at these results critically, not outright reject them, and if there’s any problem with the survey, tackle the problems for the 2012 survey, and even more: run the survey themselves. Wouldn’t that be great?

 Other suggestions

  • Reduce dependencies (specially PulseAudio)
  • Improved reliability / stability
  • Minimize / Mazimize
  • Tiling
  • alt+tab
  • Faster shell search
  • Collaborate more with other communities
  • Fix ATI fglrx issues
  • Integrate Zeitgeist
  • Render KDE apps seamlessly
  • Reduce dead space
  • Compiz compatibility

Analysis of the 2011 GNOME user survey results

Last year Phoronix launched the 2011 GNOME user survey, an effort I initiated to try to give a voice to GNOME users, a voice I though was sorely lacked. I knew the effort wasn’t going to be accepted by GNOME developers, because through the years they have always rejected anything that resembles user feedback: surveys, polls, brainstorm, or something as easy as enabling voting in bugzilla. I gave it a try anyway because I made a bet with Zeeshan Ali that this would happen, and it did (although he never accepted that).

However, in the process the effort received a lot of positive feedback and the survey improved quite a lot, it would have been a shame to throw it to the trash, so, as Alan Cox suggested, I tried to make this survey happen anyway and thanks to Michael Larabel from Phoronix, it did.

According to GNOME people, such a survey, or in fact, any survey will always give bad results, because it’s impossible to get rid of all the biases. This was discussed extensively in the GNOME desktop-devel mailing list, and despite all my efforts to convince them otherwise, I didn’t succeed (which wasn’t a surprise at all). To give an idea of the sort of resistance we are talking about, I made the argument that even president elections across the globe would have similar problems, and they argued that even those are flawed. One would have to wonder what method in their minds would give proper statistically significant user feedback, but the answer is not surprising: nothing. Yet, they claim they listen to users, somehow. Although their method of “listening to users” remains mysterious, it probably involves personal experience: talking with their friends and family, and close circles (which are more biased than any survey could).

But in this discussion there was valid criticism, mainly the non-response bias. It was argued that only people that hate GNOME would answer the survey, and thus the survey would be biased towards negative comments, and the results worthless. There is some truth to that claim, and the survey tried to identify and mitigate that factor, with success I think. I will start by explained how this was done, because otherwise the survey would indeed be worthless for many purposes, which is not the case. Other arguments against the results of the survey are invalid in my opinion, but unlike other blogs from GNOME developers, this blog has comments open, so if you disagree and find other valid criticism to this survey, you are free to comment below.

Nevertheless, the most important point that GNOME people missed, is that this was a survey v1.0, and like all open source software, it can (and will) improve. Even if they are correct (they aren’t), and the results of v1.0 is totally worthless, that doesn’t mean that v2.0, or later versions, will. Unfortunately in their mind neither v1.0, nor v100.0 can provide any helpful results at all, or for that matter any other survey in the history of humanity.

I will present the rationale of why I think these results are good and valid, and I think any rational person would agree, but you would be the judge of that.

If you want to see the straight results of the survey, go to this Phoronix page.

Non-response bias

Let’s invent a scenario where we have a population of 1000 people, and we want to figure out how many people like shopping. To do that we sample 100 people and it turns out around 25% of them say they like shopping. That’s it, right? Job done. Well, what happens if most of people interviewed where males, say 2/3 males, 1/3 females. We know from other sources that we should expect the male/female ratio to be around 50/50, so we know there was a disproportion, but is that a problem? If it turns out that say, females have a bias towards liking shopping, then yes, there is a problem.

If we find results like that, we know there is bias, but we can use our knowledge about the male/female ratio expected in the total population to calculate what would be the real amount of people that like to shop in the total population. But as GNOME people were quick to mention, we don’t have a census of Linux desktop users, so we don’t have any idea of the proportions we might expect for different biases. In the example above, if we don’t know the male/female ratio of the total population, then we have no idea if 33/66 was disproportionate or not, all we know is that there is bias. And that’s the end of the game, the 25% our survey returned is worthless. We can say how many females and males like shopping, but nothing about the total population. I wouldn’t call this totally worthless, but GNOME people would.

But that is only if there is a bias. If it turns out that males and females like shopping at roughly the same proportion, then it doesn’t matter what is the ratio of the total population, and it doesn’t matter if we are being disproportionate or not. All we can do is identify the bias, if we don’t even ask the question “What is your sex?”, then we are truly lost, because we can’t even know if there was bias or not. So all we can do is try to identify all possible sources of bias, and then check if this bias is indeed happening or not.

Even worst is what happens if no females participate in the survey at all. That is highly unlikely in this case, but consider online surveys; people without Internet would be unlikely to participate, and if they have a bias then the results would be skewed. This was also used by GNOME people as an argument against any kind of online survey.

However, I had an idea; what if we ask people to print the survey and give it to these kind of people without Internet. This in effect would be no different from a survey that is not online, the only difference is that instead of paying people to go around finding GNOME users, we ask the community to do so — crowd-sourcing the problem. We didn’t ask this directly, perhaps for v2, but I added a question to identify how people are filling the survey: on their own, on behalf of somebody else, and so on.

We did spend a considerable amount of time trying to identify all these possible biases, and then add questions in the survey to see if there’s bias or not, and that’s all anybody can do.

So all the criticism GNOME people threw was taken care one way or the other, even if they didn’t agree so. The only way the results of this survey would be worthless is if somebody comes up with a possible bias that no question in the survey would identify (we need a new survey), or if it turns out there’s a real bias (we need a census).

Was there bias?

Let’s examine the main claim of GNOME people; people that answer the survey would have a bias towards hating GNOME.

Overall, how satisfied are you with GNOME?

Barely, Completely, Halfway, Mostly, Not At All

How are you taking this survey?

I am acting on behalf of somebody else, Completely on my own, Other, Somebody is pushing for me to do it

As you can see irrespective of the way people answered the survey, there’s a clear tendency: most people avoided the extremes and answered primarily “Mostly”, and then “Halfway”. The people that answered “I am acting on behalf of somebody else” went more for the extremes, but the difference is not significantly different from the people that answered “Completely on my own”. So, fortunately it seems GNOME people were wrong; there’s no bias.

Just to be sure, lets look at another, similar question:

How do you compare your current GNOME version with the version from one year ago?

Better, No Changes, Cannot Say, Worse

Again we see no significant differences depending on how people answered the survey; most people said either better, or worse at about the same rate. The people that answered on behalf of somebody else had a tendency to answer “Better”, and the people that got pushed, answered “Worse” a bit more, but in general no big difference.

Clearly, if there is a bias, it’s not dependent on how people are answering the survey. There’s another possibility; we didn’t get enough proxy responses. Statistically speaking, we would expect the results of “I am acting on behalf of somebody else” to be ±18% off target, but “Somebody is pushing for me to do it” only ±8, based on the amount of responses with a 95% certainty. So it would be nice if we get more of these responses on the next survey, but we can be relatively certain that if there is a bias, it’s not that great. Certainly not enough to declare the whole survey worthless.

I did similar analyses on the different questions of the survey, and I couldn’t find any significant biases from groups that could be underrepresented, therefore, the results of the survey are valid.

However, there are certain interesting results.

Interesting results

This is the same improvement question, but depending on whether or not people used GNOME 3.

How do you compare your current GNOME version with the version from one year ago?

Better, No Changes, Cannot Say, WorsePeople that haven’t used GNOME 3 mostly say that it hasn’t changed, but the ones that have used it have very different opinions, which matches what we have seen in online discussions: people either hate it or love it, but it’s interesting to see that the split is 50/50.

Another theory that popped in the discussions was that people that use terminals would hate GNOME 3, but “normal people” would love it.

Overall, how satisfied are you with GNOME?

Barely, Completely, Halfway, Mostly, Not At All

How often do you use a terminal/console?

Is there anything else?, When I have no other option, I can’t live without them, What is that?

However, it seems to be the opposite: people that don’t even know what a terminal is certainly don’t like GNOME, it’s the rest that do: from people that use it when there’s no other option, from people that don’t use anything else.

One of my theories was that contributors to the project would have a bias toward liking it.

Overall, how satisfied are you with GNOME?

Barely, Completely, Halfway, Mostly, Not At All

Have you contributed to the GNOME project?

No, Yes

But it appears I was also wrong; contributors to the project have roughly the same opinion as the non-contributors.

There is a much more clear differentiator tough: people that have tried to contact the project, unsuccessfully.

Overall, how satisfied are you with GNOME?

Barely, Completely, Halfway, Mostly, Not At All

Have you ever contacted the GNOME team?

No, I don’t know how; No, never had the need; Yes, successfully; Yes, unsuccessfully

The people that have tried to contact the project unsuccessfully tend to say either they are barely satisfied, or not at all with GNOME. Of all the people that have contacted the project 2/3 of them say it was unsatisfactory. This should be worrying.

And then, people that like to use another window manager.

Overall, how satisfied are you with GNOME?

Barely, Completely, Halfway, Mostly, Not At All

Have you ever contacted the GNOME team?

Yes, successfully; Yes, unsuccessfully; No, I don’t know how; No, never had the need

Clearly GNOME has problems interacting with other window managers. Or at least people seem to think so.

What needs to change?

Unfortunately there was no easy way to ask GNOME users this question so it was more or less open form, and that’s very difficult to analyze, however, I’ve taken the time to read them one by one, and count them, and so far I have 20% of the 10000 responses, but I doubt reading the rest will change the results much. I will try to do so in the future, as time permits, but I don’t promise much.

Better customization (397)

This is by far the most requested, people don’t want to manually fiddle with gconf/dconf, extensions, or gnome-tweak, they want a whole lot more options integrated. One suggestion was to have an advanced mode, which is something I have suggested in the past, but GNOME developers are adamant against it for no rational reasons.

Not only do people want more options, but they think some of the current ones are useless, and that defaults are all wrong. They also want more options for power management, like deciding what happens when you close the lid. Also more options to change the appearance: font, icons, keyboard bindings, screensaver, etc. Also, disable the accessibility stuff.

GNOME 2 (113)

People love their GNOME 2. A lot of them suggested to have two interfaces, others requested to improve the fallback mode, but most of them demanded to get rid of GNOME shell directly. A lot of people also asked to bring back the GNOME 2 panel, and taskbar.

Improve performance and footprint (89)

A lot people think GNOME is too bloated and asked for better performance, less CPU usage, less memory usage, smoother animations, faster start-up times, etc. Specially the ones that have used GNOME shell, which apparently is a resource hog.

Nautilus (57)

People are definitely not happy with nautilus. They complain it’s too slow, takes too much time to start, and lacks a lot functionality. Many suggested to get rid of it and use some of the already existing alternatives.

Notifications (46)

The new notifications are annoying; one shouldn’t need to move the mouse cursor the corner to see them; that makes it easy to miss them, which defeats the purpose of a notification.

Shutdown / Restart / Suspend (44)

This is a no-brainer; just add this option. Yes, you can see them by pressing “alt” but people don’t want that, there’s no excuse to remove basic functionality, and users are complaining hard for this particular feature.

Improve theming (32)

Users want better themes and icons, they don’t like the default theme, also want a dark theme, and also think the amount of customization a theme can achieve is not enough.

Better multi-monitor support (32)

Another very requested feature was to improve multi-monitor support.

Others (in order)

  • Improve Evolution
  • Listen to users
  • Reduce dependencies (specially PulseAudio)
  • Improved reliability / stability
  • Minimize / Mazimize
  • Tiling
  • Get rid of Evolution (or split the calendar component
  • alt+tab
  • Faster shell search
  • Collaborate more with other communities
  • Fix ATI fglrx issues
  • Integrate Zeitgeist
  • Render KDE apps seamlessly
  • Reduce dead space
  • Compiz compatibility
  • Developer attitude

Conclusion

What is clear is that most people want more configuration options, a lot liked GNOME 2 much better than GNOME 3, and they want an option to have an interface similar to GNOME 2 with GNOME 3 technology. For every user that likes GNOME 3, there is one that hates it. There’s plenty of people that think they are ignored by GNOME developers, and that users are treated like idiots (and they don’t like that). Users want a better attitude from the developers, not only toward users, but towards other FOSS projects like KDE (GTK+ apps like fine under KDE, Qt apps don’t under GNOME), and less of the not-invented-here syndrome.

Hopefully it has become clear to rational people that there is some value in the results of this survey, even if GNOME developers reject it. All the known biases were identified, and fortunately the ones that could cause non-response bias didn’t do so, so the results are valid and nobody was underrepresented 🙂

The next survey will incorporate some of the findings here, and hopefully more people would be aware of the need to reach people that normally wouldn’t answer this survey. Perhaps at some point GNOME developers would accept that there is no significative non-response bias because of that and start listening to the results.

gst-av 0.6 released; more reliable

gst-av is a GStreamer plug-in to provide support for libav (fork of FFmpeg), it is similar to gst-ffmpeg, but without GStreamer politics, which means all libav plugins are supported, even if there are native GStreamer alternatives; VP8, MP3, Ogg, Vorbis, AAC, etc.

This release takes care of a few corner-cases, and has support for more versions of FFmpeg.

Here are the goods:
http://code.google.com/p/gst-av/downloads/list

And here’s the short-log:

Felipe Contreras (19):
      adec: flush buffer on EOS
      adec: improve timestamp reset
      adec: avoid deprecated av_get_bits_per_sample_fmt()
      adec: avoid FF_API_GET_BITS_PER_SAMPLE_FMT
      vdec: properly initialize input buffer
      parse: add more H.264 parsing checks
      parse: fix H.264 parsing for bitstream format
      get_bits: add show_bits function
      build: set runpath for libav
      vdec: fix potential leaks
      vdec: use libav pts stuff
      vdec: get delayed pictures on eos
      build: trivial improvements
      parse: trivial fix
      h264enc: fix static function
      vdec: add support for old reordered_opaque
      adec: add support for old sample_fmt
      adec: add support for really old bps()
      adec: add support for all MPEG-1 audio

Mark Nauwelaerts (1):
      parse: be less picky regarding some reserved value

EFI adventures

If you are like me, you probably have heard of EFI, but don’t know what it means in practice, how to eat it, or what to mix it with. But I recently bought a new PC, and it happened to come with EFI, so I decided to enable it and then I learned many things about it.

Whole new world

First of all, EFI is not something just can just “enable”. Even if your kernel has support for EFI, that won’t make a difference unless you boot it through EFI, and you can’t seamlessly boot through EFI unless you create a boot entry, which is done with efibootmgr, which only works once you are in EFI mode. So it’s a bit of a chicken-and-egg problem.

So how do you enter this world in the first place? First of all, you would need a shell. Some BIOSes come with one, but otherwise you would need to install one, and you can find public links on this wiki page.

But where to put this “shell”? Well, the EFI system partition–yes, you need a dedicated partition (more info here). This partition should be formatted using FAT32, and marked in a special way, so the EFI BIOS would be able to identify it, and use it’s contents.

You copy the shell to this partition on /shellx64.efi (I tried UEFI Shell 2.0 (Beta), but it didn’t work for me, so I used the old one).

Your BIOS should now be able to boot this shell… Somehow. It took me a long time to figure out that in my BIOS, that’s a hidden option on the exit menu, where you can “exit” into a shell.

Congratulations, now you are in EFI mode, and probably type some commands, like in the old DOS days, but not be able to do anything really (unless you have some EFI version of Prince of Persia).

The bootloader

There are a few EFI bootloaders out there, like ELILO, and efilinux, but I decided to go with the safest and familiar choice: GRUB2 (well GRUB was familiar, GRUB2 not so much). Arch Linux’s wiki describes in great detail how to setup GRUB2 for EFI, so I’ll just mention that you need to use a different command (‘grub_efi_x86_64-install’ in the case of Arch Linux), and the files need to be installed in the aforementioned EFI system partition.

You would end up with a file /efi/arch/grubx64.efi, and this is an EFI binary that you can use in the shell. Theoretically you should be able to enter the command “\efi\arch\grubx64.efi”, but for some reason that didn’t work for me, so I had to “cd \efi\arch”, and then “grubx64.efi”. Of course, you first need to choose the right partition, but the shell makes it easier by having aliases to the EFI system partitions, so you first type “fs0:” (or something like that), like I said; very similar to DOS.

Directly, please

But going into the shell and typing those commands every time you boot is cumbersome, which is why you would want to add a boot entry. So, once you have been able to boot Linux through EFI, you should be able to load the ‘efivars’ module, and thus access ‘/sys/firmware/efi/vars/’, which is what ‘efibootmgr’ needs.

Then you can do something like:

efibootmgr --create --gpt --disk /dev/sdX --part Y --write-signature --label "Arch Linux (GRUB2)" --loader '\efi\arch\grubx64.efi'

WARNING: You shouldn’t use this on Mac’s, there’s a different process for those.

In your BIOS you should see the option of booting into “Arch Linux (GRUB2)”, like you have the option of booting into a CD-ROM, and then you can choose that as the first one in the boot order 🙂

Stub

But wait a second, that starts to look like two bootloaders: the BIOS has a boot entry, and so does GRUB. Why not boot directly into Linux?

A couple of patches from Matt Fleming enable just that. They should be in version 3.3. Enable CONFIG_EFI_STUB, and copy bzImage to /efi/linux/linux.efi.

So now you can boot directly from the BIOS into Linux (or Windows) with no bootloader at all 🙂

Update: echo "initrd=\efi\arch\initramfs.img root=/dev/sda3 ro quiet" | iconv -t ucs2 | efibootmgr --create --gpt --disk /dev/sda --part 1 --label "Arch Linux" --loader '\efi\arch\vmlinuz.efi' --append-binary-args -

GPT

While you are on that, why not enable this new partitioning scheme? It’s more EFI friendly, and you can’t actually install a version of Windows that works with EFI without it. For Windows 7 it’s either EFI and GPT, or non-EFI and MBR; you can’t mix them (don’t ask me why).

Fortunately it’s very easy to switch from MBR to GPT; just run gdisk (from a rescue disk, of course). It’s also easy to switch back, as long ad you haven’t used more than four partitions.