The meaning of success

I once had a quite extensive discussion with a colleague about several topics related to the software industry, and slowly but methodically we reached a fundamental disagreement on what “success” means. Needless to say, without agreeing on what “success” means it’s really hard to reach a conclusion on anything else. I now believe that many problems in society — not only in the software industry — can be derived from our mismatches in our understanding of the word “success”. It is like trying to decide if abortion is moral without agreeing on what “moral” means — and we actually don’t have such an agreement — and in fact, some definitions of morality might rely on the definition of “success”.

For example: which is more successful? Android? iPhone? or Maemo? If you think a successful software platform is the one that sells more (as many gadgets geeks probably do), you would answer Android, if on the other hand you think success is defined by what is more profitable (as many business people would do), you would answer iPhone. But I contend that success is not relative only relative to a certain context; there’s also an objective success that gives a clear answer to this question, and I hope to reach it at the end of this post.

This not a meta-philosophical exercise, I believe “success” in the objective sense can be clearly defined and understood, but in order to do that, I would need to show some examples and counter-examples in different disciplines. If you don’t believe in the theory of evolution of species by natural selection, you should probably stop reading.

Definition

The Merriam-Webster dictionary defines success as (among other definitions):

  • a : to turn out well
  • b : to attain a desired object or end <students who succeed in college>

From this we can say there’s two types of success; one is within a specific context (e.g. college), and the other one is in general. In this blog post I will talk about generic success, with no specific goal, or rather with the generic ultimate goal. Since it’s highly debatable (and difficult) how to define this “ultimate goal”, I will concentrate on the opposite; to try to define the ultimate failure in a way that no rational person would deny it.

Humans vs. Bacteria

My first example is: what organisms are more successful? Humans, or bacteria? There are many angles in which we could compare these two organisms, but few would reach a definite answer. The knee-jerk reaction of most people would be to say: “well, clearly humans are more evolved than bacteria, therefore we win”. I’m not an expert in the theory of evolution, but I believe the word “evolved” is misused here. Both bacteria and humans are the results of billions of years of evolution, in fact, one could say that some species of bacteria are more evolved because Homo Sapiens is a relatively new species and only appeared a few hundred thousands years ago, while many species of bacteria have been evolving for millions of years. “Kids these days with their fancy animal bodies… I have been killing animals since before you got out of the water… Punk” — A species of bacteria might say to younger generations if such a thing would be possible. At best humans are as evolved as bacteria. “Primitive” is probably the right word; bacteria is more primitive because it closely resembles its ancestors. But being primitive isn’t necessarily bad.

In order to reach a more definitive answer I will switch the comparison to dinosaurs vs. bacteria, and come back to the original question later. Dinosaurs are less primitive than bacteria, yet dinosaurs died, and bacteria survived. How something dead can be considered successful? Strictly speaking not all dinosaurs are dead, some evolved into birds, but that’s besides the point; let’s suppose for the sake of argument that they are all dead (which is in fact how many people consider them). A devil’s advocate might suggest that this comparison is unfair, because in different circumstances dinosaurs might have not died, and in fact they might be thriving today. Maybe luck is an important part of success, maybe not, but it’s pointless to discuss about what might have been; what is clear is that they are dead now, and that’s a clear failure. Excuses don’t turn a failure into a success.

Let me be clear about my contention; anything that ceases to exist is a failure, how could it not? In order to have even the smallest hope of winning the race, whatever the race may be, even if it doesn’t have a clear goal, or has many winners; you have to be on the race. It could not be clearer: what disappears can’t succeed.

Now, being more evolved, or less primitive, is not as a trump card as it might appear; nature is a ruthless arena, and there are no favorites. The vast majority of species that have ever lived are gone now, and it doesn’t matter how “unfair” it might seem, to nature only the living sons matter, everyone else was a failure.

If we accept that dinosaurs failed, then one can try to use the same metric for humans, but there’s a problem (for our exercise); humans are still alive. How do you compare two species that are not extinct? Strictly speaking all species alive today are still in the race. So how easy is it for humans to go extinct? This is a difficult question to answer, but lets suppose an extreme event turns the average temperature of the Earth 100°C colder; that would quite likely kill all humans (and probably a lot of plants and animals), but most certainly a lot of bacterial species would survive. It has been estimated that there’s 5×1030 bacteria on Earth, countless species, and possibly surpassing the biomass of all plants and animals. In fact, human beings could not survive without bacteria, since it’s essential to the human microbiome, and if you sum the bacteria genes in a human body, it probably outranks the human genes by a factor of 100-to-1. So, humans, like dinosaurs, could disappear rather easily, but bacteria would still be around for a long long time. From this point of view, bacteria are clearly more successful than humans.

Is there any scenario in which humans would survive, and bacteria would not? (therefore making humans more successful) I can think of some, but they would be far into the future, and most definitely we are not yet there. We are realizing the importance of our microbiome only now, and in the process of running the Human Microbiome Project, so we don’t even know what role our bacteria plays, therefore we don’t know how we could replace them with something else (like nanorobots). If bacteria disappeared today, so would we. It would follow then that bacteria are more successful, and there’s no getting around that.

Fundamentals and Morality

Could we define something more fundamental about success? I believe so: a worse failure than dying, is not being able to live in the first place, like a fetus that is automatically aborted because of a bad mutation, or even worse; an impossibility. Suppose “2 + 2 = 5”; this of course is impossible, so it follows that it’s a total failure. The opposite would be “2 + 2 = 4”; this is as true as anything can be, therefore it’s a total success.

There’s a realm of mathematics that is closer to what we consider morality: game theory. But don’t be fooled by its name; game theory is as serious as any other realm of mathematics, and the findings as conclusive as probability. An example of game theory is the prisoner’s dilemma — here’s a classic version of it:

Two men are arrested, but the police do not possess enough information for a conviction. Following the separation of the two men, the police offer both a similar deal—if one testifies against his partner (defects/betrays), and the other remains silent (cooperates/assists), the betrayer goes free and the one that remains silent receives the full one-year sentence. If both remain silent, both are sentenced to only one month in jail for a minor charge. If each ‘rats out’ the other, each receives a three-month sentence. Each prisoner must choose either to betray or remain silent; the decision of each is kept quiet. What should they do? If it is supposed here that each player is only concerned with lessening his time in jail, the game becomes a non-zero sum game where the two players may either assist or betray the other. In the game, the sole worry of the prisoners seems to be increasing his own reward. The interesting symmetry of this problem is that the logical decision leads each to betray the other, even though their individual ‘prize’ would be greater if they cooperated.

There are different versions of this scenario; with different rules and more complex agents game theory arrives to different conclusions as to what rational agents should do to maximize their outcomes, but these strategies are quite factual and universal; we are not talking about human beings; they are independent of culture, or historicism; the rules are as true here as they are in the other side of the universe. So if game theory determines that a certain strategy fails in certain situation, that’s it; it’s as hard of a failure as “2 + 2 = 5”.

With this notion we might be able to dive into more realistic and controversial examples — like slavery. Nowadays we consider slavery immoral, but that wasn’t the case in the past. One might say that slavery was a failure (because it doesn’t exist (at least as a desirable concept)), but that is only the case in human society, perhaps there’s another civilization in an alien planet that still has slavery, and they are still debating, so one might be tempted to say that slavery’s failure is still contended (perhaps even more so if you live in Texas). But we got rid of slavery because of a reason; it’s not good for everybody. It might be good for the slave owners, and good for some slaves, but not good for everybody. It is hard to imagine how another civilization could arrive to a different conclusion. Therefore it is quite safe to say that in all likelihood slavery is a failure, because of its tendency to disappear. Perhaps at some point game theory would advance to the point where we can be sure about this, and the only reason it took so long to get rid of slavery is that we are not rational beings, and it takes time for our societies to reach this level of rationality.

Objective morality and the moral landscape

Similarly to the objective success I’m proposing, Sam Harris proposes a new version of objective morality in his book The Moral Landscape. I must admit I haven’t read the book, but I have watched his online lectures about the topic. Sam Harris asserts that the notion that science shouldn’t deal with morality is a myth, and that advances in neuroscience (his field of expertise) can, and should, enlighten us as to what should be considered moral. Thus, morality is not relative, but objective. The different “peaks” in the landscape of morality are points in which society aims to be, in order to be a good one, and historically the methods to find these “peaks” has been rather rudimentary, but a new field of moral science could be the ultimate tool.

Regardless of the method we use to find these “peaks”, the important notion (for this post), is that there’s an abyss; the lowest moral point. The worst possible misery for all beings is surely bad:

The worst-possible-misery-for-everyone is ‘bad.’ If the word ‘bad’ is going to be mean anything surely it applies to the worst-possible-misery-for-everyone. Now if you think the worst-possible-misery-for-everyone isn’t bad, or might have a silver lining, or there might be something worse… I don’t know what you’re talking about. What is more, I’m reasonably sure you don’t know what you’re talking about either.

I want to hijack this concept of the worst-possible-misery-for-everyone that is the basis of (scientific) objective morality, and use it as a comparison to my contention that ceasing-to-exist is the basis for objective success.

Today our society is confronted with moral dilemmas such as gay marriage and legal abortion, many of these are hijacked by religious bigotry and irrationality, and it’s hard to move forward because many still define morality through religious dogmas, and even people who don’t, and are extremely rational, still cling to the idea that morality comes from “God” (whatever that might mean). Even many scientists claim that morality cannot be found through science, and others that morality is relative. But yet others disagree and have tried to define morality in universal terms, like Sam Harris. The jury is still out on this topic, so I cannot say that morality should definitely be defined in terms of what is successful to our worldwide society, merely that it is a possibility — A rather strong one, in my opinion.

Life

It’s a little more tricky to define what constitutes a successful life, because all life ends. The solution must be one on the terms of transcendence: offspring, books, memes, etc. However people living a more hedonistic life might disagree; but lets be clear, a life can be unsuccessful in the grand scheme of things, but still be good, and the other way around. It might be tempting to define success in different terms: “if my goal is to enjoy life, and I do; I’m succeeding”, and while that is true, that’s being successful in relative terms, not general terms.

Some people might have trouble with this notion, so I would give an example: Grigori Perelman vs. Britney Spears. Most people probably don’t know Grigori, but he solved one of the most difficult problems in mathematics, and was awarded one million USD for it. Clearly this would have helped him to become famous, but he rejected interviews and rejected the money. Does this means he rejected success? Well, lets try to view this from the vantage point of 500 years into the future; both Britney Spears and Grigori Perelman would be dead by that time, so the only things that remain would be their transcendence. Most likely nobody would remember Britney Spears, nor listen to any of her music, while it’s quite likely that people would still be using Grigori Perelman’s mathematics, as he would be one of the giants upon which future scientists would stand. In this sense Grigori is more successful, and any other sense of success would be relative to something else, not objective.

Test

Hopefully my definition of success should be clear by now in order to apply it to the initial example.

iPhone

iPhone is clearly successful in being profitable, but many products have been profitable in the past and have gone with the wind. The real question is: What are the chances that the iPhone will not disappear? It is hard to defend the position that the iPhone will remain for a long period of time because it’s a single product, from a single company, and specially considering that many technology experts can’t find an explanation for its success other than the Apple Cult. While it was clearly superior from an aesthetic point of view while it was introduced, there’s many competitors that are on par today. Maybe it would not disappear in 10 years, but maybe it would. It’s totally unclear.

Android

Compared to the iPhone, Android has the advantage that many companies work on it, directly and indirectly, and it doesn’t live on a single product. So if a single company goes down, that would not kill Android, even if that company is Google. So, as a platform, it’s much more resilient than iOS. Because of this reason alone, Android is clearly more successful than the iPhone — according to the aforementioned definition.

Maemo

Maemo is dead (mostly), so that would automatically mean that it’s a failure. However, Maemo is not a single organism; it consists of many subsystems that are part of the typical Linux ecosystem: Linux kernel, X.org, Qt, WebKit, GStreamer, Telepathy, etc. These subsystems remain very much alive, in fact, they existed before Maemo, and will continue to exist, and grow. Some of these subsystems are used in other platforms, such as WebOS (also dead (mostly)), Tizen, MeeGo (also dead (kinda)), and Mer.

A common saying is that open source projects never die. Although this is not strictly true, the important thing is that they are extremely difficult to kill (just ask Microsoft). Perhaps the best analogy in the animal kingdom would be to compare Maemo to a sponge. You can divide a sponge into as many parts as you want, put it into a blender, even make sure the pieces pass through a filter with very minute holes. It doesn’t matter; the sponge would reorganize itself again. It’s hard to imagine a more resilient animal.

If this is the case, one would expect Maemo (or its pieces) to continue as Tizen, or Mer (on Jolla phones), or perhaps other platform yet to be born, even though today it seems dead. If this happens, then Maemo would be even more successful than Android. Time will tell.

Predictions

Like any a scientific theory, the really interesting bit of this idea would be it’s predictive power, so I will make a list of things in their order of success, and if I’m correct the less successful ones would tend to disappear first (or their legacy):

  • Mer > Android > iOS > WP
  • Linux > Windows
  • Bill Gates > Steve Jobs > Carlos Slim (Richest man in the world)
  • Gay marriage > Inequality
  • Rationality > Religious dogma
  • Collaboration > Institutions

To me, this definition of “success” is as true as “2 + 2 = 4” (in fact, it’s kind of based on such fundamental truths), unfortunately, it seems most people don’t share this point of view, as we still have debates over topics which are in my opinion a waste of time. What do you think? Are there examples where this definition of success doesn’t fit?

Advertisements

Android vs. Maemo power management: static vs. dynamic

Some of you might have heard about Google’s Android team proposal to introduce wakelocks (aka suspend-blockers) to the Linux kernel. While there was a real issue being solved in the kernel side, the benefits on the user-space side were dubious at best, and after a huge discussion, they finally didn’t get in.

During this discussions the dynamic and static power management were described and discussed at length, and there was a bit of talk about Maemo(MeeGo) vs Android approaches to power management, but there was so much more than that.

Some people have problems with the battery on Android devices, for some people it’s just fine, some people have problems with the Maemo, other don’t, so in general; your mileage might vary. But given the extremely different approaches, it’s easy to see in which cases you would have better luck with Maemo, and in which with Android–Although I do think it’s obvious which approach is superior, but I am biased.

An interesting achievement was shared by Thiago Maciera, who managed to get ‘5 days and a couple of minutes‘ out of the Nokia N9 while traveling, and actually using it–and let’s remember this is a 1450 mAh battery. Some Android users might have trouble believing this, but hopefully this blog post would explain what’s going on.

So lets go ahead and explore the two approaches.

Dynamic Power Management

Perhaps the simplest way to imagine dynamic power management, is the gears of a manual transmission car. You go up and down depending on how much power does the system actually needs.

In some hardware, such as OMAP chips, it’s possible to have quite a lot of fine control on the power states of quite a lot of devices, plus different operating power points on the different cores. For example, it might be possible to have some devices, such as the display on, and active, some other devices partially off, like speaker, and other completely off, like USB. And based on the status of the whole system, whole blocks can be powered off, other with low voltage levels, etc.

Linux has a framework to deal properly with this kind of hardware, the runtime power management, that originally came from the embedded world, and a lot from OMAP development, but is now available to everyone.

The idea is very simple; devices should sleep as much as possible. This means that if you have a sound device that needs chunks of 100ms, and the system is not doing anything else but playing sound, then most of the devices go to sleep, even the CPU, and the CPU is only waken up when it needs to write data for the audio device. Even better is to configure the sound device for chunks of 1 second, so the system can sleep even more.

Obviously, some co-operation between kernel and user-space is needed. Say, if you have an instant messenger program that needs to do work every minute, and a mail program that is configured to check mail every 10 minutes, you would want them to do work at the same time when they align at every 10 minutes. This is sometimes called IP heartbeat; the system wakes up for a beat, and then immediately goes back to sleep. And there are kernel facilities as well, such as range timers.

All this is possible thanks to very small latencies required for devices to go to sleep and wakeup, and have intermediary modes (e.g. on, inactive, retention, off), so, for example a device might be able to go to inactive mode in 1ms, retention in 2ms, and off in 5ms (totally invented numbers). Again, the more sleep, the better. Obviously, this is impossible on x86 chips, which have huge latencies–at least right now, and it’s something Intel is probably trying to improve effusively. All these latencies are known by the runtime pm framework in the kernel, and based on that it and the usage, it figures out what is the lowest power state possible without breaking things.

Note I’m not a power management expert, but you cant watch a colleague of mine explain the OMAP 3 power-managment on this video:

Advanced Power Management for OMAP3

And there’s plenty of more resources.

Update: That was the wrong link, here are the resources.

Static Power Management

Static power management has two modes: on and off. That’s it.

OK, that’s not exactly the case in general, but it is in the Android context; the system is either suspended, or active, and it’s easy to know in which mode you are; if the screen is on, it’s active, and if it’s off; it’ is suspended (ideally).

There’s really not much more than that. The complexity comes from the problem of when to suspend; you might have turned off the display, but there might be a system service that still needs to do work, so this service grabs a suspend blocker which, as the name suggests, prevents the system from suspending (until the lock is released). This introduces a problem; a rouge program might grab a ‘suspend blocker’ and never release it, which means your phone will never suspend, and thus the battery would drain rather quickly. So, some security mechanisms are introduced to grant permissions selectively to use suspend blockers.

And moreover, Android developers found race conditions in the suspend sequences in certain situations that were rather nasty (e.g. the system tries to suspend at the same time the user clicks a button, and the system never wakes up again), and these were acknowledged as real issues that would happen on all systems (including PC’s and servers, albeit rarely, because they don’t suspend so often), and got fixed (or at least they tried).

Versus

First of all, it’s important to know that if you have dynamic pm working perfectly, you reach exactly the same voltage usage than static pm, so in ideal cases they both behave exactly the same.

The problem is that it’s really hard for dynamic pm to reach that ideal case, in reality systems are able to sleep for certain periods of time, after which they are woken up, often times unnecessarily, and as I already explained; that’s not good. So the goal of a dynamic pm system is to increase those periods of time as much as possible, thus maximizing battery life. But there’s a point of diminished returns (read this or this for expert explanations), so, if the system manages to sleep 1s in average, there’s really not much more to gain if it sleeps 2s, or even 10s. These goals were quite difficult to achieve in the past (not these, I invented those numbers), but not so much any more thanks to several mechanisms that have been introduced and implemented through the years. So it’s fair to say that the sweet spot of dynamic pm has been achieved.

This means that today a system that has been fine-tuned for dynamic pm can reach reach a decent battery life compared to one that uses static pm in most circumstances. But for some use-cases, say, you leave your phone on your desk and you don’t use it at all, static pm would allow it to stay alive for weeks, or even months, while dynamic pm can’t possibly achieve that any time soon. Hopefully you would agree, that nobody cares about those use-cases were you are not actually using the device.

And of course, you only need one application behaving badly and waking up the system constantly, and the battery life is screwed. So in essence, dynamic pm is hard to achieve.

Android developers argued that was one of the main reasons to go for static pm; it’s easier to achieve, specially if you want to support a lot of third party applications (Android Market) without compromising battery life. While this makes sense, I wasn’t convinced by this argument; you still can have one application that is behaving badly (grabbing suspend blockers the whole time), and while permissions should help, the application might still request the permission, and the user grant it (who reads incomprehensible warnings before clicking ‘Yes’ anyway?).

So, both can get screwed by bad apps (although it’s likely that it’s harder in the static pm case, albeit not that much).

But actually, you can have both static and dynamic power management, and in fact, Android does. But that doesn’t mean Android automatically wins, as I explained, the system needs to be fine-tuned for dynamic pm, and that has never been a focus of Android (there’s no API’s or frameworks for that, etc.). So, for example, a Nokia N9 phone might be able to sleep 1s in average, while an Android phone 100ms (when not suspended). This means when you are actually using the device (the screen is on), chances are, a system fine-tuned for dynamic pm (Nokia N9) would consume less battery, than an Android device.

That is the main difference between the two. tl;dr: dynamic pm is better for active usage.

So, if Android developers want to improve the battery usage while on active usage (which I assume is what the users want), they need to fine-tune the system for dynamic pm, and thus sleep as much as possible, hopefully reaching the sweet spot. But wait a second… If Android is using dynamic pm anyway, and they tune the system to the point of diminishing returns; there is not need for static pm. Right? Well, that’s my thinking, but I didn’t manage to make Android developers realize that in the discussion.

Plus, there’s a bunch of other reasons while static pm is not palatable for non-Android systems (aka. typical Linux systems), but I won’t go into those details.

Nokia’s bet was on dynamic, Google’s bet was on static, and in the end we agreed to disagree, but I think it’s clear from the outcome in real-world situations who was right–N9 users experiencing more than one day of normal usage, and even more than two. Sadly, only Nokia N9 users would manage to experience dynamic pm in full glory (at the moment).

Upstream

But not all is lost, and this in my opinion is the most important aspect. Dynamic pm lives on the Linux kernel mainline through the runtime power management API. This is not a Nokia invention that will die with the Nokia N9; it’s a collaborative effort where Nokia, TI, and other companies worked together, and now not only benefits OMAP, but other embedded chips, and even non-embedded ones. Being upstream means it’s good, and it has been blessed by many parties, and has gone through many iterations, and finally it probably doesn’t look like the original OMAP SRF code at all. Sooner or later your phone (if you don’t have an N9) will benefit from this effort, and it might even be an Android phone, your netbook (if not already benefiting in some way), and even your PC.

Android’s suspend blockers are not upstream, and it’s quite unlikely that they will ever be, or that you would see them used in any system other than Android, and there’s good reasons for that. Matthew Garrett did an excellent job of summarizing what went wrong with the story of suspend blockers on his presentation ‘Android/Linux kernel: Lessons learned’, but unfortunately the Linux foundation seems to be doing a poor job of providing those videos (I cannot watch them any more, even though I registered, and they haven’t been helpful through email conversations).

Update: I managed to get the video from the Linux Foundation and pushed it to YouTube:
[youtube:http://www.youtube.com/watch?v=A4psPP67YMU%5D

Here is part of the discussion on LKML, if you want to read it for some strange reason. WARNING; it’s huge.

Android looks more than good

Anroid has been released:

Following last week’s announcement of the Open Handset Alliance, today we’re happy to offer an early look at the Android SDK. We’re eager to get feedback from lots of developers in our discussion group. We’re also providing developer documentation and this blog to help you get up and going.

It’s looking pretty good:

Here is a detailed explanation about the platform:

It looks like it has a lot of potential. They have just launched and they already have a very extensible platform, way beyond what we have in Maemo, and even in GNOME or any Linux DE. I give it a month before we have unbelievably-crazy, original and useful killer-applications.

I like to develop in Linux, with vim, in C or in Ruby, so I’m not so excited about their Eclipse based SDK, but I guess it makes it easier for a lot of people. I like Java, but I like to write stuff without any fancy IDE. Anyhow $25,000 looks like a pretty good motivation to try installing it 🙂

Two thumbs up.

More videos here.