Advanced Git concepts; the upstream tracking branch

Probably one of most powerful and under-utilized concepts of Git is the upstream tracking branch, and to be honest it probably was too difficult to use properly in the past, but not so much any more.

Here I’ll try to explain what it is, and how you can take the most advantage out of it.

Remote tracking branches

Before trying to understand what the upstream tracking branch is, you need to be familiar with remote branches (e.g. origin/master). If you are not, you probably want to read the section about them in the Pro Git book here.

To see all your remote tracking branches, you can use ‘git branch –remotes’.

The upstream tracking branch

Even if you have never heard of the concept, you probably already have at least one upstream tracking branch: master -> origin/master. When you clone a repository the current HEAD (usually ‘master’) is checked out for you, but also, it’s setup to track ‘origin/master’, and thus ‘origin/master’ is the “upstream” of ‘master’.

This has some implications on some Git tools, for example, when you run ‘git status‘ you might see a message like this:

# Your branch is behind 'origin/master' by 1 commit.

Also, if you run ‘git branch -vv‘:

* master 549ca22 [origin/master: behind 1] Add bash_profile

This is useful in order to keep your local branches synchronized with the remote ones, but it’s only scratching the surface.

Once you have realized that your local branch has diverged from the remote one, you will probably want to either rebase or merge, so you might want to do something like:

git rebase origin/master

However, ‘origin/master’ is already configured as the upstream tracking branch of ‘master’, so you can do:

git rebase master@{upstream}

Maybe you think @{upstream} is too much to type, so you can do @{u} instead, and since we are already on ‘master’ we can do HEAD@{u}, or even simpler:

git rebase @{u}

But Git is smarter than that, by default both ‘git merge’ and ‘git rebase’ will use the upstream tracking branch, so:

git rebase

Configuring the upstream branch

So now you know that upstream tracking branches are incredibly useful, but how to configure them? There’s many ways.

By default, when you checkout a new branch, and you are using a remote branch as the starting point, the upstream tracking branch will be setup automatically.

git checkout -b dev origin/dev

Or:

git checkout dev

If the starting point is a local branch, you can force the tracking by specifying the –track option:

git checkout --track -b dev master

If you already created the branch, you can update only the tracking info:

git branch --set-upstream-to master dev

There’s a very similar option called –set-upstream, however, it’s not intuitive, and it’s now deprecated in favor of –set-upstream-to, to be sure and avoid confusion, simply use -u.

You can also set it up at the same time as you are pushing:

git push --set-upstream origin dev

Finally, you can configure Git so they are always created, even if you don’t specify the –track option:

git config --global branch.autosetupmerge always

Conclusion

So there you have it, go nuts and configure the upstream branch for all your branches ;)

git branch --vv with upstream

An in-depth analysis of Mercurial and Git branches

I’ve discussed the advantages of Git over Mercurial many times (e.g. here, and here), and I even created a challenge for Mercurial supporters, but in this blog post I’ll try to refrain from doing judgments and concentrate on the actual facts (the key-word being try).

Continuing this full disclosure; I’ve never actually used Mercurial, at least on a day-to-day basis, where I actually had to get something done. But I’ve used it plenty of times testing many different things, precisely to find out how to do things that I can do easily in Git. In addition, I’ve looked deep into the code to figure out how to overcome some of what I considered limitations of the design. And finally, I wrote Git’s official GitMercurial bridge; git-remote-hg (more here).

So, because I’ve spent months figuring out how to achieve certain things in Mercurial, and after talking with the best and the brightest (Git, gitifyhg, hg-git, and Mercurial developers), and exploring the code myself, I can say with a good degree of confidence that if I claim something cannot be done in Mercurial, that’s probably the case. In fact, I invited people from the #mercurial IRC channel in Freenode to review this article, and I invite everyone to comment down below if you think there’s any mistake (comments are welcome).

Git vs. Mercurial branches

Now, I’ve explained before why I think the only real difference between Git and Mercurial is how they handle branches. Basically; Git branches are all-purpose, all-terrain, and Mercurial have different tools for different purposes, and can almost do as much as Git branches, but not quite.

I thought the only real limitation was that Mercurial branches (or rather bookmarks), didn’t nave a per-repository namespace. For example: in Git the branch “development” can be in different repositories, and point to different commits, and to visualize them, you can refer to “max/development” (Max’s development branch), “sarah/development” (Sarah’s), “origin/development” (The central repository version), “development” (your own version). In Mercurial you only have “development”, and that’s it. I consider that a limitation of Mercurial, but feel free to consider it a “difference”. But it turns out there’s more.

In Git, it’s easy to add, remove, rename, and move branches. In Mercurial, bookmarks are supposed to work like Git branches, however, they don’t change the basics of how Mercurial works, and in Mercurial it doesn’t matter if you have a bookmark or not pointing to a commit, it’s still there, and completely visible; in Mercurial, each branch can have multiple “heads”, it doesn’t matter if there’s a bookmark pointing to it or not. So in order to remove a bookmark (and its commits), you need to use “hg strip” command, and to use that command, you need to enable the MqExtension, however, that’s for local repositories, for remote ones you need to cross your fingers, and hope your server has a way to do that — Bitbucket does through its web UI, but it’s possible that there is just no way.

Mercurial advocates often repeat the mantra “history is sacred”, and Mercurial’s documentation attempts to explain why changing history is hard, that shows why it’s hard to remove bookmarks (and it’s commits); it’s just Mercurial’s design.

On the other hand, if you want to remove a branch in git; you can just do “git push :feature-a“. Whether “history is sacred” or not is left for each project to decide.

Solving divergence

In any version control system, divergence is bound to happen, and in distributed ones, even more. Mercurial and Git solve this problem in very different ways, lets see how by looking at a very simple divergent repository:

Diverged

As you can see we have a “Fix” in our local branch, but somebody already did an “Update” to this branch in the remote repository. Both Mercurial and Git would barf when you try to push this “Fix” commit, but lets see how to solve it in each.

In Git this problem is called a “non fast-forward” push, which means that “Fix” is not an ancestor of the tip of the branch (“Update”), so the branch cannot be fast-forwarded to “Fix”. There are three options: 1) force the push (git push --force), which basically means override “origin/master” to point to “master”, which effectively dumps “Update” 2) merge “Update” and “Fix” and then push 3) rebase “Fix” on top of “Update” and then push. Obviously dropping commits is not a good idea, so either a merge or a rebase are recommended, and both would create a new commit that can be fast-forwarded from “Update”.

In Mercurial, the problem is called “multiple heads”. In Git “origin/master” and “master” are two different branches, but in Mercurial, they are two heads of the same branch. To solve the problem, you can start by running “hg heads“, which will show you all the heads of all the branches, in this case “Fix” and “Update” would be the heads of the “default” branch (aka. “master”). Then you have also three options: 1) force the push (hg push --force), although in appearance it looks the same as the Git command, it does something completely different; it pushes the new head to the remote 2) merge and push 3) rebase and push (you need the rebase extension). Once again, the first option is not recommended, because it shifts the burden from one developer to multiple ones. In theory, the developer that is pushing the new commit would know how to resolve the conflicts in case they arise, so (s)he is the one that should resolve them, and not take the lazy way out and shift the burden to other developers.

Either way solves the problem, but Git uses remote namespaces, which I already shown are useful regardless, and the other requires the concept of multiple heads. That is one reason why the concept of “anonymous heads”, that is used as an example of a feature Mercurial has over Git, is not really needed.

Mercurial bookmarks and the forced push problem

The biggest issue (IMO) I found with Mercurial bookmarks is how to create them in the first place. The issue is subtle, but it affects Git-like workflows, and specially Git<->Mercurial bridges, either way it’s useful to understand Mercurial’s design and behavior.

Suppose you have a very simple repository:

Simple repository

In Git, “feature-a” is a branch, and you can just push it without problems. In Mercurial, if “feature-a” is a bookmark, you can’t just push it, because if you do, the “default” branch would have two heads. To push this new bookmark, you need to do “hg push --force“. However, this only happens if the commit “Update” is made, also, you can push “feature-a” if it points to “Init”, and after pushing the bookmark, you can update it to include the “Feature A” commit. The end result is the same, but Mercurial barfs if you try to push the bookmarks and the commits at the same time, and there’s an update on the branch.

There’s no real reason why this happens, it’s probably baggage from the fact that Mercurial bookmarks are not an integral part of the design, and in fact began as an extension that was merged to the core in v1.8.

To workaround this problem in git-remote-hg, I wrote my own simplified version of the push() method that ignores checks for new heads, because in Git there cannot be more than one head per branch. The code still checks that the remote commit of this branch is an ancestor of the new one, if not, you would need to do ‘git push –force’, just like in Git. Essentially, you get exactly the same behavior of Git branches, with Mercurial bookmarks.

Fixing Git

All right, I’m done trying to avoid judgement, but to try to be fair, I’ll start by mentioning the one (and only one) feature that Git lacks in comparison to Mercurial; find the branch-point of a branch, that is; the point where a branch was created (or rebased onto). It is trivial to figure that out visually, and there are scripts that do a pretty good job of finding that out from the topology of the repository, but there are always corner-cases where this doesn’t work. For more details on the problem and proposed solutions check the stackoverflow question.

Personally I’ve never needed this, but if you absolutely need this, it’s easy to patch Git, I wrote a few patches that implement this:

https://github.com/felipec/git/commits/fc/base

This implements the @{tail} notation, which is similar to the official @{upstream} notation, so you can do something like “development@{tail}”, which will point to the first commit the “development” branch was created on.

If this was really needed, the patches could be merged to upstream Git, but really, it’s not.

Fixing Mercurial

On the other hand fixing Mercurial wouldn’t be that easy:

  1. Support remote ‘hg strip’. Just like Git can easily delete remote commits, Mercurial should be able to.
  2. Support remote namespaces for bookmarks. Begin able to see where “sarah/development” points to, is an invaluable feature.
  3. Improve bookmark creation. So the user doesn’t need to force the push depending on the circumstances

Thanks to git-remote-hg, you can resolve 2) and 3) by using Git to work with Mercurial repositories, unfortunately, there’s nothing anybody can do for 1), it’s something that has to be fixed in Mercurial’s core.

Conclusion

I often hear people say that what you can achieve with Git, you can achieve with Mercurial, and vice versa, and at the end of the day it’s a matter of preference, but that’s not true. Hopefully after reading this blog post, you are able to distinguish what can and cannot be done in each tool.

And again, as usual, all comments are welcome, so if you see a mistake in the article, by all means point it out.

Cheers.

What’s new in Git v1.8.4 Mercurial bridge

Git v1.8.4 has been released, and git-remote-hg received a lot of good updates, here’s a summary of them:

Precise branch tracking

Git is able to find out and report if a branch is new, if it needs to be updated, if it can be fast-forward or not, etc.

   b3f6f3a..c0d1c89  master -> master
 * [new branch]      new -> new
 ! [rejected]        bad -> bad (non-fast-forward)
 ! [rejected]        updated -> updated (fetch first)

Unfortunately, Mercurial’s code doesn’t make this easy (you can’t just push new bookmarks), but it has been worked around by writing a custom push() method.

In addition, if you use my patched version of Git (here), you can also use –force and –dry-run when pushing.

+ 51c1c5f...0faf0ed bad -> bad (forced update)

In short, git-remote-hg now makes interacting with Mercuerial repositories exactly the same as with Git ones, except for deleting remote branches (Mercurial just cannot do that).

Shared repository

One of the most useful features of Git (and that Mercurial doesn’t have), is remote name-spaces. So you can easily track “max/development”, “sarah/development”, etc. however, to properly track multiple Mercurial repositories, git-remote-hg needs to create a clone of the Mercurial repo, and if the repository is a big one, having multiple unrelated clones wastes a lot of space.

The solution is to use the Mercurial share extension, which is not really an extension, as it’s part of the core (but can only be used by activating the extension), so you can add as many Mercurial remotes as you want, and they would all share the same object store.

Use SHA-1’s to identify revisions

Previously, Mercurial revisions were stored as revision numbers (e.g. the tenth commit is stored as 10), which means if history is rewritten there’s no way to tell that the revision changed, so the Git commit wouldn’t change either (as it’s cached).

By using SHA-1’s, Mercurial revisions are always tracked properly.

Properly update bookmarks

Previously, Mercurial bookmarks were only fetched once, this is now fixed to always update them.

All extensions are loaded

This way all kinds of extensions the user has configured will affect git-remote-hg, for example the keyring extension.

Make sure history rewrites get updated

Before, Git would complain that a non-fast-forward updated happened–not any more.

Always point HEAD to “default”

Mercurial properly reports which is the current branch and bookmark, but only for local repositories. To get rid of the mismatch we always track “default”

Don’t force bookmark updates

We were inadvertently forcing the update of bookmarks, effectively overriding the previous one even if the update was not fast-forward.

Use Git author for lightweight tags

Unannotated tags don’t have an author in Git, but it’s needed for Mercurial, so instead of providing an empty author, use the one configured for Git.

Fix replacing a file with a directory

What’s next

There are few features missing, and they might not land in upstream Git any more, but:

Support for revision notes

This feature allows showing the Mercurial revision as Git notes:

commit 6c88a31540012991de3add247a958fd83531256f
Author: Felipe Contreras 
Date:   Fri Aug 23 13:00:30 2013 -0500

    Test

Notes (hg):
    e392886b34c2498185eab4301fd0e30a888b5335

If you want to have the latest fixes and features, you need to use my personal repository:

https://github.com/felipec/git

Unfortunately, you not only need the python script, but to compile Git itself to have all the benefits (like push –force and –dry-run).

Wiki

Also, there’s more information and detailed instructions about how to install and configure this remote-helper.

https://github.com/felipec/git/wiki/git-remote-hg

I’m now quite confident git-remote-hg is by far the best bridge between Git and Mercurial, and here’s a comparison between this and other projects.

Enjoy :)

What it takes to improve Git or: How I fixed zsh completion

I’ve used Git since pretty much day one, and I use it all the time, so it’s important to me that it’s easy to type Git commands quickly and efficiently. I use zsh, which I believe is way superior to bash, unfortunately I found many issues with its Git completion.

In this blog post I will try to guide you through the ordeal from how I identified a problem, and how I ended up fixing it years after, for everyone’s benefit.

The issue

I work on the Linux (kernel) source tree from time to time, and I noticed that sometimes completion took a long, looong time. Specifically, I found that typing ‘git show v’ took several seconds to complete.

I decided to bring that issue up to the zsh developers, and it caused a lot of fuzz. I won’t go to every detail of the discussion, but long story short; they were not going to fix the issue because of their uncompromising principles; correctness over functionality, even if very few people use that correctness, and the functionality is almost completely broken to the point the completion is not usable in certain cases. I argued that completion is meant to make typing commands more efficient, and if completing a command takes longer than what it would have taken me to type it manually, the completion is failing its purpose. I thought any sane person would see the problem with that, but apparently I was wrong (or was I?).

Fortunately zsh has bash completion emulation, so it’s possible to use Git’s official bash completion in zsh. You loose some of the features of zsh completion, but it works very efficiently (‘git show v’ was instantaneous).

Unfortunately, zsh’s bash emulation, and zsh’ bash completion emulation (two different things), are not perfect, so some workarounds were needed in Git’s bash completion script, and those workarounds were not working properly by the time I started to use such completion, so that’s when my involvement begin.

Fixing the bridge

Each time I found a bug, I tried to fix it in Git (patch), and made sure that zsh folks fixed in their side too (commit), so eventually no workarounds would be needed, and everything would work correctly.

The completion worked for the most part, but with workarounds, and not exactly as good as bash’s. So I decided to fix zsh’s bash completion emulation once and for all. After my patches were applied by zsh developers, Git’s official completion worked much closer to how it did in bash, but there were still minor issues.

Moreover, Git’s bash completion was constantly changing, and it was only a matter of time before one change broke zsh’s completion, so I decided to get involved, understand the code and simplify it to minimize the possibility (e.g. d79f81a, 583e4d5). I saw a lot of areas of improvement, but in order to make sure nothing got broken in the process of simplification, I thought it would make sense to have some tests (5c293a6). Git’s testing framework is one of the most powerful and simple there is, so it was a pleasure to write those tests. Eventually the completion tests were good enough that I became confident in changing a lot of the completion code.

At the same time I realized most of zsh’s bash completion emulation code was not needed at all, so I wrote a very small version of it that only worked with Git’s completion. The result was very simple, and it worked perfectly, yet it could be even simpler, if only I could simplify Git’s completion even more.

The culmination of that work was the creation of __git_complete (6b179ad), a helper that has nothing to do with zsh, but it solved a long standing problem with Git completion and aliases. It’s not worth going into details about what was the problem, and why it received so much push-back from Git developers (mostly because of naming issues), what is important is that I implemented it with a wrapper function, a wrapper function that was *exactly* what my zsh simple completion wrapper needed.

Now that everything was in place, the final wrapper script ended up very small and simple (c940786), it didn’t have any of the bugs zsh’s bash completion emulation had, and was under full control of the Git project, so it could be improved later on.

Finally. I had Git completion in zsh that worked *perfectly*; it worked exactly the same as it did on bash. But that was not enough.

Now that Git completion worked just like in bash, it was time to implement some extras. zsh completion is extremely powerful, and does things bash cannot even dream of doing, and with my custom wrapper, it was possible to have the best of both worlds, and that’s exactly what I decided to do (4911589).

git-zsh-shot

Finally

So there it is, after years of work, several hundreds of mails, tons of patches through different iterations… Git now has nice zsh completion that not only works as efficiently as in bash without any difference, but in fact it even has more features.

If you want to give it a try, just follow the instructions: contrib/completion/git-completion.zsh

;)

Felipe Contreras (54):
      git-completion: fix regression in zsh support
      git-completion: workaround zsh COMPREPLY bug
      completion: work around zsh option propagation bug
      completion: use ls -1 instead of rolling a loop to do that ourselves
      completion: simplify __gitcomp and __gitcomp_nl implementations
      tests: add initial bash completion tests
      completion: simplify __gitcomp_1
      completion: simplify by using $prev
      completion: add missing general options
      completion: simplify __git_complete_revlist_file
      completion: add new __git_complete helper
      completion: rename internal helpers _git and _gitk
      completion: add support for backwards compatibility
      completion: remove executable mode
      completion: split __git_ps1 into a separate script
      completion: fix shell expansion of items
      completion: add format-patch options to send-email
      completion: add comment for test_completion()
      completion: standardize final space marker in tests
      completion: simplify tests using test_completion_long()
      completion: consolidate test_completion*() tests
      completion: refactor __gitcomp related tests
      completion: simplify __gitcomp() test helper
      completion: add new zsh completion
      completion: start moving to the new zsh completion
      completion: fix warning for zsh
      completion: add more cherry-pick options
      completion: trivial test improvement
      completion: get rid of empty COMPREPLY assignments
      completion: add new __gitcompadd helper
      completion: add __gitcomp_nl tests
      completion: get rid of compgen
      completion: inline __gitcomp_1 to its sole callsite
      completion: small optimization
      prompt: fix untracked files for zsh
      completion: add file completion tests
      completion: document tilde expansion failure in tests
      completion; remove unuseful comments
      completion: use __gitcompadd for __gitcomp_file
      completion: refactor diff_index wrappers
      completion: refactor __git_complete_index_file()
      completion: add hack to enable file mode in bash < 4
      completion: add space after completed filename
      completion: remove __git_index_file_list_filter()
      completion: add missing format-patch options
      complete: zsh: trivial simplification
      complete: zsh: use zsh completion for the main cmd
      completion: zsh: don't override suffix on _detault
      completion: cleanup zsh wrapper
      completion: synchronize zsh wrapper
      completion: regression fix for zsh
      prompt: fix for simple rebase
      completion: zsh: improve bash script loading
      completion: avoid ls-remote in certain scenarios

The problem with GNOME 3

Since I started using Linux I used GNOME, v1.2 in those times. It has always done what I needed, maybe not perfectly, and not fully, but for the most part. GNOME 3 changed all that.

I complained about GNOME 3 since day one, and I discussed with GNOME 3 developers many problems with their rationale about why what they were doing made sense, and I foresaw many of the problems they are now facing.

I even started the GNOME user survey, in an effort to make GNOME developers see the light.

I blogged once before about GNOME 3, but only superficially, and even then GNOME developers didn’t take it well. Some GNOME developers might be tired of my arguments in Google+ and other media, but it’s time to dive into the issues with GNOME 3 in this blog.

Two years after the first release of GNOME 3, once GNOME developers have had the time to polish the rough edges, it’s time to make the call; GNOME 3 sucks. Here’s why.

The purpose of GNOME

Before we start, we need to clarify the purpose of GNOME, or rather; the purpose of any software project in general, which is to be useful to the users. Your software might the most efficient at doing certain task, it might have the simpler code, and the best development practices, but it’s all for naught if it’s not useful to anybody.

Software has to be useful for other people, otherwise nobody will use your software, and nobody will contribute to your software. Quite likely there will always be other software that does something similar to yours, so you need to convince users that your software is better than the alternatives, usually by providing a good user experience.

Once the user has spent time analyzing the alternatives and has chosen your software, the user expects your software to keep working, and in the same way. If the software keeps changing the way it behaves from one version to the next, it’s not achieving it’s main goal; to be useful to the users. If you do this, your users will move on to another project that might not have as many features, but at least they can rely on it and they don’t have to spend any more of their valuable time learning what broke now in the new release.

More important than providing a good user experience, is to not break existing user expectations.

You would think this is as obvious as a leopard’s spots, yet many software projects, including GNOME, don’t seem to understand this fact.

No project is more important than the users of the project. — Linus Torvalds

What users want

The problems I’ll talk about are not my invention, but conformed by GNOME user surveys, which show the most important issues according to the users:

  1. Lack of configuration options
  2. Developers don’t listen to users
  3. Bring back traditional interface

I knew these were important issues myself before the survey, but a survey was needed in order to know for sure. Of course, even after the survey, GNOME developers reject these issues are important to their users.

Developer attitude

In my opinion the main problem with GNOME is not just the code, code can be fixed, but the attitude from the developers (which is reflected in the code), and as users made it clear in the survey.

Many open source projects would kill to have the user-base GNOME had, and welcome their input with open arms, but GNOME neglected their users, they thought they were irrelevant, and they tried to dismiss their complaints with typical defenses, which of course don’t make sense.

no-feedback

People hate new things

By far the most widely used defense for the changes in GNOME 3 is that some people are backwards, and always hate new things, so it’s no surprise that they hate GNOME 3.

Any rational person would see the problem with this claim, but unfortunately I must spell it, because I had to explain it at great length to GNOME developers, and based on the responses, one would think they are incapable of seeing hasty generalization fallacies.

Say 50% of users hate GNOME 3, how many of them have legitimate reasons, and how many hate it just because they always hate new things? It’s impossible to know. So to dismiss every complaint saying “people hate new things” is totally stupid; you cannot know beforehand if they belong to this group or not, you first need to hear their complaints, and then decide if the complaints are invalid or not.

Saying “Jon doesn’t have legitimate complaints” makes sense (after analyzing his complaints), saying “people hate new things” does not, and GNOME developers do the later when people complain about GNOME 3.

Users don’t know what they want

Really? People don’t know if they want a punch in the face or not? Some people might not know what they want, but they definitely know what they don’t want.

Moreover, that’s a very rudimentary notion of user choice. Malcolm Gladwell explains rather well how it’s true that people don’t know what they want, or rather; they can’t tell you what they want, but when they get a taste of it, they most definitely can tell you: this is what I want.

If it wasn’t clear by now; GNOME 3 is not what a lot of people wanted.

gnome-hate

People complained about GNOME 2 too

Did this actually happen? If you google GNOME 2 for results between 2002 and 2004 you would find barely (if any) results that show negative comments, so if it did happen, the backlash probably wasn’t that bad, and it probably died rather quickly. Check the thread in Slashdot, and try to find one comment suggesting to go back to GNOME 1; I couldn’t find any.

Did anybody create forks after GNOME 2 was released the same way people forked after GNOME 3? No.

Yes, people complained about GNOME 2, but not nearly as much as they do about GNOME 3. There’s even an entire wikipedia article dedicated to the controversy over GNOME 3.

 

Sorry GNOME developers, none of your excuses for not listening to your users make sense. You should always listen to your users. Without them your project is nothing.

Configuration

The biggest complaint users had was the lack of configuration options, which could have been solved very easily, if the developers had more than two neurons in their brain, but instead, there’s only sub-par alternatives.

Advanced mode

GNOME 3 got rid of a bunch of useful configurations users relied on, and to defend themselves they said you could use gsettings manually, or install (or write) an extension, and so on, but I argued that there should be an “advanced mode” that the user can optionally turn on in order to further configure the system the way they wanted.

GNOME developers argued that this was a bad idea, because more code would need to be maintained which is absolutely not true for gsettings; the configurations are already there in the code, the only thing missing is the UI. Then they argued that it would mess up the configuration UI, which they wanted to keep clean, but adding a single check box can’t possibly mess with anything. In the end, I never received a single satisfactory answer of why an advanced mode didn’t make sense.

Tweak tool

The tweak tool is the closest thing to a sane configuration interface, however, it should be cleaned up, distributed by default, and integrated in the GNOME control center by adding an “Advanced mode” checkbox that when switched on enables these settings which should be distributed among the different “capplets”.

Even then, the amount of configurations available is not enough.

Shell extensions

Finally, there’s shell extensions which sound like a great idea at first, but if you think about it for thirty seconds, you realize the problem; there’s no APIs for extensions. This means extensions bring many problems.

First, there’s no guarantee that an extension will work the next version of GNOME, because it’s modifying the JavaScript code directly, and there’s no guarantee that the code will remain the same. There’s no extension API like with Chrome, or Firefox. The result is not unexpected; extensions break very easily from one version of GNOME to the next.

The second issue is that extensions can easily break GNOME, because they can modify absolutely anything in the shell.

gsettings

Another option is to manually change the configurations through the command line. This is obviously not user-friendly, and not ideal either.

 

GNOME developers made the mistake of not allowing configurability directly into GNOME, and as a result they have made the much needed configurations second class citizens, than no matter what you do, they don’t work as expected. The problem is very easy to solve; the configurations should be integrated into GNOME, and activated by a single switch, so their normal configurations are not disturbed. But talking sense to these guys is next to impossible.

Classic mode

Finally, after so much backlash from the users for years, GNOME developers had to implement the classic mode, so users could be able to run their beloved GNOME 2 interface with GNOME 3 technology.

Unfortunately, GNOME classic is more of a hack than anything else; it’s full of bugs and inconsistencies. Sure, this mode is still in its infancy, and will probably improve in the next releases, but it’s safe to bet that it would never be as polished and usable as GNOME 2 was.

Sorry GNOME devs; too little, too late.

Suckage

I am not going to list every single reason why GNOME 3 sucks, I would never finish this post, and there’s plenty of people that have already added their share (I’ll list some of them).

Personally for me the most obvious way to see that GNOME 3 defaults are brain-dead is the way alt+tab works. The whole purpose of work-spaces is to separate work, instead of having all your browser windows, terminals, editors, email client, and so on, on the same work-space, you split them among different ones. Even their own design references for GNOME shell say so. So, to make alt-tab cycle through the windows in all works-paces instead of only the current one is idiotic; it undermines the whole purpose of work-spaces.

Instead of accepting this obvious fact (which many users complained about), they disregard any feedback, and say “oh, you can use an extension for that”. Never mind the fact that a huge percentage of their users need to install the extension just to have a sane behavior, and that the extension can break at any moment.

gnome-3-is

Another quite obviously stupid behavior is to switch to an already running “application” instead of opening a new one. For example; if I press the Windows key, and type chromium, I expect a new browser window to open, but no, instead I’m unexpectedly dragged to another work-space where there’s already a browser window. This makes sense on single-window applications, but not on other ones.

Linus Torvalds and others complained about this, and I proposed a solution; add a menu item when right clicking on the application that says “Always open new window”; problem solved. Now each time the user tries to open this application, it’s done in a new window. But no, users don’t know what they want; GNOME developers know better what’s best for us.

The list of stupidities in GNOME 3 is never ending:

gnome-3

The Linux way

What GNOME should have done is simple; don’t ever, ever break user experience. This is how the Linux project has managed to become the most successful software project in history. There are no forks of the Linux kernel, simply because there’s no need; user experience is never broken; what works in v2.0, works in v3.0, and will work in v4.0. New features are added carefully, making sure nothing breaks, and users always have a way to make the system work the way they expect it to work.

GNOME shell should have been developed alongside GNOME 2 and users should have given the option to turn this mode on and off. Slowly, GNOME shell would get more features from GNOME 2, until it was a full replacement, and at the same time GNOME 2 would evolve to integrate GNOME 3 technologies (like GTK+ 3.0). Eventually GNOME shell becomes the default, but always with the option to use the classic interface. After some cycles, the GNOME shell interface is bound to be a full replacement for the classic one.

Unfortunately, they chose not to do this.

Going this way requires more effort, of course, but it’s the only way to move forward without loosing developers and users along the way. GNOME developers argued that they didn’t have the resources to do such careful move, but that is clearly false, as the developers working on MATE, Cinnamon and GNOME classic show; there’s developers interested in making things work for the traditional mode.

They chose not to listen to the warnings, and they became impatient and went forward with a move that had the potential of angering a lot of users. Well, this potential was fully realized, and now they are paying the consequences.

They were wrong

They argued this was like GNOME 2; the people complaining about it will eventually learn the new ways, and most will be happy. They were wrong.

Even to this day people keep complaining about GNOME 3, how the interface doesn’t make sense, how the developers don’t listen, and how the design is brain-dead.

There’s no other way to put it; GNOME 3 was a mistake.

Bridge support in git for mercurial and bazaar

I’ve been involved in the git project for some time now, and through the years I’ve had to work with other DSCMs, like mercurial and monotone. Not really as a user, but investigating the inner workings of them, mostly for conversion purposes, and I’ve written tools that nobody used, like mtn2git, or pidgin-git-import, but eventually I decided; why not write something that everybody can use?

So, I present to you git-remote-hg, and git-remote-bzr. What do they do?

git clone "hg::http://selenic.com/repo/hello"
git clone "bzr::lp:gnuhello"

That’s right, you can clone both mercurial and bazaar repositories now with little to no effort.

The original repository will be tracked like any git repository:

% git remote show origin
* remote origin
  Fetch URL: hg::http://selenic.com/repo/hello
  Push  URL: hg::http://selenic.com/repo/hello
  HEAD branch: master
  Remote branches:hg and mercurial. 
    branches/default tracked
    master           tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (create)

You can pull, and in fact push as well. Actually, you wouldn’t even notice that this remote is not a git repository. You can create and push new branches… everything.

You might be thinking that this code being so new might have many issues, and you might screw up a mercurial repository. While that is true to some extent, there’s already a project that tries to act as a bridge between mercurial and git: hg-git. This project has a long history, and a lot of people use it successfully. So, I took their test framework, cleaned it up, and used to test my git-remote-hg code. Eventually all the tests passed, and I cloned big mercurial repositories without any issue, and the result were exact replicas, and I know that because the SHA-1’s were the same :)

So you can even keep working on top of clones you have created with hg-git, and switch back and forth between git-remote-hg and hg-git as you wish. The advantage over hg-git, is that you don’t need to use the mercurial interface at all, you can use git :)

To enable the hg-git compatibility mode you would need this:

% git config --global remote-hg.hg-git-compat true

What about the others?

Surely I must not be the first one try something this cool, right? I don’t want to dwell into all the details why none of the other tools suited my needs, but let’s give a quick look:

hg-git

It’s for mercurial, not git. So you need to use it through the mercurial UI.

fast-export

This is a very nice tool, but only allows you to export stuff (fetch an hg repo), and you have to manually run the ‘git fast-import’ command, setup the marks, and so on. Also, it’s not very well maintained.

Another git-remote-hg

It needs hg-git.

Yet another git-remote-hg

You need quite a lot of patches on top of git, so your vanilla version of git is not going to cut it. In addition to that it doesn’t support as many features; no tags, no bookmarks. Plus it fails all my extensive tests for git-remote-hg.

git-hg

It needs separate tools, such as ‘hg convert’, and fast-export, and it doesn’t use remote helper infrastructure, so it’s not transparent: you have to run git-hg clone, git-hg fetch, and so on.

Another git-hg

It needs hg-git, and doesn’t use git’s remote helper infrastructure either: you have to do git hg-clone, git hg-pull, etc.

There might be others, but you get the point.

The situation in bazaar is not much better, but I’m not going to bother listing the tools.

More coolness

In fact, you can push and pull from and to mercurial and bazaar :)

% git remote -v
bzr	bzr::lp:~felipec/+junk/test (fetch)
bzr	bzr::lp:~felipec/+junk/test (push)
gh	gh:felipec/test.git (fetch)
gh	gh:felipec/test.git (push)
hg	hg::bb+ssh://felipec/test (fetch)
hg	hg::bb+ssh://felipec/test (push)

Isn’t that cool?

So, are there any drawbacks? Surely some information must be lost.

Nope, not really. At least not when using the hg-git compat mode, because all the extra information is saved in the commit messages.

Update: I forgot to mention the only bidirectionality problem: octopus merges. In git a merge can have any number of parent commits, but in mercurial only two are allowed. So you might have problem converting from a git repository to a mercurial one. It’s the only feature that hg-git has, that git-remote-hg doesn’t. As for bazaar, it also allows multiple parents, so it should work fine, but this hasn’t been thoroughly tested.

The only consideration is that in the case of mercurial branches are quite different from git branches, so you need to be really careful to get the behavior you want, which is why it’s best to just ignore mercurial branches, and use bookmarks instead. When you create git branches and push them to mercurial, bookmarks will be created on whatever branch is current.

Trying it out

Just copy them files (git-remote-hggit-remote-bzr) to any location available in your $PATH (.e.g ~/bin), and start cloning :)

Soon they might be merged into upstream git, so they would be distributed as other contrib scripts (e.g. /usr/share/git), and eventually possibly enabled by default. They were distributed as other contrib scripts, and they were going to be enabled by default, until the maintainer decided not to, for no reason at all.

Enjoy!

Progress through the years, and Ruby awesomeness

We’ve all have noticed that when looking back at code you wrote years ago, you realize how much you have progressed–or how much you sucked in the past, depending on your perspective–so it’s always a fair assumption to say, that in the future you will look back at the code you are working now, and feel ashamed. Perhaps this becomes logarithmically less of an issue as time goes on, but at least I still experience this.

I want to share what I think is a good example of this. Ruby code is particularly special in this respect since there’s always tons of ways to write code that do exactly the same.

Back in 2007 when I started msn-pecan, I wanted a script to create a ChangeLog, not because I thought it was needed, but mainly to shut up the people that said the git log wasn’t good enough, and that you still needed to update the ChangeLog in every commit. Few of those people remain today, but my script is still here.

The objective was to output something like this:

== Release 0.0.2 ==

2007-11-30	Felipe Contreras 
	* Add readme.
	* Update copyright stuff.
	* Fix to allow aliases with special characters.

== Release 0.0.1 ==

2007-11-29	Felipe Contreras 
	* Fix warnings.
	* Add display name support.

2007-11-28	Felipe Contreras 
	* Add .gitignore file.
	* Update info.
	* Fix compilation.
	* Stuff required to compile.
	* Initial import.

For some reason I was forced to look at it:

#!/usr/bin/env ruby

require "stringio"

revs=`git rev-list master`
refs=`git for-each-ref --format="%(objectname) %(refname)" refs/tags`

commits = revs.map { |e| e.chomp }

tags = {}

list = {}
list_order = []
$tag = ""

refs.each do |l|
  tag, name = l.chomp.split(" ")
  name = name.slice(/refs\/tags\/v(.*)/, 1)
  tags[tag] = name
end

commits.each do |e|
  commit = StringIO.new(`git show --pretty="format:%an ::%ai::%s\n%b" --quiet #{e}`)
  author, date, subject = commit.readline.chomp.split("::")
  date = date.split(" ")[0]
  if tags[e]
    $tag = tags[e]
    list_order << {:id => $tag}
  end
  id = "#{date}\t#{author}"
  uid = "#{date}\t#{author}\t#{$tag}"
  if not list[uid]
    list[uid] = []
    list_order << {:id => id, :value => list[uid]}
  end
  list[uid] << subject
end

list_order.each do |i|
  id = i[:id]
  value = i[:value]

  if value
    puts "#{id}"
    puts value.map { |e| "\t* #{e}" }.join("\n")
    puts "\n"
  else
    puts "== Release #{id} ==\n\n"
  end
end

If you are familiar with Ruby, you would see that there’s tons of things that don’t make sense there, unnecessary complexity, simpler idioms, etc. So I decided to clean it up:

#!/usr/bin/env ruby

require 'date'

tags = {}
tag = nil
commits = []

class Commit
  attr_reader :author, :date, :tag

  def initialize(id, subject, author, date, tag)
    @id = id
    @author = author
    @date = Date.parse(date)
    @subject = subject
    @tag = tag[1..-1] if tag
  end

  def to_s
    return @subject
  end

end

open("|git for-each-ref --format='%(objectname) %(refname:short)' refs/tags").each do |l|
  commit, id = l.chomp.split(" ")
  tags[commit] = id
end

open("|git log --oneline --format='%H%x00%an %x00%ai%x00%s'").each do |l|
  commit, author, date, subject = l.chomp.split("")
  tag = tags[commit] if tags[commit]
  commits << Commit.new(commit, subject, author, date, tag)
end

commits.group_by(&:tag).each do |tag, commits|
    puts "== Release #{tag} ==\n\n"
    commits.group_by(&:date).sort.reverse.each do |date, commits|
      commits.group_by(&:author).each do |author, commits|
        puts "#{date}\t#{author}\n"
        puts commits.map { |e| "\t* #{e}" }.join("\n")
        puts "\n"
      end
    end
end

So we went from code that took more than 10s to complete, to one that barely took 100ms, it’s much easier to read, smaller, and it’s actually more correct (there was a bug in sorted dates). Not bad!

I was particularly surprised by the method group_by, which takes any Enumeratable and does what the name says:

[
  {:id => 0, :name => 'foo'},
  {:id => 0, :name => 'bar'},
  {:id => 1, :name => 'roo'} ].group_by { |e| e[:id] }
=> {
  0=>[{:id=>0, :name=>"foo"}, {:id=>0, :name=>"bar"}],
  1=>[{:id=>1, :name=>"roo"}] }

And then, in the functional tradition, you can sort, reverse, etc. the result.

Moreover, instead of getting the full output of a command with `git command`, it’s much better to pipe it with open("|git command"), something obvious, but a bit tedious to do in other languages.

The rest is git magick–I’ve been progressing on that front as well :)

Did I mention I love ruby?