git update: the odyssey for a sensible git pull

While git pull kind of works, over the years a slew of people have pointed out many flaws, and proposed several fixes. Some have even suggested that newcomers should be discouraged from using the command (as they often use it wrongly), and others to remove it entirely.

I spent several days digging through the entire history of the Git mailing list in order to document all the discussions related to git pull and its default mode. It’s a long story, so grab some coffee (a whole pot of it).

Update: I wrote a shorter article that only talks about why git pull is broken.

Preamble

The entire post is going to be talking about fast-forward merges so I’ll attempt to explain them briefly.

Basically when your history and the remote history have diverged, like:

non-fast-forward

a fast-forward merge is not possible: you need to create a real merge that will have two parents: C and B.

The fast-forward case is much simpler:

fast-forward

While you can create a merge, in this case it’s not necessary: the “master” branch pointer can simply move forward to B.

The whole story is about what happens when a novice does git pull when a fast-forward is not possible.

--ff-only

The story begins in 2008 with a request from Sverre Hvammen Johansen to add the --ff-only option to git merge (then called git-merge), since in his workflow a certain branch was only allowed to fast-forward.

A few weeks later we get the first patch to implement the --ff=only merge strategy. Unfortunately the patch was more complicated that it needed to be, and it wasn’t merged.

Sverre gave it another go a few months later. Junio C. Hamano (the maintainer) said “with vastly improved documentation and justification compared to the previous rounds, I am beginning to actually like this series”. Unfortunately even though Sverre sent several rounds, they were never merged. Good attempt by Sverre.

Another attempt was made at the beginning of 2009 by Yuval Kogman, this time only adding the --ff-only option. What it did was simple: if a fast-forward is possible then do it, if not then fail. Everyone was in favor. Initial versions of the patch had some issues, those were fixed, but unfortunately the last iteration received no feedback. Thanks for trying though.

A few months later, the question was asked “how do you fast-forward your local dev to sync up with origin/dev?“. If the previous patch was merged, the answer would have been simple: git merge --ff-only, but since that wasn’t the case, Junio replied with:

git push . origin/dev dev

This is not user-friendly if you ask me, and later on turned out it wasn’t even correct.

Months later Randal L. Schwartz brings up the same issue and demands a simple command that can be told to users in the #git IRC channel who constantly asked this question. It is pointed out such thing has been implemented twice already, but never merged.

To which Junio replied:

Do you mean twice they were both found lacking, substandard, useless, uninteresting, buggy, incorrect, or all of the above?

Geez! Eyvind was just trying to help. They clearly were not useless nor uninteresting, since many people had rallied in favor of such a feature, and yet more joined saying +1, even in this very thread.

The first big one

At the end of the year appears the first big discussion about the asymmetry of pull and push. Thomas Rast makes very bold suggestions, like deprecating git fetch completely. These drastic changes would have worked using the suggestion of Wesley J. Landaker to add a configuration, and deprecation warnings.

This proposal was not taken seriously; Junio considered the suggestion to turn git pull into git fetchmindless mental masturbation“, and did not see how git pull and git push could ever be symmetric.

On the other hand an interesting piece of data was provided by Björn Steinbrink. He investigated 10 days of IRC activity in the #git channel and found plenty of users confused by git pull, many expected not only there to be a --ff-only option, but this behavior to be the default.

Aside from --ff-only, other main ideas were:

  1. Add a --merge option to git pull
  2. Add a pull.merge configuration

Björn Gustavsson used this big thread as inspiration to implement --ff-only for both git merge and git pull.

Finally. After three implementations, Junio accepted the idea and merged the patches in October 2009.

Begging for a configuration

Not long after the introduction of git pull --ff-only (6 months later), the first request for a configuration to make it the default appears in 2010. Peter Kjellerstedt argues that pull.options and merge.options configurations would fit him well. Not only that, but he thinks --ff-only should be the default. Nothing comes of this.

In 2013 people in the Linux Kernel Mailing List found issues with git pull and Linus Torvalds brought them to the Git mailing list. Linus argued that depending on whether you are a maintainer or a submaintainer you might want to do either --ff-only, or a true merge. An alias could do the trick (for pull --ff-only), but asked the Git community for ideas.

One of the interesting ideas came from Theodore Ts’o; have a per-repository configuration. Junio did follow this thread closely, but nothing happened.

Only days later, Ramkumar Ramachandra introduces the idea of a pull.default configuration which ideally could be specified on a repository-specific basis. Ramkumar doesn’t seem to have pursued his idea further.

Second big one

A couple of months later Andreas Krey triggered a huge discussion when he pointed out that the order of the parents when doing a merge was wrong. What does that mean?

So this is what git pull does:

The new merge commit D merges B into C, this is fine if you are a maintainer, and instead of “origin/master” you are pulling something like “john/topic” into your “master” branch, in that case your “master” (C) is the first parent. But it’s not OK for normal developers (i.e. non-maintainers).

A normal developer would want to do the opposite: to merge his “master” (C) to the upstream “origin/master” (B), so the first parent is “origin/master” (B). This is not a trivial thing. The order of the parents completely changes how history is visualized, and even traversed, plus it’s important when resolving conflicts, along many other things.

Junio’s response:

Don’t do that, then.

For the record: there’s no way to tell git pull not to reverse the parents, and the four people that expressed an opinion said they would rather have the order reversed (John Szakmeister, Jeremy Rosen, John Keeping, and of course Andreas Krey).

Junio said he would not be opposed to an option to reverse the order of the parents, but as you’ll see later on, when I sent a proposal to do precisely that he ignored it.

Yet again another suggestion to make --ff-only the default, this time from John Keeping. This round I chimed in, and so did many others. Junio was not convinced this was the way to go, because integrators would be affected:

It is not about passing me, but about not hurting users like kernel folks we accumulated over 7-8 years.

Since I have worked on the Linux kernel, I explained to Junio that most lieutenants (submaintainers) don’t do git pull; they send pull requests to Linus, so essentially Linus is the only person that does git pull as an integrator (as Junio does). Linus is certainly capable of configuring git pull to something other than --ff-only (which I don’t know why Junio switched for --rebase in his responses).

That is not something I can agree or disagree without looping somebody whose judgement I can trust from the kernel circle ;-).

That somebody was none other than Linus Torvalds, who agreed with us:

It would be a horrible mistake to make “rebase” the default, because it’s so much easier to screw things up that way.

That said, making “[--ff-only]” the default, and then if that fails, saying [a big warning message], [would make sense].

Linus suggested it would be even better to make this configuration per-branch, which is similar to the suggestion of Theodore Ts’o’s idea of making it per-repository.

Eventually, using Mercurial as inspiration, I suggested the following error message:

The pull was not fast-forward, please either merge or rebase.

This seemed to land on Junio, who finally saw why --ff-only made sense:

Initially I said limiting “git pull” to “--ff-only” by default did not make sense, but when we view it that way, I now see how such a default makes sense.

John Keeping came with another good idea by using a hypothetical command git update as an example. This command, unlike git pull would have the correct order of the parents, and would be similar to svn update, and others. While John didn’t propose such command, eventually I did.

After Junio realized the value of --ff-only he event sent a patch integrating Linus’ suggestions and mine. However, I came up with a simpler approach that in my opinion was better, and that triggered another big discussion.

Third big one

My response to Junio’s patch triggered yet another big discussion.

Richard Hansen argued that git pull was straight up dangerous, and he recommended this instead:

git remote update -p
git merge --ff-only @{u}'

He suggested people have that as a git up alias.

In September of 2013 I sent a patch series to add a pull.mode configuration which can be set to merge-ff-only. It was completely ignored, however Junio soon after made a similar proposal on the other thread: pull.integrate with an option of fail back. I argued that my proposal of pull.mode was better, and other people agreed.

Another point I argued is that we needed a deprecation warning before changing the default mode, something like “in the future you would have to choose a merge or a rebase, merging automatically for now”.

Jeff King provided some useful information from GitHub trainers, essentially they didn’t like the rebase option by default because most people don’t know what a rebase is. This is what prompted me to add the following line to the warning:

If unsure, run “git pull --merge“.

v3 of my patch series did receive some feedback, but it got stuck with trivial details. I tried to fix them for v4, but didn’t receive any feedback at all.

Moving on

At the beginning of 2014 David Aguilar sent a couple of patches to add the pull.ff configuration, and Junio simply merged them with no comment.

This ignored all the previous feedback, and doesn’t add a configuration per-repository, or per-branch, but well… At least there’s a configuration now (sort of).

Pull is Evil (first huge one)

In April of 2014 Marat Radchenko pointed out a list of problems 20 newcomers in his organization faced after trying everyday tasks. Most of the problems came from the order of the parents in git pull. Marc Branchaud pretty early on renamed the thread to “Pull is Evil” since in his opinion git pull is pretty much broken, due to the fact that different people use it in different ways.

I think the pull command should be deprecated and quietly retired as a failed experiment.

Junio brought up again the idea of a git update command, which would demarcate the behavior meant for integrators, and normal developers.

Another interesting idea came from W. Trevor King: add an --interactive option (like git add or git rebase) so the user can choose to a) merge, b) rebase, or c) fast-forward.

I pointed that this has been discussed before, everyone agrees the default should change, and I had already sent the patches. Matthieu Moy rejected the notion that everyone was in favor, since he himself wasn’t, but I pointed that he was the only person not in favor:

But in fact, he was in favor, not with changing the default, but with the configuration I proposed. So he wasn’t against all my patches, just one.

Either way I still think Matthieu Moy is wrong. Yes he is right in questioning why ask if to merge or rebase if most newcomers would not want to know at that point in time, but the answer is simple: they don’t have to know, they can simply do git pull --merge and find out later why that --merge is necessary. Exactly the same thing happens with git commit --all.

Pull is Mostly Evil (Pull is Evil part 2)

Marc Branchaud din’t find a good subthread to reply to, so he started yet another thread. In it he suggested that since git pull can only do the right thing if configured properly, it should be an advanced command that must be configured to be used, and when it’s not configured it does nothing by default.

Junio once more pondered about a git update command that would separate the two clearly distinct ways of using git pull.

Being frustrated with the lack of follow-up I made the prediction that the consensus and the proposals will likely be ignored and nothing would change:

And it has been discussed before. If history is any indication, it will be discussed again.

Spoiler: I was right.

Jeff King did not like my comment, and also stated that he wasn’t sure the majority of users would want the parents on a merge done with git pull to be the other way. This despite many big threads started precisely because the order of the parents was wrong, and many developers stating that this was the only reason why they didn’t use git pull, nor did they recommend newcomers to use it. I offered to point him to the many endless discussions in the mailing list.

Richard Hansen wrote a comprehensive description of the issues, and how they could be solved with different new commands: git update and git integrate, as well as other options.

The rest of the discussion was pretty much Richard Hansen and me hashing out different use-cases and ways to deal with them.

Noise

The discussion did not go without hiccups. In particular David Kastrup and James Denholm threw some personal attacks at me that had absolutely nothing to do with the discussion. Back then I was much more abrasive that I am today, in many instances throughout the entire history of the mailing list (including these discussions) the present me would have reacted differently, but that doesn’t change the fact that I’ve always focused on the code, not so much the feelings of participants of the discussion.

I quickly ignored them and focused on the code itself.

One more try

Since this issue popped yet again, I sent v5, and a v6 of my pull.mode approach. They did receive some amount of feedback, but in particular there was a comment from Richard Hansen that showed yet another issue with git pull.

Supposing we are currently on the branch “master”, and the upstream branch is “origin/master”.

  1. git pull --rebase (rebase master onto origin/master)
  2. git pull --merge (merge origin/master into master)
  3. git pull --rebase github john (rebase master onto github/john)
  4. git pull --merge github john (merge github/john into master)

Half of these are wrong. #1 is correct, since when we do git pull --rebase we want to update our “master” to “origin/master”, so we rebase on top of, but #2 is wrong, because when we use --merge we want a similar thing: we want to merge “master” to “origin/master”, not the other way around. This is what Andreas Krey pointed out, which started the second big thread. But in addition #3 is wrong too, because when we specify a repository and a branch to pull from, we want that branch to rebase on top of master, not the other way around.

So basically it’s all wrong.

Summary

After listening and responding to everyone, all the issues with git pull were clear on my mind, but I realized not everyone had the patience, the stamina, or the time to go through all the responses, so I wrote a summary.

I reduced the main problems with git pull to two:

  1. It does unwanted merges
  2. It does merges with reversed parents

The deeper issue is that git pull is in fact used in two very different ways: to integrate and to update. So when you try to fix the defaults for one, you stumble into unintended consequences for the other. While I do see clearly in my mind how git pull can be disentangled in a way that the defaults would work correctly for everyone, it is pointless if I’m the only one who sees that, because I can’t convince anyone (except perhaps Richard Hansen).

Instead I chose an escape hatch, a completely new command: git update, which would integrate the input from everyone in order to do what git pull can’t do cleanly.

By default git update would be essentially the same as git pull --ff-only (or more like my pull.mode=ff-only), which does solve the main issue: newcomers won’t do real merges by mistake. But it also solves the issue about the order of the parents:

  • git update --merge (merge “master” into “origin/master”)
  • git pull --merge github john (merge “github/john” into “master)

Not only that, but we could fix the order of the parents for rebase too:

  • git update --rebase (rebase “master” onto “origin/master”)
  • git pull --rebase github john (rebase “github/john” onto “master”)

Now everything is fixed. Additionally the default mode for update could be configured with update.mode, and the mode for pull with pull.mode (e.g. update.mode=rebase, pull.mode=merge). Not only that, but this could be configured per-branch (as Linus Torvalds and Theodore Ts’o suggested) with branch.master.updatemode and branch.master.pullmode. This was not only a proposal, I wrote the patches for all of that.

All users agreed my proposed git update was good, as Philippe Vaucher stated:

I know patches from Felipe are sensitive subject and that a lot of people probably simply ignore him entirely, however I think this patch deserves more attention.

No developer commented on it.

Moving on

Nothing resulted from the big Pull is Evil thread, except after it was finished, Jim Garrison asked how would pull do the wrong thing? Junio sent a comprehensive reply and Stephen P. Smith’s proposed to pretty much copy-paste the email from Junio into the list of howtos, and was accepted without any pushback. Not only was the fist version accepted, but Junio himself fixed the patch so no v2 was necessary from Stephen.

Unnecessary merge

At the beginning of 2018 Linus Torvalds once again contacted the mailing list due to an “unnecessary merge” in the Linux kernel. He explained the difference between a merge and a “back-merge”, and why in one you want a fast-forward, but not the other. Unfortunately git doesn’t know which situation we are in.

He even suggested the following configuration:

   git config --global alias.update pull --ff-only

This is precisely what my proposed git update did by default. Linus Torvalds even used the exact same name.

Junio then proceeds to suggest a per-repository configuration pullFF=only, when in fact Linus had already proposed a per-branch configuration back in 2013, and I had already implemented it. Nothing happens.

Why merge?

In 2020 Robert Dailey asks why does git pull use merge by default, he argues most people prefer rebases. Once again Elijah Newren argues that it would be desirable to consider a transition to ff-only by default. Once again Konstantin Tokarev tells Junio that in reality what newbies often do is wrong merges, and that ff-only would be better. Robert Dailey also gets convinced that ff-only is better. Alex Henrie mentions a patch he wrote but was reluctant to send it in order to avoid a big discussion.

The warning

Alex Henrie finally decided to send his patch which adds a warning telling users that running git pull without specifying to either merge or rebase is discouraged. Initially Alex wrote the warning to state that the default behavior would change in a future version of git, but Junio objected to that because there was “no consensus”:

Sorry for not catching this in the earlier round, but I do not think anybody argued, let alone the community came to a concensus on doing, such a strong move. Am I mistaken?

After five rounds of comments from Junio, he accepted the patch.

Interestingly enough this change immediately caused people to ask what it was:

Also, we have this lovely comment on GitHub:

Thanks for confusing everyone instead of just doing the default strategy when no parameter is given.

Imagine when every (CLI) application warns the user about not having each default value within the configuration file or given as parameter, despite the user is OK with (or does not care about) the default behaviour.

And we have a bug report from Wolfgang Fahl completely in German. Roughly translated: the bug is the warning.

Clearly people did not like this warning.

Pick the right default

On November of 2020 Vít Ondruch complained about the warning and stated that the Git project should choose a sensible default and not bother users with a warning.

I’d like to keep my trust in Git upstream, but this is was not trustworthy decision IMO.

This started because of a bug report on Fedora.

The main argument from Junio is that there was no perfect default:

There is no clear “default that would suit everybody”.

I pointed out that this is a false dichotomy because the options are not 1) implement the perfect solution, and 2) do nothing; we have another option 3) implement a better solution than the current one, and we all know what that solution is: ff-only. Plus we had the patches for that.

Jeff King pointed out that now there’s a pull.ff=only configuration, and he didn’t see how my pull.mode=ff-only was different. However, I explained that my version provides a more friendly error message, also works correctly with pull.rebase=true, and in addition allows a default mode that throws a more correct warning than the current one.

Junio proposed to make git pull --rebase honor pull.ff=only and fail if the pull is not fast-forward:

And then making an unconfigured pull.ff to default to pull.ff=only may give a proper failure whether you merge or rebase. I dunno.

This is obviously the wrong thing to do. As I explained to both Junio and Jeff, if you make git pull --rebase respect that, and you make pull.ff=only the default, then git pull --rebase will always fail (unless it’s a fast-forward case).

Once again I stated what should be the default.

  • git pull (fail by default unless it’s a fast-forward)
  • git pull --merge (force a merge [unless it’s a fast-forward,
    depending on pull.ff])
  • git pull --rebase (force a rebase [unless it’s a fast-forward,
    depending on pull.ff])

Vít Ondruch agreed.

Both Jeff and Alex Henrie argued that what my pull.mode does could be achieved with pull.rebase and pull.ff, and that’s the direction Junio wanted to take. I wasn’t convinced, and at this point I started to convert my patches from 2014 which was no easy task because before git pull was written in shell, but now it had been converted to C, so essentially I had to rewrite everything from scratch. Alex started to write his approach too.

Junio welcomed the competition:

Let’s see how well the comparison between two approaches play out.

Pretty quickly I realized what they suggested was not possible. If I removed my pull.mode and made pull.ff=only the default, then git pull --merge would fail. We want git pull fail by default, but not git pull --merge. To make this approach possible we would need to change the semantics of pull.ff=only, and that didn’t seem a clean solution to me.

pull.rebase=ff-only

In order to avoid all the hurdles from 2013 I decided to write the simplest patch possible that would achieve the desired result. Instead of creating a new pull.mode=ff-only, I reused an existing configuration, and named the mode pull.rebase=ff-only. This obviously was not ideal–since it left the interface in an inconsistent state (didn’t fix all the issues my original patch did)–but it was better than the current situation.

Junio responded that the name looked funny:

It looks quite funny for that single variable that controls the way how pull works, whether rebase or merge is used, is pull.REBASE.

Yes Junio, that’s why I suggested pull.mode instead, but that patch was never merged.

Raymond E. Pasco also agreed ff-only was the way to go.

pull.mode revampled

My first attempt to revive my old series recreated most of the same old behavior, but not all. It only received one comment for a tiny correction on the documentation.

Since my patches for pull.mode were ignored again, on my v2 I sent a bunch of improvements that were not 100% related to my approach, and instead of using my proposed pull.mode, I changed the semantics of --ff-only, which was not my preferred solution, and I clearly demarcated that particular patch with “tentative“. Elijah Newren provided a ton of valuable feedback, and I incorporated most of it, but there was one point where I did not agree:

If unsure, run “git pull --merge“.

I believe that escape hatch is useful, even if it undermines the value of the warning because 1) the warning must be temporary anyway 2) people are complaining about the warning already, 3) input from GitHub trainers already showed many people don’t even know what a rebase is, and 4) the merge is happening already anyway.

I explained my point of view to Elijah using an analogy. If a passenger in a plane is ignoring the safety demonstration he is doing something wrong, however, he should be completely free to do it, it’s his volition. The crew should allow him to ignore the demonstration. Similarly git users should be able to ignore the warning, and they can skip it by simply doing git pull --merge.

I argued that even if he didn’t agree on that particular line of that particular patch, the rest of the changes in the series don’t suddenly make the situation worse. He said he didn’t agree.

Junio used my analogy to liken users doing git pull --merge when they shouldn’t to be the same as passengers that should be removed from a plane:

A team member who runs “pull --[no-]rebase” pushes the result, which may be a new history in a wrong shape, back to the shared repository probably falls into a different category. … Or perhaps in the same public hazard category?

I don’t see how one line in a temporary warning can potentially create a “public hazard”. It’s users doing wrong merges what creates problems, and removing one line from the warning wouldn’t magically fix that. The warning–which if we take Stack Overflow and reddit as indication–isn’t even properly understood by many users, and doesn’t stop the code from creating a merge anyway. If these merges did indeed create a “public hazard”, shouldn’t git simply not do them by default? In other words: make ff-only the default.

Sadly I did not think of that argument at that time, I merely responded that what constitutes a public hazard is up to each individual project to determine, not git. Which is true.

On the other hand he said something that was completely untrue:

I hope “his comment” does not refer to what I said. Redefining the semantics of --ff-only (but not --ff=no and friends) to make it mutually incompatible with merge/rebase was what Felipe wanted to do, not me.

That’s not true. I wanted to introduce pull.mode=ff-only, not change the semantics of --ff-only. But as I had already explained before, --ff-only cannot be made the default without changing its semantics. The patch that introduced pull.mode was ignored (again), so I took a shot at making --ff-only work, and the only way is changing its semantics, which I did in a patch clearly labeled as “tentative” because I was not advocating for that.

And to make it crystal clear I repeated it yet again:

I don’t want to change the semantics of --ff-only; I want to introduce a new “pull.mode=ff-only“.

Moreover, I explained once again that this patch series was merely part 1. In part 2 I would introduce pull.mode. And there’s more parts after that.

For some reason Junio didn’t seem to understand what I was proposing. I explained that with pull.mode=ff-only I wanted to introduce a simple error message, but with pull.mode=default a warning to give users a heads up.

I quite don’t get it. They say the same thing. At that point the user needs to choose between merge and rebase to move forward and ff-only would not be a sensible choice to offer the user.

Other people like Jacob Keller did not have any issue understanding my proposal.

After multiple rounds eventually Junio gets my proposal, but now he doesn’t see the point of taking it slow instead of flipping the default to ff-only right away. Junio said that he assumed everyone has already configured pull.rebase therefore nobody would notice the change. I disagreed and pointed why this whole thread started: because a user refused to configure git and expected git to do the right thing–choosing a useful default.

Junio responded as he did at the start:

Which is basically asking for impossible, and I do not think it is a good idea to focus our effort to satisfy such a request in general. There is no useful default that suits everybody in this particular case.

This is once again the Nirvana fallacy. Nobody asked for a perfect solution, merely a better default.

Additionally he argued the pull.mode=ff-only would only be useful for users that don’t use git “for real”:

But for anybody who uses git for real (read: produces his or her own history), it would be pretty much a useless default that forces the user to say rebase or merge every time ‘git pull’ is run.

I mean… didn’t he just argue that he wanted to make ff-only the default right away? Literally in his last reply. Very confusing.

I showed to him a real example of me using git pull --ff-only “for real”. Additionally I asked him in the most polite way I could a question that should be very valid at this point:

How often do you send pull requests?

He didn’t answer.

Since once again Junio stated that he didn’t see a good reason why certain classes of users would want to configure pull.mode=ff-only, I offered to do some mail archaeology which lead to the creation of this blog post. It is obvious this is needed.

Once again Junio misinterpreted what I was proposing, and once again I explained in great detail. This time however Junio correctly identified that without pull.mode the user cannot explicitly ask for the default behavior, and that in my opinion is a crucial thing the user must be able to do in order to test this proposed new default. Moreover, I explained that the name is irrelevant, it could be pull.mode=ff-only, or pull.rebase=ff-only, or pull.ff=only, but one option should turn on the new behavior we are proposing to users, and currently no option does.

To cut to the chase I explained that we want:

  1. git pull to eventually fail in non-fast-forward cases
  2. A grace period before that switch is flipped

And so far no other proposal but mine achieved that.

Since Junio insisted pull.mode was not necessary (which I had already demonstrated multiple times it is), I asked him a poignant question to unequivocally demonstrate that:

So, after your hypothetical patch, there would be no difference between:

git -c pull.rebase=no -c pull.ff=only pull

and:

git -c advice.pullnonff=false pull

?

Junio answered that both would error out, which is correct. But then he has to concede my point, because:

  • git -c pull.ff=only pull (fail)
  • git -c pull.ff=only pull --merge (fail)

If both fail we cannot tell the user:

Not a fast-forward, either merge or rebase.

Because the user would do git pull --merge and that command would fail as well because it will try to do git merge --ff-only and it’s not a fast-forward. That’s it. pull.ff=only can’t be used for this. Period.

Junio did not reply.

Part 2 was completely ignored.

When Junio sent his regular mail with the status of all patch series he didn’t include any of my proposals. Since Junio himself was the one that asked for Alex and me to provide our competing proposals I replied to him asking for the status of the ff-only topic. He didn’t reply.

Alex’s approach

Junio asked for both my approach and Alex’s approach which he sent in two patches. One changing the warning to state that pull.ff=only will be the new default, and the other actually changing the default.

Junio did not think the approach was correct, and neither did I. When I explained why, Junio mis-read my response, but this time he himself caught the mistake, and then said the following:

If we instead introduced a separate command, say “git update“, that is “fetch followed by rebase” (just like “git pull” is “fetch followed by merge”), to rebase your work on top of updated upstream, there wouldn’t be a need for us to be having this discussion.

Yes, if my suggestion to add git update was heard we wouldn’t be having this discussion (yet again).

But to move forward I argued that there’s three things to do:

  1. Improve the annoying warning
  2. Consider changing the semantics of --ff-only, or implement pull.mode=ff-only
  3. Consider a new “git update” command

Since my patches for pull.mode were not being reviewed, I suggested to focus only on #1 for now.

Jacob Keller chimed in and also agreed the order of the parents with git pull was a significant problem.

I also reminded Junio of the purpose of the patch, to which he replied:

Sorry, I know you keep repeating that “keep in mind”, but I do not quite see why anybody needs to keep that in mind. Has a concensus that the repurposed --ff-only should be the default been established?

This is the title of the patch from Alex we were supposed to be reviewing “pull: default pull.ff to "only" when pull.rebase is not set either“. The whole point is to make pull.ff=only the default, and in that case git pull --merge would fail, which is not what we want. Unless the semantics of pull.ff=only are changed, which is not my proposal, my proposal (pull.mode=ff-only) is better because it doesn’t need that change.

He did not reply again.

Alex stated that he had trouble following the discussion and simply left it at the table.

A fast-forward

In v3 I focused on fixing all the issues without committing to any particular approach. However, Junio objected to my use of the word “fast-forward”:

… I find the phrase “in a fast-forward way” a bit awkward. Perhaps use the ‘fast-forward’ as a verb

I argued that if fast-forward was a verb, we would have git fast-forward command (which I actually think is a good idea), but currently we don’t have it, so fast-forward is a modifier. In particular if git merge is a verb, then --ff is an adverb.

Elijah disagreed:

A square is a special type of a rectangle, but that doesn’t make “square” an adjective; both square and rectangle are nouns.

Since I’m deeply interested in language, this is a discussion I could really weigh in:

Words have multiple definitions. The word “square” is both a noun, and an adjective. It’s perfectly fine to say “square corner”.

Moreover, I found plenty of instances in the documentation which use fast-forward as an adverb. Even the git-merge man page uses “fast-forward merge”.

  • non-fast-forward update
  • fast-forward merges
  • fast-forward check
  • “fast-forward-ness”
  • fast-forward situation
  • fast-forward push
  • non-fast-forward pushes
  • non-fast-forward reference
  • fast-forward cases
  • “fast-forward” merge

This time I managed to convince him.

However it didn’t escape to me the fact that the documentation already talks of “fast-forward update”, so this suggests that the git update command could fill a whole already present in the documentation.

Everything at once

Since Junio had trouble figuring out my proposal in full, I decided to send the 19 patches together at once, and in addition I wrote a detailed explanation of what I was proposing to happen at every step of the way.

Is it more clear what is my proposal?

No reply.

Improve the default warning

Since I didn’t see much hope in fixing git pull‘s default, for v5 I decided to drop everything except what fixes the most urgent issue: the warning on every single pull.

Junio objected to the name of a variable I used: default_mode. He said the word “mode” was overused, and he proposed a “more focused” word, he came with rebase_unspecified. Additionally he didn’t want this variable to be global, even though there were already 38 global variables, so why did he feel that 39 was too much? I have no idea.

Moreover he made some comments on the test changes:

We are merely allowing fast-forward merges without unnecessary merge commits, but we are faced to merge c1 into c2, which is not ff. The command goes ahead and merges anyway, but why shouldn’t we be seeing the message? I am puzzled.

Now, for the most part I haven’t gone into code in this post, but I think it’s important you feel a little bit of my pain here.

git reset --hard c2 &&
test_config pull.ff true &&
git pull . c1 2>err &&
test_i18ngrep ! "Pulling without specifying how to reconcile" err

The name of the test gives a hint to what it’s trying to do “pull.rebase not set and pull.ff=true“. As you can see there’s no pull.rebase configuration (or --rebase), but there is pull.ff=true. So clearly it has something to do with that pull.ff.

What this test is doing is trying to pull c1 into c2, and since the history has diverged the merge cannot be a fast-forward, therefore the code should throw a warning…

except that only happens if the user has not specified any --ff* option (--ff, --no-ff, or --ff-only), either through to command line, or from the pull.ff configuration, and the test does set pull.ff=true. Therefore there’s no warning.

This is behavior Alex Henrie added and Junio accepted with no comment. In my opinion this behavior is wrong, but that’s what the code does at this point, and even Junio did not expect this, that’s why he was puzzled.

I explained to him that the code only checks for opt_ff, and I tried to fix that behavior in v1, v2, v3, and v4 of my patch series, but he did not comment on those patches.

So the tests used to do a fast-forward from c0 to c1, and that would normally throw a warning (as all pulls did), but because pull.ff=true is set, it doesn’t. This test written by Alex Henrie ensures that pull.ff=true doesn’t generate a warning. But after my patch doing a fast-forward from c0 to c1 would not generate a warning anyway, so the pull.ff=true doesn’t do anything, and therefore the test doesn’t do anything. That’s why I changed the test to start from c2, because then the merge is not fast-forward, and a warning is thrown unless pull.ff=true is set, which is what the test was supposed to be testing in the first place.

To show Junio that the tests are doing what they originally intended to do, I modified all the tests to remove the pull.ff configurations:

git reset --hard c2 &&
git pull . c1 2>err &&
test_i18ngrep ! "Pulling without specifying how to reconcile" err

Now the test fails.

Junio was not convinced. He stated that the original test was checking that the fast-forward would work fine, but that’s not true since from the test we can see that the only thing it’s doing is checking there’s no error message. The code could very well have done a real merge instead of a fast-forward, and the test would still pass. I explained to him that the tests that actually check for the fast-forward are in another file. This particular test is only about the warning message.

Junio claimed to know what the original author of the test intended to do and what he cared about, and that was to test that pulling c1 into c0 issued a message, except none of the tests actually did that, they checked the opposite: that the message is not there. So now that my patch skips the message, the tests pass, but for the wrong reason.

git reset --hard c0 &&
git pull . c1 2>err &&
test_i18ngrep ! "Pulling without specifying how to reconcile" err

If instead of starting from c2 we start from c0 as the original code, then the pull is fast forward, and there’s no message, so this test pass. But this test was supposed to be checking the output with pull.ff=true, that’s why it was called “pull.rebase not set and pull.ff=true“, if you remove the configuration it still passes. So in the fast-forward case what is this test supposed to be doing if it cannot possibly fail now? Nothing.

I explained that and in addition I said that I’m not in the habit of attempting to read minds, what the test are trying to do is very clear from the code.

Junio replied.

You do not have to read mind. What is written in the test is clear enough: run “git pull --rebase . c1” into c0 and see what happens.

Yes, that was precisely my point: no need to read minds (although my reading of the code is different).

Do you feel my pain? I don’t mind going as many rounds as necessary to improve a patch series, even ten or more, but these rounds are improving nothing. The name of the variable doesn’t matter, because in a few patches later I remove it, and even these tests for the warning don’t matter because eventually we want to remove the warning completely.

And this is only one test, of one the patches, of one of simplest patch series.

But fine, if Junio wants to keep a bunch of tests that literally test nothing, I’ll oblige. For v6 I left the useless tests untouched and added the real ones separately.

Junio isn’t entirely happy

Junio sent v7 himself, stating he wasn’t happy with the way the --ff* options were handled. He suggested that these options should imply a merge, but I showed situations in which that would become tricky.

Additionally he wrote a patch to remove my global variable, and seized the opportunity to state:

There is no reason to make the code deliberately worse by ignoring improvement offered on the list.

Really? I did not ignore the comments, I responded to all of them, but I disagreed with some. And so if I disagree with any of Junio’s suggestions for improvement I’m making the code “deliberately worse”? If I have to do what he says, then those are not suggestions, those are orders. Not to mention the v7 he is sending contains the patches I sent in v6, which have the useless tests–a change he suggested–and even though I was 100% against, I included it.

Anyway, since Junio didn’t want to discuss his proposal to imply merges any further, finally he merged this patch series.

After this patch the warning becomes much less annoying. So that’s progress.

What happened to the rest of the patches? Nothing.

Today

After this latest attempt to fix git pull, I distanced myself from the project for a few months, and then came back. I decided to resend some of my patches, but now separately, to minimize the possibility of them being rejected.

  1. Cleanups
  2. Documentation
  3. --merge

Of these only #1 was merged (though not yet in “master”). #2 generated yet another discussion of the semantics of “fast-forward” but nobody was against the changes themselves. #3 everyone is in favor, but no comment from Junio.

If you think any of these will eventually be merged you are more hopeful than I am. Also, there’s this comment from Junio: (emphasis mine):

If some contributors get too annoying to be worth our time interacting with, it is OK to ignore them.

He seems to be referring to me.

So where does that leave us? The order of the parents is still wrong. And the default mode still isn’t fast-forward-only. Regardless of what you may think of my communication style, I’m the one that has actually sent patches to fix all these issues. And unlike Junio and many other git developers–I’m not being paid to work on this, I’m doing this altruistically (I don’t even use git pull myself). If I’m not the person to fix this, who is? I left the project for about 6 years and nobody fixed it.

Recently I challenged Alex Henrie to send a patch to try to fix the situation. He took me up on the challenge, and when I reviewed his patch he immediately realized that more work is needed (as I predicted he would find).

What to do now?

The final solution

While writing this post I realized just how many suggestions were proposed, but not implemented, so I decided to try once again to fix the situation, but now consider absolutely all the feedback in a holistic way.

Here’s there result.

git fast-forward

The documentation about what a fast-forward is, and how to attempt one, is scattered all over. Additionally the simplest way to attempt one is with git merge --ff-only which isn’t user-friendly, and some people don’t even consider a fast-forward to be a modifier of a merge (i.e. adverb).

By having git fast-forward command we solve both problems and fast-forward in the git context now becomes a verb: something independent of git merge.

Additionally there’s an advice for newcomers:

hint: Diverging branches can't be fast-forwarded, you need to either:
hint:
hint:   git merge
hint:
hint: or:
hint:
hint:   git rebase
hint:
hint: For more information check "git help fast-forward".
hint:
hint: Disable this message with "git config advice.diverging false"

Advanced users can disable this specific advice (or all advice).

As the advice says, anyone who wants to learn more about fast-forwards can simply do git help fast-forward.

git update

I converted my old tool from shell to C, and although it doesn’t have yet all the features, it does have plenty of functionality.

git update without arguments simply does git fetch + git fast-forward. If the fast-forward fails, we get the above advice. This solves the first problem that newcomers often do merges by mistake.

Also, it’s possible to do git update --merge (or configure update.mode=merge), and on those cases the order of the parents is reversed properly, as everyone wants (everyone that is not an maintainer).

There’s also git update --rebase, and also a per-branch a configuration branch.<name>.updateMode.

pull.mode

And of course all the other patches that improve git pull, including pull.mode=fast-forward, a better warning that shows that in the future git pull will do fast-forwards by default, along with a per-branch configuration branch.<name>.pullMode.

Also, thanks to git fast-forward we also get the same divergence advice, which will be permanent, but easy to turn off.

In total it’s 33 patches, and one additional to flip the default in the future (after users have had a chance to configure pull.mode to whatever they prefer).

The patches have been sent, but I wouldn’t hold my breath.

Sadly this is where the story ends, even though everyone has agreed on what is the path forward “for some reason” the patches will not be merged, and new users will be forever doomed to keep making mistakes because git pull was not designed for them.

Wait a second…

Git is a distributed version control system. You are not forced to use Junio’s repository, you can use mine. Everything I’ve talked about is fixed there. If you are interested why not give it a try?

Note: Some people are thinking that the purpose of this article is to point fingers, but that could not farther from the truth. The only reason names are named is to be able to follow the discussions (or at least attempt to). I am not seeking to assign any blame to any one person for any action or inaction. In any discussion disagreements are to be expected, and it doesn’t have to mean anything more than that.

5 thoughts on “git update: the odyssey for a sensible git pull

  1. Typo I think:

    “without specifying to either merge or rename is discouraged.”

    should be

    “without specifying to either merge or rebase is discouraged.”

    Like

  2. Pingback: Why is git pull broken? – Felipe Contreras

  3. Pingback: The git staging area, the term literally everyone agrees with | Felipe Contreras

  4. Pingback: How not to ban a prolific git developer | Felipe Contreras

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.