How a proper git commit is made (by a git developer)

Throughout my career of 20 years as an open source software developer I’ve seen all kinds of projects with all kinds of standards, but none with higher standards as the Git project (although the Linux kernel comes pretty close).

Being able to land a commit on git.git is a badge of honor that not all developers would be able to get, since it entails a variety of rare skills, like for example the ability to receive criticism after criticism, and the persistence to try many times–sometimes reaching double-digit attempts, not to mention the skill necessary to implement the solution to a problem nobody else saw before–or managed to implement properly–in C, and sometimes the creativity to come up with alternative solutions that address all the criticism received previously.

The purpose of this post is not to say “this is why Git is better than your project” (the Git project has many issues), the purpose is to showcase parts of a fine-tuned development process so you as a developer might consider adopting some yourself.

Discussion

The first part of the process (if done correctly) is inevitably discussion. The git mailing list is open, anyone can comment on it, and you don’t even need to be subscribed, just send a mail to the address and you will be Cc’ed in all the replies.

This particular story starts with a mail by Mathias Kunter in which he asks why git push fails with certain branches, but only if you don’t specify them. So for example if you are in a branch called “topic”, this would fail:

git push

But not this:

git push origin topic

Why? His question is a valid one.

The first reply comes from me, and I explain to him how the code views the two commands, for which a basic understanding of refspecs is needed. Briefly, a refspec has the form of src:dst, so “test:topic” would take the “test” branch and push it to the remote as “topic”. In the first command above, the refspec would be the equivalent of “topic:” (no destination), while the second command would be “topic:topic”. In other words: git does’t assume what name you want as destination.

Did I know this from the top of my head? No. I had to look at the code to understand what it’s doing before synthesizing my understanding in as succinct as a reply as I possibly could write. I often don’t look at the official documentation because I find it very hard to understand (even for experts), and it’s often inaccurate, or ambiguous.

Notice that I simply answered his question of why the first command fails, and in addition I offered him a solution (with push.default=current git would assume the name of the destination to be the same as the source), but at no point did I express any value judgement as of what was my opinion of what git ought to actually do.

Mathias thanked me for my reply, and pushed back on the solution to use the “current” mode because he thought the “simple” mode (which is the default), should behave the same way in this particular case. For his argument he used the documentation about the simple mode:

When pushing to a remote that is different from the remote you normally pull from, work as current.

This is where the nuance starts to kick in. If the “topic” branch has an upstream branch configured (e.g. “origin/topic”), then git push would behave the same in both the “simple” and “current” modes, however, if no upstream branch is configured (which is usually the case), then it depends on the remote. According to Mathias, if he has no upstream configured, then there’s no “remote that is different from the remote you normally pull from” (the remote you normally pull from in this case is “origin”, because upstream branch is “origin/topic”), so “simple” should work like “current”.

In my opinion he is 100% right.

Additionally, I have to say we need more users like Mathias, who even though he knows how to fix the issue for himself, he is arguing that this should be fixed for everyone.

Elijah Newren suggested that perhaps the documentation could be changed to explain that this only happens when “you have a remote that you normally pull from”, in other words: when you have configured an upstream branch. But this doesn’t make sense for two reasons. If you don’t have configured an upstream branch, then the other mode that would be used is “upstream”, but “upstream” fails if you don’t have configured an upstream branch, so it would always fail. Secondly, the reason why it doesn’t always fail is that the former is not possible: when you don’t have configured an upstream branch, “origin” is used, therefore you always have a “remote that you normally pull from”.

I explained to Elijah that the remote is never null, since by default it’s “origin”, and suggested a trivially small modification to the documentation to mention that (since even experts like him miss that), but he suggested an even bigger one:

If you have a default remote configured for the current branch and are pushing to a remote other than that one (or if you have no default remote configured and are pushing to a remote other than ‘origin’), then work as ‘current’.

Oh boy! I cannot even being to explain why I find this explanation so wrong on so many levels, but let me start by saying I find this completely unparseable. And this is the biggest problem the official git documentation has: it’s simply translating what the code is doing, but if the code is convoluted, then the documentation is convoluted as well. I shred this paragraph piece by piece and showed why changing the logic would make it much more understandable.

But at this point I got tired of reading the same spaghetti code over and over again. It’s not just that I found the code hard to follow, it’s that I wasn’t actually sure of what it was doing. And so my reorganization patch series began.

The series

I am of the philosophy of code early, and code often. I don’t believe in design as separate from code, I believe the design should evolve as the code evolves. So I just went ahead and wrote the thing. The first version of my patch series reorganized the code so that the “simple” mode was not defined in terms of “current” and “upstream”, but as a separate mode, and then once it became clear what “simple” actually did, redefine “current” in terms of “simple”, rather than the other way around. It consisted of 11 patches:

  1. push: hedge code of default=simple
  2. push: move code to setup_push_simple()
  3. push: reorganize setup_push_simple()
  4. push: simplify setup_push_simple()
  5. push: remove unused code in setup_push_upstream()
  6. push: merge current and simple
  7. push: remove redundant check
  8. push: fix Yoda condition
  9. push: remove trivial function
  10. push: flip !triangular for centralized
  11. doc: push: explain default=simple correctly

I could describe each one of these patches in great detail, and in fact I did: in the commit messages of each of these patches, but just to show an example of the refactorization these patches do, let’s look at patch #7, #8, and #9.

push: remove redundant check

 static int is_workflow_triangular(struct remote *remote)
 {
-	struct remote *fetch_remote = remote_get(NULL);
-	return (fetch_remote && fetch_remote != remote);
+	return remote_get(NULL) != remote;
 }

There is no need to do two checks: A && A != B, because if A is NULL, then NULL != B (B is never NULL), so A != B suffices.

push: fix Yoda condition

 static int is_workflow_triangular(struct remote *remote)
 {
-	return remote_get(NULL) != remote;
+	return remote != remote_get(NULL);
 }

There’s a lot of Yoda conditions in the git code, and it’s very hard for many people to parse what that code is supposed to do in those situations. Here we want to check that the remote we are pushing to is not the same as the remote of the branch, so the order is the opposite of what’s written.

push: remove trivial function

-static int is_workflow_triangular(struct remote *remote)
-{
-	return remote != remote_get(NULL);
-}
-
 static void setup_default_push_refspecs(struct remote *remote)
 {
 	struct branch *branch = branch_get(NULL);
-	int triangular = is_workflow_triangular(remote);
+	int triangular = remote != remote_get(NULL);

Now that the code is very simple, there’s no need to have a separate function for it.

The rest

Notice that there’s absolutely no functional changes in the three patches above: the code after the patches ends up doing the exactly same as before. The same applies for all the other patches.

There’s three things that I think are important of the overall result: 1. the new function setup_push_simple is a standalone function, so to understand the behavior of the “simple” mode, all you have to do is read that function, 2. the existing function is_workflow_triangular is unnecessary, has the wrong focus, and is inaccurate, and 3. now that it’s easy to follow setup_push_simple, the documentation becomes extremely clear:

pushes the current branch with the same name on the remote.

If you are working on a centralized workflow (pushing to the same repository you pull from, which is typically origin), then you need to configure an upstream branch with the same name.

Now we don’t need to argue about what the “simple” mode does, there’s no confusion, and the documentation while still describing what the code does is easy to understand by anyone, including the original reporter: Mathias.

Great. Our work is done…

Not so fast. This is just the first step. Even though this patch series is perfectly good enough, in the Git project the first version is very rarely accepted; there’s always somebody with a comment.

The first comment comes from Elijah, he mentioned that in patch #3 I mentioned that I merely moved code around, but that wasn’t strictly true, since I did remove some dead code too, and that made it harder to review the patch for him. That wasn’t his only comment, additionally he stated that in patch #4 I should probably add a qualification to explain why branch->refname cannot be different from branch->merge[0]->src, and in patch #10 he said the commit message seemed “slightly funny to [him]” but he agreed it made the code easier to read, and finally in the last patch #11 which updates the documentation:

Much clearer. Well done.

I replied to all of Elijah’s feedback mostly stating that I’m fine with implementing his suggestions, but additionally I explained the reasoning behind changing “!triangular” to “centralized”, and that’s because I heard Linus Torvalds state that the whole purpose of git was to have a decentralized version control system so it makes no sense to have a conditional if (triangular) when that conditional is supposed to be true most of the time.

Additionally Bagas Sanjaya stated that we was fine with the changes to the documentation, since the grammar was fine.

v2

OK, so the first version seemed to be a success in the sense that everyone who reviewed it didn’t find any fatal flaws in the approach, but that doesn’t mean the series is done. In fact, most of the work is yet ahead.

For the next version I decided to split the patch series into two parts. The first part includes the patches necessary to reach the documentation update which was the main goal, and the second part would be everything that makes the code more readable but which is not necessary for the documentation update. For example changing !triangular to centralized is a nice change, but not strictly necessary.

Now, v2 requires more work than v1 because not only do I have to integrate all the suggestions from Elijah, but I have to do it in a way that it’s still a standalone patch series, so anyone who chooses to review v2 but hasn’t seen v1, can understand it. So essentially v2 has to be recreated.

This is why git rebase is so essential, because it allows me to choose which commits to update, and the end result would look like all the suggestions from Elijah had always been there.

But not only that, I have to update the description of the series (the cover letter in git lingo), to explain what the patch series does for the people that haven’t reviewed it before, and in addition explain what changed from v1 to v2 for the people that have already reviewed the first version.

This is where one tool which I think is mostly unknown comes into play: git rangediff. This tool compares two versions of a patch series (e.g. v1 and v2), and then generates a format similar to a diff delineating what changed. It takes into consideration all kinds of changes, including changes to the commit message of every commit, and if there are no changes either in the code or the commit message, that’s shown too.

So essentially people who have already reviewed v1 can simply look at the rangediff for v2 and based on that figure out if they are happy with the new version. The reason why it’s called rangediff is because it receives two ranges as arguments (e.g. master..topic-v1 master..topic-v2).

This time the result was 6 patches. If you check the rangediff you can see that I made no changes in the code, whoever, I changed some of the commit messages, and 5 patches were dropped.

Let’s look at one change from the rangediff for patch #3.

3: de1b621b7e ! 3: d66a442fba push: reorganize setup_push_simple()
    @@ Metadata
      ## Commit message ##
         push: reorganize setup_push_simple()
     
    -    Simply move the code around.
    +    Simply move the code around and remove dead code. In particular the
    +    'trivial' conditional is a no-op since that part of the code is the
    +    !trivial leg of the conditional beforehand.
     
         No functional changes.
     
    +    Suggestions-by: Elijah Newren <newren@gmail.com>
         Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
     
      ## builtin/push.c ##

It should be pretty straightforward to see what I changed from the commit message.

The second part is what is most interesting. Not only does it includes better versions of the 5 patches that were dropped from the previous series, but it includes 10 patches more. So this is something that has to be reviewed from scratch, since it’s completely new.

  1. push: create new get_upstream_ref() helper
  2. push: return immediately in trivial switch case
  3. push: reorder switch cases
  4. push: factor out null branch check
  5. push: only get the branch when needed
  6. push: make setup_push_* return the dst
  7. push: trivial simplifications
  8. push: get rid of all the setup_push_* functions
  9. push: factor out the typical case
  10. push: remove redundant check
  11. push: fix Yoda condition
  12. push: remove trivial function
  13. push: only get triangular when needed
  14. push: don’t get a full remote object
  15. push: rename !triangular to same_remote

I’m not going go through all the different attempts I made locally to arrive to this series, but it was way more than one. I used git rebase over and over to create a sequence of commits to arrive to a clean result, where each commit implemented a logically-independent change that I thought would be easy to review, would not break previous functionality, and would make the code easier to read from the previous state.

The highlights from this series is that I added a new function get_upstream_ref that would be used by both the “simple” and “upstream” modes, but additionally by reorganizing the code I was able to remove all the setup_push_* functions, including the one I introduced: setup_push_simple. By moving everything into the same parent function now it’s easy to see what all the modes do, not only the “simple” mode. Additionally, instead of changing !triangular to centralized, I decided to change it to same_remote. Upon further analysis I realized that you could be in a triangular workflow and yet be pushing into the same repository (that you pull from), and the latter is what the code cared about, not triangular versus centralized. More on that later.

This time much more people commented: Mathias Kunter, Philip Oakley, Ævar Arnfjörð Bjarmason, and Junio C Hamano himself (the maintainer).

Philip Oakley pointed out that the documentation currently doesn’t explain what a triangular workflow is, but that isn’t a problem of this patch series. Ævar Arnfjörð Bjarmason provided suggestions for the first part of the series, but those were not feasible, and overridden by the second part of the series. Mathias Kunter said he actually preferred the explanation of the “simple” mode provided in another part of the documentation, but he incorrectly thought my patches changed the behavior to what he suggested, but that wasn’t the case.

Junio provided comments like:

  • Much cleaner. Nice.
  • Simpler and nicer. Good.
  • Nice.
  • Nicely done.
  • Well done.

If you want to take a look at the change from the most significant patch, check: push: get rid of all the setup_push_* functions.

His only complaints were my use of the word “hedge” (which he wasn’t familiar with), of “move” (when in fact it’s duplicated), “reorder” (when in his opinion “split” is better), and that he prefers adding an implicit break even if the last case of a switch is empty.

I explained to Junio that the word “hedge” has other meanings:

* to enclose or protect with or as if with a dense row of shrubs or low trees: ENCIRCLE

* to confine so as to prevent freedom of movement or action

Some synonyms: block, border, cage, confine, coop, corral, edge, fence, restrict, ring, surround.

So my use of it fits.

For most of his comments I saw no issue implementing them as most didn’t even involve changing the code, but additionally he suggested changing the order so the rename of triangular is done earlier in the series. While I have no problem doing this and in fact I already tried in an interim version of the series, I knew it would entail resolving a ton of conflicts, and worse than that, if later on somebody decides they don’t like the name same_remote I would have to resolve the same amount of conflicts yet again. I said I would would consider this.

Moreover, I recalled the reason why I chose same_remote instead of centralized, and I explained that in detail. Essentially it’s possible to have a triangular workflow that is centralized, if you pull and push to different branches but of the same repository. The opposite of triangular is actually two-way, not centralized.

centralized = ~decentralized
triangular = ~two-way

This triggered a discussion about what actually is a triangular workflow, and that’s one of the benefits of reviewing patches by email: discussions turn into patches, and patches turn into discussions. In total there were 36 comments in this thread (not counting the patches).

v3

For the next version I decided not just to move the same_remote rename sooner in the series as Junio suggested, but actually in the very first patch. Although a little risky, I thought there was a good chance nobody would have a problem with the name same_remote, since my rationale of triangular versus two-way seemed to be solid.

Part 1:

  1. push: rename !triangular to same_remote
  2. push: hedge code of default=simple
  3. push: copy code to setup_push_simple()
  4. push: reorganize setup_push_simple()
  5. push: simplify setup_push_simple()
  6. push: remove unused code in setup_push_upstream()
  7. doc: push: explain default=simple correctly

Part 2:

  1. push: create new get_upstream_ref() helper
  2. push: return immediately in trivial switch case
  3. push: split switch cases
  4. push: factor out null branch check
  5. push: only get the branch when needed
  6. push: make setup_push_* return the dst
  7. push: trivial simplifications
  8. push: get rid of all the setup_push_* functions
  9. push: factor out the typical case
  10. push: remove redundant check
  11. push: remove trivial function
  12. push: only check same_remote when needed
  13. push: don’t get a full remote object

Moving the same_remote patch to the top changed basically the whole series, as can be seen from the rangediff, but the end result is exactly the same, except for one little break.

Junio mentioned that now the end result matches what he had prepared in his repository and soon included then in his “what’s cooking” mails.

* fc/push-simple-updates (2021-06-02) 7 commits

 Some code and doc clarification around "git push".

 Will merge to 'next'.

* fc/push-simple-updates-cleanup (2021-06-02) 13 commits

 Some more code and doc clarification around "git push".

 Will merge to 'next'.

So there it is, unless somebody finds an issue, they will be merged.

To be honest this is not representative of a typical patch series. Usually it takes more than 3 tries to got a series merged, many more. And it’s also not common to find so many low-hanging fruit.

switch (push_default) {
default:
case PUSH_DEFAULT_UNSPECIFIED:
case PUSH_DEFAULT_SIMPLE:
    if (!same_remote)
        break;
    if (strcmp(branch->refname, get_upstream_ref(branch, remote->name)))
        die_push_simple(branch, remote);
    break;

case PUSH_DEFAULT_UPSTREAM:
    if (!same_remote)
        die(_("You are pushing to remote '%s', which is not the upstream of\n"
              "your current branch '%s', without telling me what to push\n"
              "to update which remote branch."),
            remote->name, branch->name);
    dst = get_upstream_ref(branch, remote->name);
    break;

case PUSH_DEFAULT_CURRENT:
    break;
}

refspec_appendf(&rs, "%s:%s", branch->refname, dst);
Documentation/config/push.txt | 13 +++----
builtin/push.c | 91 +++++++++++++++++++------------------------
2 files changed, 47 insertions(+), 57 deletions(-)

The commit

These two patch series cleaned up the code and improved the documentation, but they didn’t actually change any functionality. Now that it’s clear what the code is actually doing, Mathias Kunter’s question is easily answered: why is git push failing with “simple”? Because get_upstream_ref always fails if there’s no configured upstream branch, but should it?

All we have to do is make it not fail. In other words: if there’s no upstream branch configured, just go ahead, and thus it would behave as “current”.

Then this:

If you are working on a centralized workflow (pushing to the same repository you pull from, which is typically origin), then you need to configure an upstream branch with the same name.

Becomes this:

If you are working on a centralized workflow (pushing to the same repository you pull from, which is typically origin), and you have configured an upstream branch, then the name must be the same as the current branch, otherwise this action will fail as a precaution.

Which makes much more sense.

Right now pushing to master works by default:

git clone $central .

git push

But not pushing to a new topic branch:

git clone $central .
git checkout -b topic

git push

It makes no sense to allow pushing “master” which is much more dangerous, but fail pushing “topic”.

And this only fails because git push uses “origin” by default, and so does git clone. If you clone with --origin foobar the above will work too, because same_remote will be false.

This is the patch that changes the behavior to make it sensible, where I explain all of the above, and the change in the code is simple, basically don’t die().

It took 47 days from the moment Mathias sent his question to the point where Junio merged my patches to master, but we are close the finish line, right?

No, we are not.

Convincing Junio of some code refactoring is relatively easy, because it’s simply a matter of showing that 2 + 2 = 4; it’s a technical matter. But convincing him of what git ought to do is much more difficult because it requires changing his opinion using arguments, this part is not technical, but rhetorical.

For reference, even though several years ago I managed to convince everyone that @ is a good shortcut for HEAD, Junio still complains that it “looks ugly both at the UI level and at the implementation level“, so for some reason my arguments failed to convince him.

So can Junio be convinced of this obvious fix? Well, everything is possible, but I wouldn’t be holding my breath, especially since he has decided to ignore me and all my patches.

Either way the patch is good. It’s simple thanks to the many hours I spent cleaning up the code, and benefited from all the input from many reviewers. And of course if anyone still has any comments on it they are free to state them on the mailing list and I’d be happy to address them.

Why is git pull broken?

A lot of people complained that my previous post–git update: the odyssey for a sensible git pull–was too long (really? an article documenting 13 years of discussions was long?), and that a shorter version would be valuable. The problem is that the short version is actually too short:

Do not use git pull.

That’s it, really.

But why? Even thought it’s obvious for me, and many other developers why git pull is broken and should not be used by most users, presumably a lot of people don’t know that, since they continue to use it.

Here it is.

It creates merge commits

What most people want to do is synchronize their local branch (e.g. “master”) with the corresponding remote branch (e.g. “origin/master”), in particular because if they don’t, git push fails with:

To origin
! [rejected] master -> master (non-fast-forward)
error: failed to push some refs to 'origin'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull …') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

OK, so we need to “integrate” the remote changes with git pull, so presumably git pull is the mirror of git push, so it makes sense.

Except it’s not, and git pull was never designed with this use case in mind. The mirror of git push is git fetch which simply pulls the remote changes locally so you can decide later on how to integrate those changes. In Mercurial hg pull is the equivalent of git fetch, so in Mercurial hg push and hg pull are symmetric, but not in git.

At some point in time a path to make git pull be symmetric to git push was delineated, but the maintainer of the Git project considered it “mindless mental masturbation“, so forget about it.

After you have pulled the changes with git fetch, then there’s two possibilities: fast-forward and diverging.

fast-forward

A fast-forward is simple; if the local branch and the remote branch have not diverged, then the former can be easily updated to the latter.

In this case “master” (A) can be fast-forwarded to “origin/master” (C) (only possible if the branches have not diverged).

merge

However, if the branches have diverged, it’s not that easy:

In this case “master” (D) has split of “origin/master” (C) so a new commit (E) is needed to synchronize both.

rebase

There’s another more advanced possibility if the branches have diverged:

In this case the diverging commit of the local branch “master” (D) is recreated on top of “origin/master” (C) so the resulting history is linear (as if it never had diverged in the first place and the base of the local branch was C).

Choices

OK, so if the branches have diverged you have two options (merge or rebase), which one should you pick? The answer is: it depends.

Some projects prefer a linear history, in those cases you must rebase. Other projects prefer to keep the history intact, so it’s fine if you merge. If you don’t do many changes then most of the time you can fast-forward.

Most experts would do a rebase, but if you are new to git a merge is easier.

We are still nowhere near a universal answer, and what do most people do when the answer is not clear? Nothing. By default git pull does a merge, so that’s what most people end up doing by omission, but that’s not always right.

So that’s the first problem: git pull creates a merge commit by default, when it shouldn’t. People should be doing git fetch instead and then decide whether to merge or rebase if the branches have diverged (a fast-forward is not possible).

Merges are created in the wrong order

Let’s say the project allows merges, in that case it’s OK to just do git pull (since the default action is merge) right?

Wrong.

This is what git pull does by default: a merge commit. However, it’s merging “origin/master” (C) into “master” (D), but upstream is the remote repository, not the local one.

The order is wrong:

This is a correct merge: the local “master” (D) is merged into the remote “origin/master” (C). A similar result would happen if you had created a topic branch for D, and then merged that into “master”

In git, merge commits are commits with more than one parent, and the order matters. In the example above the first parent of E is C, and the second one is D. To refer to the first parent you do master^1, the second is master^2.

Proper history

Does it really matter which is the first parent? Yes it does.

Correct vs. incorrect order

In the correct history (left) it’s clear how the different topic branches are integrated into “master” (blue). Visualization tools (e.g. gitk) are able to represent such history nicely. Additionally you can do git log --first-parent to traverse only the main commits (blue).

In the incorrect history (right) the merges are a mess. It’s not clear what merged into what, visualization tools will show a mess, and git log --first-parent will traverse the wrong commits (green ones).

Better conflict resolution

If that wasn’t enough, at the time of resolving conflicts it makes more sense to think of integrating your changes to upstream (“origin/master”) rather than the other way around. Mergetools like meld would present the flow correctly: from right to the middle.

Consensus

Update: In the original version of the article I only concentrated on the facts, and I didn’t include the opinion of other developers, but since there seems to be a lot of people ignoring the facts, and distrusting my judgement, I’ve decided to list some of the developers who agree git pull isn’t doing what it should be doing (at least by default, for non-maintainers).

Conclusion

So every time you do a merge, you do it wrong. The only way to use git pull correctly is to configure it to always do a rebase, but since most newcomers don’t know what a rebase is, that’s hardly a universal solution.

The proper solution is my proposal for a git update command that creates merge commits with the correct order of the parents, does only fast-forwards by default, and can be properly configured.

So there you have it. Now you know why git pull is definitely broken and should not be used. It was never intended to be used by normal users, only maintainers.

Do git fetch instead, and then decide how to integrate your changes to the remote branch if necessary.

git update: the odyssey for a sensible git pull

While git pull kind of works, over the years a slew of people have pointed out many flaws, and proposed several fixes. Some have even suggested that newcomers should be discouraged from using the command (as they often use it wrongly), and others to remove it entirely.

I spent several days digging through the entire history of the Git mailing list in order to document all the discussions related to git pull and its default mode. It’s a long story, so grab some coffee (a whole pot of it).

Update: I wrote a shorter article that only talks about why git pull is broken.

Preamble

The entire post is going to be talking about fast-forward merges so I’ll attempt to explain them briefly.

Basically when your history and the remote history have diverged, like:

non-fast-forward

a fast-forward merge is not possible: you need to create a real merge that will have two parents: C and B.

The fast-forward case is much simpler:

fast-forward

While you can create a merge, in this case it’s not necessary: the “master” branch pointer can simply move forward to B.

The whole story is about what happens when a novice does git pull when a fast-forward is not possible.

--ff-only

The story begins in 2008 with a request from Sverre Hvammen Johansen to add the --ff-only option to git merge (then called git-merge), since in his workflow a certain branch was only allowed to fast-forward.

A few weeks later we get the first patch to implement the --ff=only merge strategy. Unfortunately the patch was more complicated that it needed to be, and it wasn’t merged.

Sverre gave it another go a few months later. Junio C. Hamano (the maintainer) said “with vastly improved documentation and justification compared to the previous rounds, I am beginning to actually like this series”. Unfortunately even though Sverre sent several rounds, they were never merged. Good attempt by Sverre.

Another attempt was made at the beginning of 2009 by Yuval Kogman, this time only adding the --ff-only option. What it did was simple: if a fast-forward is possible then do it, if not then fail. Everyone was in favor. Initial versions of the patch had some issues, those were fixed, but unfortunately the last iteration received no feedback. Thanks for trying though.

A few months later, the question was asked “how do you fast-forward your local dev to sync up with origin/dev?“. If the previous patch was merged, the answer would have been simple: git merge --ff-only, but since that wasn’t the case, Junio replied with:

git push . origin/dev dev

This is not user-friendly if you ask me, and later on turned out it wasn’t even correct.

Months later Randal L. Schwartz brings up the same issue and demands a simple command that can be told to users in the #git IRC channel who constantly asked this question. It is pointed out such thing has been implemented twice already, but never merged.

To which Junio replied:

Do you mean twice they were both found lacking, substandard, useless, uninteresting, buggy, incorrect, or all of the above?

Geez! Eyvind was just trying to help. They clearly were not useless nor uninteresting, since many people had rallied in favor of such a feature, and yet more joined saying +1, even in this very thread.

The first big one

At the end of the year appears the first big discussion about the asymmetry of pull and push. Thomas Rast makes very bold suggestions, like deprecating git fetch completely. These drastic changes would have worked using the suggestion of Wesley J. Landaker to add a configuration, and deprecation warnings.

This proposal was not taken seriously; Junio considered the suggestion to turn git pull into git fetchmindless mental masturbation“, and did not see how git pull and git push could ever be symmetric.

On the other hand an interesting piece of data was provided by Björn Steinbrink. He investigated 10 days of IRC activity in the #git channel and found plenty of users confused by git pull, many expected not only there to be a --ff-only option, but this behavior to be the default.

Aside from --ff-only, other main ideas were:

  1. Add a --merge option to git pull
  2. Add a pull.merge configuration

Björn Gustavsson used this big thread as inspiration to implement --ff-only for both git merge and git pull.

Finally. After three implementations, Junio accepted the idea and merged the patches in October 2009.

Begging for a configuration

Not long after the introduction of git pull --ff-only (6 months later), the first request for a configuration to make it the default appears in 2010. Peter Kjellerstedt argues that pull.options and merge.options configurations would fit him well. Not only that, but he thinks --ff-only should be the default. Nothing comes of this.

In 2013 people in the Linux Kernel Mailing List found issues with git pull and Linus Torvalds brought them to the Git mailing list. Linus argued that depending on whether you are a maintainer or a submaintainer you might want to do either --ff-only, or a true merge. An alias could do the trick (for pull --ff-only), but asked the Git community for ideas.

One of the interesting ideas came from Theodore Ts’o; have a per-repository configuration. Junio did follow this thread closely, but nothing happened.

Only days later, Ramkumar Ramachandra introduces the idea of a pull.default configuration which ideally could be specified on a repository-specific basis. Ramkumar doesn’t seem to have pursued his idea further.

Second big one

A couple of months later Andreas Krey triggered a huge discussion when he pointed out that the order of the parents when doing a merge was wrong. What does that mean?

So this is what git pull does:

The new merge commit D merges B into C, this is fine if you are a maintainer, and instead of “origin/master” you are pulling something like “john/topic” into your “master” branch, in that case your “master” (C) is the first parent. But it’s not OK for normal developers (i.e. non-maintainers).

A normal developer would want to do the opposite: to merge his “master” (C) to the upstream “origin/master” (B), so the first parent is “origin/master” (B). This is not a trivial thing. The order of the parents completely changes how history is visualized, and even traversed, plus it’s important when resolving conflicts, along many other things.

Junio’s response:

Don’t do that, then.

For the record: there’s no way to tell git pull not to reverse the parents, and the four people that expressed an opinion said they would rather have the order reversed (John Szakmeister, Jeremy Rosen, John Keeping, and of course Andreas Krey).

Junio said he would not be opposed to an option to reverse the order of the parents, but as you’ll see later on, when I sent a proposal to do precisely that he ignored it.

Yet again another suggestion to make --ff-only the default, this time from John Keeping. This round I chimed in, and so did many others. Junio was not convinced this was the way to go, because integrators would be affected:

It is not about passing me, but about not hurting users like kernel folks we accumulated over 7-8 years.

Since I have worked on the Linux kernel, I explained to Junio that most lieutenants (submaintainers) don’t do git pull; they send pull requests to Linus, so essentially Linus is the only person that does git pull as an integrator (as Junio does). Linus is certainly capable of configuring git pull to something other than --ff-only (which I don’t know why Junio switched for --rebase in his responses).

That is not something I can agree or disagree without looping somebody whose judgement I can trust from the kernel circle ;-).

That somebody was none other than Linus Torvalds, who agreed with us:

It would be a horrible mistake to make “rebase” the default, because it’s so much easier to screw things up that way.

That said, making “[--ff-only]” the default, and then if that fails, saying [a big warning message], [would make sense].

Linus suggested it would be even better to make this configuration per-branch, which is similar to the suggestion of Theodore Ts’o’s idea of making it per-repository.

Eventually, using Mercurial as inspiration, I suggested the following error message:

The pull was not fast-forward, please either merge or rebase.

This seemed to land on Junio, who finally saw why --ff-only made sense:

Initially I said limiting “git pull” to “--ff-only” by default did not make sense, but when we view it that way, I now see how such a default makes sense.

John Keeping came with another good idea by using a hypothetical command git update as an example. This command, unlike git pull would have the correct order of the parents, and would be similar to svn update, and others. While John didn’t propose such command, eventually I did.

After Junio realized the value of --ff-only he event sent a patch integrating Linus’ suggestions and mine. However, I came up with a simpler approach that in my opinion was better, and that triggered another big discussion.

Third big one

My response to Junio’s patch triggered yet another big discussion.

Richard Hansen argued that git pull was straight up dangerous, and he recommended this instead:

git remote update -p
git merge --ff-only @{u}'

He suggested people have that as a git up alias.

In September of 2013 I sent a patch series to add a pull.mode configuration which can be set to merge-ff-only. It was completely ignored, however Junio soon after made a similar proposal on the other thread: pull.integrate with an option of fail back. I argued that my proposal of pull.mode was better, and other people agreed.

Another point I argued is that we needed a deprecation warning before changing the default mode, something like “in the future you would have to choose a merge or a rebase, merging automatically for now”.

Jeff King provided some useful information from GitHub trainers, essentially they didn’t like the rebase option by default because most people don’t know what a rebase is. This is what prompted me to add the following line to the warning:

If unsure, run “git pull --merge“.

v3 of my patch series did receive some feedback, but it got stuck with trivial details. I tried to fix them for v4, but didn’t receive any feedback at all.

Moving on

At the beginning of 2014 David Aguilar sent a couple of patches to add the pull.ff configuration, and Junio simply merged them with no comment.

This ignored all the previous feedback, and doesn’t add a configuration per-repository, or per-branch, but well… At least there’s a configuration now (sort of).

Pull is Evil (first huge one)

In April of 2014 Marat Radchenko pointed out a list of problems 20 newcomers in his organization faced after trying everyday tasks. Most of the problems came from the order of the parents in git pull. Marc Branchaud pretty early on renamed the thread to “Pull is Evil” since in his opinion git pull is pretty much broken, due to the fact that different people use it in different ways.

I think the pull command should be deprecated and quietly retired as a failed experiment.

Junio brought up again the idea of a git update command, which would demarcate the behavior meant for integrators, and normal developers.

Another interesting idea came from W. Trevor King: add an --interactive option (like git add or git rebase) so the user can choose to a) merge, b) rebase, or c) fast-forward.

I pointed that this has been discussed before, everyone agrees the default should change, and I had already sent the patches. Matthieu Moy rejected the notion that everyone was in favor, since he himself wasn’t, but I pointed that he was the only person not in favor:

But in fact, he was in favor, not with changing the default, but with the configuration I proposed. So he wasn’t against all my patches, just one.

Either way I still think Matthieu Moy is wrong. Yes he is right in questioning why ask if to merge or rebase if most newcomers would not want to know at that point in time, but the answer is simple: they don’t have to know, they can simply do git pull --merge and find out later why that --merge is necessary. Exactly the same thing happens with git commit --all.

Pull is Mostly Evil (Pull is Evil part 2)

Marc Branchaud din’t find a good subthread to reply to, so he started yet another thread. In it he suggested that since git pull can only do the right thing if configured properly, it should be an advanced command that must be configured to be used, and when it’s not configured it does nothing by default.

Junio once more pondered about a git update command that would separate the two clearly distinct ways of using git pull.

Being frustrated with the lack of follow-up I made the prediction that the consensus and the proposals will likely be ignored and nothing would change:

And it has been discussed before. If history is any indication, it will be discussed again.

Spoiler: I was right.

Jeff King did not like my comment, and also stated that he wasn’t sure the majority of users would want the parents on a merge done with git pull to be the other way. This despite many big threads started precisely because the order of the parents was wrong, and many developers stating that this was the only reason why they didn’t use git pull, nor did they recommend newcomers to use it. I offered to point him to the many endless discussions in the mailing list.

Richard Hansen wrote a comprehensive description of the issues, and how they could be solved with different new commands: git update and git integrate, as well as other options.

The rest of the discussion was pretty much Richard Hansen and me hashing out different use-cases and ways to deal with them.

Noise

The discussion did not go without hiccups. In particular David Kastrup and James Denholm threw some personal attacks at me that had absolutely nothing to do with the discussion. Back then I was much more abrasive that I am today, in many instances throughout the entire history of the mailing list (including these discussions) the present me would have reacted differently, but that doesn’t change the fact that I’ve always focused on the code, not so much the feelings of participants of the discussion.

I quickly ignored them and focused on the code itself.

One more try

Since this issue popped yet again, I sent v5, and a v6 of my pull.mode approach. They did receive some amount of feedback, but in particular there was a comment from Richard Hansen that showed yet another issue with git pull.

Supposing we are currently on the branch “master”, and the upstream branch is “origin/master”.

  1. git pull --rebase (rebase master onto origin/master)
  2. git pull --merge (merge origin/master into master)
  3. git pull --rebase github john (rebase master onto github/john)
  4. git pull --merge github john (merge github/john into master)

Half of these are wrong. #1 is correct, since when we do git pull --rebase we want to update our “master” to “origin/master”, so we rebase on top of, but #2 is wrong, because when we use --merge we want a similar thing: we want to merge “master” to “origin/master”, not the other way around. This is what Andreas Krey pointed out, which started the second big thread. But in addition #3 is wrong too, because when we specify a repository and a branch to pull from, we want that branch to rebase on top of master, not the other way around.

So basically it’s all wrong.

Summary

After listening and responding to everyone, all the issues with git pull were clear on my mind, but I realized not everyone had the patience, the stamina, or the time to go through all the responses, so I wrote a summary.

I reduced the main problems with git pull to two:

  1. It does unwanted merges
  2. It does merges with reversed parents

The deeper issue is that git pull is in fact used in two very different ways: to integrate and to update. So when you try to fix the defaults for one, you stumble into unintended consequences for the other. While I do see clearly in my mind how git pull can be disentangled in a way that the defaults would work correctly for everyone, it is pointless if I’m the only one who sees that, because I can’t convince anyone (except perhaps Richard Hansen).

Instead I chose an escape hatch, a completely new command: git update, which would integrate the input from everyone in order to do what git pull can’t do cleanly.

By default git update would be essentially the same as git pull --ff-only (or more like my pull.mode=ff-only), which does solve the main issue: newcomers won’t do real merges by mistake. But it also solves the issue about the order of the parents:

  • git update --merge (merge “master” into “origin/master”)
  • git pull --merge github john (merge “github/john” into “master)

Not only that, but we could fix the order of the parents for rebase too:

  • git update --rebase (rebase “master” onto “origin/master”)
  • git pull --rebase github john (rebase “github/john” onto “master”)

Now everything is fixed. Additionally the default mode for update could be configured with update.mode, and the mode for pull with pull.mode (e.g. update.mode=rebase, pull.mode=merge). Not only that, but this could be configured per-branch (as Linus Torvalds and Theodore Ts’o suggested) with branch.master.updatemode and branch.master.pullmode. This was not only a proposal, I wrote the patches for all of that.

All users agreed my proposed git update was good, as Philippe Vaucher stated:

I know patches from Felipe are sensitive subject and that a lot of people probably simply ignore him entirely, however I think this patch deserves more attention.

No developer commented on it.

Moving on

Nothing resulted from the big Pull is Evil thread, except after it was finished, Jim Garrison asked how would pull do the wrong thing? Junio sent a comprehensive reply and Stephen P. Smith’s proposed to pretty much copy-paste the email from Junio into the list of howtos, and was accepted without any pushback. Not only was the fist version accepted, but Junio himself fixed the patch so no v2 was necessary from Stephen.

Unnecessary merge

At the beginning of 2018 Linus Torvalds once again contacted the mailing list due to an “unnecessary merge” in the Linux kernel. He explained the difference between a merge and a “back-merge”, and why in one you want a fast-forward, but not the other. Unfortunately git doesn’t know which situation we are in.

He even suggested the following configuration:

   git config --global alias.update pull --ff-only

This is precisely what my proposed git update did by default. Linus Torvalds even used the exact same name.

Junio then proceeds to suggest a per-repository configuration pullFF=only, when in fact Linus had already proposed a per-branch configuration back in 2013, and I had already implemented it. Nothing happens.

Why merge?

In 2020 Robert Dailey asks why does git pull use merge by default, he argues most people prefer rebases. Once again Elijah Newren argues that it would be desirable to consider a transition to ff-only by default. Once again Konstantin Tokarev tells Junio that in reality what newbies often do is wrong merges, and that ff-only would be better. Robert Dailey also gets convinced that ff-only is better. Alex Henrie mentions a patch he wrote but was reluctant to send it in order to avoid a big discussion.

The warning

Alex Henrie finally decided to send his patch which adds a warning telling users that running git pull without specifying to either merge or rebase is discouraged. Initially Alex wrote the warning to state that the default behavior would change in a future version of git, but Junio objected to that because there was “no consensus”:

Sorry for not catching this in the earlier round, but I do not think anybody argued, let alone the community came to a concensus on doing, such a strong move. Am I mistaken?

After five rounds of comments from Junio, he accepted the patch.

Interestingly enough this change immediately caused people to ask what it was:

Also, we have this lovely comment on GitHub:

Thanks for confusing everyone instead of just doing the default strategy when no parameter is given.

Imagine when every (CLI) application warns the user about not having each default value within the configuration file or given as parameter, despite the user is OK with (or does not care about) the default behaviour.

And we have a bug report from Wolfgang Fahl completely in German. Roughly translated: the bug is the warning.

Clearly people did not like this warning.

Pick the right default

On November of 2020 Vít Ondruch complained about the warning and stated that the Git project should choose a sensible default and not bother users with a warning.

I’d like to keep my trust in Git upstream, but this is was not trustworthy decision IMO.

This started because of a bug report on Fedora.

The main argument from Junio is that there was no perfect default:

There is no clear “default that would suit everybody”.

I pointed out that this is a false dichotomy because the options are not 1) implement the perfect solution, and 2) do nothing; we have another option 3) implement a better solution than the current one, and we all know what that solution is: ff-only. Plus we had the patches for that.

Jeff King pointed out that now there’s a pull.ff=only configuration, and he didn’t see how my pull.mode=ff-only was different. However, I explained that my version provides a more friendly error message, also works correctly with pull.rebase=true, and in addition allows a default mode that throws a more correct warning than the current one.

Junio proposed to make git pull --rebase honor pull.ff=only and fail if the pull is not fast-forward:

And then making an unconfigured pull.ff to default to pull.ff=only may give a proper failure whether you merge or rebase. I dunno.

This is obviously the wrong thing to do. As I explained to both Junio and Jeff, if you make git pull --rebase respect that, and you make pull.ff=only the default, then git pull --rebase will always fail (unless it’s a fast-forward case).

Once again I stated what should be the default.

  • git pull (fail by default unless it’s a fast-forward)
  • git pull --merge (force a merge [unless it’s a fast-forward,
    depending on pull.ff])
  • git pull --rebase (force a rebase [unless it’s a fast-forward,
    depending on pull.ff])

Vít Ondruch agreed.

Both Jeff and Alex Henrie argued that what my pull.mode does could be achieved with pull.rebase and pull.ff, and that’s the direction Junio wanted to take. I wasn’t convinced, and at this point I started to convert my patches from 2014 which was no easy task because before git pull was written in shell, but now it had been converted to C, so essentially I had to rewrite everything from scratch. Alex started to write his approach too.

Junio welcomed the competition:

Let’s see how well the comparison between two approaches play out.

Pretty quickly I realized what they suggested was not possible. If I removed my pull.mode and made pull.ff=only the default, then git pull --merge would fail. We want git pull fail by default, but not git pull --merge. To make this approach possible we would need to change the semantics of pull.ff=only, and that didn’t seem a clean solution to me.

pull.rebase=ff-only

In order to avoid all the hurdles from 2013 I decided to write the simplest patch possible that would achieve the desired result. Instead of creating a new pull.mode=ff-only, I reused an existing configuration, and named the mode pull.rebase=ff-only. This obviously was not ideal–since it left the interface in an inconsistent state (didn’t fix all the issues my original patch did)–but it was better than the current situation.

Junio responded that the name looked funny:

It looks quite funny for that single variable that controls the way how pull works, whether rebase or merge is used, is pull.REBASE.

Yes Junio, that’s why I suggested pull.mode instead, but that patch was never merged.

Raymond E. Pasco also agreed ff-only was the way to go.

pull.mode revampled

My first attempt to revive my old series recreated most of the same old behavior, but not all. It only received one comment for a tiny correction on the documentation.

Since my patches for pull.mode were ignored again, on my v2 I sent a bunch of improvements that were not 100% related to my approach, and instead of using my proposed pull.mode, I changed the semantics of --ff-only, which was not my preferred solution, and I clearly demarcated that particular patch with “tentative“. Elijah Newren provided a ton of valuable feedback, and I incorporated most of it, but there was one point where I did not agree:

If unsure, run “git pull --merge“.

I believe that escape hatch is useful, even if it undermines the value of the warning because 1) the warning must be temporary anyway 2) people are complaining about the warning already, 3) input from GitHub trainers already showed many people don’t even know what a rebase is, and 4) the merge is happening already anyway.

I explained my point of view to Elijah using an analogy. If a passenger in a plane is ignoring the safety demonstration he is doing something wrong, however, he should be completely free to do it, it’s his volition. The crew should allow him to ignore the demonstration. Similarly git users should be able to ignore the warning, and they can skip it by simply doing git pull --merge.

I argued that even if he didn’t agree on that particular line of that particular patch, the rest of the changes in the series don’t suddenly make the situation worse. He said he didn’t agree.

Junio used my analogy to liken users doing git pull --merge when they shouldn’t to be the same as passengers that should be removed from a plane:

A team member who runs “pull --[no-]rebase” pushes the result, which may be a new history in a wrong shape, back to the shared repository probably falls into a different category. … Or perhaps in the same public hazard category?

I don’t see how one line in a temporary warning can potentially create a “public hazard”. It’s users doing wrong merges what creates problems, and removing one line from the warning wouldn’t magically fix that. The warning–which if we take Stack Overflow and reddit as indication–isn’t even properly understood by many users, and doesn’t stop the code from creating a merge anyway. If these merges did indeed create a “public hazard”, shouldn’t git simply not do them by default? In other words: make ff-only the default.

Sadly I did not think of that argument at that time, I merely responded that what constitutes a public hazard is up to each individual project to determine, not git. Which is true.

On the other hand he said something that was completely untrue:

I hope “his comment” does not refer to what I said. Redefining the semantics of --ff-only (but not --ff=no and friends) to make it mutually incompatible with merge/rebase was what Felipe wanted to do, not me.

That’s not true. I wanted to introduce pull.mode=ff-only, not change the semantics of --ff-only. But as I had already explained before, --ff-only cannot be made the default without changing its semantics. The patch that introduced pull.mode was ignored (again), so I took a shot at making --ff-only work, and the only way is changing its semantics, which I did in a patch clearly labeled as “tentative” because I was not advocating for that.

And to make it crystal clear I repeated it yet again:

I don’t want to change the semantics of --ff-only; I want to introduce a new “pull.mode=ff-only“.

Moreover, I explained once again that this patch series was merely part 1. In part 2 I would introduce pull.mode. And there’s more parts after that.

For some reason Junio didn’t seem to understand what I was proposing. I explained that with pull.mode=ff-only I wanted to introduce a simple error message, but with pull.mode=default a warning to give users a heads up.

I quite don’t get it. They say the same thing. At that point the user needs to choose between merge and rebase to move forward and ff-only would not be a sensible choice to offer the user.

Other people like Jacob Keller did not have any issue understanding my proposal.

After multiple rounds eventually Junio gets my proposal, but now he doesn’t see the point of taking it slow instead of flipping the default to ff-only right away. Junio said that he assumed everyone has already configured pull.rebase therefore nobody would notice the change. I disagreed and pointed why this whole thread started: because a user refused to configure git and expected git to do the right thing–choosing a useful default.

Junio responded as he did at the start:

Which is basically asking for impossible, and I do not think it is a good idea to focus our effort to satisfy such a request in general. There is no useful default that suits everybody in this particular case.

This is once again the Nirvana fallacy. Nobody asked for a perfect solution, merely a better default.

Additionally he argued the pull.mode=ff-only would only be useful for users that don’t use git “for real”:

But for anybody who uses git for real (read: produces his or her own history), it would be pretty much a useless default that forces the user to say rebase or merge every time ‘git pull’ is run.

I mean… didn’t he just argue that he wanted to make ff-only the default right away? Literally in his last reply. Very confusing.

I showed to him a real example of me using git pull --ff-only “for real”. Additionally I asked him in the most polite way I could a question that should be very valid at this point:

How often do you send pull requests?

He didn’t answer.

Since once again Junio stated that he didn’t see a good reason why certain classes of users would want to configure pull.mode=ff-only, I offered to do some mail archaeology which lead to the creation of this blog post. It is obvious this is needed.

Once again Junio misinterpreted what I was proposing, and once again I explained in great detail. This time however Junio correctly identified that without pull.mode the user cannot explicitly ask for the default behavior, and that in my opinion is a crucial thing the user must be able to do in order to test this proposed new default. Moreover, I explained that the name is irrelevant, it could be pull.mode=ff-only, or pull.rebase=ff-only, or pull.ff=only, but one option should turn on the new behavior we are proposing to users, and currently no option does.

To cut to the chase I explained that we want:

  1. git pull to eventually fail in non-fast-forward cases
  2. A grace period before that switch is flipped

And so far no other proposal but mine achieved that.

Since Junio insisted pull.mode was not necessary (which I had already demonstrated multiple times it is), I asked him a poignant question to unequivocally demonstrate that:

So, after your hypothetical patch, there would be no difference between:

git -c pull.rebase=no -c pull.ff=only pull

and:

git -c advice.pullnonff=false pull

?

Junio answered that both would error out, which is correct. But then he has to concede my point, because:

  • git -c pull.ff=only pull (fail)
  • git -c pull.ff=only pull --merge (fail)

If both fail we cannot tell the user:

Not a fast-forward, either merge or rebase.

Because the user would do git pull --merge and that command would fail as well because it will try to do git merge --ff-only and it’s not a fast-forward. That’s it. pull.ff=only can’t be used for this. Period.

Junio did not reply.

Part 2 was completely ignored.

When Junio sent his regular mail with the status of all patch series he didn’t include any of my proposals. Since Junio himself was the one that asked for Alex and me to provide our competing proposals I replied to him asking for the status of the ff-only topic. He didn’t reply.

Alex’s approach

Junio asked for both my approach and Alex’s approach which he sent in two patches. One changing the warning to state that pull.ff=only will be the new default, and the other actually changing the default.

Junio did not think the approach was correct, and neither did I. When I explained why, Junio mis-read my response, but this time he himself caught the mistake, and then said the following:

If we instead introduced a separate command, say “git update“, that is “fetch followed by rebase” (just like “git pull” is “fetch followed by merge”), to rebase your work on top of updated upstream, there wouldn’t be a need for us to be having this discussion.

Yes, if my suggestion to add git update was heard we wouldn’t be having this discussion (yet again).

But to move forward I argued that there’s three things to do:

  1. Improve the annoying warning
  2. Consider changing the semantics of --ff-only, or implement pull.mode=ff-only
  3. Consider a new “git update” command

Since my patches for pull.mode were not being reviewed, I suggested to focus only on #1 for now.

Jacob Keller chimed in and also agreed the order of the parents with git pull was a significant problem.

I also reminded Junio of the purpose of the patch, to which he replied:

Sorry, I know you keep repeating that “keep in mind”, but I do not quite see why anybody needs to keep that in mind. Has a concensus that the repurposed --ff-only should be the default been established?

This is the title of the patch from Alex we were supposed to be reviewing “pull: default pull.ff to "only" when pull.rebase is not set either“. The whole point is to make pull.ff=only the default, and in that case git pull --merge would fail, which is not what we want. Unless the semantics of pull.ff=only are changed, which is not my proposal, my proposal (pull.mode=ff-only) is better because it doesn’t need that change.

He did not reply again.

Alex stated that he had trouble following the discussion and simply left it at the table.

A fast-forward

In v3 I focused on fixing all the issues without committing to any particular approach. However, Junio objected to my use of the word “fast-forward”:

… I find the phrase “in a fast-forward way” a bit awkward. Perhaps use the ‘fast-forward’ as a verb

I argued that if fast-forward was a verb, we would have git fast-forward command (which I actually think is a good idea), but currently we don’t have it, so fast-forward is a modifier. In particular if git merge is a verb, then --ff is an adverb.

Elijah disagreed:

A square is a special type of a rectangle, but that doesn’t make “square” an adjective; both square and rectangle are nouns.

Since I’m deeply interested in language, this is a discussion I could really weigh in:

Words have multiple definitions. The word “square” is both a noun, and an adjective. It’s perfectly fine to say “square corner”.

Moreover, I found plenty of instances in the documentation which use fast-forward as an adverb. Even the git-merge man page uses “fast-forward merge”.

  • non-fast-forward update
  • fast-forward merges
  • fast-forward check
  • “fast-forward-ness”
  • fast-forward situation
  • fast-forward push
  • non-fast-forward pushes
  • non-fast-forward reference
  • fast-forward cases
  • “fast-forward” merge

This time I managed to convince him.

However it didn’t escape to me the fact that the documentation already talks of “fast-forward update”, so this suggests that the git update command could fill a whole already present in the documentation.

Everything at once

Since Junio had trouble figuring out my proposal in full, I decided to send the 19 patches together at once, and in addition I wrote a detailed explanation of what I was proposing to happen at every step of the way.

Is it more clear what is my proposal?

No reply.

Improve the default warning

Since I didn’t see much hope in fixing git pull‘s default, for v5 I decided to drop everything except what fixes the most urgent issue: the warning on every single pull.

Junio objected to the name of a variable I used: default_mode. He said the word “mode” was overused, and he proposed a “more focused” word, he came with rebase_unspecified. Additionally he didn’t want this variable to be global, even though there were already 38 global variables, so why did he feel that 39 was too much? I have no idea.

Moreover he made some comments on the test changes:

We are merely allowing fast-forward merges without unnecessary merge commits, but we are faced to merge c1 into c2, which is not ff. The command goes ahead and merges anyway, but why shouldn’t we be seeing the message? I am puzzled.

Now, for the most part I haven’t gone into code in this post, but I think it’s important you feel a little bit of my pain here.

git reset --hard c2 &&
test_config pull.ff true &&
git pull . c1 2>err &&
test_i18ngrep ! "Pulling without specifying how to reconcile" err

The name of the test gives a hint to what it’s trying to do “pull.rebase not set and pull.ff=true“. As you can see there’s no pull.rebase configuration (or --rebase), but there is pull.ff=true. So clearly it has something to do with that pull.ff.

What this test is doing is trying to pull c1 into c2, and since the history has diverged the merge cannot be a fast-forward, therefore the code should throw a warning…

except that only happens if the user has not specified any --ff* option (--ff, --no-ff, or --ff-only), either through to command line, or from the pull.ff configuration, and the test does set pull.ff=true. Therefore there’s no warning.

This is behavior Alex Henrie added and Junio accepted with no comment. In my opinion this behavior is wrong, but that’s what the code does at this point, and even Junio did not expect this, that’s why he was puzzled.

I explained to him that the code only checks for opt_ff, and I tried to fix that behavior in v1, v2, v3, and v4 of my patch series, but he did not comment on those patches.

So the tests used to do a fast-forward from c0 to c1, and that would normally throw a warning (as all pulls did), but because pull.ff=true is set, it doesn’t. This test written by Alex Henrie ensures that pull.ff=true doesn’t generate a warning. But after my patch doing a fast-forward from c0 to c1 would not generate a warning anyway, so the pull.ff=true doesn’t do anything, and therefore the test doesn’t do anything. That’s why I changed the test to start from c2, because then the merge is not fast-forward, and a warning is thrown unless pull.ff=true is set, which is what the test was supposed to be testing in the first place.

To show Junio that the tests are doing what they originally intended to do, I modified all the tests to remove the pull.ff configurations:

git reset --hard c2 &&
git pull . c1 2>err &&
test_i18ngrep ! "Pulling without specifying how to reconcile" err

Now the test fails.

Junio was not convinced. He stated that the original test was checking that the fast-forward would work fine, but that’s not true since from the test we can see that the only thing it’s doing is checking there’s no error message. The code could very well have done a real merge instead of a fast-forward, and the test would still pass. I explained to him that the tests that actually check for the fast-forward are in another file. This particular test is only about the warning message.

Junio claimed to know what the original author of the test intended to do and what he cared about, and that was to test that pulling c1 into c0 issued a message, except none of the tests actually did that, they checked the opposite: that the message is not there. So now that my patch skips the message, the tests pass, but for the wrong reason.

git reset --hard c0 &&
git pull . c1 2>err &&
test_i18ngrep ! "Pulling without specifying how to reconcile" err

If instead of starting from c2 we start from c0 as the original code, then the pull is fast forward, and there’s no message, so this test pass. But this test was supposed to be checking the output with pull.ff=true, that’s why it was called “pull.rebase not set and pull.ff=true“, if you remove the configuration it still passes. So in the fast-forward case what is this test supposed to be doing if it cannot possibly fail now? Nothing.

I explained that and in addition I said that I’m not in the habit of attempting to read minds, what the test are trying to do is very clear from the code.

Junio replied.

You do not have to read mind. What is written in the test is clear enough: run “git pull --rebase . c1” into c0 and see what happens.

Yes, that was precisely my point: no need to read minds (although my reading of the code is different).

Do you feel my pain? I don’t mind going as many rounds as necessary to improve a patch series, even ten or more, but these rounds are improving nothing. The name of the variable doesn’t matter, because in a few patches later I remove it, and even these tests for the warning don’t matter because eventually we want to remove the warning completely.

And this is only one test, of one the patches, of one of simplest patch series.

But fine, if Junio wants to keep a bunch of tests that literally test nothing, I’ll oblige. For v6 I left the useless tests untouched and added the real ones separately.

Junio isn’t entirely happy

Junio sent v7 himself, stating he wasn’t happy with the way the --ff* options were handled. He suggested that these options should imply a merge, but I showed situations in which that would become tricky.

Additionally he wrote a patch to remove my global variable, and seized the opportunity to state:

There is no reason to make the code deliberately worse by ignoring improvement offered on the list.

Really? I did not ignore the comments, I responded to all of them, but I disagreed with some. And so if I disagree with any of Junio’s suggestions for improvement I’m making the code “deliberately worse”? If I have to do what he says, then those are not suggestions, those are orders. Not to mention the v7 he is sending contains the patches I sent in v6, which have the useless tests–a change he suggested–and even though I was 100% against, I included it.

Anyway, since Junio didn’t want to discuss his proposal to imply merges any further, finally he merged this patch series.

After this patch the warning becomes much less annoying. So that’s progress.

What happened to the rest of the patches? Nothing.

Today

After this latest attempt to fix git pull, I distanced myself from the project for a few months, and then came back. I decided to resend some of my patches, but now separately, to minimize the possibility of them being rejected.

  1. Cleanups
  2. Documentation
  3. --merge

Of these only #1 was merged (though not yet in “master”). #2 generated yet another discussion of the semantics of “fast-forward” but nobody was against the changes themselves. #3 everyone is in favor, but no comment from Junio.

If you think any of these will eventually be merged you are more hopeful than I am. Also, there’s this comment from Junio: (emphasis mine):

If some contributors get too annoying to be worth our time interacting with, it is OK to ignore them.

He seems to be referring to me.

So where does that leave us? The order of the parents is still wrong. And the default mode still isn’t fast-forward-only. Regardless of what you may think of my communication style, I’m the one that has actually sent patches to fix all these issues. And unlike Junio and many other git developers–I’m not being paid to work on this, I’m doing this altruistically (I don’t even use git pull myself). If I’m not the person to fix this, who is? I left the project for about 6 years and nobody fixed it.

Recently I challenged Alex Henrie to send a patch to try to fix the situation. He took me up on the challenge, and when I reviewed his patch he immediately realized that more work is needed (as I predicted he would find).

What to do now?

The final solution

While writing this post I realized just how many suggestions were proposed, but not implemented, so I decided to try once again to fix the situation, but now consider absolutely all the feedback in a holistic way.

Here’s there result.

git fast-forward

The documentation about what a fast-forward is, and how to attempt one, is scattered all over. Additionally the simplest way to attempt one is with git merge --ff-only which isn’t user-friendly, and some people don’t even consider a fast-forward to be a modifier of a merge (i.e. adverb).

By having git fast-forward command we solve both problems and fast-forward in the git context now becomes a verb: something independent of git merge.

Additionally there’s an advice for newcomers:

hint: Diverging branches can't be fast-forwarded, you need to either:
hint:
hint:   git merge
hint:
hint: or:
hint:
hint:   git rebase
hint:
hint: For more information check "git help fast-forward".
hint:
hint: Disable this message with "git config advice.diverging false"

Advanced users can disable this specific advice (or all advice).

As the advice says, anyone who wants to learn more about fast-forwards can simply do git help fast-forward.

git update

I converted my old tool from shell to C, and although it doesn’t have yet all the features, it does have plenty of functionality.

git update without arguments simply does git fetch + git fast-forward. If the fast-forward fails, we get the above advice. This solves the first problem that newcomers often do merges by mistake.

Also, it’s possible to do git update --merge (or configure update.mode=merge), and on those cases the order of the parents is reversed properly, as everyone wants (everyone that is not an maintainer).

There’s also git update --rebase, and also a per-branch a configuration branch.<name>.updateMode.

pull.mode

And of course all the other patches that improve git pull, including pull.mode=fast-forward, a better warning that shows that in the future git pull will do fast-forwards by default, along with a per-branch configuration branch.<name>.pullMode.

Also, thanks to git fast-forward we also get the same divergence advice, which will be permanent, but easy to turn off.

In total it’s 33 patches, and one additional to flip the default in the future (after users have had a chance to configure pull.mode to whatever they prefer).

The patches have been sent, but I wouldn’t hold my breath.

Sadly this is where the story ends, even though everyone has agreed on what is the path forward “for some reason” the patches will not be merged, and new users will be forever doomed to keep making mistakes because git pull was not designed for them.

Wait a second…

Git is a distributed version control system. You are not forced to use Junio’s repository, you can use mine. Everything I’ve talked about is fixed there. If you are interested why not give it a try?

Note: Some people are thinking that the purpose of this article is to point fingers, but that could not farther from the truth. The only reason names are named is to be able to follow the discussions (or at least attempt to). I am not seeking to assign any blame to any one person for any action or inaction. In any discussion disagreements are to be expected, and it doesn’t have to mean anything more than that.

Adventures with man color

As it’s usually the case with me, a simple fix sends me to an unending rabbit hole of complex issues. And this was no exception.

It all started when I tried to help the Git project to move towards asciidoctor, a program that generates documentation from text files using a markdown language. The initial project was asciidoc, but it’s in a bit of a rot. The original asciidoc is written in Python (a language I detest), and the new asciidoctor in Ruby (a language I love), so I clearly saw an edge.

The problem starts with a feature asciidoctor has, but not asciidoc: generate man pages. Of course asciidoc can generate man pages, but it does so by first generating a docbook XML, and then docbook stylesheets can be used to convert those to man pages. The same can be done with asciidoctor, but additionally there’s an option to generate man pages directly.

Using docbook is very slow; generating man pages directly is way faster.

When I sent the initial patch (very trivial), a Git developer mentioned some “groff issues”. Apparently if your system uses groff (GNU troff), you are supposed to build git with GNU_ROFF=1:

Define GNU_ROFF if your target system uses GNU groff. This forces apostrophes to be ASCII so that cut&pasting examples to the shell will work.

It’s actually not “GNU groff”, but “GNU troff”, so this option is wrongly named, but regardless; virtually nobody is using it.

My Arch Linux system doesn’t build git with GNU_ROFF, it does use groff, and yet I don’t see any issues with apostrophes… So what’s going on?

The context

The original commit “Quote ‘ as \(aq in manpages” comes from 2009, and mentions the problem comes from “docbook/xmlto”, and apparently only affects groff. In the original thread “quote in help code example” you can see they mention the output of the git filter-branch command specifically, but when I look at that man page, it looks fine.

So I started to dig in, and for starters I don’t even know what groff is.

I tried different versions of asciidoc, they worked fine. Then I tried different versions of docbook stylesheets, they worked fine. I even tried different versions of asciidoctor, to see when the fixed the issue; the all worked fine. Weird.

So I decided to compile groff myself and find the point where it started to fail. First I tried a version two years ago, and it did not compile correctly, so I had to do some hacking to make it compile, after I did, everything worked fine. I continued going more and more into the past, fixing the compilation issues, and not finding any problem. I reached 2006, and still did not see any change.

This was strange. Clearly at some point in time, with some combination of tools there was a problem, but I couldn’t find either of those.

Then I decided to manually modify a man page, and put quotes directly… Bingo. I could see that ' was actually rendered as `, but what caused it?

I then moved forward in versions and to my surprise they all had this issue, even the most recent version of groff.

What on Earth is going on here?

While doing this I noticed something different from my compiled version of groff, and Arch Linux’s version. In the compiled version of groff I saw links rendered as blue. When I saw the generated man pages I saw \m[blue], but I assumed that was for some other kind of troff program or something totally unrelated. But no, here I was seeing blue, but not with all the groff binaries.

So I tried to build groff with the same options as Arch Linux… Still blue. After trying a few things I eventually found a difference: Arch Linux installs a file /usr/share/groff/site-tmac/man.local, if I remove that file the blue color returns. Inside the file there’s:

\" Shut off SGR by default (groff colors)
\" Require GROFF_SGR envvar defined to turn it on
if '\V[GROFF_SGR]'' \
  output x X tty: sgr 0

That’s it! If you export GROFF_SGR=1 on Arch Linux, you see man pages with colors, just like my compiled version. The reason my compiled version does this by default is that it doesn’t have Arch Linux’s man.local file.

GROFF_SGR

If you google GROFF_SGR you find that it’s not properly documented. Some distributions such as Debian and Arch Linux do disable groff’s colors, but they don’t document doing so. Debian “fixed it”. However, I don’t think most people are going to read the entirety of grotty’s man page, not even a little bit, so that doesn’t help, even if you are running Debian–where it’s documented.

However, if you read the man page, you will find another variable: GROFF_NO_SGR. Unlike GROFF_SGR, this one is standard, and it’s respected in all distributions.

This reminded me of trick I learned while reading Arch Linux’s installation guide:

man() {
    LESS_TERMCAP_md=$'\e[01;31m' \
    LESS_TERMCAP_me=$'\e[0m' \
    LESS_TERMCAP_so=$'\e[01;44;33m' \
    LESS_TERMCAP_se=$'\e[0m' \
    LESS_TERMCAP_us=$'\e[01;32m' \
    LESS_TERMCAP_ue=$'\e[0m' \
    command man "$@"
}

This code automatically converts parts of man pages to color (e.g. bold and underline), which looks much better than normal man pages, but it turns out it only works if you have groff’s SGR disabled, so… In other distributions you need to do GROFF_NO_SGR=1, for the above to work.

Cool. We found something.

Back to apostrophes

There’s something else in Arch Linux’s man.local:

char \' \N'39'

This converts \' to ', instead of groff’s default: \(aa (acute accent: ´). This is the reason why I could not reproduce the problem: Arch Linux was hiding it. However, this is the wrong way of fixing it. There is a reason groff developers decided to pick \(aa; they know better. Distribution packagers should not be overriding this.

Why did they do this? The change came due to task FS#9643 – man PKGBUILD shows slanted single quotes. This happened in 2008, which suggests there was indeed an issue around that time, and pacman documentation was built with asciidoc too.

Arch Linux fixed the issue in the wrong way, though. Debian chose a different path. In their bug report #507673 Shouldn’t parse ‘ to \’ they discussed the issue at length, and they correctly identified that the issue was in docbook-xsl (not groff), and if you are going to convert ' it should be to \(aq, but that would only work in groff. They also found that Pod::man had a portable solution:

.ie \n(.g .ds Aq (aq
.el .ds Aq '

This creates an alias: from Aq to (aq, but only when the program is groff, in all other programs it gets converted to '.

This is the correct solution in the correct layer, and generates the proper output everywhere.

But to check that it is the correct solution it would behoove us to understand what groff actually is. groff (or GNU troff) is a document formatting system; it receives text mixed with commands, and generates documents. A man page is just one of the many types of documents it can generate. It can for example generate a PDF.

So, let’s write a simple groff document:

.nf
single quotes: 'text'
single quoted quotes: \'text\'
apostrophe quote: \(aqtext\(aq
.fi

We can generate a PDF document using groff -T pdf test.groff > test.pdf. But this of course is not what we ultimately want, we want to generate a man page, in the same way as man does. To do that we need to specify the output device as utf8: groff -T utf8 test.groff > test.txt. This generates the following:

single quotes ': ’text’
single quoted quotes \': ´text´
apostrophe quote \(aq: 'text'

As you can see the output text is quite different from the input text; that’s what groff does. But this is only the case on a utf-8 system, if you specify the ascii output, then all the quotes above get translated to simply '.

The output with \(aq is correct in both utf-8 and ascii. And if we add the Aq alias:

apostrophe quote alias \(Aq: 'text'

That is indeed correct. The Debian fix seems to work. To make sure we would need a non-GNU troff, like in Solaris, but alas, I don’t have access to something like that, so I’m just going to assume it works in other systems (as other people reported it did).

This proper fix was eventually picked by docbook in 2010: Fixed bug #2412738 (apostrophe escaping) by applying the submitted patch. If you take a look at the code of git-filter-branch.1 you can see the fixed code in action:

git filter-branch --tree-filter *\(Aqrm filename*\(Aq HEAD

Therefore both Arch Linux’s and git’s workarounds are not necessary anymore. Yet they remain there ten years later.

Digging deeper

OK, so we found out what the issue was, and how it got fixed: in docbook, and also unnecessarily in git and Arch Linux (three different levels). But what caused it? Going back to groff from 2006 didn’t cause the issue, so what happened?

By looking back at docbook stylesheet’s history with git blame, I found out the commit that caused the issue: Reverted necessary escaping of backslash, dot, and dash. This commit happened in 2007, and it was made due to an internal limitation of docbook’s architecture.

So from 2007 to 2010 docbook stylesheets were generating wrong man pages, different projects worked-around the issue in different ways, but today–in 2021–these workarounds are not needed, and yet they remain in place.

Back to Git

After all this investigation I sent a patch to the Git project (doc: remove GNU troff workaround) to remove the GNU_ROFF option which clearly was not needed since at least ten years. But I also sent a comment about what I found regarding GROFF_SGR, and the trick to colorize man pages. In reply I received a suggestion to implement the LESS_TERMCAP trick into git help (which is basically an alias for man).

So I sent a patch (help: colorize man pages), and a big discussion propped up (typical due to the bike-shed effect). In that thread it was mentioned: “why not let the user configure man to do this?”. The problem is that you have too many moving parts; groff, man, git, less, distribution configurations, environment variables, aliases, workarounds, docbook and asciidoc bugs… And of course the thing that started it all: asciidoctor.

But it made me think: what is indeed the best way to configure man to do this?

After several days of investigation, and several days of trying options I arrived to what I think is the actual solution.

Solution

export MANPAGER="less -R --use-color -Dd+r -Du+b"
export MANROFFOPT="-c"

Unlike Arch Linux’s hack, the -D arguments to less are much more succinct, and they allow adding color (in addition to the style (e.g. underlined)), not removing information. So --color=d+r (long option for -D) converts d (Bold text) to r (red), and the + signifies add color (i.e. don’t remove the bold attribute). Moreover --use-color adds other colors to the less interface; the prompt is cyan, searches are in green, and warnings in yellow.

And instead of the the ugly GROFF_SGR=1, we can tell man to pass -c to groff.

So the full command is:

groff -T utf8 -m man -c git-filter-branch.1 | less -R --use-color -Dd+r -Du+b

No man involved. This is way simpler… Why is nobody using this?

After I updated my patch and other people tested it, it became clear it didn’t always work. In particular older versions of less did not have the -D options (at least not for Linux). So I checked the history of less and I found out that they enabled -D for Linux systems in 2021.

No wonder everyone is still using the LESS_TERMCAP_* variables. Nobody knows of the new option, because it’s too new!

So the patch to remove the GNU_ROFF option in Git (totally necessary in 2021) is there. And so are the updated Arch Linux instructions to use the new -D flags of less.

If you want to properly colorize man pages, you do this:

export MANPAGER="less -R --use-color -Dd+r -Du+b"
export MANROFFOPT="-c"

If you want to colorize other similar documentation (like Ruby’s documentation):

export RI_PAGER="less -R --use-color -Dd+r -Du+b"

And so on. And if you want less to format everthing:

export LESS='-RXF --use-color -Dd+r$Du+b'

That’s it. If you are running a recent enough version of less, everything works perfectly with a simple configuration.

Oh, also, I realized asciidoctor didn’t have the portable fix, so I sent them a patch that is now merged. I found an issue with less colors and searches that is fixed now. And I reported their unnecessary workaround to Arch Linux too.

The visual style of a programmer

Recently I heard a person say that us “geeks” don’t have a good sense of style, presumably because we typically wear very plain clothes (check pictures of Steve Jobs or Mark Zuckerberg), however, what I think many people don’t see is that we do have style, but where it matters; our computer screens, not clothes.

Most of the time a programmer is staring at code, and the program that shows code and allows you to edit it properly is called a text editor.

This is vim, one of the most popular text editors for programmers with the default configuration.

By default it works, however, staring at these colors for hours gets tedious; I want better colors. Fortunately vim has the concept of “color schemes”, so you have several hundreds of themes to choose.

After trying tons of those, I decided none were exactly what I wanted, so I decided to create my own.

Color theory

I have been choosing colors for websites for about 20 years, so I am familiar with the ways colors are programmed, but many people are not.

While sometimes you can tell a program “red” and it will use the right color, sometimes you need a slightly darker red, or something between orange and red. So in order to be perfectly specific, the most common system to tell a computer a color is called RGB (red, green, blue). In this system, red is 100%, 0%, 0% (100% of the red component), green would be 0%, 100%, 0%, and yellow (which is a combination of red and green), 100%, 100%, 0%.

But computers don’t naturally deal with percentages; they are digital, so they need concrete numbers, which is why 100% is translated to 255 (the maximum value), thus 50% would be 128. And they don’t even use the decimal system; they use binary, and the closest between decimal and binary is hexadecimal, in which 255 is “FF”. Just like in decimal 9 is the biggest digit (1 less than 10), in hexadecimal F is the biggest digit representing 15 (1 less than 16).

So, red in hexadecimal RGB (the standard) is “FF0000”.

I can do the translation in my mind between many hexadecimals to their corresponding human colors, and do some alterations, like for example making an orange more red, or make a cyan darker, or less saturated.

This method of selecting colors has served me well for several years, and I have created aesthetically pleasing colors for many interfaces, but it’s always trial and error, and although the colors look OK, I could never be sure if they are precisely what I wanted.

For example if yellow is “FFFF00” (100% red and 100% green), I could figure out orange would be “FF8000” (50% green). But for more complicated colors, like a light red “FF8080”–where green is already halved–it’s not so clear how to combine it with a light yellow “FFFF80” where green is full, or how to make a purple that looks similar.

I wanted a programmatically precise method of generating the colors I wanted, and in order to do that I researched about color wheels and learned that in fact there’s many systems of colors, and many different color wheels.

What I wanted was a way to generate the RGB color wheel, but without using the RGB color model. It turns out there’s two alternate representations of the RGB model; HSL (hue, saturation, lightness) and HSV (hue, saturation, value). I was familiar with HSV, but it turns out HSL is the one that better serves my purposes.

In HSL red is 0°, 100%, 50%, yellow is 60°, 100%, 50%, orange is 30°, 100%, 50%; the saturation and lightness are not changing, only the hue. So now it’s clear how to generate the light orange, since light red is 0°, 100%, 75%, light yellow is 60°, 100%, 75%, so obviously light orange is 30°, 100%, 75%.

I can easily generate the full color wheel by simply changing the hue: red 0°, orange 30°, yellow 60°, chartreuse green 90°, green 120°, spring green 150°, cyan 180°, azure 210°, blue 240°, violet 270°, magenta 300°, rose 330°.

My color scheme

I have been using my own color scheme for about 10 years, but armed with my new-found knowledge, I updated the colors.

I cannot stress enough how incredibly different this looks to my eyes, especially after hours of programming.

These are the colors I ended up picking.

Is this not style?

If you are a programmer using vim, here’s my color scheme: felipec.

Font

But wait, there’s more. Colors are part of the equation, but not the whole. When reading so much text it’s important in what font that text is rendered.

Generally speaking there’s three kinds of typefaces, “serif”, “sans-serif”, and “monospace”. The kind virtually everyone uses for code is monospace, which looks like: this.

There’s tons of different monospace fonts, many created specifically to read code. In fact, there’s even sites that allow you to compare code in different programming languages with different fonts to see which one you like best, for example: Coding Fonts.

It’s this way I found my new favorite coding font: Input. Not only has the font been carefully designed, but it can be configured to accommodate different preferences, such as the shape of the letter “g”, which I decided to change. You can play with different preferences and preview how it looks in different languages (and in fact different vim color schemes).

This is what it looks like:

Probably most people don’t notice the difference between the DejaVu and Input fonts, but I do, and plenty of programmers do too, which is why these fonts were created in the first place.

There there

So there is it, just because most people don’t see it, doesn’t mean there’s no there there.

Programmers do have style. It’s just that we care more about the color of a conditional more than we do about the color of our shirt.

Why renaming Git’s master branch is a terrible idea

Back in May (in the inauspicious year of 2020) a thread in the Git mailing list with the tile of “rename offensive terminology (master)” was started, it lasted for more than a month, and after hundreds of replies, no clear ground was gained. The project took the path of least resistance (as you do), and the final patch to do the actual rename was sent today (November).

First things first. I’ve been a user of Git since 2005 (before 1.0), and a contributor since 2009, but I stopped being active, and only recently started to follow the mailing list again, which is why I missed the big discussion, but just today read the whole enchilada, and now I’m up-to-date.

The discussion revolved around five subjects:

  1. Adding a new configuration (init.defaultbranch)
  2. Should the name of the master branch be changed?
  3. Best alternative name for the master branch
  4. Culture war
  5. The impact to users

I already sent my objection, and my rationale as to why I think the most important point–the impact to users–was not discussed enough, and in fact barely touched.

In my opinion the whole discussion was a mess of smoke screen after smoke screen and it never touched the only really important point: users. I’m going to tackle each subject separately, leaving the most important one at the end, but first I would like to address the actual context and some of the most obvious fallacies people left at the table.

The context

It’s not a coincidence that nobody found the term problematic for 15 years, and suddenly in the height of wokeness–2020 (the year of George Floyd, BLM/ANTIFA uprising, and so on)–it magically becomes an issue. This is a solution looking for a problem, not an actual problem, and it appeared precisely at the same time the Masters Tournament received attention for its name. The Masters being more renowned than Git certainly got more attention from the press, and plenty of articles have been written explaining why it makes no sense to link the word “masters” to slavery in 2020 in this context (even though the tournament’s history does have some uncomfortable relationship with racism) (No, the masters does not need renaming, Masters Name Offensive? Who Says That?, Will Masters Be Renamed Due to BLM Movement? Odds Favor “No” at -2500, Calls for The Masters to change its name over ‘slave’ connotations at Augusta). Few are betting on The Masters actually changing its name.

For more woke debates, take a look at the 2 + 2 = 5 debate (also in 2020).

The obvious fallacies

The most obvious fallacy is “others are doing it”. Does it have to be said? Just because all your friends are jumping off a cliff doesn’t mean you should too. Yes, other projects are doing it, that doesn’t mean they don’t have bad reasons for it. This is the bandwagon fallacy (argumentum ad populum).

The second one comes straight out of the title “offensive terminology”. This is a rhetorical technique called loaded language; “what kind of person has to deny beating his wife?”, or “why do you object to the USA bringing democracy to Iraq?”. Before the debate even begins you have already poisoned the well (another fallacy), and now it’s an uphill battle for your opponents (if they don’t notice what you are doing). It’s trying to smuggle a premise in the argument without anyone noticing.

Most people in the thread started arguing why it’s not offensive, while the onus was on the other side to prove that it was offensive. They had the burden of proof, and they inconspicuously shifted it.

If somebody starts a debate accusing you of racism, you already lost, especially if you try to defend yourself.

Sorry progressives, the word “master” is not “offensive terminology”. That’s what you have to prove. “What kind of project defends offensive terminology?” Is not an argument.

Adding a new configuration

This one is easy. There was no valid reason not to add a new configuration. In fact, people already had configurations that changed the default branch. Choice is good, this configuration was about making it easier to do what people were already doing.

The curious thing is that the only places in the thread where the configuration was brought up was as a diversion tactic called motte and bailey.

What they started with was a change of the default branch, a proposition that was hard to defend (bailey), and when opponents put enough pressure they retreated to the most defensible one (motte): “why are you against a configuration?”

No, nobody was against adding a new configuration, what people were against was changing the default configuration.

Should the name of the master branch be changed?

This was the crux of the matter, so it makes sense that this is where most of the time debating was spent. Except it wasn’t.

People immediately jumped to the next point, which is what is a good name for the default branch, but first it should be determined that changing the default is something desirable, which was never established.

You don’t just start discussing with your partner what color of apartment to choose. First, your girlfriend (or whatever) has to agree to live together!

Virtually any decision has to be weighted in with pros and cons, and they never considered the cons, nor established any real pro.

Pro

If the word “master” is indeed offensive, then it would be something positive to change it. But this was never established to be the case, it was just assumed so. Some arguments were indeed presented, but they were never truly discussed.

The argument was that in the past (when slavery was a thing), masters were a bad thing, because they owned slaves, and the word still has that bad connotation.

That’s it. This is barely an argument.

Not only is very tenuously relevant in the present moment, but it’s not actually necessarily true. Slavery was an institution, and masters simply played a role, they were not inherently good or bad. Just because George Washington was a slave owner, that doesn’t mean he was a monster, nor does it mean the word “master” had any negative connotation back then. It is an assumption we are making in the present, which, even if true; it’s still an assumption.

This is called presentism. It’s really hard to us to imagine the past because we didn’t live it. When we judge it we usually judge it wrong because we have a modern bias. How good or bad masters were really viewed by their subjects is a matter for debate, but not in a software project.

Note: A lot of people misunderstood this point. To make it crystal clear: slavery was bad. The meaning of the word “master” back then is a different issue.

Supposing that “master” was really a bad word in times of slavery (something that hasn’t been established), with no other meaning (which we know it isn’t true) this has no bearing in the modern world.

Prescriptivism

A misunderstanding many people have of language, is the difference between prescriptive and descriptive language. In prescriptivism words are dictated (how they ought to be used). In descriptivism words are simply described (how they are actually used). Dictionaries can be found on both camps, but they are mainly on the descriptive side (especially the good ones).

This misunderstanding is the reason why many people think (wrongly) that the word “literally” should not mean “virtually” (even though many people use it this way today). This is prescriptiveness, and it doesn’t work. Words change meaning. For example, the word “cute” meant “sharp” in the past, but it slowly changed meaning, much to the dismay of prescriptivists. It does not matter how much prescriptivists kick and scream; the masses are the ones that dictate the meaning of words.

So it does not matter what you–or anyone–thinks, today the word “literally” means “virtually”. Good dictionaries simply describe the current use, they don’t fight it (i.e. prescribe against it).

You can choose how you use words (if you think literally should not mean virtually, you are free to not use it that way). But you cannot choose how others use language (others decide how they use it). In other words; you cannot prescribe language, it doesn’t matter how hard you try; you can’t fight everyone.

Language evolves on its own, and like democracy; it’s dictated by the masses.

So, what do the masses say about the word “master”? According to my favorite dictionary (Merriam-Webster):

  1. A male teacher
  2. A person holding an academic degree higher than a bachelor’s but
    lower than a doctor’s
  3. The degree itself (of above)
  4. A revered religious leader
  5. A worker or artisan qualified to teach apprentices
  6. An artist, performer, or player of consummate skill
  7. A great figure of the past whose work serves as a model or ideal
  8. One having authority over another
  9. One that conquers or masters
  10. One having control
  11. An owner especially of a slave or animal
  12. The employer especially of a servant
  13. A presiding officer in an institution or society
  14. Any of several officers of court appointed to assist a judge
  15. A master mechanism or device
  16. An original from which copies can be made

These are not all the meanings, just the noun meanings I found relevant to today, and the world in general.

Yes, there is one meaning which has a negative connotation, but so does the word “shit”, and being Mexican, I don’t get offended when somebody says “Mexico is the shit”.

So no, there’s nothing inherently bad about the word “master” in the present. Like all words: it depends on the context.

By following this rationale the word “get” can be offensive too; one of the definitions is “to leave immediately”. If you shout “get!” to a subordinate, that might be considered offensive (and with good reason)–especially if this person is a discriminated minority. Does that mean we should ban the word “get” completely? No, that would be absurd.

Also, there’s another close word that can be considered offensive: git.

Prescriptives would not care how the word is actually used today, all they care about is to dictate how the word should be used (in their opinion).

But as we saw above; that’s not how language works.

People will decide how they want to use the word “master”. And thanks to the new configuration “init.defaultbranch”, they can decide how not to use that word.

If and when the masses of Git users decide (democratically) to shift away from the word “master”, that’s when the Git project should consider changing the default, not before, and certainly not in a prescriptive way.

Moreover, today the term is used in a variety of contexts that are unlikely to change any time soon (regardless of how much prescriptivists complain):

  1. An important room (master bedroom)
  2. An important key (master key)
  3. Recording (master record)
  4. An expert in a skill (a chess master)
  5. The process of becoming an expert (mastering German)
  6. An academic degree (Master of Economics)
  7. A largely useless thing (Master of Business Administration [MBA])
  8. Golf tournaments (Masters Tournament [The Masters])
  9. Famous classes by famous experts (MasterClass Online Classes)
  10. Online tournament (Intel Extreme Masters [IEM])
  11. US Navy rank (Master-at-Arms [MA])
  12. Senior member of a university (Master of Trinity College)
  13. Official host of a ceremony (master of ceremonies [MC])
  14. Popular characters (Jedi Master Yoda)
  15. A title in a popular game (Dungeon Master)
  16. An important order (Grand Master)
  17. Vague term (Zen master)
  18. Stephen Hawking (Master of the Universe)

And many, many more.

All these are current uses of the word, not to mention the popular BDSM context, where having a master is not a bad thing at all.

Subjectiveness

Even if we suppose that the word is “bad” (which is not), changing it does not solve the problem, it merely shuffles it around. This notion is called language creep (also concept creep). First there’s the n-word (which I don’t feel comfortable repeating, for obvious reasons), then there was another variation (which ends in ‘o’, I can’t repeat either), then there was plain “black”, but even that was offensive, so they invented the bullshit term African-American (even for people that are neither African, nor American, like British blacks). It never ends.

This is very well exemplified in the show Orange Is The New Black where a guard corrects another guard for using the term “bitches”, since that term is derogatory towards women. The politically correct term now is “poochies”, he argues, and the proceeds to say: “these fucking poochies”.

Words are neither good or bad, is how you use them that make it so.

You can say “I love you bitches” in a positive way, and “these fucking women make me vomit” in a completely derogatory way.

George Carlin became famous in 1972 for simply stating seven words he was forbidden from using, and he did so in a completely positive way.

So no. Even if the word “master” was “bad”, that doesn’t mean it’s always bad.

But supposing it’s always bad, who are the victims of this language crime? Presumably it’s black people, possibly descended from slaves, who actually had masters. Do all black people find this word offensive? No.

I’m Mexican, do I get offended when somebody uses the word “beaner”? No. Being offended is a choice. Just like nobody can make you angry, it’s you the one that gets angry, nobody inflicts offense on other people, it’s the choice of the recipients. There’s people with all the reason in the world, who don’t get offended, and people that have no reason, and yet they get easily offended. It’s all subjective.

Steve Hughes has a great bit explaining why nothing happens when you get offended. So what? Be offended. Being offended is part of living in a society. Every time you go out the door you risk being offended, and if you can’t deal with that, then don’t interact with other people. It’s that simple.

Collective Munchausen by proxy

But fine, let’s say for the sake of argument that “master” is a bad word, even on modern times, in any context, and the people that get offended by it have all the justification in the world (none of which is true). How many of these concerned offended users participated in the discussion?

Zero.

That’s right. Not one single person of African descent (or whatever term you want to use) complained.

What we got instead were complainers by proxy; people who get offended on behalf of other (possibly non-existent) people.

Gad Saad coined a term Collective Munchausen by proxy that explains the irrationality of modern times. He borrows from the established disorder called Munchausen Syndrome by Proxy.

So you see, Munchausen is when you feign illness to gain attention. Munchausen by proxy is when you feign the illness of somebody else to gain attention towards you. Collective Munchausen is when a group of people feign illness. And collective Munchausen by proxy is when a group of people feign the illness of another group of people.

If you check the mugshots of BLM activists arrested, most of them are actually white. Just like the people pushing for the rename (all white), they are being offended by proxy.

Black people did not ask for this (the master rename (but probably many don’t appreciate the destruction of their businesses in riots either)).

Another example is the huge backlash J. K. Rowling received for some supposedly transphobic remarks, but the people that complained were not transgender, they were professional complainers that did so by proxy. What many people in the actual transgender community said–like Blair White–is that this was not a real issue.

So why on Earth would a group of people complain about an issue that doesn’t affect them directly, but according to them it affects another group of people? Well, we know it has nothing to do with the supposed target victim: black people, and everything to do with themselves: they want to win progressive points, and are desperate to be “on the right side of history”.

It’s all about them.

The careful observer probably has already noticed this: there are no pros.

Cons

Let’s start with the obvious one: it’s a lot of work. This is the first thing proponents of the change noticed, but it wasn’t such a big issue since they themselves offered to do the work. However, I don’t think they gauged the magnitude of the task, since just changing the relevant line of code basically breaks all the tests.

The tests are done now, but all the documentation still needs to be updated. Not only the documentation of the project, but the online documentation too, and the Pro Git book, and plenty of documentation scattered around the web, etc. Sure, a lot of this doesn’t fall under the purview of Git developers, but it’s something that somebody has to do.

Then we have the people that are not subscribed to the mailing list and are completely unaware that this change is coming, and from one day to the next they update Git and they find out there’s no master branch when they create a new repository.

I call these the “silent majority”. The vast majority of Git users could not tell you the last Release Notes they read (probably because they haven’t read any). All they care about is that Git continues to work today as it did yesterday.

The silent majority doesn’t say anything when Git does what it’s supposed to do, but oh boy do they complain when it doesn’t.

This is precisely what happened in 2008, when Git 1.6.0 was released, and suddenly all the git-foo commands disappeared. Not only did end-users complained, but so did administrators in big companies, and distribution maintainers.

This is something any project committed to its user-base should try to avoid.

And this is a limited list, there’s a lot more than could go wrong, like scripts being broken, automated testing on other projects, and many many more.

So, on one side of the balance we have a ton of problems, and in other: zero benefits. Oh boy, such a tough choice.

Best alternative name for the master branch

Since people didn’t really discuss the previous subject, and went straight to the choice of name, this is where they spent a lot of the time, but this is also the part where I paid less attention, since I don’t think it’s interesting.

Initially I thought “main” was a fine replacement for “master”. If you had to choose a new name, “main” makes more sense, since “master” has a lot of implications other than the most important branch.

But then I started to read the arguments about different names, and really think about it, and I changed my mind.

If you think in terms of a single repository, then “main” certainly makes sense; it’s just the principal branch. However, the point of Git is that it’s distributed, there’s always many repositories with multiple branches, and you can’t have multiple “main” branches.

In theory every repository is as important as another, but in practice that’s not what happens. Humans–like pretty much all social animals–organize themselves in hierarchies, and in hierarchies there’s always someone at the top. My repository is not as important as the one of Junio (the maintainer).

So what happens is that my master branch continuously keeps track of Junio’s master branch, and I’d venture to say the same happens for pretty much all developers.

The crucial thing is what happens at the start of the development; you clone a repository. If somebody made a clone of you, I doubt you would consider your clone just as important as you. No, you are the original, you are the reference, you are the master copy.

The specific meaning in this context is:

an original from which copies can be made

Merriam-Webster

In this context it has absolutely nothing to do with master/slaves. The opposite of a master branch is either a descendant (most branches), or an orphan (in rare cases).

The word “main” may describe correctly a special branch among a bunch of flat branches, but not the hierarchical nature of branches and distributed repositories of clones of clones.

The name “master” fits like a glove.

Culture war

This was the other topic where a lot of time was spent on.

I don’t want to spend too much time on this topic myself–even though it’s the one I’m most familiar with–because I think it’s something in 2020 most people are faced with already in their own work, family, or even romantic relationships. So I’d venture to say most people are tired of it.

All I want to say is that in this war I see three clear factions. The progressives, who are in favor of ANTIFA, BLM, inclusive language, have he/him in bio, use terms like anti-racism, or intersectional feminism, and want to be “on the right side of history”. The anti-progressives, who are pretty much against the progressives in all shapes or forms, usually conservatives, but not necessarily so. But finally we have the vast majority of people who don’t care about these things.

The problem is that the progressives are trying to push society into really unhealthy directions, such as blasphemy laws, essentially destroying the most fundamental values of modern western society, like freedom of speech.

The vast majority of people remain silent, because they don’t want to deal with this obvious nonsense, but eventually they will have to speak up, because these dangerous ideologies are creeping up everywhere.

For more about the subject I can’t recommend enough the new book of Gad Saad: The Parasitic Mind: How Infectious Ideas Are Killing Common Sense.

It really is a parasitic mindset, and sensible people must put a stop to it.

Update: The topic has been so controversial that as a result of this post reddit’s r/git decided to ban the topic completely, and remove the post. Hacker News also banned this post.

The impact to users

I already touched on this on the cons of the name change, but what I didn’t address are the mitigation strategies that could be employed.

For any change there’s good and bad ways of going about it.

Even if the change from “master” to “main’ was good and desirable (which it isn’t), simply jumping to it in the next version (Git 2.30) is the absolute worst way of doing it.

And this is precisely what the current patch is advancing.

I already briefly explained what happened in 2008 with the v1.6.0 release, but what I find most interesting is that looking back at those threads many of the arguments of how not to do a big change, apply exactly in the same way.

Back then what most people complained about was not the change itself (from git-foo to “git foo”) (which they considered to be arbitrary), but mainly the manner in which the change was done.

The main thing is that there was no deprecation period, and no clear warning. This lesson was learned, and the jump to Git 2.0 was much smoother precisely because of the warnings and period of adjustment, along with clear communication from the development team about what to expect.

This is not what is being done for the master branch rename.

I also find what I told Linus Torvalds very relevant:

What other projects do is make very visible when something is deprecated, like a big, annoying, unbearable warning. Next time you deprecated a command it might be a good idea to add the warning each time the command is used, and obsolete it later on.

Also, if it’s a big change like this git- stuff, then do a major version bump.

If you had marked 1.6 as 2.0, and added warnings when you deprecated the git-foo stuff then the users would have no excuse. It would have been obvious and this huge thread would have been avoided.

I doubt anyone listened to my suggestion, but they did this for 2.0, and it worked.

I like to refer to a panel Linus Torvalds participated in regarding the importance of users (educating Lennart Poettering). I consider this an explanation of the first principles of software: the main purpose of software is that it’s useful to users, and that it continues to be useful as it moves forward.

“Any time a program breaks the user experience, to me that is the absolute worst failure that a software project can make.”

Linus Torvalds

Now it’s the same mistake of not warning the users of the upcoming change, except this time it’s much worse, since there’s absolutely no good reason for the change.

The Git project is simply another victim of the parasitic mindset that is infecting our culture. It’s being held hostage by a tiny amount of people pushing for a change nobody else wants, would benefit no one, would affect negatively everyone, and they want to do it in a way that maximizes the potential harm.

If I was a betting man, my money would be on the users complaining about this change when it hits them on the face with no previous warning.

The amount fallacy

Finding a new star nobody has found before is rare, but it happens—the same goes for fallacies. Errors in reasoning happen all the time, and most of those times people don’t bother looking up the specific name of that error; identifying it as an error suffices. When an error is too common, somebody eventually bothers to name it and thus a fallacy is born. It’s convenient to name fallacies because it saves time trying to disentangle the logic; you can just google the fallacy, and soon enough you will find examples and explanations.

I believe I have found a new fallacy, but unlike most new fallacies, this one has been under our nose for god knows how long.

I’ve decided to coin it the “amount fallacy”, although a close second was “toxic fallacy”, and also “sweet spot fallacy”. This concept is far from new, but it doesn’t seem to have a name. It has already been spread around in toxicology for at least four centuries with the aphorism “the dose makes the poison”. The idea is simple: everything is toxic. Yes, even water can be toxic, it all depends on the amount.

This concept applies absolutely everywhere, which is perhaps why nobody has bothered to name it. Is hot water good or bad? It depends on what precisely you mean by “hot”; it can be 40°C, 60°C, 1000°C, or just about any amount. Since language is often imprecise, the fallacy can sneak by very inconspicuously.

It can be spotted much more easily by realizing sweet spots; too little or too much of something is bad. Water above a certain temperature is bad, but so is water below certain temperature. A similar notion is the Goldilocks principle.

As obvious as this fallacy is, it’s committed all the time, even by very intelligent people.

Take for example income inequality. The debate about inequality is still raging in 2020, perhaps more than ever, and the positions are very clear: one side argues it’s possible for income inequality to be “too high” (and in fact it already is), the other side argues income inequality is inevitable (and in fact desirable). These two positions don’t contradict each other; all you have to do is accept that there is a sweet spot. It’s that simple.

Income inequality for different Gini coefficients

Surely it cannot be that easy. Surely people must have realized this obvious fallacy while discussing income inequality. Of course they have! But they also haven’t named it. This makes it so people fall into the same fallacy over and over, and it has to be explained why it’s a fallacy over and over.

What a piece of work is a man! How noble in reason, how infinite in faculty! In form and moving how express and admirable! In action how like an angel, in apprehension how like a god!

William Shakespeare

People often aggrandize the intellectual capabilities of the human mind, so they assume intelligent people surely can’t be making fallacies this ludicrous, and if they do; surely they would realize when somebody points that out, and if they don’t; surely somebody would record this kind of fallacy so others don’t fall for it. But remember that it took thousands of years after the invention of the wheel before humans came up with the brilliant idea of putting them on luggage (around 50 years ago). So don’t be too quick to assume the grandioseness of the human mind.

Here’s another example: immigration. One side argues immigration enriches the culture of a country, the other side argues immigration dilutes the national identity. Perhaps there’s an amount of immigration which isn’t too much or too little? Yes, there’s some people that argue precisely this, except without naming the fallacy.

Another example: exposure to germs. Too many germs can certainly overwhelm any immune system, but too little weakens it. The immune system requires constant training, and in fact there’s a theory that the current allergy epidemic is due to children’s underexposure to germs (hygiene hypothesis).

A more recent example: epidemic mitigation measures. Many people argue that masks must be compulsory, because not wearing them “kills people”, this is of course very likely true. But what part is missing in the argument? The amount. Everything kills people. Just driving a car increases the probability that you will kill somebody. Driving cars kill people; that’s a fact. But how many? Richard Dawkins—a famous evolutionary biologist, and author—recently made precisely this fallacy in a popular tweet.

The same applies to anything antifragile, but the examples are countless: recreation, safety, criticism, politeness, solitude, trust, spending, studying, exercise, thinking, planning, working, management, circumlocution, sun exposure, child play, child discipline, vitamin consumption, etc.

Technically this falls into the category of hasty generalization fallacies; the fact that some rich people are greedy doesn’t mean all rich people are greedy. In particular it’s an imprecision fallacy, similar to the apex/nadir fallacies, except in terms of amounts.

The form is:

  • 1. Some amounts of X are bad
  • 2. Some amounts of X don’t represent all amounts of X (ignored)
  • ∴ All amounts of X are bad

The amount fallacy happens because premise 2 is ignored. An exception could be cheating: a small amount of cheating is bad, even more so a large amount; the amount of cheating doesn’t change its desirability.

Perhaps this fallacy already has a name, but I don’t think so; I’ve looked extensively. Even if it’s too obvious, it needs to have a name, because people commit it all the time.

So there you have it. Any time somebody says something is bad (or good) because some amount of it is bad, be on your guard; that doesn’t mean any amount of it is bad.

What does “flatten the curve” really mean?

One of the most common phrases we hear of late when discussing about the Coronavirus disease (COVID-19), is that we must try our best to flatten the curve. While this is true, many people don’t understand the reason why. It is important to manage our expectations about where we are, and what’s coming, and that’s what I will try to show in this article.

I will try to show that:

  1. We need to flatten the curve
  2. Most of us are going to get infected anyway

The curve

This is the curve many have seen:

I wrote a simulation using real data from USA as of today (SIR model). And as you can see it resembles the “flatten the curve” graphs you might have seen before.

USA model simulation

However, what you don’t see are the magnitudes; 12,000,000 people in the worst case scenario, and 3,000,000 people in the best. This Y axis is the number of active cases at any given day, provided that 50% of the cases are not noticeable, and only 15% might need hospitalization.

The total number of active cases would be 164,000,000 in the worst scenario, and 40,000,000 in the best.

And the time span is one year. The peak of the worst scenario would be at day 78, and in the best case scenario that day there would be 30,000, so sure; 30,000 is better than 12,000,000, but the true objective of flattening the curve is to delay the peak, which would happen at day 240 instead of 78.

So yes, it’s good to delay the peak from day 78 to day 240, and to reduce the active cases that need hospitalization from 12,000,000 to 4,000,000, so social distancing is good, but that will not be enough; the healthcare system will still be overwhelmed, and people will die as a result.

Healthcare limit

The totals

However, one thing is the number of active cases, which doesn’t look very good, another is the total number of cases.

Active cases
Total cases

In the worst case scenario there would be 326,000,000 cases, and in the best case 250,000,000. So to think that you will not get infected if everyone performs social distancing is delusional.

This is why experts say 70%-80% of the population will get infected (76% in this case) (even in the best case scenario).

So this is the actual curve people should keep in mind:

Active vs. total cases

The numbers

The difference in my model between the worst case scenario and the best, is the growth factor (worst: 1.2, best: 1.04), but what does that number mean?

If yesterday the total number of active cases was 32,859, and today they are 43,112 (as of 2020-03-23), that means dividing today by yesterday you get 1.312 (32,859 * 1.312 = 43,111). Is this good or bad?

Right now South Korea is 0.98, and Italy is 1.07, so yeah, 1.312 is pretty bad, so bad in fact that it’s 10% worse than my worst case scenario, and the worst case scenario is 15% worse than the best. When the growth factor is 1.0 that means the curve is at its peak; South Korea presumably has passed it already.

When people say “flatten the curve” what they really mean is reduce the growth factor; when you reach 0.0 the curve is flat.

Imagine you are in a car, and the accelerator pedal is stuck; not only will you be moving forward, you will be moving forward with an ever increasing speed. Surely you wouldn’t feel safe until the accelerator is unstuck, and that is the inflection point; the point in which the velocity stops increasing.

It’s easier to see the two points by visualizing the new cases per day (velocity); the point in magenta is the inflection point (deacceleration), the blue one is the top of the curve.

When you visualize the total number of cases the inflection point is different, and when reached you should expect the total number of cases to be twice as they are at that point.

The important thing to note here is that as long as no inflection point is reached, the growth is exponential, and there may be many orders of magnitude to go, so basically there’s no end in sight.

Where are we?

Have we reached an inflection point? The short answer is: we can’t say yet. There’s too much day-to-day volatility, and conditions in every country are drastically different.

USA growth factor rolling average

As you can see USA not only is in bad shape, but it’s getting worse; the graph should be moving closer to 1.0, not away.

Growth factor rolling average

Fortunately not every country is in the situation of USA; some countries are getting significantly closer to 1.0, even though not quite there yet.

World growth factor rolling average (except China)

Worldwide we are all over the place.

I think it’s safe to say we are nowhere near any inflection point.

But wait

Say that somehow miraculously we reach an inflection point and we are on our way to a perfectly flat curve. Can we be content now?

Well, no, the virus can make a comeback, depending on the seasons, or even mutations. The Spanish flu of 1918 did in fact do so; the second wave was much deadlier than the first, and it wasn’t the last one.

Spanish flu waves

Even in the most ideal of situations like in South Korea, the nature of a pandemic makes it so not any country is “safe” until the whole world is safe; the virus can be reintroduced into the country, in fact, many times over.

As a simplification you can think of the world as a neighborhood. You can choose to stay home and delay the inevitable, but if everyone else is infected you will eventually be too, unless you stay inside for years.

The solution

The only realistic solution (other than let things run their natural course) is vaccination, but as of today no vaccine is expected in less than 18 months.

So we can try to delay the worst by social distancing, closing airports, and pretty much everything you can think of, and we may be able to delay the worst (reduce the growth factor to less than 1.04), but even so it might not buy us enough time for the vaccine.

Conclusion

It’s too soon to tell where we are and where we are heading. Many of the parameters needed to make reasonable predictions are not known with any real confidence. The model I used is one of the simplest models, and the growth factors I used are pretty much guessed. Even so we know something for certain: it can be really bad, even if we try our best.

So yeah, you should still perform social distancing in order to help others, particularly those that are more susceptible such as the elderly; by reducing the growth factor we give breathing room to the healthcare system.

But you will get infected, or at least you should operate under that assumption, it’s only a matter of when.

To keep track of the number of cases per country in real time you can use this graph I developed. I will publish more graphs as I have them.

How a modern troll argues

Traditionally an Internet troll is a person who starts quarrels or upsets people on the Internet to distract and sow discord, whether for the troll’s amusement, or a specific gain. The classical lone troll wasn’t hard to deal with, once identified all you had to do is follow the classical response: “do not feed the troll”. This worked well in the past, but times have changed.

Nowadays there’s more than one kind of troll.

We have brigading trolls (such as Chapo Trap House trolls), who bad together pushing the same kind of inflammatory commentary; the community might ignore one or two, but if a band of users say the same thing that might give credence to their claims, and a good actor might be fooled into giving a response, thinking there might be some valid criticism there.

Then there’s concern trolls who act pretty much as good actors, except their advice to the group would cause harm rather than good. They use sockpuppet accounts to hide their true agenda; for example a Republican might create an account faking to be a Democrat, and then propose a witch hunt to identify the “true” Democrats, and thus in effect dividing and weaking the group. The usual advice to ignore the trolls doesn’t work because concern trolls are almost indistinguishable from good actors.

And finally there’s professional trolls; which as their name suggest, they are being paid by a corporation, a political entity, or special-interest group to achieve specific goals. There have been many studies that prove these kinds of trolls exist, the most famous group being Russia’s Internet Research Agency. The tactics and goals of these groups vary, sometimes from day to day, but in general their tactics include: flip the narrative, cancel, position, counter-offend, and oppress usually with the goal of polarizing a community. Therefore it wouldn’t be unusual to play both sides of the fence; have both alt-right and far-left sock puppets in order to get the real users to fight between themselves.

The community

Recently I became a moderator (mod) in an online community in reddit (called a “sub”) about the author and public intellectual Sam Harris. I wasn’t paying close attention to this sub, but yet I noticed a steady stream of irrelevant content, I complained about it, and thus I learned I was merely one of many to do so.

When I became a mod I started receiving a lot of feedback about the dire status of this online community. It turns out it was so bad that a group of users decided to create an alternate community in order to get away from toxic behavior and a sea of irrelevance. Also, there were sister and related communities that recognized the problem, and the need to solve it.

I had my work cut out for me, but what I didn’t know at the time was the amount of pushback I would receive by every little attempt to improve the community. Right away I received complaints merely by removing content about what I considered to be clearly irrelevant topics; topics Sam Harris barely had talked about, and in my opinion didn’t belong in the sub.

Virtually all the users that complained about the moderation decisions shared a similar style of arguing. At the time I couldn’t deal with the sheer amount of comments, but slowly and steadily I discussed the issues with each and every user, and while doing so I realized many of them were dishonest. I would say A, and they would claim I said B and run with it. For example I would say “In my opinion the detractors that are engaging in bad faith shouldn’t be welcome”, and they would say I wanted to ban all detractors (what I said, what they say I said).

A misunderstanding here and there is to be expected, but not so many misunderstandings from so many sources, constantly. More importantly; these users didn’t seem to be interested in the least in being corrected. It looked like they knew what they truth was, they were just not interested in accepting it.

So in order to keep my sanity and avoid wasting time I arrived to a rule of thumb; the moment a user makes it clear he/she isn’t interested in what I am actually saying; I end the conversation, and I avoid future ones. Generally I give people the benefit of the doubt, but when it is clear they are not interested in what I say, they are merely interested in what they can claim I said, there’s no point in discussing with that person. So I labeled these people as intellectually dishonest, and moved on.

This is of course the traditional approach—ignore the trolls—and it worked for me, but not for the community, because these trolls kept spreading lies, even if I didn’t engage with them, and they kept derailing conversations, and sowing discord.

Something had to be done about these trolls, I just didn’t know what. I didn’t even want to accept these were trolls.

The setup

I asked for advice in different communities—there’s even a community of moderators—and I received good advice, however, most if it couldn’t be applied to our particular community because we have a strong commitment to freedom of speech (in the spirit of Sam Harris and others in the Intellectual Dark Web).

So how could we both hold on to our strong commitment to freedom of speech, and at the same time stop the trolls from destroying the community? If a user is obviously acting in bad faith, the solution is easy; ban that user. But many of these trolls would do everything in their power to appear as good actors. So even if a moderator is pretty sure a user is engaging in bad faith, he/she can never be sure. The fact that a user appears to be a troll is not enough.

I roamed the Internet for inspiration, and I encountered tips to actively deal with trolls, mostly in the form of trolling the trolls. However, I didn’t want to reduce myself to their level. I tried different tactics, mostly engaging with the trolls, but not as if I was dealing with real people, and then I saw the light.

Trolls have a major disadvantage, that any good actor engaging in good faith doesn’t have; they don’t care. Their interest in any particular subject is manufactured, it isn’t real. So if you spend time writing a really good argument they would not be able to counter it; they don’t have the intellectual tools, nor the interest in doing so. What they will do is go to their troll toolkit, and pick any of their well-practiced tactics to deviate the conversation. The most common one is the smoke screen.

A good actor might inadvertently use a smoke screen, but a troll will use it over and over, to the point that the times he avoids an argument are more than the times he engages in it. This is not an accident, this is deliberate.

After engaging with trolls in this matter I realized how ridiculously often they do this. All you have to do is ignore all the red herrings they throw, all the ad hominems attacks, don’t drink from the poisoned well, ignore the smoke screen, and concentrate on the argument. Don’t say anything extra they might reply to, don’t ask any follow up questions, don’t answer their irrelevant questions; stay on point.

Any person acting in good faith will reply to your questions, even if it might mean losing the argument. A troll will not.

So when I realized this trend, I decided to engage with a suspected troll to see how far the rabbit hole could go, and I honestly didn’t expect a nonsensical discussion of such epic proportions.

The discussion

The context of the discussion is a little tricky. First, there was a discussion between Sam Harris and Eric Weinstein in Weinstein’s relatively new podcast: The Portal. In this discussion they touched on the lack of effort some people make to try to understand people they disagree with, and they mentioned examples such as Sam Seder, and David Pakman. Funnily enough, both Seder and Pakman replied about these comments in their respective podcasts, and their conclusions couldn’t be more different.

The ironic part is that Pakman was mentioned as an example of a person who does make an effort to understand what his opponent is saying, and he did understood what was being said by Weinstein and Harris. On the other hand Seder was brought up as an example of a person that does not make an effort, and lo and behold in his podcast he did indeed misrepresent what Weinstein and Harris said.

This was the topic of the post I made to reddit’s Sam Harris community: Good and bad faith actors behaved in predictable ways that Sam Harris & Eric Weinstein accurately described.

In my post I made it crystal clear what was in my opinion the argument Weinstein made:

Let’s get the premise right; the premise is that some people would rather mock a straw man, than get correctly the actual gist of what is being said. That’s it.

This is the argument. This is what Harris and Weinstein are talking about, this is what Pakman replies to, and this is what Seder is attempting to address. This argument for brevity and analysis purposes I’m going to call argument W.

Right off the bat user BloodsVsCrips starts with this attack:

If you rank Tim Pool as a 4 out of 5 your definition of “good faith” becomes useless.

This is in reference to another discussion in which users were supposed to rank political commentators, and I did rank Tim Pool with an overall grade of 4/5. This of course has absolutely nothing to do with the argument at hand; neither what Weinstein said, nor what Seder said about what Weinstein said. So this is a smokescreen, an ad hominem, a genetic fallacy, and poisoning the well. The thing he didn’t do is address argument W.

I mocked his obvious attempt at not addressing the argument, mrsamsa accused me of not addressing the argument, I asked what was BloodsVsCrips supposed argument, and mrsamsa replies:

That people with a demonstrably bad barometer for determining good and bad faith might be inaccurately judging the people described in the OP.

Now, this is not an accurate representation of BloodsVsCrips’s argument, and yet it commits the same fallacies. I chose to concentrate on the genetic fallacy, which has this form:

  • X said Y is true
  • X is a bad source
  • ∴ Y is false

Of course X is me, but Y is a little bit tricky, because mrsamsa’s argument is also a smoke screen, so he wants to change Y from the original topic (argument W) to “Sam Seder acts in bad faith”. To be clear, I did say Sam Seder acted in bad faith, but I did so with an argument:

Sam Seder didn’t show any signs of understanding Eric Weinstein’s argument, therefore he misrepresented Eric Weinstein’s argument.

We can call this argument F, which depends on argument W.

In fairness to mrsamsa if Y is in fact “Sam Seder acts in bad faith”, then his argument wouldn’t be a genetic fallacy, but we know I did provide an argument (argument F) for my claim, to ignore that would be falling into his smoke screen.

So the syllogism would be:

  • felipec did put forward argument F
  • felipec is a bad source
  • Argument F is false

That is an obvious genetic fallacy.

It should not matter what did or didn’t say about Tim Pool, it shouldn’t matter how good I am at representing anybody’s good faith, it doesn’t matter who I am at all. The only thing that matters is; what was Weinstein’s argument? (argument W), and did Sam Seder represent Weinstein’s argument correctly or not? (argument F).

Is mrsamsa going to focus on the argument, like any good actor would do? Or is he going to do something else?

Dancing around the genetic fallacy

So the first thing I tried to do is nail down the definition of a genetic fallacy; I asked mrsamsa two direct questions, and he evaded them both:

Answer these questions.

(1) Is this a genetic fallacy?
> P1: X said Y is true
> P2: X is a bad source
> C: Y is false

(2) Is this a genetic fallacy?
> P1: felipec said Sam Seder is a bad faith actor
> P2: felipec doesn’t have good judgement
> C: Sam Seder is not a bad faith actor

mrsamsa:

If you’re just asking hypothetically then yes, presenting it as a proof can be fallacious but that’s obviously not what happened…

Notice he is responding in terms of yes, argument X can be fallacious, but he doesn’t want to say it is. I insisted I am not interested in him saying if it can be, but if it actually is:

mrsamsa:

…I did answer, yes it can be fallacious…

This continues:

mrsamsa:

I’ve already answered above. Now stop with this bad faith nonsense and continue with the discussion.

Then he tries to throw smoke screen:

mrsamsa:

Because you don’t understand the topic very well (as evidenced by thinking the genetic fallacy wasn’t an informal fallacy) your response is to misconstrue my answer, rather than to realise “that makes sense”.

Unfortunately I made a mistake of categorizing the genetic fallacy as a formal fallacy, and we’ll see later he will use this as a smoke screen, and an ad hominem.

In addition, he continues taunting the “can be”.

mrsamsa:

Again, someone who believes that the genetic fallacy isn’t an in informal fallacy is unlikely to teach me anything about logic.

It can be, certainly.

More of the same:

mrsamsa:

Do one easy thing right now: admit you made an error by claiming that the genetic fallacy isn’t an informal fallacy.

I can’t change my answer though, I can’t forget everything I know about fallacies and logic and pretend the answer is “yes” even though such a response is nonsensical…

It should be a very easy (1) yes, (2) yes. Why on Earth would any good actor avoid such obvious questions? Maybe mrsamsa does indeed possess a superior knowledge of logic, and the textbook example of a genetic fallacy doesn’t indeed contain a genetic fallacy, but if that is the case why isn’t he enlightening us? Why isn’t he explaining in which cases argument (1) can’t be a genetic fallacy?

The answer is simple; he doesn’t care. In the best case scenario, just like a hostile witness he is simply not answering the question, nor venturing any useful information; he is wasting our time. In the worst case scenario, he is a troll who doesn’t have a good answer for our questions, and is also wasting our time.

And that of course doesn’t translate to argument (2), because that is a specific argument; it either has a genetic fallacy, or it doesn’t. No can, if, then, or buts.

Now, this is just a taste, because the amount of times he applies the same tactic over and over, not just to me, but anyone trying to have a rational discussion is just mind blowing.

Argument within an argument

I wasn’t the only one engaging with him; DwightVSJim unfortunately also got caught in his web of nonsense. He also avoided to answer any of DwightVSJim’s criticisms, but the amount of nonsense is too much to summarize, so I’ll just make a list of comments:

  • mrsamsa: A liar is more likely to lie than someone who doesn’t lie.
  • mrsamsa: You can’t be serious..
  • mrsamsa: Come on, just think a little bit about this.
  • mrsamsa: Yes, if I said something different then it would be different.
  • mrsamsa: I don’t know what this is supposed to mean.
  • mrsamsa: Keep up.
  • mrsamsa: It can be, definitely.
  • mrsamsa: Perfectly valid argument, no fallacy.
  • mrsamsa: Remember that arguments are about judging the persuasiveness of the support for a conclusion…
  • mrsamsa: You’ve just proved yourself wrong there.
  • mrsamsa: You’ve just said that my argument based on the origin of the claim isn’t fallacious.
  • mrsamsa: I’m showing that an argument based on the origin of the claim can reject the conclusion while not being fallacious.
  • mrsamsa: The origin of the claim is Ham, not me.
  • mrsamsa: I am, using Ken Ham as the origin.
  • mrsamsa: The fact that I’m making the argument is obviously irrelevant to whether something is or isn’t a fallacy.
  • mrsamsa: No, the origin in my argument is Ken Ham.
  • mrsamsa: I’m arguing that.
  • mrsamsa: Please just take two seconds to realise how stupid this argument is.
  • mrsamsa: C Felipec is probably wrong about Seder acting in bad faith.

Of course, while avoiding anything presented to him, he didn’t stop making snarky remarks. And at no point in time does mrsamsa addresses the actual arguments: argument F and argument W. He just goes on an on inventing new irrelevant arguments such as a fake argument from Ken Ham.

Finally, I arrive to save DwightVSJim from mrsamsa’s black hole. I point out if in “felipec says Y”, Y is an argument itself, there’s no need to even check if there’s a genetic fallacy:

felipec:

We also have logical argument X which can be evaluated on it’s own. So if logical argument X is:

  • All men are mortal
  • All Greeks are men
  • ∴ All Greeks are mortal

We know logical argument X is valid, so it’s irrelevant if “felipec is probably wrong about logical argument X”, because we can look at logical argument X and see that it is valid. It’s like getting two 6 dices and somebody else is saying; “I don’t need to look at the dices, because I know getting two 6s is very unlikely”.

This destroys mrsamsa’s red herrings, because it forces us to look into the actual argument X (argument F), and not run around irrelevant probabilities of “me” being right.

Miraculously, mrsamsa agrees:

mrsamsa:

All you’ve argued is that there is another way to challenge the conclusion of X. Okay sure, yeah there are. That doesn’t make it fallacious.

So he finally accepted one thing, but in doing so I got him, because there’s an easy way to demonstrate the smoke screen fallacy:

felipec:

Which one would be better to tackle if we want to discern the truth of the conclusion of X? [Argument X, or argument Y?]

All he has to do is say “X”, and he would be admitting that in this whole thread we should be looking at argument F, not my bona fides. You can guess what happened.

mrsamsa:

As for which is better, it depends on the specific claim and the strength of the evidence for each argument.

What a surprise. He is backed into a corner, he has two specific options, and instead of choosing X or Y, he throws another smoke screen and say “it depends”. I didn’t let him go that easily, and I insist he gives an answer:

felipec:

X, or Y?

He goes back his usual cop out:

mrsamsa:

And I said we can say your proposal.

This is the point where I decided to end the discussion, because I thought I had gathered more than enough evidence. Note however that I’m not going through the threads in the order they happened.

After dozens of comments on this particular subject, mrsamsa hasn’t answered a single important question directly. He always avoided the topic, specially when he was backed into a corner. Any good actor would have to answer my question honestly, and say “X”, but that’s not what he did.

Can’t own mistakes

We also got completely derailed when mrsamsa failed to convert one of his fake “arguments” to a syllogism, and then failed to accept his mistakes.

First, mrsamsa provided this example of a supposedly valid argument:

“I doubt Ken Ham is accurately describing that principle of evolution because he has history of misrepresenting and misunderstanding them”

It was pointed out to him the many reasons why this “argument” is essentially useless, but mrsamsa kept defending it, and provided a supposed syllogism for it:

mrsamsa:

P1 Ken Ham thinks X about evolution

P2 Ken Ham has a history of misrepresenting and misunderstanding claims about evolution

P3 we should doubt claims made by people with a history of misrepresenting and misunderstanding claims

C we should doubt Ken Ham’s claim X about evolution.

This was in fact not a “perfect” argument nor an accurate mapping from his original argument. I pointed out the many issues this argument has:

  1. He used different verbs in the premises: thinks vs. claims
  2. A similar “valid” argument can be made with the premise “we should doubt claims made by people with a mustache”
  3. Changed “I doubt X” to “we should doubt X”
  4. It’s a red herring trying to distract us away from argument W

So I destroyed the notion that this was in any way a “perfect”, along with getting us out of the black hole of ignoring argument F. I made an analogy of an embedded argument and his Ken Ham “argument” to arrive to the following syllogism that puts a nail in his coffin:

felipec:

And since we obviously can’t trust you to put a correct syllogism, it would be like:

  • P1: Ken Ham made argument A about evolution
  • P2: Ken Ham has a history of misrepresenting and misunderstanding claims about evolution
  • C: Argument A about evolution is unsound

This “argument” has a genetic fallacy.

What mrsamsa does do is everything, but acknowledge the fact that he made two different arguments, and that an argument can be embedded in another argument, and we should talk about the original embedded argument, not the outer one:

mrsamsa:

Holy shit, pedantry through the charts.
Yes, let’s act like normal human beings, se how that works out for you.
How is that a “correct” syllogism? It’s an invalid argument.
You’re the embodiment of dunning-Kruger.

The rest of the responses keep evading the issue:

  • mrsamsa: Of course I didn’t address the argument that has a conclusion that’s irrelevant to my argument.
  • mrsamsa: What are you talking about?
  • mrsamsa: Semantics aren’t the damning blow that you seem to think they are.
  • mrsamsa: I mean, you keep pretending you can’t see my argument so why should I engage with your strawman of my argument?

Until finally:

mrsamsa:

I’ll take that as a “I’m going to continue to troll rather than engage in any kind of meaningful discussion”.

Very classy.


I take the conclusion reached in this subthread that is pretty hidden by now, and repeat it on the main thread:

felipec:

Hmm. “We should doubt X” versus “I doubt X”. Somebody isn’t reading his own arguments.

Of course he handled it very graciously:

mrsamsa:

Don’t get hung up on semantics and try to address the argument itself.

Another user, SailOfIgnorance, came to mrsamsa’s rescue and made the argument that there’s no distinction between “I doubt X” and “we should doubt X”, but if course there is, and I answered the challenge:

felipec:

You want me to state the distinction between (1) “we should doubt X” and (2) “I doubt X”? Easy, (1) requires reasoning, (2) doesn’t.

This explanation was addressed in perfect faith by mrsamsa:

mrsamsa:

Easy, /u/sailofignorance. I assume that clears up any confusion you may have had!

mrsamsa:

Oh. Maybe meditate more?

I event went to the trouble of providing different syllogisms to show the pointlessness of “I doubt X”. Essentially conclusions like “I believe X”, or “I doubt X” are pointless, because they don’t require reasoning, so you don’t need any argument, or any other premises.

When SailOfIgnorance challenged this notion, I explained it to him:

felipec:

> Give me an example where “I doubt X” doesn’t require reasoning

Easy: “I doubt I’ll have eggs on my breakfast”.

SailOfIgnorance did eventually acknowledge this difference, but did mrsamsa? No.

Why can’t he just accept that he made a mistake? (multiple ones actually) Why doesn’t he want to acknowledge any of the main arguments? (argument F, or argument W). This tendency should be clear by now, and we are only just getting started.

Nonsense of origin

In another thread with DwightVSJim, mrsamsa to spin and spin around the same issue:

  • mrsamsa: That honestly makes no sense and is a bizarre interpretation of the genetic fallacy…
  • mrsamsa: The genetic fallacy is a fallacy of irrelevance…
  • mrsamsa: Both attack the origin, but only one is fallacious.
  • mrsamsa: Shooting the paper boy for the contents of the paper would be an error, shooting the editor/ journalist would not.
  • mrsamsa: No they don’t need to bring it as well.
  • mrsamsa: Obviously the only relevant person to the genetic fallacy would be the person making the original claim…
  • mrsamsa: You understand that your understanding means that unless someone makes an argument that they themselves are a biased source to reject the conclusion they’re arguing for, a genetic fallacy is impossible, right?
  • mrsamsa: That’s literally what you’re arguing.
  • mrsamsa: So you’re not saying that I is the “origin” that’s relevant to the genetic fallacy?
  • mrsamsa: But if you’re not saying that I is the “origin” that’s relevant to the genetic fallacy then what was your argument?
  • mrsamsa: When I make an argument that dismisses a claim made by Bob because it came from Bob, I’m pointing out that you seem to believe that I is the “origin” relevant to the genetic fallacy rather than Bob.
  • mrsamsa: I just summarized it there and we’ve gone over it in detail above, address whichever is easier for you.
  • mrsamsa: Just highlight what is the origin that would be relevant when assessing if it commits the genetic fallacy.
  • mrsamsa: So you’re now agreeing with my characterization of your position that you think the arguer is the origin that the genetic fallacy refers to?
  • mrsamsa: Jesus, this is such an insane view.
  • mrsamsa: Except remember that the genetic fallacy is a fallacy of irrelevance so this is only true when we’re rejecting it because it’s coming from Bob for irrelevant reasons.

This is what happens when a good actor tries to engage honestly as if the troll is a real person. If nobody challenges his bullshit, the troll can go on and on about irrelevant stuff.

When I show up to this thread I end by showing to mrsamsa exactly in which kind of arguments the source of a claim matters, and where it doesn’t, by providing different unquestionable examples:

felipec:

No, you don’t understand. The genetic fallacy occurs when we accept or reject a claim. Normally I would ask you for syllogism explaining when the truthfulness of claim is relevant, and when it’s irrelevant, but we all know you are going to botch it, and it will take us 50 comments for you to not accept that you did, so let me do that for you:

  • Bob says X
  • People that say X are racist
  • ∴ Bob is a racist

Here’s another example:

  • Bob says X
  • If anybody say X we drink
  • ∴ We drink

In both these cases the conclusion has nothing to do with the X being true or false, and the other premise relies on Bob’s claim (regardless if it’s true).

Now let’s make it a fallacy:

  • Bob says X
  • Bob usually lies
  • If anybody says a lie we drink
  • ∴ We drink

Why is this a fallacy? Because we assuming the truthfulness of a claim based on its source, if we reject X because Bob said it we are committing the genetic fallacy. It’s irrelevant who said X; we need to find out if X is true or not.

Tell me /u/mrsamsa, how is this not the case? Do we or do we not need to know if X is true before accepting that conclusion?

This destroys all the irrelevant nonsense about who is the origin of what that mrsamsa kept going on and on about. He replied dozens of times to DwightVSJim when it was easy to avoid the actual issue, but when I make it difficult for him to ignore what actually is a genetic fallacy, what does he do?

He didn’t bother to reply.

Words don’t matter

In yet another thread mrsamsa argued that the names of fallacies don’t matter, instead, we should point out the errors in reasoning themselves, and he did so in his usual classy manner:

  • mrsamsa: Trust me, don’t try to explain to dwight how fallacies work and what they mean.
  • mrsamsa: Yeah that’s exactly how I see it now.
  • mrsamsa: Bringing up the names of fallacies is indeed a troll talking point.
  • mrsamsa: Non trolls realise that the names of fallacies are pointless in themselves because it doesn’t help describe your position or highlight the supposed problem in the argument you’re replying to.
  • mrsamsa: Fallacies have nothing to do with the conclusions…
  • mrsamsa: Of course it does, fallacies aren’t about whether a conclusion is true or not…
  • mrsamsa: Tell me what you think a fallacy is.
  • mrsamsa: Very good.
  • mrsamsa: Fallacies are only related to the structure of the reasoning, they don’t determine the truth or falsity of the conclusion.
  • mrsamsa: You can’t be serious?
  • mrsamsa: Why are you dodging your false claim that I corrected in my post above?

I can’t help but notice the sweet irony of mrsamsa lecturing us in how non-trolls argue, but unfortunately he managed to one again derail the discussion away from the arguments (argument F and argument W) into purse nonsense.

When I jump into the discussion his tone changes:

felipec:

The guy who says the names of fallacies are irrelevant is claiming the categories of fallacies matter. Makes sense.

mrsamsa:

I assume you’re trolling rather than intentionally being so blatantly dishonest here, but if you’re genuinely confused, the names of fallacies are less important in a debate than actually explaining what you think the flawed reasoning is.

This whole argument that the names of fallacies don’t matter is obviously self-defeating, since words are basically the only thing we have to communicate, there is a reason we don’t explain what a napkin is every time we want one. So obviously there’s a reason why common fallacies have names.

felipec:

> I assume you’re trolling rather than intentionally being so blatantly dishonest here, but if you’re genuinely confused, the names of fallacies are less important in a debate than actually explaining what you think the flawed reasoning is.

And what is the quickest way to explain that there’s a flaw in reasoning?

> If someone says “it’s spelt “ab hominid” not “ad hominem” loser!” then while the names of fallacies still aren’t useful in a debate, it’s still worthwhile pointing out that the person isn’t right.

If I say “to say that a person didn’t finish high-school therefore his argument is invalid; it’s an X fallacy”.

1. ab hominid
2. hasty generalization fallacy

What do you think is closer to the truth? Which one do you think is more worth correcting?

I ask mrsamsa two questions. Care to venture how many he will answer?

mrsamsa:

> And what is the quickest way to explain that there’s a flaw in reasoning?

Not by using the name of a fallacy as that doesn’t explain the flaw in reasoning in the person’s argument.

> What do you think is closer to the truth? Which one do you think is more worth correcting?

I don’t really understand how this is relevant, both need to be corrected. Fortunately we can do two things and aren’t limited to one.

This is nonsense. Obviously saying “argument X has flaw Y” is explaining the flaw in reasoning, even if Y is misspelled. But if he answers my questions he would have to concede, so he just doesn’t.

The rest of the discussion is more of the same; mrsamsa evading questions, derailing, and smoke screening:

  • mrsamsa: Hey, you answer a single question I’ve posed to you in this thread and then I’ll take your tantrums more seriously. Otherwise it just looks like bad faith trolling.
  • mrsamsa: You’re the definition of bad faith troll.
  • mrsamsa: How does this relate to anything I’ve said?
  • mrsamsa: And how does that relate to my argument?
  • mrsamsa: I explained why naming a fallacy doesn’t help you identify the specific flaw in an argument. And you replied that naming the fallacy is identifying the flaw.

I didn’t let him get away with such obvious smoke screening. I tried to force him to address the argument, but of course he didn’t.

felipec:

Your argument was that the name of a fallacy doesn’t help you find out the flaw in an argument, I just showed you it does.

If you can’t see that there’s no more reason to discuss. Good bye.

Why are we even discussing this? Of course we can say “argument X has a genetic fallacy”, and the words “genetic fallacy” are useful to identify the flaw in reasoning. If mrsamsa’s wasn’t intentionally derailing the conversation why does he keep bringing red herrings, and always avoiding not only the main arguments (argument F, and argument W), but he avoids any arguments he himself brings up.

This is meta-arguing; mrsamsa is arguing about arguing, and he is derailing the very same tangent he has put us on.

The actual argument

There’s only one point in the whole discussion where mrsamsa actually tried to engage with the actual argument:

mrsamsa:

Seder is responding to Weinstein’s metaphor of the IDW doing maths and how people like Seder are ignorant of alternative forms of maths so jump to ridicule and thinking people are crazy.

Seder is criticizing the idea that when people mock others who seriously propose race realism etc that really what’s happening is that the race realist is privy to some kind of information that Seder is unaware of, and what appears to be crazy is in fact just an evidence based position that could be uncovered by a serious investigation rather than dismissal.

His criticism is essentially of the idea: a) that it’s appropriate to compare things like race realism to the factual nature of maths, and b) that Seder thinks it’s crazy because he’s unaware of the logic behind race realism, rather than the fact that they’re wrong.

This is obviously a misrepresentation of Weinstein’s argument (argument W), in fact, he didn’t say what was the argument, he just mentioned a “metaphor”, so I ask him to at least make an attempt:

felipec:

> Seder is responding to Weinstein’s metaphor of the IDW doing maths

That’s not what Eric Weinstein did. Can you do at least a poor man’s job of a steel man?

Will mrsamsa finally answer a direct question? You already know the answer, don’t you?

mrsamsa:

Use your words buddy, if you disagree with something then explain it.

Why do you think Weinstein wasn’t using a maths analogy?

felipec:

I’m not going to fall for that. Can you make a steel man argument for Eric Weinstein or not?

mrsamsa:

You’re not going to fall for engaging in a productive good faith discussion?

So there it is. When it was time to actually address the argument that mattered, he didn’t even make an attempt to explain it with his own words.

My bad

I don’t want to be a locus of attention; I want to focus on what mrsamsa did, but I’ve had to rely on my comments, since they are directly related to many of mrsamsa’s comments.

And even if I made argument F, and I brought up argument W; I want the focus of the discussion to be on those arguments, not me.

Unfortunately I made a mistake in saying that the genetic fallacy was a formal fallacy. In my defense I probably had more than a few beers when I made that comment, I did a quick Google search and found no evidence that the genetic fallacy was informal.

But if it’s true that I made such mistake, it doesn’t matter in the discussion if the genetic fallacy is formal or informal; it’s still a fallacy. In retrospect I shouldn’t have fell in such an obvious bait, but I did; I did respond to a red herring.

Now, a good actor would simply say: “you made a mistake” and move on. In fact mrsamsa made many of such mistakes, and I didn’t punish him eternally for them. But that’s not what a troll does.

What a troll does is hold on to this trivial mistakes as evidence that a person is “bad at logic”. This troll tactic is another fallacy called poisoning the well. Since I made one mistake, that means that forever and ever any argument I make is flawed. Indeed, after I accepted the mistake mrsamsa and other possible trolls have weaponized this mistake and brought it up in other threads.

Even after I ended the conversation, mrsamsa piled on with other users on the formal fallacy mistake:

mrsamsa:

But to him, I think he views it all as some grand conspiracy to derail the sub, so he seems to view it as a moral duty to not give an inch on anything…

mrsamsa:

These people are so unaware of how little they know about a topic that they’re practically incapable of even understanding how little they know about the topic. It becomes a black hole where trying to explain it to them fails because they lack any foundation to even make sense of the possibility that other people might know more than them, and worse still, those people might think they’re wrong.

mrsamsa:

But the genetic fallacy not being an informal fallacy claim is just a really easy thing to put your hand up over, especially as it doesn’t affect his argument or position at all. It’s just a “whoops, yeah, don’t know why I said that, of course it isn’t. Let’s move on now” moment that doesn’t need to be a big deal.

mrsamsa:

If it wasn’t abundantly clear, he’s barely read the wiki page on the topic nevermind an actual textbook.

mrsamsa:

And coming from the guy whose ego is too big to even accept that they didn’t know the genetic fallacy was an informal fallacy, that insult falls flat.

And of course, I did accept I made a mistake later on:

felipec:

> Will you admit you were mistaken?

Yes, I made a mistake. Not that it matters, because the category of this fallacy is completely irrelevant, and focusing on this is a red herring.

As expected other users used this as ammunition:

zemir0n:

But it is relevant. Your knowledge or lack thereof of logical fallacies is definitely relevant when it comes to talking about logical fallacies.

Did mrsamsa praise what he supposedly wanted me to do in good faith? You know the answer:

mrsamsa:

Replying to comments is a fallacy.

Beating on a dead horse

As I said, once I thought I gathered enough evidence to convince any rational person that mrsamsa was in fact a troll, I ended the conversation. This was taken by mrsamsa and his allies a “victory”.

It doesn’t have to be said that they took this “victory” graciously:

mrsamsa:

So I mean… a win? Even if for the most bizarre reason.

mrsamsa:

That was probably one of the most cringey things I’ve ever seen on the internet. It wouldn’t have been quite so bad if, as you point out, the entire essay long argument didn’t boil down to this bizarre point based entirely on semantics.

Once again bringing up the informal fallacy red herring:

mrsamsa:

It definitely makes sense in light of him being unaware that the genetic fallacy is an informal fallacy, which is fine if it’s a slip of the tongue but egregious as a genuine error.

And he goes on and on:

mrsamsa:

He doesn’t seem to know about a lot of things.

But instead of just listening to the other person he seems to scan posts for any little thing that he can pinpoint and say “Ah ha! You said “doubt” instead of “reject”, therefore you’re making two completely different arguments and I can’t take you seriously any more!”.

We already explored why “I doubt X” is different than “we should doubt X”, he exploiting the fact that a casual observer might have missed the difference and then claiming I just did a “gotcha!” “just semantics” comment. No, there was a difference, the difference has been explained, and he is framing the narrative to make it seem otherwise.

This is dishonest behavior.

mrsamsa:

Being on the receiving end, it’s definitely [extremely boring and frustrating].

mrsamsa:

I feel like he’s got this caricature of what it means to be a “rational person”, and he’s confused being pedantic with being rational. It’s like someone who’s idea of how to be a lawyer comes from TV and so they just yell “objection!” at everything.

mrsamsa:

Destroyed!

mic drop

Is this the way a good actor treats a “win”?

Other conversations

Not content with derailing this conversation with his genetic fallacy nonsense, he sprang up a discussion about the same nonsense in a completely unrelated post. In order to protect the sanity of the reader, I’m not going to go in detail through those, but there have been several dozens of such comments.

In a post titled: Full Subscription Model and Amount of Hatred Sam Receives on this Site:

  • mrsamsa: I followed it fine, you were simply wrong for all the reasons I explained to you.
  • mrsamsa: I’m happy if you agree now but you explicitly argued before that BvC was the origin of the claim subject to the genetic fallacy.
  • mrsamsa: You called me dishonest for saying that, and then repeated what I just said…
  • mrsamsa: What’s not a genetic fallacy?
  • mrsamsa: Take a step back and read what I’m writing.
  • mrsamsa: Why do you think I “take pride in pissing people off”?
  • mrsamsa: The arguer is irrelevant to assessing whether a genetic fallacy occurred.
  • mrsamsa: But if you mean the origin of the argument then you accept that who is arguing that “Bob is a liar” is irrelevant, right?
  • mrsamsa: We’re trying to figure out what you mean by “origin”, that’s all.
  • mrsamsa: I don’t have a problem with the concept of the genetic fallacy, we’re debating your interpretation of it that isn’t consistent with how it’s actually understood.
  • mrsamsa: So when is the person making the argument ever the issue?
  • mrsamsa: Do you need me to dig up our past conversation where you explicitly told me the opposite?
  • mrsamsa: When you argued that BvC was relevant to assessing if his argument was a genetic fallacy.
  • mrsamsa: So, in your view, the arguer is the source that the genetic fallacy refers to.
  • mrsamsa: Of the issue yes, not the arguer.
  • mrsamsa: I don’t know how I can make it any simpler.
  • mrsamsa: Where did I claim that you believed it was always the case?
  • mrsamsa: The arguer isn’t relevant to the genetic fallacy. That’s what you’re struggling to understand.
  • mrsamsa: I’ve told you a million times that you’re wrong but I don’t imagine that you’d agree that it’s bad faith for you to not accept that.
  • mrsamsa: The person making the argument (eg BvC in this case) is irrelevant and is never the origin of the issue that matters.
  • mrsamsa: Stop being evasive and wasting time.
  • mrsamsa: The issue is the claim being debated, ie felipecs ability to judge bad faith.
  • mrsamsa: Present your evidence, no more evasion and smokescreens.
  • mrsamsa: Yes great, link me to a source that supports your interpretation.
  • mrsamsa: The idea in BvCs argument is felipecs barometer.
  • mrsamsa: What part of his argument would we analyze to determine whether it’s based on dismissing a source or not?
  • mrsamsa: He’s dismissing an argument of felipecs.
  • mrsamsa: His argument about judging bad faith. That’s why he appealed to his barometer, as that was a criticism of the argument.
  • mrsamsa: He’s trying to convince us of his methods for judging bad faith, that’s the argument.
  • mrsamsa: Read his OP, it spells it all out in detail.
  • mrsamsa: What part are you pretending not to understand?What part are you pretending not to understand?
  • mrsamsa: What part of his OP did you not understand?
  • mrsamsa: Stop fucking evading, make your argument and stop throwing up smokescreens.
  • mrsamsa: Read the OP and tell me specifically what you don’t understand.
  • mrsamsa: Instead of constantly demanding people to answer your questions, show some good faith and do so in return.
  • mrsamsa: Answer the question or stop responding. That’s the end of the conversation.
  • mrsamsa: You’ve just proven yourself wrong with your own timeline.
  • mrsamsa: Holy shit this is getting ridiculous.
  • mrsamsa: Jesus christ. If you’d stop with the mental gymnastics these conversations would go so much easier.
  • mrsamsa: It’s not, because it’s only fallacious when the appeal to the origin is irrelevant.

In yet another post: Why is Felipec even a mod? Who made him a mod?

  • mrsamsa: Semantics won’t save you or felipec here.
  • mrsamsa: So for it to be a genetic fallacy, BvC would need to reject his own argument based on his own history.
  • mrsamsa: Careful, felipec bans people for not saying yes or no to that question.
  • mrsamsa: Why are you trying to turn it into a deductive proof?
  • mrsamsa: Why not just answer my question?
  • mrsamsa: I am rather cynically assuming that the reason you keep pretending not to be able to read my position every time I spell it out for you though is because you realise that you can’t actually refute it, and you don’t want to admit that after all this fucking bullshit I was actually right.
  • mrsamsa: If we’re in agreement that reaching a conclusion that felipec is probably wrong about a claim because he has a history of being wrong on that topic isn’t fallacious then good, I’m happy with that outcome.
  • mrsamsa: No, saying felipec is probably wrong because of his history is coming from him, he’s the origin referred to in the argument.
  • mrsamsa: As I say in my other comment, using the word “doubt” here confuses the comparison a little bit.
  • mrsamsa: It doesn’t change the origin. The origin is “felipec’s history of judging bad faith”.
  • mrsamsa: It’s a fine argument, there’s nothing fallacious about it and it wouldn’t require evidence to avoid being a fallacy.
  • mrsamsa: It’s a little confusing to use the first one as a general argument and then to use a specific example in the second.
  • mrsamsa: Now do you agree that the “origin” that would be relevant to the genetic fallacy is premise 2 in both those arguments?
  • mrsamsa: And just to be clear, this is a hypothetical, right?
  • mrsamsa: The point wasn’t about Tim Pool, the point was about Felipec’s ability to judge bad faith actors in this sub.
  • mrsamsa: I’m probably jumping ahead, continue.
  • mrsamsa: I’ll have to see why he thinks changing the conclusion from “probably wrong” to “definitely wrong” changes the origin.
  • mrsamsa: That’s not particularly relevant for the genetic fallacy.
  • mrsamsa: Remember that to dismiss a claim we don’t need to state it in absolute terms.
  • mrsamsa: If I say that creationism is probably false, I am claiming that it’s untrue.
  • mrsamsa: If I say felipec’s argument is 99.9999% likely to be false, are you saying “Well… that’s not a dismissal of his argument, he’s saying he could be right!”?
  • mrsamsa: How can someone argue that something is probably untrue without arguing it’s untrue?
  • mrsamsa: If argued as definitely then yes, if probably then no. Agreed.
  • mrsamsa: Both are arguing it’s untrue. One says it’s probably untrue, the other definitely untrue.
  • mrsamsa: Saying something is probably untrue is a dismissal of a claim.
  • mrsamsa: We’re talking about the concept of untrue, not definitely untrue. You’re conflating the two.
  • mrsamsa: It means that something is probably true or false, since truth isn’t binary.
  • mrsamsa: Wait… are you saying arguments with the conclusion of “probably” aren’t even arguments?…
  • mrsamsa: I think you’re going to have to support some of these claims, I can’t see what you’re possibly basing them on.
  • mrsamsa: Can you give me an example of an argument that bases its conclusion on a probability?
  • mrsamsa: So when scientists say that creationists are probably wrong, they aren’t dismissing their claims as true or false?
  • mrsamsa: I’m not sure how this relates to my question.
  • mrsamsa: Since truth is binary in your view, if they aren’t fully dismissing of creationism, and they aren’t fully accepting of evolution, then surely they’re both in the same position of being “neither true or false”?
  • mrsamsa: You’ve just said that if an argument concludes that something is “probably wrong/false/untrue/etc” then they aren’t dismissing that claim.
  • mrsamsa: You explicitly said that people can’t dismiss claims as true or false if they’re talking in a probabilistic way.
  • mrsamsa: I don’t see how any of this addresses my question.
  • mrsamsa: And yet clearly I ended up being correct.
  • mrsamsa: I think you’ve misspoken there – a sound conclusion can’t be false.

Can you imagine trying to argue with him? Such a joy.

Notice that this is just the genetic fallacy argument he is using to derail other conversations. He does a similar style of arguing with other arguments, but I’m not going to bother going into more trouble trying to understand what can’t be understood.

The ban

In my opinion, there’s more than enough evidence than mrsamsa is most likely a troll, as so I decided to temporarily ban him for a month. There are some considerations that must be pondered before banning somebody, depending on the sub, it might be perfectly fine to ban somebody that we can’t be 100% sure is a troll.

Part of this analysis was sent to the mod team for a second opinion, but since no strong opinions against the ban were voiced I decided to go ahead with it.

Unfortunately due to some internal issues, his ban got reverted. I would attribute this as a mistake, and miscommunication, but it is quite telling what has been mrsamsa’s behavior after the ban.

He has been consistently lying about the reason why he was banned, and why he was subsequently unbanned:

mrsamsa:

Yeah because felipec banned me for disagreeing with him, and the other mods overturned it because that’s not against the rules.

This is a lie. He knows what was the rationale behind the ban, since part of this analysis was sent to him, so to attribute the rationale to a “disagreement” is disingenuous at best. He was banned because he engaged in bad faith, and that’s against the rules.

It is also a lie that the other mods decred he didn’t violate the rules. For starters only one mod did engage with the situation.

mrsamsa:

It’s pretty crazy that being wrong (in the eyes of the mod) is now an instant bannable offence. I assume the other mods must have thought there were others reason that justify the ban, otherwise hopefully it’s hashed out between them and the user.

He is lying about the reason why he was banned; it wasn’t because he was “wrong”, and he knows that.

mrsamsa:

What do you think of him banning me for “not answering questions” to his satisfaction (where he wanted a yes or no answer to a question I explained didn’t have a yes or no response? Or for “snarky” remarks like “keep up”, “think about it” and “this is a stupid argument”?

As you can see in this analysis, he wasn’t banned because of that; it was because he didn’t answer virtually any question directly. It is his behavior in aggregate that shows this.

mrsamsa:

Yeah he said the same thing about my ban which turned out to be him asking another mod and then banning me before they replied. So I don’t know, but the mods seemed fair in dealing with me so I assume they’ll look at both sides of the issue there.

That’s yet another lie. Some back and forth happened before the ban was applied.

mrsamsa:

Felipec banned me for disagreeing with him, I appealed it and the other mods agreed that there didn’t seem to be any basis to it and overruled him.

Another lie, and knows it. He saw the discussion that lead to him been unbanned, and no point did anyone say there wasn’t any basis.

Suspect

If his behavior in conversations wasn’t enough evidence, there’s also reason to believe that he has more than one user, who also engage in these discussions. In addition his comments constantly get more than five upvotes, which is extremely rare in deep discussions. This suggests that he is tricking the system. But not only that, but he gets upvotes on comments on old threads that are supposed to be invisible for other users. This means there’s practically zero doubt he is gaming the system.

I am also in contact with at least three users that have noticed the same behavior; each time they discuss with mrsamsa their comments are always voted down, and the ones of mrsamsa up. Not to mention the endless discussions in which mrsamsa never addresses an actual point.

The weird thing to find would be a productive discussion with him.

Conclusion

Let’s summarize what user mrsamsa did:

  • Didn’t talk about Eric Weinstein’s argument (argument W)
  • Didn’t address my argument about Sam Seder (argument F)
  • Didn’t accept a textbook definition of a genetic fallacy
  • Didn’t accept that focusing on the argument X, is better than Y
  • Didn’t accept that he provided two different arguments “I doubt X” vs “We should doubt X”
  • Didn’t accept that “I doubt X” arguments are useless
  • Wasted everyone’s time
  • Rehashed the same discussion in other conversations
  • Lied about the reasons of his ban

Regardless of what the actual motives of mrsamsa are, it’s fair to say this is not the kind of behavior anyone should accept in their online community.

This kind of behavior is not easy to see unless one is engaging in the actual discussion, and not simply avoiding it following the traditional advice “don’t feed the troll”.

Moderators most likely would never see the obvious tricks being used over and over, because they don’t typical engage with the trolls, and looking at any individual comment there’s always the possibility to be taken in good faith. But in aggregate there shouldn’t be any doubt.

Modern trolls rely on these man-power limitations, exploit the good faith of moderators and users, specially in communities that value freedom of speech. They hide in plain sight, and constantly derail conversations claiming to be good actors.

What they don’t expect is somebody keeping track of the amount of times they engage in troll tactics, such a smoke screens, ad hominems, and poisoning the well. So that’s precisely what we should do; actively deal with them.

First principles of income distribution

Anyone attempting to live well within a society must have a concept of fairness; there’s a limit to how much debasement you should tolerate. You wouldn’t just let somebody cut in line in front of you, not only because you would have to wait longer, but it’s a matter of principle: it’s not fair. This isn’t unique to humans, even monkeys wouldn’t do the same job another monkey is doing, if the other monkey is receiving a grape, while he a cucumber slice. I wouldn’t do it either; I prefer grapes too.

If your potatoes are as good as the neighbor’s potatoes, but he sells them at $20, while you can only sell them at $10, either he is doing something right, you are doing something wrong, or your view of the world is incorrect, and your potatoes are not as good as you thought. It is a puzzle that must be solved, because those extra $10 might be the difference between survival, attracting a potential mate, or death.

You might not be able to solve every mystery of life and the universe, but surely you should be able to sell your potatoes at $20, and so you must.

This is the simplest explanation why we are hard-wired for fairness, and we refuse to be part of a system that exploits us. We might not understand every aspect of an extremely complex socio-economic system, but we recognize we should be paid the same amount as other cogs in the machine of the same level, at least.

Inequality

We understand we can’t all have the same wealth, we can see people that work less than we do, or have less ability, so we should be paid more than them. If follows that other people provide more value than us, therefore certain level of inequality might be tolerated, if not even favored.

Many staunch capitalists shrug at the question of inequality. “You want everyone to have the same wealth?” they belch. Few things in this world are black-and-white, and inequality is no exception. The question is not “equality vs. inequality”, the question is “how much inequality?”.

It should not be surprising that high levels of inequality create social instability; large masses of people don’t like to be screwed over. If high levels of inequality are not rectified: crime increases, and eventually revolutions erupt. Ask Marine Antoinette if leveling the playing field a bit more wouldn’t have been a good policy, or rather–ask her detached head.

Therefore it should not be surprising either that elites pay close attention to inequality metrics as it stands to reason that nobody is fond of surprise revolutions. But more pressing than inequality metrics, are perceptions of inequality, because it doesn’t matter if large masses of people are being screwed over… If they don’t know it.

However, extremely high levels of inequality can’t be ignored forever, and eventually society descends into chaos. But what is that level? How much is way way too much?

Morality

The first thing a right-wing capitalist would tell you is that “it doesn’t matter”. It doesn’t matter how much money your neighbor is making selling potatoes, as long as you are making good money. This feels wrong, just like it feels wrong to receive a cucumber slice instead of a grape, but perhaps it is our base instincts at play, and in fact there is nothing inherently wrong.

The phrase they often use is “a rising tide lifts all boats”. The idea is that rich people are the ones that provide the most value to the economy, so if they have a lot of profit, they will know how to use that money best, and therefore the whole economy would benefit, including you. This is also called trickle-down economics; the earnings of the rich trickle down to the poor.

However, it doesn’t take a genius to find a caveat: what if the rich hedge 100% of the earnings? How much do you get in that case? Well, nothing. Right-wing governments have tried time and time again to decrease the taxes for the rich, in order to incentivize the supposed “job-creators”, increase the economy, and receive more total taxes as a result. The latest instance is Trump’s Tax Reform. It has never worked.

What many dogmatic capitalists seem to forget is the first principle of economics: resources are limited. So therefore naturally there’s a limit to how many resources the rich may hedge before the poor classes start to starve.

There is no magic bullet: there is a limit to the amount of value an economy can create. And how you distribute the fruits of that value does matter, and that is the distribution of income.

Numbers

So we start with two premises a) too much inequality is bad, and b) income is finite. We have to find a number to express how much inequality there is, that is certain, and anybody that lives in the real world understands you can’t give to four people half the pie each (4/2 ≠ 100%) (staunch capitalists seem to forget that).

I can tell that a nation has a R/P 10% of 23.05, a Gini of 48.86, or a top 1% share of 13.5, but what does that really say? I would have to explain what each metric means, and you would still not get a good picture. I could show the Lorenz curve as well, but I would have to explain it, and it still would be hard to see what is the problem, if there is any.

Example 1

Let’s say there’s an economy of two people, a total of value created of $100,000 (it doesn’t matter the units), and we divide that total evenly ($50,000, $50,000). This is perfect equality, or no inequality, something nobody is advocating for, or even possible. The Gini index is 0, but we’ll see later how to get that number in a more realistic example.

Example 1 (Gini: 0.0).
Median: $50,000 – Richest: $50,000 – R/M: 1.0

Example 2

A slightly more realistic example divides the value unevenly ($20,000, $80,000). In this case there is inequality, but how much?

Example 2 (Gini: 30.0).
Median: $20,000 – Richest: $80,000 – R/M: 4.0

The Gini index is often referred as a representation of the Lorenz curve of an income distribution, but we don’t need extra layers of complexity to understand what the value means. Another way to define Gini is in terms of the relative mean absolute difference: we find all the relative differences, and divide by n.

The total is $100,000, x₁ is $20,000, x₂ is $80,000, so: |x₁x₂| / total →|$20,000 – $80,000| / $100,000 →$60,000 / $100,000 → 60%. The relative difference of x₁ and x₂ is 60%, and the other way around (x₂x₁) is the same, so the sum is 120%, we divide that by n (2), and the result is 60%. The Gini index is half of the RMAD, so: 30.

So when you see a Gini index of 30, you can picture the above distribution (20, 80), but is that a fair distribution? Well, 30 or above is considered medium inequality (30 < x < 50), but I leave it to you to decide if it actually is.

Example 3

Let’s move to a more complicated example ($7,000, $13,000, $20,000, $60,000):

Example 3 (Gini: 41.5).
Median: $13,000 – Richest: $60,000 – R/M: 4.6

At first this looks like it has more inequality, but in fact the economy follows the same distribution as the previous example, except with more granularity: x₁ + x₂ = $20,000, x₃ + x₄ = $80,000.

It’s much more tedious to calculate the Gini mathematically by hand, just the first element would be: (|x₁x₁| + |x₁x₂| + |x₁x₃| + |x₁x₄|) / total → ($0 + $6,000 + $13,000 + $53,000) / $100,000 → $72,000 / $100,000 → 72%. The whole RMAD is (72% + 60% + 60% + 140%) / 4 → 332% / 4 → 83%. So the Gini is 41.5.

But wait a second! Why is the Gini higher in this case, if the distribution is the same? Well, that’s the first caveat of the Gini index: it depends entirely on the number of samples of the population: the more samples, the more precise it is.

But that’s not the only caveat. If you have been paying attention, you might have deduced already that there’s more than one set of four numbers whose relative absolute difference equals to 332%. Which means there’s many income distributions that result in the same Gini index, and there are:

Alt 1 (Gini: 41.5).
Median: $17,000 – Richest: $54,000 – R/M: 3.2
Alt 2 (Gini: 41.5).
Median: $11,000 – Richest: $46,000 – R/M: 4.2
Alt 3 (Gini: 41.5).
Median: $12,000 – Richest: $51,000 – R/M: 4.2
Alt 4 (Gini: 41.5).
Median: $11,000 – Richest: $64,000 – R/M: 5.8

So that’s the second caveat: a single Gini index cannot represent entirely a distribution of income. It is by far the best way to represent the economic inequality in a single number, but it cannot give you the whole picture.

The last example is particularly interesting, as the richest person earns 5.8 times more income than the average person, yet the Gini is exactly the same because the bottom 75% is quite homogeneous.

Example 4

Finally we arrive to the most realistic example ($2,000, $3,000, $4,000, $5,000, $6,000, $7,000, $8,000, $11,000, $15,000, $39,000):

Example 4 (Gini: 46.2).
Median: $6,000 – Richest: $39,000 – R/M: 6.5

This again follows the same distribution of the previous examples: add up the first five elements and it will give you $20,000, add the rest and it will give you $80,000. But the granularity makes the inequality more visible, the Gini index is increasing, and so is the ratio between the richest and the average person.

At this point I must confess that this is not an entirely fake economy; this is in fact a simplification of the economy of Mexico, and each example follows exactly the distribution of income in Mexico, which has a Gini of 48.86. As the granularity increases, the Gini index gets closer to the real value. Unfortunately even official sources list the Gini index as 47.13, but that’s because the economy has been simplified to ten values, when the real Gini is 48.86 (if you use the whole surveyed sample).

So we actually have reached the limit of official sources and we are going beyond.

Real numbers

It’s time to move away from fake numbers to real ones, instead of sets of 4 or 10, to hundreds or millions. Values are adjusted to have a mean of $10,000, but the proportions are the same.

Real 10

If we divide the real sample into 10 values, we get a graph closely following our fake example #4 (these numbers are not rounded):

Real 10 (Gini: 47.1).
Median: $5,791 – Richest: $39,476 – R/M: 6.8

Real 100

Dividing the real sample into 100 values we start to see how the inequality shapes up. Also, the Gini index is very close to the real value.

Real 100 (Gini: 48.8).
Median: $6,259 – Richest: $134,956 – R/M: 21.6

If you pay attention to the richest person you would see his income keeps increasing as we add more samples. At this point he receives 21.6 times more income than the average person.

Real

Finally, if we plot the real sample as it is (122,643,890 weighted values), we get the following graph:

Real (Gini: 48.9).
Median: $6,313 – Richest: $4,406,353 – R/M: 698

Does that graph looks remotely similar to a fair distribution of income? The richest person has an income of $4,400,000; 700 times what the average person gets. That doesn’t even reach the 50 index needed to be considered high inequality. 48.9 is still considered medium. And yes; this is real.

There is a final caveat to income surveys: the richest of the rich are extremely underrepresented. The richest person in Mexico doesn’t receive an income of $4,400,000, it’s closer to $4,000,000,000 (400,000 times the media), but the chances of interviewing that person in a random survey are virtually zero. The real number of entries in the survey are 70,000, with a mean household size of 3.6, so you can’t say much about the top 0.001%, except: they have an insane amount of income.

At which point does an economy becomes ridiculously unfair? Well, apparently it’s not with a Gini of 48.9 (or at least this distribution), because Mexico has not exploded into a revolution, although that might be due to ignorance. Perhaps if the population of Mexico knew how unfair the distribution of income is, they would do something about it. But at the moment it seems a Gini index of 50 is manageable.

Conclusion

Hopefully after reading this article you have a better understanding of what the Gini index is, and why it’s a good measure of inequality, although not a perfect one. And what a distribution of income with a Gini of 50 looks like.

This article only scratches the surface of income distribution measurements. There are many ways to stratify the data: by area, by urban vs. rural areas, by number of habitants, by age, by work status (full-time vs. part-time), by sex, etc. The per capita income can be recalculated through equivalization, which increases it dramatically. And the top incomes can be calculated through other means. Plus, there are confidence intervals to take into consideration.

And we didn’t even mention wealth and income dynamics. The income distribution is the number that is more easily obtained, but what is most important is how that number changes, and increases the wealth of each individual. The distribution of wealth is a much more complicated subject, but suffice to say: it’s much more unequal than the distribution of income.

But all this doesn’t change the fact that an inequality in the distribution of income can be measured and visualized. Personally I think anyone with a pair of working eyes can say with confidence: yes, some distributions of income are unfair.

Blog at WordPress.com.

Up ↑