My tone doesn’t make me wrong, or how I convinced the Ruby project to fix an inconsistency

Back in 2012 I stumbled upon an inconsistency in Ruby’s handling of time zones which took me a lot of discussions to get fixed, but eventually I managed to get it done.

Essentially this didn’t work:

DateTime.strptime('0 +0100', '%s %z')

The time zone part (+0100) was ignored.

If you are not familiar with strptime/strftime, the former parses a string with a time, and the later formats it, for example:

date_str = '1970-1-1'
DateTime.strptime(date_str, '%Y-%m-%d').strftime('%Y-%m-%d')
=> "1970-01-01"

The format “%Y-%m-%d” specifies the format of the date, and in this case it works fine, but when “%s %z” was used, it didn’t. “%s” is the number of seconds since the epoch (1970-1-1), so, these are the same:

DateTime.strptime('0', '%s')
DateTime.strptime('1970-1-1', '%Y-%m-%d')

%z” simply adds the time zone information (or at least it should), and since strftime is the inverse of strptime, this should work:

DateTime.strptime('0 +0100', '%s %z').strftime('%s %z')

But it didn’t (the time zone was ignored).

I reported the bug in 2012, a patch that fixed the issue was provided hours later. So that’s it, right? Not so fast.

Tadayoshi Funaba

The purpose of this post is not to chastise Funaba, but he is a central part of the story, and his actions speak for themselves.

Immediately after the bug report was created, Funaba stated that “%s” implied UTC, so adding a time zone was “odd as a date”, but perhaps could be considered as an option for the future.

I explained to him that “%s %z” works perfectly fine in C.

He replied with this:

you told about C’s stdlib.
why not talk about ruby’s Time?
Time is based on stdlib.
why not want it with Time?

Followed by:

there is no evidence of bug.

Then he marked the bug report as rejected.

Right away we start to see an issue: Funaba is not precisely fluent in English. That’s OK, we can’t ask everyone to have perfect English (I’m sure I’ve uttered many barely parsable English sentences), but if he is willing surely we can find a way to communicate effectively.

I explained to Funaba that we were talking about C’s strptime(3), not mktime(3). Additionally, the documentation of Ruby’s Date.strptime did mention C’s strptime(3) specifically.

At this point it’s worth mentioning that Ruby has two different classes: Date, and DateTime. Although Date doesn’t represent time internally, it did use the time zone, and strptime worked correctly. Not so with DateTime.

What possible reason could there be for Date to honour the time zone, but not DateTime?

I asked Funaba this question, and also, if “%s %z” is so wrong, why not then fail with an error, in both Date and DateTime, not just with strptime, but also with strftime?

Funaba decided to reply in Japanese:

俺の英語は通用しないらしいから日本語で書いてあげます。(My English doesn’t seem to be understood by you, so I will write it in Japanese for you.)

I used Google Translate to try to understand what he was trying to say, which still didn’t make any sense.

He mentioned lcoaltime(3), time_t, and Time, which have absolutely nothing to do with what were talking about (plus there’s a typo).

And then he presented this code (code I can understand):

DateTime.strptime('%F %T %p', "2001-02-03 16:05:06 PM")
ArgumentError: invalid date

This is the documentation of DateTime.strptime:

DateTime.strptime([string[, format]])

Can you spot the problem? Funaba (who I later learned was the maintainer of this part of the code) didn’t even realize he put the format instead of the date string. 🤦

He used his mistake as an argument that some combinations don’t make sense, but of course if you swap the arguments the command works fine.

Additionally, he tried to say (presumably) that “%s %z” as a representation of a date doesn’t make sense–he was wrong of course, since this is precisely the representation git uses internally. You can verify any commit with “git cat-file -p @“:

author Junio C Hamano gitster@email.com 1623644608 +0900
committer Junio C Hamano gitster@email.com 1623645268 +0900

Do you see that “1623644608 +0900“? That’s a “%s %z” format. See Git Internals – Git Objects.

Since I developed both Mercurial (git-remote-hg) and Bazaar (git-remote-bzr) bridges for Git, I know that’s all the information they need as well.

Surely after learning this fact Funaba reconsidered his position… No, he stopped responding.

I noticed that Time had a similar issue, so I sent a patch to the mailing list, and it was applied without problems.

I also fixed Rubinius and RubySL, none of them had any problem applying my patches.

I mentioned those fixes, and additionally I provided examples in C and Perl showing how they have absolutely no problem with “%s %z” in a comprehensive summary. Still silence.

One year later–at the end 2013–I gave up on bug 7445.

Second round

Since this was not merely a whim, but something that did actually affect my code, I was bitten again by this bug, and I eventually tried again on February of 2014 on issue 9794.

Funaba rejected the issue with no explanation half an hour later.

This angered me and prompted me to write an email to the mailing list titled Who the f*heck is Tadayoshi Funaba and why can he reject sensible patches unilaterally?

Now, I understand many people don’t like this tone, I was angry and I wasn’t afraid of letting the community know, I’m not the same person any more, I meditate and I can easily calm myself down before sending an email (although I still take combative positions when I feel it’s warranted). However, my conclusion has not changed: I still think Tadayoshi Funaba was wrong and too stubborn to have the power to reject patches. Yes, there probably were better ways to express that sentiment, but this doesn’t change the facts: strptime was broken, a fix was available, and a maintainer was being obtuse. Take me out of the equation and those facts still remain true.

matz–the creator of the Ruby language and leader of the main implementation–weighted in, and he said there had been similar troubles with Japanese programmers before. He provided two arguments why he thought Funaba was right: 1) %s with %z is undefined, and 2) %s means epoch, therefore offsetting it is wrong. matz said:

If you were lucky to read Japanese, you’d have understood it.

First of all, it’s true that “%s %z” is undefined according to the C standard and POSIX, however, so is “%s“. Should Ruby stop supporting “%s“? Of course not! If everybody agrees what “%s” means, and Ruby already supports it, what’s the problem with keeping that support? (even if it’s undefined in the C standard) Moreover, undefined doesn’t mean you shouldn’t support it, it just means that: the C standard doesn’t define it. GNU libc does support it, so does Perl, PHP, Rubinius, etc. If Ruby supports the undefined “%s“, and “%z“, why not “%s %z” as well?

The second point–that according to matz I wasn’t able to understand because I wasn’t lucky enough to read Japanese–is that “%s” means seconds since the epoch. He explained to me what the epoch was (as if I didn’t know already), but he explained it wrong: he said it was a fixed point in 1979, but it’s actually 1970. Additionally, he misunderstood date-time equivalences.

Just like 0xF in hexadecimal represents 15 in decimal (the exact same number), so does 00:00 in New York represents 05:00 in London (the exact same time). When you talk to somebody in a different time zone, you are not experiencing different times, it’s the same time represented differently in different places. The epoch (1970-01-01T00:00:00 UTC) did not happen only in one part of the world, it happened everywhere, for example 1 a.m. in Berlin (1970-01-01T01:00:00 +0100).

You can easily check that they are equivalent:

DateTime.parse('1970-01-01T00:00:00 UTC') ==
DateTime.parse('1970-01-01T01:00:00 +0100')
=> true

Not only that, but if you asked ruby to represent the epoch in Berlin with “%s %z“, it did it correctly:

DateTime.parse('1970-01-01T01:00:00 +0100').strftime('%s %z')
=> "0 +0100"

Therefore “0” and “0 +0100” are exactly the same time, just one in London, and the other in Berlin. The time zone is just extra information and doesn’t affect “%s“.

So it wasn’t me the one who was unlucky not to read Japanese, it was Funaba the unlucky one to not read my rationale in English.

Also, I specifically asked if anybody could come up with a single reason why the time zone should be ignored. Nobody was able to do that.

A mess

matz did concentrate on the actual issue but others concentrated only on my tone, and even threw accusations of racism:

Rubyists are open and welcoming people able to navigate cultural issues without resorting to racist rants.

I think you should find a different language.

Others did defend me, and accepted there was a language problem, and a real bug. But too much time was spent on me (which wasn’t my intention), and too little on the actual issue.

Then even matz stopped responding on the mailing list, because a separate discussion was taking place in the bug report, but exclusively in Japanese.

Japanese verbosity

Now, I’m actually somewhat familiar with Japanese culture, and I even know a little bit of Japanese. One feature of Japanese communication is that you have to say much more than just what is actually necessary. So for example you never just say Felipe, you have to say “Contreras-san”. Why? Just because.

When you meet somebody the equivalent of “Hi, I’m Felipe. Nice to meet you.” is:

Hajimemashite, watashi wa Felipe desu. Yoroshiku onegaishimasu.

“Hajimemashite” roughly means “it is the first time meeting you”, “watashi wa Felipe desu” means “I’m Felipe”, and “yoroshiku onegaishimasu” means “please be kind to me”. Why do you have to say so many things? Just because.

That’s why when I tried to follow the discussion between Funaba-san and Tanaka-san in Google Translate, I wasn’t particularly surprised when I realized it took them about 25,000 words to arrive at the conclusion that %s and %z are orthogonal. Or at least Tanaka arrived at it.

Note: Apparently some people think that I’m deriding the Japanese language. I’m most definitely not. If I didn’t like the language I would not have studied it. All I’m trying to say is that it’s hard to summarize the Japanese discussion. If you don’t believe me fire Google Translate and try it yourself.

One of the weird things in the Japanese discussion is that Funaba said he had no idea what was the purpose of the proposed change. Tanaka provided him links where I explained it was to parse git dates. Also, Tanaka himself had trouble understanding Funaba, even in Japanese. Tanaka said he didn’t understand what was the problem with “%s %z“, Funaba said it was “funny”, Tanaka asked him what he meant by that, and Funaba was never actually able to express what he meant.

I will show a full response by Funaba, just to exemplify the kind of verbosity I’m talking about (I’ll do my best to try to parse the translation):

Update: Somebody provided translations so I’ve updated what I originally parsed from Google Translate.

Chunk 1

Akira Tanaka wrote:

> tadayoshi funaba wrote:
> 
> 曜日と年月日とは異なり、%s と %z は独立なので問題ないと思います。
> ここで独立というのは直交しているという意味で、片方を変えてももう一方は変わりません。

問題があるとかでなく、別のものなってるという事です。
たとえば、可能性としては、UTCで書いた日付を地方時にという事も出来るという事ですよね。
今回 %s だけですが。

俺は今、日本にいてUTCの時刻を扱っているとします。
それを例えば以下のように記述する事にします。
与えられた時刻はUTCだけど、それを +09:00 で参照する事にします。

2001-02-03T04:05:06 UTC (+0900)

一般的でないので、
DateTime.parse('2001-02-03T04:05:06 UTC').new_offset('+0900')
といったような操作で説明する事になります。

今現在このような記述は ruby ではないですが、'%s %z' の件で求められているのは、俺からするとこういうものを含む、これまで扱っていたものとは別のものだと感じています。
Translation

It doesn’t mean there is a problem, it just means they are different. For example, it is possible to write a date in UTC and then change it to local time. But this time it’s only %s.

Let’s say I’m in Japan and I’m dealing with UTC time. The given time is in UTC, but I’ll refer to it as +09:00.

2001-02-03T04:05:06 UTC (+0900)

Since this is not common, the description would be something like DateTime.parse('2001-02-03T04:05:06').new_offset('+0900').

Now, this kind of description is not currently in ruby, but I feel that what is required for the ‘%s %z‘ thing is something different from what we have been dealing with, which from my point of view includes this kind of thing.

Chunk 2

> 日付とはなにかとか、そういうことをもうすこし詳しく述べていただけると、理解が可能になるのかもしれません。

俺が日付と言ってるのはそういう時刻も含まれてます。
この日付というのはある規則に基づいて時間軸上に振られた名前のようなものだと思っています。空間に置き換えるとマイルストーンのようなものです。
実際にはもっと不完全な日付もあって、それも日付です。
文脈やその他の情報から特定できる場合もあるし、出来ない事もあります。
Translation

When I say date, I am including such time. I think of this date as a kind of name assigned on a time axis based on certain rules. In space, it is like a milestone. There are actually more imperfect dates, which are also dates. Sometimes they can be identified by context and other information, and sometimes they can’t.

Chunk 3

> > 時差に特別な意味があると思えば地方時で記述すればいいと思います。
> 
> どこの地方時かというのが問題で、OS に設定された地方時じゃなくて、
> UTC からの時差が固定されている人工的な地方時を選んだほうが、
> 与えられた時差をそのまま記録できる、という話だと理解しています。

ちょっと意味が判りませんでした。

%s がそもそも不完全と思えるので、なぜこんなに拘るのはわかりません。

%s を空間に置き換えてみると、これでマイルストーンは置けず、ただ距離が示されているだけ、距離の定義がわかない状態ではないですか。

あと、ついでなので、田中さんが固定時差と言ってるものも万能でない事は確認しておきたいです。

与えられた自明の地方時も有用で、時差ではなくタイムゾーン=時間帯の情報は
単なる時差以上の情報を含んでいます。

夏時間など実際に我々が参照している時系と形式は重要じゃないですか。
そういう意味も含めて、%s 偏重の流れに違和感を感じます。
Translation

I’m not sure what you mean.

Since %s seems incomplete to begin with, I don’t know why you are so hung up on it.

If we replace %s with space, we can’t place milestones with it, it just shows the distance, and we can’t define the distance, can we?

Also, just for the record, I would like to confirm that what Mr. Tanaka is referring to as fixed time difference is not universal.

The given self-evident local time is also useful, and the time zone = time zone information, not time difference. It contains more information than just time difference.

Isn’t it important to have the actual time system and format we are referring to, such as daylight saving time? In that sense as well, I feel uncomfortable with the trend of %s bias.

So…

Even trying my best I can’t really for the life of me even get a rough idea of what is being said. I suppose he means he doesn’t like %s, and that using a time difference from %s can’t deal with all cases (all of which is irrelevant).

And this is just one response, not even the longest, or the most confusing.

I did ask them to please provide a summary of their discussion in English, but they just ignored me.

matz reopens

After 23 replies back-and-forth between Funaba and Tanaka–and god knows how many thousands of words–matz weighted in (in Japanese):

If the format is so frequently used by tools that are actually widely used, it is not inconsistent information as in the case of the day of the week, so it is good to leave the offset information in this case. I think.

It’s up to you to use the patch you received or write your own.

Funaba didn’t seem to like that response:

I feel like I’ve been told that I don’t have any decision-making power.
I really realized it today.

Since matz didn’t see any harm in the change, and also failed to understand why Funaba was against it, he reopened the issue.

Eventually Funaba pushed the change:

a new feature (not a bug fix) of strptime. applies offset even if the given date is not local time (%s and %Q). This is an exceptional feature and I do NOT recommend to use this at all. Thank you git community.

Not content with this passive-aggressive commit message, he left these final words:

行末の空白を削ったりして、のんびり余生を過そうと思います。(I’m going to spend the rest of my life leisurely, removing whitespaces at the end of lines.)

I mean, he not only failed to convince a single person that the change was problematic–in any language–nobody even understood if he ever presented an actual problem.

I really don’t think the trouble was with me.

I didn’t have anything against Funaba, I just don’t think he should have had such kind of power over an important part of the Ruby core.

After that incident he only made 3 commits more over the next few days and never touched the project since.

Conclusion

Regardless of what you may think of me or the tone I used back in 2014, I did fix the date-time parsing code of multiple projects, and the only reason I wasn’t able to do so in Ruby MRI’s DateTime was Funaba, and only Funaba.

Some people make the wrong assumption that being polite always takes you further, despite the ample evidence that in many instances the opposite is the case: in business, politics, foreign policy, dealing with ex-spouses, neighbors, flatmates, bosses, colleagues, people in tech support, or at a convenience store, and many, many more.

In this case when I was polite in the bug report Funaba simply rejected it saying “there is no evidence of bug“, and then proceeded to ignore me. When I sent the patches to the mailing list with nothing but plain comments absolutely nothing happened. It’s only when I sent a fiery response that people started to pay attention.

Despite everything, I was right.

However, I understand. In this case it was Funaba against the world, in many instances I have been against the world, and sometimes the world is wrong. But this wasn’t Funaba’s code, it was the Ruby community’s code, if the Ruby community wanted a change, he should have just stated his objection for the record, and applied it. Sometimes you get to see a change blow up, and tell everybody
“see? I was right”, but this was not one of these cases. Sometimes the world is right.

When it comes to politeness though, the world is wrong. Sometimes you have to be bold, and this is one of those examples. Like Linus Torvalds says: “on the Internet nobody can hear you being subtle”.

A lot of people misunderstood the point of the post, it wasn’t to rehash an old argument in which I was right, it is to show an example in which ignoring tone and focusing on technical merit is beneficial to the project, and the reason is that the ruby project is doing it again, see: Fixing Ruby gems installation once and for all. Once again the maintainers are wrong, and it will be up to you to decide if I’m right.

5 thoughts on “My tone doesn’t make me wrong, or how I convinced the Ruby project to fix an inconsistency

  1. Maybe it is due to this kind of conflict that Python was chosen by many companies… I personally like Ruby a lot better.

    In terms of “you have to be bold and combative”, one time I had an interview at Lyft in San Francisco.  The interviewer was holding a dog and brushing him while interviewing me.  When I finished the interview and walked down the street in San Francisco, it felt like a street of gangsters.  It kind of was in line with some coworkers I had.  Forming small circles (called clique) and playing power and politics in the office.   Never writing anything down they know, so that they have the power and cannot be fired easily and you have to bow to them in order that they tell you something.  30% or 50%… it depends on their mood and how they are “pleased”.

    In the 80’s and 90’s, people who worked in the hi-tech were really computer majors and engineers.  These days, they can go to wild cat university or was making coffee and go to a bootcamp for 6 months and now they are “developers” or “code ninjas”.  The company may simply be do sugar water and potato chip delivery.  Or it can be letting people find a plumber.  How hi-tech is it?

    I am not sure about other places… but in Silicon Valley, companies are going towards this pirates, gangster, wolves type.  Also known as Pirates of Silicon Valley, Gangs of New York, Wolves of Wall Street.  You said you have to be bold and combative. Think about how you deal with pirates and gangsters.

    Liked by 1 person

  2. Thank you for this interesting insight into workflow in open source ecosystem.

    Forgive my ‘boldness’, but when you say
    > 0xF in hexadecimal represents 16 in decimal.
    it doesn’t impact your point, but I think the world is right on this one and you are wrong 🙂

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.