Why bugzilla sucks for handling patches

Since I started to use email to send patches (thanks to ‘git format-patch’ and ‘git send-email’) I’ve never looked back. Using email to send, review, and merge patches is superior to anything else, including bugzilla, and here I’ll mention the most important reasons.

Contributing convenience

Who doesn’t find email convenient? Chances are you have an email account, and a SMTP server to send email with, then all you need to do is configure ‘git send-email’ (once), and you are ready to contribute patches to any project. A typical workflow is: ‘git clone’, commit some patches, ‘git format-patch’, find project’s email address, ‘git send-email’, done.

Proponents of bugzilla argue that submitting patches there is equally easy; all you need is ‘git bz’: an utility to automate common tasks, such as bug creation, and attaching new patches. This sounds good in theory, however, unlike what you might think, most proponents of ‘git gz’ don’t even use it, and therefore don’t notice the drawbacks of this approach, mainly: ‘git bz’ is not really mature.

If you manage to get ‘git bz’ working (took me quite a while), then you stumble with another problem: you need a bugzilla account for every project you want to contribute, and of course, you need to configure ‘git bz’ to use each bugzilla account. But wait, I’m not talking about tracker, product, and component, but also default field values, such as assigned-to, operating system, platform, version, priority, and even whether https is used or not. Unfortunately, ‘git bz’ doesn’t even allow to configure these fields, so you would need to patch it. And finally, you need to login to the respective bugzilla with your configured browser before using ‘git bz’.

So no, ‘git bz’ is nowhere nearly as convenient as ‘git send-mail’. In fact, anybody suggesting it without trying it is making a disservice, so please, don’t.

Distributed

If you know anything about git, you know that being distributed is a defining factor. It only makes sense that the patch flow is also distributed; for example, if Turk creates an internal repository for his company team, he should be able to receive patches from the team, and at certain point in time push the patches upstream. It would be ridiculous for Turk to setup a bugzilla server just to receive patches from the team. No, he doesn’t need to do anything, his teammates can use an already familiar channel (email) to send the patches.

As I mentioned, I had to patch ‘git bz’ just to make it work, it took me a whole day, and after that I was so happy with the result that I decided to contact the author and send him the patches, and I did; through email. Isn’t it ironic that the only way to contribute to ‘git bz’ is through email patches? After all, it’s just 2K lines of code, does it really deserve it’s own component in say, GNOME’s bugzilla?

My patches were ignored, and the last commit was 3 months ago. To this date I don’t know what’s the recommended way to contribute, so if you want to take a look at my repo here it is, but honestly, I don’t see much hope for this project.

In conclusion, patches in bugzilla is an awful way to work in a distributed manner, and also for small, or new projects.

Scalability

Teams that use bugzilla usually work on a centralized manner: there’s a team of “official” developers who have commit access to the central repository, and outsiders must submit patches through bugzilla. This is a very traditional way of working, however, has two big disadvantages.

First, being an outsider means each commit will be painful, therefore, even if you are perfectly comfortable with the code-base, your commit bandwidth will be very low. If the project is open enough, you will stay in the “outsider” realm for a short time, and when commit access is granted, the real work begins, but still that’s a waste of resources.

Second, “given enough eyeballs all bugs are shallow”, IOW; patch reviews are good. Now, a special mailing list for post commits (in typical CVS fashion) is not good enough, and here I’m going to make a guess; there’s psychological difference between commenting on a change that is proposed, as opposed to a change that is already done; that is, in the first case the other person has to make the change, and in the latter you have to make the change, at least if you want to be a good contributor and make sure the comment is not lost. Most of the time people avoid these “would be nice if…” comments because the change is already done, and think “nah… it’s not that bad”.

Moreover, it creates a messy commit history. For example, it’s not uncommon to see a considerable amount of tries for each patch series in linux (kernel), say 5 tries, before everyone is happy. Multiple commits fixing the previous ones look ugly, and confusing, on the other hand, one commit that’s small nice and clean makes everybody think: this is perfect!

Anyway, the true test of scalability is to let everyone contribute the same way. Would it be sensible for a project to force everyone to submit changes through bugzilla? No, that would be insane. How about email? That’s exactly how many projects work: linux, ffmpeg, qemu, etc.

Reviewing convenience

Suppose somebody goes through all the pain of creating a bug report and manually attaching a patch series, and adds you to CC (hopefully you have a bugzilla account), then, you would receive an email notification, which you most probably will wish you can just reply, but no, you have to click on a link, login to bugzilla, then, for each comment you would have to manually copy the context, add attribution lines, and then your comment.

With email, you receive an email, you reply, and add your comments inline. What’s more convenient than that? Moreover, each patch can be reviewed individually, and have it’s own separate thread, unlike in bugzilla, which comments follow a single continuous line.

Nowadays there’s splinter for bugzilla, which allows to comment directly on the patches, but it’s still nowhere near intuitive as email; the pain of logging in, finding the patch, etc. is all there, as well as the cumbersomeness of comments on top of comments.

Moreover what happens if you are offline? In email: no problem, bugzilla: you are screwed.

Statefulness

So we arrive to the only relatively valid argument in favor of bugzilla: bug entries don’t disappear, so it’s possible to ignore patches for one year, and work on them later on, IOW: “they will not be lost in my inbox”. The immediate counter-argument is: perhaps if the full patch arrived to your inbox it would be harder to ignore it, and easier to just comment on it. But let’s assume that statefulness is desirable for the sake of argument.

First, if the patch is important to the submitter, and it seems to be ignored, the solution is easy: resend. So, important patches will not be lost. Patches are also ignored in bugzilla, and in such cases people usually ping. Here is an example of a patch ignored in GNOME bugzilla for almost 3 years: 426326; chances are gnome-mount will become obsolete before that patch is applied. Aren’t you glad bugzilla is keeping track of it?!

Secondly, if you really want to keep track of all the patches so nothing gets dropped to the floor, then how hard would it be to have a script that parses incoming email? For example, there’s patchwork, which handles all incoming email of a mailing list, provides a web interface, and allows you to keep track of the status of all patches: accept, reject, under review, etc. For example, see the linux-omap patches list. This way you have all the advantages of email, plus the stateful nature of bugzilla.

So, no, this is not an excuse to use bugzilla.

Conclusion

Don’t get me wrong: bugzilla is an awesome bug tracker, but being awesome in one thing doesn’t mean the same will happen on a different area.

It seems people still don’t understand what distributed development means; it means development doesn’t depend on a centralized entity; it doesn’t depend on an “official” group of developers with commit access, and it doesn’t depend on a centralized patch tracker where “official” developers review and apply patches. So, please forget about this historic baggage; email is simple, natural, distributed, and doesn’t discriminate against one-weekend-contributors.

16 thoughts on “Why bugzilla sucks for handling patches

  1. Using email for patch handling might work if you set up separate mailing lists etc. for them like your patchwork example. Problem is when people advocating emailing patches start mass emailing them to people whose only sin was touching some related code 9 years ago.

    More important than whether to use bugzilla or email is to think of the social consequences and impact of ones action and only take such actions if they actually solve problems instead of just aggravate people. Sometimes it is better to use what one considers a less optimal method sometimes than just make a lot of people irritated and annoyed if one want to achieve ones primary goal, which is to get a patch merged upstream.

  2. @Christian Those comments are completely unrelated to this post.

    But anyway, IMO CC’ing people that have previously worked on the modified code is a good practice, and that’s exactly what they do in the linux kernel (see scripts/get_maintainer.pl). However, heuristics can be adjusted; let’s say, only CC people that have made more than 5 commits touching these lines, etc.

    If you are of the opinion that “Linux does it” doesn’t constitute an argument in favor of certain practice, then I’m not interested in discussing this argument here (wouldn’t lead anywhere). I’ll probably make another post about Linux’s good practices, and I think it would be more appropriate to comment then.

  3. @Christian Oh, and one’s primary goal is far from being “get a patch merged upstream”, that’s just a means towards an end. Besides, I’m not suggesting people to boycott projects using bugzilla, I’m merely expressing an opinion.

    IMO if you care about your contributors you should think about the pain caused by the contribution process, so bugzilla vs email does matter, just like git vs CVS.

  4. I fully agree that Bugzilla sucks, but e-mail sucks too. It sucks a lot less, but it still sucks. It may be better than any other tool out there for patch management, but it still sucks.

    Just as Bugzilla is a bug tracker and nothing more, e-mail is a communication tool and no more. Using e-mail doesn’t mean you are tracking or handling patches. It just means you are communicating patches and commenting about them. There is no management, other than what each individual may do after receiving the e-mail message.

    Simply put, with e-mail you can’t query what patches are pending review or assign status or priority to them or flag them or what patches are older than N days. You can flag the message, just like any non-patch message but not the patch itself. Also, for a clean tracking of comments you would need all MUAs to have perfect thread handling for both replying and reading and most are really far from perfect and incompatibilities may easily occur.

    I’m not defending Bugzilla and I agree that e-mail is excellent for publicly code reviewing a particular piece of code (it matches that “code review” is essentially “communication”). It’s just that it is different from what we should expect from a real distributed patch handling tool.

    What about sending two patching options for a particular bugfix? What is the tool to easy and quickly see the difference among the patches?

    In my head, the system has the comments and flags directly attached to a commit or a branch or a tag or even a hunk and you don’t need e-mail to handle them. If integrated to git, I imagine doing “git pull” and having all the comments, dates, flags (if any) status changes brought together with it.

    Or maybe not, maybe a distributed system completely parallel and separate to git where you can use the same system to track patches for different repos in a truly distributed way, who knows.

    So there is still a lot to be done in this field. It is evident: looking for “distributed patch tracking” yields results related to DVCS instead (except for Patchwork).

  5. @alvarezp email is not perfect, that’s true, but I don’t agree with your comment “e-mail is a communication tool and no more”; a good communication tool is exactly what you need to review code. Besides, most of your comments assume that you don’t have supplementary tools such as patchwork to track the status of the patches.

    Regarding the MUA; if somebody is using a sucky one, that’s their problem, just like it’s their problem when somebody tries to use bugzilla with a text-based browser.

    Also, supposing git included commit comments it still would require a communication tool to say “hey, check my comments; fetch from my repo”, and that would require a context switch from the communication tool. Then, you would reply “hey, check my comments on your comments, on this other repo” which requires yet another context switch. It quickly becomes tedious.

    And finally, it seems you are looking for something like ditz; a distributed issue tracker. It would be cool to let ditz manage patches, but still, I think email should be one way to interface with it.

  6. @felipec: Just to clarify over “I don’t agree with your comment “e-mail is a communication tool and no more”; a good communication tool is exactly what you need to review code”:

    That’s exactly what I said in the fourth paragraph. Reviewing code means “commenting” about code which is essentially communication, but “handling patches” also involve tracking (mostly querying and everything that derives from it).

    Tried ditz… still don’t know what to say. It is not vcs-based, but vcs-friendly (semifriendly). Though it is still an issue tracker, it is a good idea.

    BTW, read about Patchwork and looks like a great solution. Have you tried it?

  7. @onkar yes I tried it some time ago, the code was organized very badly and I didn’t bother any more. In any case, the interface is a bit better, like bugzilla+splinter, but still has the same problems: centralized, requires account, login, etc.

  8. While I agree that the patch review process sucks in bugzilla, I disagree that email works better at scale. At Mozilla we do in fact require everyone to go through bugzilla to get patches landed (whether you have commit access or not), and I assure you that scaling that process is not one of our big problems. (We have plenty of other problems to deal with!) I believe that any large project will have standard ways of doing things, and learning those customs is the hard part for new contributors, not the technical details of where to put a patch.

    If we had to handle the flood of bugs we have in a single mailing list, we would be overwhelmed and everything would get lost. We simply have too many bugs filed for the number of developers we have to handle them. We have some great people who do triage, trying to sort out the relevant bugs from the irrelevant and getting them in front of the right people.

    Keeping patches in bugzilla is good for us, because it keeps our bug tracking and our bug fixing in the same place. If we had patches on a mailing list, then they’d be in different places, and it would become a lot harder to determine whether something had been fixed, or whether a certain fix made it to a certain stable branch.

  9. @Ted The fact that you do worst in other areas doesn’t mean you do well in this one. What is your definition of “scale”? How many patches per day?

    I didn’t suggest for bugs to be tracked through email, only patches.

    Also, your argument that separating the patches makes things a lot harder is very weak. Wouldn’t the issue be solved by adding a simple line like “Fixes bug #123456” to the commit message, and a bot that parses this?

  10. Pingback: using bugzilla in the right way « Herostratus’ legacy

  11. Pingback: Why Linux is the most important software project in history « Felipe Contreras

  12. Pingback: Why Linux is the most important software project in history | Maemo Nokia N900

  13. Pingback: Why Linux is the most important software project in history | Daringsearch

  14. Ted Mielczarek never replied, but in any case the argument is flawed. The Linux kernel is certainly a huge project, it uses e-mail for patch reviewing, and it scales just fine. The rate is 5-6 patches per hour, no problem–and that is just the final version, it’s not strange to have patches with 20 revisions before being accepted and committed.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s