How msn-pecan fixed a 6 year old bug, how Pidgin didn’t, and stole the fix

The bug we are talking about is the infamous switchboard timeout error which was very elusive, it happened randomly, and very often for some users in unknown conditions. Essentially you send a message, and after one minute you receive a notification telling you the message never arrived, after which you need to resend the message, and hope it will arrive this time.

Let’s see how the two projects approached this bug.

Pidgin

There have probably been many bug reports regarding this issue, but it’s very difficult to find old historic bugs in Pidgin’s new and old tracker. The modern version is reported in Pidgin’s tracker as #3330. There you can see people saying it happens a lot, that the priority should be increased, and many tickets were marked as duplicate. Developers however stayed in denial mode: they say it doesn’t happen to them, and then turn to the usual strategy: ask for irrelevant information such as a valgrind log, and to try again as it might have been magically fixed.

Then they try a simplistic workaround; re-send the message on failure. This doesn’t work on most cases, and even when it does, the message arrives more than 1 minute late. As usual, no developer did much else about the bug.

In the mean-time, Adium had many reports (2475, 2395, 6316, 6708, 6952, 7288, 9978, 11045, 11398, 11478) of the same bug. At this point something was very clear: it happens more often in OS X.

The interesting point is when Rasmus Hummelmose, an Adium user, logged to IRC to rant about the problem. He received the same response on both #adium and #pidgin; it’s a server problem, or it’s your slow connection, there’s nothing we can do. That didn’t convince him (it wasn’t true) and he effusively tried to explain that the issue was real and was affecting many users. He didn’t achieve anything more than upsetting the developers.

This is not the way to solve an important bug.

msn-pecan

The msn-pecan team on the other hand thought: hey, there’s a bug, and this guy can reproduce it, let’s fix it. I invited him to #msn-pecan. Rasmus was a bit reluctant; Why loose time with msn-pecan developers? Surely Pidgin developers must be capable enough to do the job. He changed his mind when I explained that the core parts of libpurple’s (Pidgin) msn were either developed or refined by me anyway, and therefore, Pidgin devs probably didn’t have the expertise required to identify this problem.

With that we started an endeavour to fix the problem through the weekend. I started by providing some infrastructure changes in order to visualize what was actually happening, Devid Antonio Filoni created Adium builds, and Rasmus tested, and provided feedback. We made some conjectures and discarded them with further testing and fixed some bugs along the way until we found a reliable way to reproduce: send a message, wait for 15 minutes of inactivity, and send another message.

After this is was clear that something bogus was happening with the network connection, but since we cannot fix all the elements involved, we implemented a simple fix: close the connection after 1 minute. That worked perfectly. Rasmus was happy, and we were too :)

That‘s how you do it.

The stealing

Logically after our success, Rasmus decided to rub the fix on the face of Pidgin and Adium developers, after all, they were the ones that said it was not a bug. But they were not impressed.

However, Daniel Ljungborg (aka Dimmuxx) was interested in the fix, and in good faith I pointed the commit message that explains the issue in detail.

Then I find out Ka-Hing Cheung, a Pidgin developer, implemented the fix as I described, but thought it was OK to not thank anybody, explain where the fix came from, or mention the msn-pecan project, or any external source at all. We (Rasmus, Devid and I) spent a weekend of our free time working hard to identify, fix, and verify the issue, and if you read the commit message you would think they came out with the solution:

Author: khc@pidgin.im
Timeout switchboard connections at 60 seconds, should Fixes #3330 for most people.

That is plagiarism, pure and simple, and unfortunately, it’s not the first time.

msn-pecan 0.1.0-rc1 ready for testing; on the way to the first serious release

The next msn-pecan release started as 0.0.20 but there are so many changes that
it’s going to be 0.1.0. It is way more stable than 0.0.19 but we still would
like to do more extensive testing, so we are rolling a release candidate in
order to fix critical bugs that might be lingering. Hopefully it will be the
only release candidate before the actual release.

The aim of 0.1.0 is going to be our “first serious release”, that doesn’t mean
the previous releases were bad, it just means that we were never truly
confident about the code being delivered until now.

Compared to 0.0.19:

  • Timeout issues fixed (switchboard error)
  • Better offline messages receiving support
  • Offline message sending support
  • Reorganization of P2P code (less crashes)
  • Several crash fixes
  • Adium improvements
  • Performance improvements
  • Massive code reorganization

Special thanks for Devid Antonio Filoni, and Andrea Piccinelli who have been
very active fixing issues and making sure msn-pecan is rock-solid. Also Rasmus
Hummelmose who was essential in fixing the timeout issues, it wouldn’t have
been possible without his testing. Also thanks to the Pidgin developers (we
picked some patches), and many other contributors.

This is the list of issues fixed so far:

  • 37: Pidgin leaves handle on files after transfers
  • 82: Implement sending of offline messages
  • 117: Received offline messages are being cut
  • 138: Translation is not whole integrated from Launchpad
  • 144: Unable to chat after message timed out
  • 155: Pidgin crashes after connecting (using NTLM Authorization Proxy Server)
  • 156: msn-pecan crash in msg_ack() at cvr/slplink.c:321
  • 157: msn-pecan crash in msn_switchboard_can_send() at switchboard.c:779
  • 158: msn-pecan crash in msn_switchboard_free() at switchboard.c:262
  • 159: Pidgin crash when connecting to MSN
  • 161: 0.0.19 ubuntu package
  • 163: Translations not working on win32
  • 164: msn-pecan crash in pecan_contact_get_personal_message() at ab/pecan_contact.c:616
  • 170: Crash upon sign in
  • 171: crash when disabling account
  • 174: Windows 7 RC and Pecan
  • 177: Offline messages of blocked contacts should not be displayed
  • 181: Too many timeout messages
  • 183: msn-pecan should use audio:// links with pidgin 2.6.0
  • 184: already showed OIM message show again using another client
  • 185: Add support for receiving winks
  • 133: pidgin crashed with SIGSEGV in msn_message_destroy()
  • 154: Pidgin Randomly Crashes
  • 166: proxy authorization support missing
  • 153: User Adding Problems

The diffstat is huge:

44 files changed, 3423 insertions(+), 3116 deletions(-)

For the source tarball, win32 installer and maemo package check the usual location:
http://code.google.com/p/msn-pecan/downloads/list

And the Adium build is here:
http://code.google.com/p/msn-pecan/wiki/AdiumBuilds

So, start the testing! And please report back any issues :)

Here is the current list of pending issues for 0.1.0 final:
http://code.google.com/p/msn-pecan/issues/list?q=label%3Amilestone-0.1.0

Finally here’s the shortlog:

     6  Andrea Piccinelli
     1  Chris Stafford
     1  David Geary
    29  Devid Antonio Filoni
     1  Devid Filoni
     4  Elliott Sales de Andrade
   214  Felipe Contreras
     2  Mike Ruprecht

msn-pecan 0.0.19 released, more stability and “now playing” support

This is mostly a bugfix release that should make msn-pecan much more stable. Bug also brings some nice features such as “now playing” support.

Everyone is encouraged to update.

I’ve been busy with other projects so I have not been able to work as
much as I would have wanted, but all in all, this is a good release.

The features (thanks to Devid) include support for “now
playing” (ported from libpurple), and a new way to be notified when a
contact has deleted you (you’ll see on the conversation window). Also,
some issues in Adium were fixed, so we will probably roll an Adium
build with msn-pecan enabled.

I’ve updated my system so it might take a while before the win32
binaries are uploaded, anyone willing to take this one?

Download from the usual place at googe code.

Bugs fixed

  • 99 Unable to retrieve display pictures when invisible or logged as invisible
  • 109 Protocol priority
  • 120 Aliases not cleared properly
  • 133 pidgin crashed with SIGSEGV in msn_message_destroy()
  • 154 Pidgin Randomly Crashes
  • 135 russian translation doesn’t work

Enhancements

  • 140 Support for “Now Playing” status field
  • 129 Can’t verify installation, or uninstall
  • 124 Notify when deleted by a contact

Taks

  • 121 Make dbg package for the launchpad repo

Diffstat

105 files changed, 1295 insertions(+), 749 deletions(-)

Contributors to this release

Albert Cervin (1):
     po: update Swedish translation

Devid Antonio Filoni (14):
     Fix use_userdisplay option using Adium.
     Change PACKAGE to GETTEXT_PACKAGE.
     Translate msn-pecan also using Adium.
     Fix use server alias option
     Move utils/msgplustags.{c,h} to pecan_util.{c,h}.
     Update COPYRIGHT.
     Add PURPLE_STATUS_TUNE support and parse contacts's CurrentMedia.
     Update MsnClientCaps in msn.h.
     Add "Has Space" to get_info.
     Show "You are not in A's contact list" message opening a conv
window if needed.
     Set the passport also to session->user.
     ab/pecan_contact.c: cleanup.
     slp: destroy object at the right time
     po: update Italian translation

Elias Julkunen (1):
     po: update Finnish translation

Felipe Contreras (44):
     Cleanup login steps.
     Show up 'connecting' state.
     session: Cleanups.
     Trivial cleanups.
     Cleanup configuration stuff.
     session: Random cleanups.
     Remove bad Russian translation.
     cmdproc: trivial cleanup
     slpcall: avoid extra error cb call
     status: trivial cleanups
     status: improve "available" state mapping
     Trivial cleanups
     userdisplay: report failure
     Log general server notifications
     Makefile: add debugging symbols by default
     session: destroy ud manager after freeing slplinks
     contact: store pecan status instead of pidgin one
     Reorganize how bad statuses are handled
     notification: store user state
     contaclist: add function foreach_contact
     Get user displays when returning from 'hidden' state
     status: trivial cleanup
     contact: use "invisible" for the HIDDEN status
     Use pidgin mobile status only when offline
     Update copyright notices
     command: trivial cleanups
     command: remove unused refcount
     command: general cleanups
     contact: enable workaround for server aliases
     fix_purple: trivial cleanups
     fix_purple: rename purple_buddy_set functions
     Reorganize switchboard handling.
     Close the http node when not needed.
     Make sure the switchboard is not freed when it's still used by slplink.
     transaction: trivial cleanup
     transaction: add ref/unref
     Properly ref/unref transactions
     transaction: fix initial ref_count
     sb: set error properly
     Fix wrong memory access
     Optimize GType retrieval
     Fix SLP DATA message creation
     po: general translation updates
     Bump version to 0.0.19

Geoffrey Antos (1):
     slplink: fix buffer overflow vulnerability

Luís Neto (1):
     po: update Portuguese translation

Wei Hsiang Hung (1):
     po: update Traditional Chinese translation

msn-pecan 0.0.18 released, now with voice clips support

Hello!

It’s time for msn-pecan 0.0.18. There are many bugfixes and important new features like voice clips support.

Again, Devid did most of the work. The most important fix is that personal status messages now work properly (can be enabled and disabled), the most important feature is voice clip support; thanks to Devid Filoni for the port, original patch from Chris Stafford, and Youness Alaoui for libsiren.

Also important is a fix for WLM 2009 user displays (Thomas Gibson-Robinson), P4-context support for groups.im (Thiago Silva and Devid), a new option to hide Plus! tags (Devid) and support for Plus! sounds (Devid and Thiago).

Also, some cleanups and reorganization heading towards a standalone library.

An interesting event happened on January 12; for some reason the MSN servers changed something that prevented many clients (>MSNP13) from working, while msn-pecan kept working just fine. That day the number of downloads increased dramatically (100010,000 a day), but unfortunately it lasted only a day.

12,552 page views.

12,552 page views.

New translations: Turkish, Portuguese (standard and Brazil) and Arabic. If your favorite language is not translated you can help at Launchpad’s site.

Here’s the diffstat:
diffstat

Download from the usual place at googe code.

PSM option poll

On version 0.0.17 there’s a new option called “independent status message” turned on by default which overrides Pidgin’s status message, and creates an account action “Set personal message” that works exactly like “Set friendly name”.

It’s a little controversial, and Devid doesn’t like the name, so here’s a poll to see what you guys think should be the name of the option. If you have another suggestion, please say so.

libmsn-api plan

Ok,

This is a response to my previous post, since Tiago (libmsn’s author) replied and I think I should explain in more detail.

libmsn-api plan

libmsn-api plan

I think there should be a libmsn-api, in C, which is basically a bunch of .h files. Several things could implement this API, for example pymsn, which is written in Python, but with C bindings that implement this API it could be used by both telepathy-msn and pidgin-msn effortlessly. Of course, there’s already telepathy-butterfly, but there’s nothing that allows libpurple to use pymsn.

In a similar way libmsn could implement the exact same API. The only problem is that libmsn AFAIK doesn’t handle the connection stuff (sockets, proxy, ssl), some something else would need to handle that. In order to write a telepathy CM effortlessly in C, somebody should handle this connection stuff. That would be libim-util, which could be shared by libmsn-pecan and libmsn, and actually other protocols libraries could use the same library, like Skype.

One option which I’m depicting in the diagram, is to wrap libmsn and libim-util together and provide this connection functionality through the libmsn-api. This connection stuff is not required by libpurple since it already provides this connections stuff, however, it’s far from ideal, so it would be nice to have the option to use libim-util for that. This approach also has the advantage of making it easy to write standalone applications, like bots :)

The other option would be to use libim-util directly from telepathy-msn, and remove that connection stuff from the libmsn-api.

Keep in mind that this is not just theoretical stuff, I’ve actually implemented simplified versions of many of these pieces, including the pymsn’s c wrapper.

The purpose is to give more options to the users, more users to the msn backends, and avoid the current situation where each msn application has it’s own msn implementation.

msn-pecan 0.0.17, the beginning of a new era

Hello!

It has been a long time since the previous release, and there’s a reason for that, but first, on with the release.

This release would have pretty dull without Devid, since he pretty much did the two new features: custom emoticon sending support, and a new feature called for a lack of better name ‘independent personal status messages’.

I’ll try to explain ‘independent psm’. It’s related to issue 51 (Easier way to setup personal messages), now you set psm just like your ‘Friendly name’, since Pidgin lacks of a unified way to set nicknames, and psm, then both are now available as account operations.

This means the old way of setting psm would not work. Some people might not like this, so you can turn this feature off.

Ideally, something like Craig Harding’s personal bar should be implemented in Pidgin, but since they are stubborn as rocks you’ll better help Craig and use it as a plug-in.

Also, we have new translations: Danish, German, Hungarian, Norwegian Bokmal, Russian, Simplified Chinese, and Traditional Chinese. If your favorite language is not translated you can help at Launchpad’s site, it’s pretty easy!

Here’s the diffstat:
Makefile | 191 +++++++++++++++++++-------------------
cvr/pecan_slp_object.c | 71 ++++++++++++++
cvr/pecan_slp_object.h | 15 +++
cvr/slp.c | 26 +++++-
io/pecan_http_server.c | 2 +-
msn.c | 238 +++++++++++++++++++++++++++++++++++++++++++++---
pecan_oim.c | 2 +-
pecan_status.c | 42 +++++---
switchboard.c | 67 +++++++++++++-
9 files changed, 518 insertions(+), 136 deletions(-)

Download from the usual place at googe code.

Status of pidgin’s official plug-in

Now, if you remember Adium decided to switch to the new official msn plugin, how is that going?

Well, if you go to Pidgin’s site, you’ll see a whole lot of bug reports. There’s a bunch of ‘account not connecting’, ‘crashing since 2.5′, just as I expected. If you go Adium’s tracker, you’ll see something similar.

Of course, msn-pecan also has some crash issues, but those have been identified as issues with buddy icons, if you disable them (which you can), then it works perfectly fine. On the other hand there are no connection issues.

So yeah, the official plug-in has MSNP15 support (which give it absolutely no extra features compared to msn-pecan btw) but some people aren’t being able to connect! Focus on the essential features, then think about the fancy ones, that’s what msn-pecan does.

Kopete’s libmsn and the future

So Tiago Salem Herrmann has managed to get his libmsn-2 into Kopete, congratulations!

I have been playing with a standalone libmsn-pecan, that other projects can use, not just Pidgin, so libmsn-2 seemed like a possible replacement, no need to re-invent the wheel, right? But unfortunately libmsn-2 is written in C++ and there are no C bindings.

So I thought on creating C bindings for libmsn-2, actually the same C API can be used for libmsn-pecan and libmsn-2.

So this is the plan:

msn-pecan plan

msn-pecan plan

I already have been working on a separate libmsn-pecan, that can be used by Telepathy, and Pidgin. But now there will be the extra option of using libmsn-2′s code instead of libmsn-pecan, but the plugins wouldn’t notice any difference, hence, libmsn-2′s code can be shared among Kopete, Pidgin, and Empathy.

There’s plenty of work to do before this can be sueful, but I already started it:

Who wants to join?