Felipe Contreras

So, it looks like Pidgin developers have a little trouble believing that I’ve
contributed substantially to the msn prpl, so I decided to prove it once and
for all.

Read the rest of this entry »

The bug we are talking about is the infamous switchboard timeout error which was very elusive, it happened randomly, and very often for some users in unknown conditions. Essentially you send a message, and after one minute you receive a notification telling you the message never arrived, after which you need to resend the message, and hope it will arrive this time.

Let’s see how the two projects approached this bug.

Pidgin

There have probably been many bug reports regarding this issue, but it’s very difficult to find old historic bugs in Pidgin’s new and old tracker. The modern version is reported in Pidgin’s tracker as #3330. There you can see people saying it happens a lot, that the priority should be increased, and many tickets were marked as duplicate. Developers however stayed in denial mode: they say it doesn’t happen to them, and then turn to the usual strategy: ask for irrelevant information such as a valgrind log, and to try again as it might have been magically fixed.

Then they try a simplistic workaround; re-send the message on failure. This doesn’t work on most cases, and even when it does, the message arrives more than 1 minute late. As usual, no developer did much else about the bug.

In the mean-time, Adium had many reports (2475, 2395, 6316, 6708, 6952, 7288, 9978, 11045, 11398, 11478) of the same bug. At this point something was very clear: it happens more often in OS X.

The interesting point is when Rasmus Hummelmose, an Adium user, logged to IRC to rant about the problem. He received the same response on both #adium and #pidgin; it’s a server problem, or it’s your slow connection, there’s nothing we can do. That didn’t convince him (it wasn’t true) and he effusively tried to explain that the issue was real and was affecting many users. He didn’t achieve anything more than upsetting the developers.

This is not the way to solve an important bug.

msn-pecan

The msn-pecan team on the other hand thought: hey, there’s a bug, and this guy can reproduce it, let’s fix it. I invited him to #msn-pecan. Rasmus was a bit reluctant; Why loose time with msn-pecan developers? Surely Pidgin developers must be capable enough to do the job. He changed his mind when I explained that the core parts of libpurple’s (Pidgin) msn were either developed or refined by me anyway, and therefore, Pidgin devs probably didn’t have the expertise required to identify this problem.

With that we started an endeavour to fix the problem through the weekend. I started by providing some infrastructure changes in order to visualize what was actually happening, Devid Antonio Filoni created Adium builds, and Rasmus tested, and provided feedback. We made some conjectures and discarded them with further testing and fixed some bugs along the way until we found a reliable way to reproduce: send a message, wait for 15 minutes of inactivity, and send another message.

After this is was clear that something bogus was happening with the network connection, but since we cannot fix all the elements involved, we implemented a simple fix: close the connection after 1 minute. That worked perfectly. Rasmus was happy, and we were too

That’s how you do it.

The stealing

Logically after our success, Rasmus decided to rub the fix on the face of Pidgin and Adium developers, after all, they were the ones that said it was not a bug. But they were not impressed.

However, Daniel Ljungborg (aka Dimmuxx) was interested in the fix, and in good faith I pointed the commit message that explains the issue in detail.

Then I find out Ka-Hing Cheung, a Pidgin developer, implemented the fix as I described, but thought it was OK to not thank anybody, explain where the fix came from, or mention the msn-pecan project, or any external source at all. We (Rasmus, Devid and I) spent a weekend of our free time working hard to identify, fix, and verify the issue, and if you read the commit message you would think they came out with the solution:

Author: khc@pidgin.im
Timeout switchboard connections at 60 seconds, should Fixes #3330 for most people.

That is plagiarism, pure and simple, and unfortunately, it’s not the first time.

The next msn-pecan release started as 0.0.20 but there are so many changes that
it’s going to be 0.1.0. It is way more stable than 0.0.19 but we still would
like to do more extensive testing, so we are rolling a release candidate in
order to fix critical bugs that might be lingering. Hopefully it will be the
only release candidate before the actual release.

The aim of 0.1.0 is going to be our “first serious release”, that doesn’t mean
the previous releases were bad, it just means that we were never truly
confident about the code being delivered until now.

Compared to 0.0.19:

• Timeout issues fixed (switchboard error)
• Better offline messages receiving support
• Offline message sending support
• Reorganization of P2P code (less crashes)
• Several crash fixes
• Adium improvements
• Performance improvements
• Massive code reorganization

Special thanks for Devid Antonio Filoni, and Andrea Piccinelli who have been
very active fixing issues and making sure msn-pecan is rock-solid. Also Rasmus
Hummelmose who was essential in fixing the timeout issues, it wouldn’t have
been possible without his testing. Also thanks to the Pidgin developers (we
picked some patches), and many other contributors.

This is the list of issues fixed so far:

• 37: Pidgin leaves handle on files after transfers
• 82: Implement sending of offline messages
• 117: Received offline messages are being cut
• 138: Translation is not whole integrated from Launchpad
• 144: Unable to chat after message timed out
• 155: Pidgin crashes after connecting (using NTLM Authorization Proxy Server)
• 156: msn-pecan crash in msg_ack() at cvr/slplink.c:321
• 157: msn-pecan crash in msn_switchboard_can_send() at switchboard.c:779
• 158: msn-pecan crash in msn_switchboard_free() at switchboard.c:262
• 159: Pidgin crash when connecting to MSN
• 161: 0.0.19 ubuntu package
• 163: Translations not working on win32
• 164: msn-pecan crash in pecan_contact_get_personal_message() at ab/pecan_contact.c:616
• 170: Crash upon sign in
• 171: crash when disabling account
• 174: Windows 7 RC and Pecan
• 177: Offline messages of blocked contacts should not be displayed
• 181: Too many timeout messages
• 183: msn-pecan should use audio:// links with pidgin 2.6.0
• 184: already showed OIM message show again using another client
• 185: Add support for receiving winks
• 133: pidgin crashed with SIGSEGV in msn_message_destroy()
• 154: Pidgin Randomly Crashes
• 166: proxy authorization support missing
• 153: User Adding Problems

The diffstat is huge:

44 files changed, 3423 insertions(+), 3116 deletions(-)

For the source tarball, win32 installer and maemo package check the usual location:
http://code.google.com/p/msn-pecan/downloads/list

And the Adium build is here:
http://code.google.com/p/msn-pecan/wiki/AdiumBuilds

So, start the testing! And please report back any issues

Here is the current list of pending issues for 0.1.0 final:
http://code.google.com/p/msn-pecan/issues/list?q=label%3Amilestone-0.1.0

Finally here’s the shortlog:

     6  Andrea Piccinelli
     1  Chris Stafford
     1  David Geary
    29  Devid Antonio Filoni
     1  Devid Filoni
     4  Elliott Sales de Andrade
   214  Felipe Contreras
     2  Mike Ruprecht

Thanks to Devid, we now have a build of Adium with msn-pecan, see the announcement at cocoaforge.

Enjoy

This is mostly a bugfix release that should make msn-pecan much more stable. Bug also brings some nice features such as “now playing” support.

Everyone is encouraged to update.

I’ve been busy with other projects so I have not been able to work as
much as I would have wanted, but all in all, this is a good release.

The features (thanks to Devid) include support for “now
playing” (ported from libpurple), and a new way to be notified when a
contact has deleted you (you’ll see on the conversation window). Also,
some issues in Adium were fixed, so we will probably roll an Adium
build with msn-pecan enabled.

I’ve updated my system so it might take a while before the win32
binaries are uploaded, anyone willing to take this one?

Download from the usual place at googe code.

Bugs fixed

• 99 Unable to retrieve display pictures when invisible or logged as invisible
• 109 Protocol priority
• 120 Aliases not cleared properly
• 133 pidgin crashed with SIGSEGV in msn_message_destroy()
• 154 Pidgin Randomly Crashes
• 135 russian translation doesn’t work

Enhancements

• 140 Support for “Now Playing” status field
• 129 Can’t verify installation, or uninstall
• 124 Notify when deleted by a contact

Taks

• 121 Make dbg package for the launchpad repo

Diffstat

105 files changed, 1295 insertions(+), 749 deletions(-)

Contributors to this release

Albert Cervin (1):
     po: update Swedish translation

Devid Antonio Filoni (14):
     Fix use_userdisplay option using Adium.
     Change PACKAGE to GETTEXT_PACKAGE.
     Translate msn-pecan also using Adium.
     Fix use server alias option
     Move utils/msgplustags.{c,h} to pecan_util.{c,h}.
     Update COPYRIGHT.
     Add PURPLE_STATUS_TUNE support and parse contacts's CurrentMedia.
     Update MsnClientCaps in msn.h.
     Add "Has Space" to get_info.
     Show "You are not in A's contact list" message opening a conv
window if needed.
     Set the passport also to session->user.
     ab/pecan_contact.c: cleanup.
     slp: destroy object at the right time
     po: update Italian translation

Elias Julkunen (1):
     po: update Finnish translation

Felipe Contreras (44):
     Cleanup login steps.
     Show up 'connecting' state.
     session: Cleanups.
     Trivial cleanups.
     Cleanup configuration stuff.
     session: Random cleanups.
     Remove bad Russian translation.
     cmdproc: trivial cleanup
     slpcall: avoid extra error cb call
     status: trivial cleanups
     status: improve "available" state mapping
     Trivial cleanups
     userdisplay: report failure
     Log general server notifications
     Makefile: add debugging symbols by default
     session: destroy ud manager after freeing slplinks
     contact: store pecan status instead of pidgin one
     Reorganize how bad statuses are handled
     notification: store user state
     contaclist: add function foreach_contact
     Get user displays when returning from 'hidden' state
     status: trivial cleanup
     contact: use "invisible" for the HIDDEN status
     Use pidgin mobile status only when offline
     Update copyright notices
     command: trivial cleanups
     command: remove unused refcount
     command: general cleanups
     contact: enable workaround for server aliases
     fix_purple: trivial cleanups
     fix_purple: rename purple_buddy_set functions
     Reorganize switchboard handling.
     Close the http node when not needed.
     Make sure the switchboard is not freed when it's still used by slplink.
     transaction: trivial cleanup
     transaction: add ref/unref
     Properly ref/unref transactions
     transaction: fix initial ref_count
     sb: set error properly
     Fix wrong memory access
     Optimize GType retrieval
     Fix SLP DATA message creation
     po: general translation updates
     Bump version to 0.0.19

Geoffrey Antos (1):
     slplink: fix buffer overflow vulnerability

Luís Neto (1):
     po: update Portuguese translation

Wei Hsiang Hung (1):
     po: update Traditional Chinese translation

Well, it seems 100k views is a good milestone to talk about some stats of my blog, so here they are. I’m also trying to understand what are the sources of the traffic and how you might increase your viewers too.

Blog grow graph

General stats

• Total views: 102,436
• Busiest day: 1,903 — Monday, January 12, 2009
• Views today: 112
• Posts: 110
• Comments: 458
• Spam: 21,913 comments

As you can see I don’t post too often, but somehow I still manage to get some views. The day I got most views was the day the MSN servers did some strange update that made all new clients stop working. The new clients use MSNP13 and newer, while msn-pecan uses MSNP12. So everybody rushed to download the only Pidgin plug-in that worked. The number of downloads increased to 10,000 and many of those viewers ended up in my blog. Sadly, it lasted only one day.

But where does the normal traffic comes from?

Top 10 referring sites

• code.google.com/p/msn-pecan: 3,389
• maemo.org: 3,037
• maemo.org/news/planet-maemo: 974
• beagleboard.org: 694
• dzone.com/rsslinks/ruby_vs_python_on_web20_twitter.html: 677
• maemo.org/news: 578
• google.com/reader/view: 310
• fconfig.wordpress.com/2006/08/17/setting-up-a-fedora-nfs-server/: 276
• dzone.com/links/ruby_vs_python_on_web20_twitter.html: 245
• pplware.com/2008/03/29/pidgin-ainda-melhor/: 205

So, important planets like maemo.org and beablegoard.org constitute a big chunk of the visits. Maemo.org, specially, has a good voting system so good stories go to the top. Unfortunately I don’t blog so often about maemo-related stuff, and I try to not spam topic-centric planets.

Somehow that list seems not enough to fill 100k. Could be that Google is helping here?

Top 10 Google searches

• penn and teller bullshit episodes: 2,176
• tortillas: 1,375
• fedora 8 network install: 807
• gstreamer tutorial: 800
• fedora: 785
• fedora 7: 742
• bullshit episodes: 696
• pidgin msn: 594
• classic torrents: 590
• msn-pecan: 588
• public domain torrents: 527
• penn teller bullshit episodes: 527

Still some stuff seems to be missing. My guess? RSS feed subscribers. At least 66 people are subscribed to my blog through Google Reader. So, thanks to all the people that follow my blog. If you have any recommendation on how to improve it, please share it

Also, special thanks to WordPress for making this blog possible, and always providing great stats to help me figure out what to blog about next.

Finally, here is the list of top 20 posts:

Top 20

• Free TV episodes: Penn & Teller: Bullshit! (fun and enlightening): 10,681
• Fedora 8 network install from USB: 6,440
• Public Domain Torrents: Classic Movies and B-Movies For FREE!: 5,859
• Fedora 7 install from usb: 4,220
• Alternative MSN plugin for Pidgin: 4,011
• Get to know a little bit more about Mexican culture: 3,598
• Automounting a storage device with GNOME: 3,126
• Gaim/Pidgin rants, MSN history: 2,985
• Ruby vs Python on Web2.0: Twitter: 2,603
• NFS in Fedora 7 (and iptables): 2,363
• GStreamer hello world: 2,098
• Fedora 7 test 3 struggles: 2,031
• GStreamer hands-on introduction: 2,016
• gst-openmax demo on the beagleboard: 1,892
• Ruby on Rails on Fedora Core 6: 1,818
• msn-pecan 0.0.14 released: 1,669
• Transcoding for the Internet Tablets the smart way: 1,475
• Ogg Vorbis and Maemo 5; technical standpoint: 1,446
• msn-pecan 0.0.12 released: 1,399
• How to update to Fedora 7: 1,381

Hello!

It’s time for msn-pecan 0.0.18. There are many bugfixes and important new features like voice clips support.

Again, Devid did most of the work. The most important fix is that personal status messages now work properly (can be enabled and disabled), the most important feature is voice clip support; thanks to Devid Filoni for the port, original patch from Chris Stafford, and Youness Alaoui for libsiren.

Also important is a fix for WLM 2009 user displays (Thomas Gibson-Robinson), P4-context support for groups.im (Thiago Silva and Devid), a new option to hide Plus! tags (Devid) and support for Plus! sounds (Devid and Thiago).

Also, some cleanups and reorganization heading towards a standalone library.

An interesting event happened on January 12; for some reason the MSN servers changed something that prevented many clients (>MSNP13) from working, while msn-pecan kept working just fine. That day the number of downloads increased dramatically (100010,000 a day), but unfortunately it lasted only a day.

12,552 page views.

New translations: Turkish, Portuguese (standard and Brazil) and Arabic. If your favorite language is not translated you can help at Launchpad’s site.

Here’s the diffstat:

Download from the usual place at googe code.

Thanks to Nicolas Deschildre, the Pidgin brainstorm is now back to life. He provided the infraestructure and ideatorrent setup. There’s some work needed to have a fancier theme, but for now it should work.

The next step is to spread the word so as many Pidgin users as possible hear about this, since it’s not officially supported by Pidgin developers (won’t be on the Pidgin site), although they are OK with it in “unofficial” form.

I strongly believe an active brainstorm would be very useful for the Pidgin community. We can for example link trac tickets to brainstorm ideas, in order to make them more visible so they are not forgotten to oblivion. Or we could suggest a pack of ideas for a Google SoC student. This could open the doors to many things.

So let’s organize and spread the word! Digg it, slashdot it, or whatever

http://pidgin.ideatorrent.org/

Recently Casey Ho emerged from the Pidgin community doing many interesting things trying to gather feedback from the users, but what really got me interested was the new brainstorm based on Ubuntu’s brainstorm.

In a very short time it received hundreds of votes and very good feedback on Casey Ho’s blog. Apparently it was too convenient, and Pidgin devs didn’t like that.

Just like in a totalitarian rule, the brainstorm was quickly shut-down regardless of what users thought. Luke Schierer explained the reasons, which can be narrowed down to: developers whined about yet another place to receive feedback.

So, when users organize that doesn’t help to build a community, but a “clamoring mob of users”. Instead, feedback should be received in the usual way, which is slow, difficult and very restrictive; exactly how Pidgin devs like to work.

It looks like a rational response, except when you think about it for two seconds and ask yourself: what is the difference between a user and a dev? The official blessing of the Pidgin’s junta? X number of blessed patches?

The truth is that in functional open source communities, the line between devs and users is blurry; community members are both devs and users at the same time.

Let’s take an example:

About 4 years ago, voice and video was the most requested feature, but there was no way users could convince developers of that and it was clearly stated by the developers that voice and video would never be implemented, because Pidgin (at that time Gaim) was an instant messaging client, and just that.

However, some users did not accept that and started a friendly fork called gaim-vv which gained some support from the the core developers, but eventually was killed by some decisions by the main developer. Ubuntu Brainstorm made obvious that vv was the most requested feature by far, and eventually a proposal came to Google SoC, that’s how Maiku became a core developer.

It’s quite easy; users vote for features, some features are so relevant that some users decide to implement them, and if they like the development process they keep submitting patches until they are trusted and get commit access. Google SoC facilitates the process for some users.

But Pidgin devs are somehow unable to see that; what users want is irrelevant, only what devs want is relevant; therefore the brainstorm was a bad idea. And they still claim that user feedback is welcome.

I added a track trac ticket to add back the brainstorm section. You can vote there to fight back

BTW. If they are so open to user feedback why all their blogs have comments closed except Casey Ho’s?

Update

The ticket was changed from “Add a brainstorm section” to “Make trac provide various features like Brainstorm”, carrying out the 2 votes it already had, which is like to change “I want a cheetah” to “I want a falcon with features like a cheetah”, no, I want a “brainstorm section”; don’t put words on my mouth.

In a brainstorm, subjects can’t be changed precisely because of this reason, so the first thing to change in trac would be that, which doesn’t really make sense.

Ok,

This is a response to my previous post, since Tiago (libmsn’s author) replied and I think I should explain in more detail.

libmsn-api plan

I think there should be a libmsn-api, in C, which is basically a bunch of .h files. Several things could implement this API, for example pymsn, which is written in Python, but with C bindings that implement this API it could be used by both telepathy-msn and pidgin-msn effortlessly. Of course, there’s already telepathy-butterfly, but there’s nothing that allows libpurple to use pymsn.

In a similar way libmsn could implement the exact same API. The only problem is that libmsn AFAIK doesn’t handle the connection stuff (sockets, proxy, ssl), some something else would need to handle that. In order to write a telepathy CM effortlessly in C, somebody should handle this connection stuff. That would be libim-util, which could be shared by libmsn-pecan and libmsn, and actually other protocols libraries could use the same library, like Skype.

One option which I’m depicting in the diagram, is to wrap libmsn and libim-util together and provide this connection functionality through the libmsn-api. This connection stuff is not required by libpurple since it already provides this connections stuff, however, it’s far from ideal, so it would be nice to have the option to use libim-util for that. This approach also has the advantage of making it easy to write standalone applications, like bots

The other option would be to use libim-util directly from telepathy-msn, and remove that connection stuff from the libmsn-api.

Keep in mind that this is not just theoretical stuff, I’ve actually implemented simplified versions of many of these pieces, including the pymsn’s c wrapper.

The purpose is to give more options to the users, more users to the msn backends, and avoid the current situation where each msn application has it’s own msn implementation.