RSS Feed Subscribe via RSS 2.0 or Atom!

Planet Pidgin

Building libpurple3 on OSX with homebrew

August 26, 2016 05:01 AM by Gary Kramlich

I've spent a far amount of time this week working on making it easier (or even possible in some cases) to build libpurple3 on OSX.  I haven't gotten to Gtk+ yet, so I haven't even tried compiling Pidgin yet.

Typically OSX isn't one of first party supported platforms, but I'm on vacation this week and only have a MacBook at my immediate disposal.  So here we are ;)

Most of the issues have been in the generation of the configure script and landed in PR #103.  But there was a side effect from an earlier PR that became a problem on OSX since homebrew does not have farstream.  PR #107 has been submitted to fix that.

When PR #107 is merged everything should be buildable.  But it does not and will not work out of the box.  Why you ask? Politics!  (of course...)

Homebrew attempts to play things very safe, which is admirable, but a giant pain when you're actually trying to compile something that isn't already in Homebrew.  So we go about making this easier by using a little known feature of Pidgin's build system.

Years ago I got tired of trying to remember what arguments I was passing to autogen.sh and configure.  So like any programmer, I added code that would do it for me!  That code sources a shell script named autogen.args from the directory you invoke autogen.sh from.  So in the root of my build directory I have a file named autogen.args with the following content:

export PKG_CONFIG_PATH=$(brew --prefix libffi)/lib/pkgconfig:$(brew --prefix libxml2)/lib/pkgconfig
CONFIGURE_FLAGS="--disable-gtkui --disable-consoleui --disable-vv --disable-kwallet --disable-meanwhile --disable-avahi --disable-dbus --disable-gnome-keyring --enable-introspection"
As you can see, I'm disabling a ton of stuff to make this build work, but that's not all.  Notice the PKG_CONFIG_PATH environment variable being exported.  This is dealing with Homebrew's policy of staying out of the system's way.  OSX has it's own version of libffi and libxml2 so Homebrew will not install any part of those packages where the system will look for them.  This has the unfortunate side effect of causing weird build failures until you realize this is the problem.

At any rate, I hope this has been helpful to someone :)

Junk security reporting for cash — and credit?

June 25, 2015 03:14 PM by Ethan Blanton

A few times a year, Pidgin gets a rash of junk security reports from “independent security researchers”. Typically we get 3–5 or more reports in the span of just a couple of days, reporting “vulnerabilities” such as HTTP POST submission of plain-text passwords to TLS-encrypted GNU Mailman subscription preferences login. (As many of you will know, by default Mailman will email your password to you every month. In addition, password retrieval is as simple as entering a subscribed email address and waiting for the password to arrive in that address’s inbox.) The report typically also asks for consideration in the form of a bounty or finder’s fee.

The typical defining characteristics of such an email are:

  1. The security flaw being reported is trivial-to-irrelevant.
  2. The mechanism by which the flaw operates is often protocol- or configuration-related, such as the presence or absence of certain HTTP headers or the method used for a form submission.
  3. The report does not indicate that the “researcher” understands the flaw or even personally verified its existence; often one gets the feeling from this and the previous point that it was identified by an automated tool.
  4. The reporter shows no understanding of the entity being contacted, often (for example) referring to the Pidgin project as a “company” or suggesting that it is a large and wealthy organization.
  5. The reporter asks for some sort of reward.
  6. All of the reports in a given burst come from the same general locale, often India.

The first three points on this list make the reports seldom useful and somewhat of a nuisance. The third point, in particular, means that when the reporter receives a polite reply to the effect of “we know, it’s not a problem and here’s why,” they often persist in repeating textbook explanations of the flaw with no reference to the effective implications of the problem (or lack thereof). Sometimes this degenerates into threats of exploitation or disclosure to the highest bidder, though not often.

The fourth and fifth points conflict somewhat with the sixth point in trying to understand why this happens and attempting to address the problem. Points 4 & 5 suggest that the reporter’s primary objective is financial gain, and coupled with 1–3 that they recently found out about this issue (or issues like it) and saw an opportunity to make a quick buck off large organizations with mediocre security skills. This would lead me to believe that the rashes of back-to-back reports are prompted by, perhaps, media coverage of bug bounties or reporting of a new-and-interesting configuration error in large numbers of sites. Unfortunately, the error or flaw being reported is often not new or interesting.

I believe that the sixth point is most instructive, and I think there’s something the community can do about it. I suspect that these rashes of reports are in fact course assignments for some sort of course on computer security being taught at an academic institution (and probably various courses at various institutions). I suspect that students are instructed to learn about some common vulnerabilities, find an example on the web, and contact the site administrators with their findings.

I cannot object to the general principle behind such an assignment. However, I can object to the way it is being specifically handled. If my suspicions are correct, then I believe that it is the responsibility of the course instructor to vet the students’ individual reports for relevance and correctness before sending them on to the affected organizations. I would also leave out the bounty-seeking in an educational setting, but I recognize that requesting a bounty for valid flaws is an accepted practice in the commercial world, and that identifying valid flaws is a valuable service.

An alternate, but related, possibility is that some security course is discussing vulnerabilities and disclosures, and students are taking it upon themselves to contact various organizations with flaws they found using automated tools to try to make some cash on the side. I find this less likely given the very short time frames in which such reporting bursts occur (often just a few days).

In either of these education-related cases, there’s something that can be done about the problem. If you are an instructor teaching such a course, please emphasize responsible reporting, which includes understanding both the mechanism and the impact of the flaw, and determining whether it is likely to be relevant to the affected organization. In addition, as mentioned above, screening reports before they are sent out would be the neighborly thing to do.

In the case of an organization like Pidgin, we take vulnerability reports seriously and expend some effort validating their existence and/or relevance. As we are an all-volunteer organization with a limited amount of time at our disposal, dealing with bogus and junk reports waste valuable resources. Report responsibly!

More Windows, more features

May 13, 2014 04:29 PM by Tomasz Wasilczyk

With a great help from LRN, who sent initial set of patches for fixing autotools-based build for win32 and assisted in the work, I have finally managed to make 3.0.0 cross compilation possible and easy. It also involved fixing minor win32 problems and enabling features that were not accessible for this platform before.

The most important change is making linux-to-windows cross-compilation almost as easy as normal build. The whole effort (not counting installation of mingw* packages) is reduced to setting two environmental variables (PKG_CONFIG and PKG_CONFIG_PATH) and adding a single switch to the configure script: ./configure --host=i686-w64-mingw32 && make. That’s all! For now, there is no option to cross-build the installer yet.

Finch working on Windows On the other hand, the most end-user-attractive change is a Finch win32 build. It required both libgnt and Finch fixes, which made those two quite usable on this system.

Moreover, I finally implemented all remaining features required for using OTR plugin with Finch. There is still one missing – you cannot browse key list yet, but it’s not crucial for a daily use. This change is not related to Windows and there may be minor problems when running the new OTR plugin on win32, but I will face all of them by the chance of integrating it with Pidgin tree.

An example of less significant, but still useful Windows related change is GTK3/gstreamer-1.0 compatibility, which was an easy to achieve with the fixed autotools build. I also removed the Bonjour SDK dependency from the win32 build, as there were license issues with it – for now, you can build Bonjour prpl without it.

The last feature may be found useless by some of you, while some might like it – that’s why it’s optional. You can enable Filesystem Hierarchy Standard directory structure with a single configure switch: --with-win32-dirs=fhs. For now, it’s the easiest way to prepare a working Windows build, since there is no cross-built installer yet.

Few steps towards a stable release

Pidgin 3 is not API/ABI compatible with Pidgin 2, and is only partially configuration-compatible. While the first incompatibility is necessary to move forward, the second might be really frustrating for users. Because of ABI incompatibility, libpurple2 plugins won’t work with libpurple3 – their authors will eventually convert them for the new version. Configuration incompatibility may lead to loosing your data – preferences, contacts etc.

Pidgin 3 was almost backward-compatible, which allowed to switch to it flawlessly. On the other hand, Pidgin 2 was not forward-compatible. For example, it dropped all encrypted passwords, that were set with Pidgin 3. Now, it won’t be able to read those (since keyring support is not implemented for 2.x.y branch), but at least it leaves them in place. I also fixed more forward-incompatibilities: handling for GTalk and Facebook XMPP accounts created with 3.0.0 version and internationalization issues related to default group (“Buddies”) naming. All of these will be released in 2.x.y branch, so you will be able to switch from 3.0.0 to 2.10.10 and back without loosing your data.

There were also minor, but annoying issues fixed. Nick colors for chat participants in XMPP MUC or irc should not suffer from a low-contrast issue. I have finally made spell checking usable, by implementing a language selection sub menu for WebKitGTK. It still has some flaws, that I will work on some day: the biggest one is that the selected language is global, not per-conversation.

To make development branch more stable, I decided to focus on Coverity bug reports. Since we are allowed to maintain just one branch at once, I decided to fix all 2.x.y reports before switching to 3.0.0. For now, I fixed almost all of them (and merged fixes to the 3.0.0 branch), so it should be a bit more stable. I also updated all win32 dependencies, which should also improve stability.

Don’t get me wrong, there is still a long way to the stable Pidgin 3.0.0. But it’s already usable right now.

Show me your desktop

An example usage of Screen Capture plugin I have switched from Konnekt (a local Polish instant messenger) to Pidgin around the year 2007. It provided a feature absent in Pidgin, that I missed very much: an easy way to send screenshots. There were a plugin for Pidgin 2.x.y, but I didn’t liked it and it wasn’t working with Pidgin 3.

Ultimately, I decided to invest two days of my life and create my own plugin. It just works, covering all attributes I liked in other instant messengers with such feature. It’s simple, stable and fits into the Pidgin UI well.

I plan to extend its functionality with another plugin. For now, it’s only possible to send screenshots over protocols with inline images support. Thus, you can not use it with XMPP or IRC. The second plugin will allow for uploading images to imgur (or similar services) for protocols that doesn’t support images.

Smile!

April 21, 2014 03:37 PM by Tomasz Wasilczyk

I have put a lot of effort into a feature that I don’t even use: graphical emoticons. As usual, the small issue resulted in a large code refactoring. I had the task of fixing regression bugs from Pidgin 2.x.y, so I took care of broken remote smileys. It’s a pleasant feature, that allows defining the list of custom smileys to use in outgoing messages. If we send one in our message and the remote client doesn’t have it, the image will be automatically transferred. For protocols not supporting this feature, the bare textual representation will be shown instead of the picture.

The code responsible for emoticon handling was horrid. In 2.x.y branch it was already quite messy, but became even worse after replacing GtkIMHtml with WebKitGTK for displaying a conversation. The problem was, the old parsing code was optimised to be used with GtkIMHtml. It was totally incompatible with WebKitGTK, so the GSoC student who did a conversion made his own provisional smiley parser. It just added insult to injury.

Inline images

Both GtkIMHtml and WebKitGTK use purple’s “stored images” to provide data for emoticon smiley. It’s a generic container to hold raw image file contents, without any decoding. Its main purpose was pretty simple: to reference this data by a single integer number. Thus, an image may be embedded in HTML with <img src="purple-image:5">. It’s quite convenient, but the API was a bit messy. Not that bad, but we had plans to convert it to a GObject. Instead of duplicating the exact behavior of the old implementation, I took the usual way: I wrote entirely new PurpleImage class, much richer, but also simpler to use.

A conversation with remote smiley and remote inline image used

PurpleImages, aside from the old simple referencing feature, have several new ones. Remote images support is the one which improves end-user experience the most. It allows defining an empty image and providing a data for it later. Now, if we receive a message with an inline image embedded (not necessarily a smiley), it will be displayed without it and the image will be shown when ready – just like its done in web browsers. It heavily impacts protocol plugins: with the old API, they had to queue messages and wait for images being transferred. Now, it’s fully handled by libpurple. By the chance of deploying it, I fixed inline images support in every protocol that had it.

Tries

As I mentioned before, the smiley parser for 3.0.0 branch was provisional and the old one was not suitable for WebKitGTK. GtkIMHtml (a Pidgin 2.x.y component, despite its Gtk namespace) exposed its internal parsing mechanism and allowed to process every literal in parsed HTML. Thereby, a libpurple routine was called on every word and replaced them into an image, if hit the emoticon. The lookup was quite fast, because of trie-like implementation of GtkSmileyTree.

An example of trie

A Trie is a tree structure for holding multiple strings and searching them by prefixes. The primary idea of trie is that two strings with the common prefix will share the part of tree for the common segment and branch out the remaining parts. It’s only a fundamental part of structure, which may be used for completely different purposes. In fact, it was used both in the old GtkSmileyTree and my new smiley parser implementation. Except for this single similarity, these two are barely related.

Instead of writing fat-but-fast smiley parser, I decided to implement independent PurpleTrie class – a trie-based text search algorithm. It allows for searching multiple strings in multiple source texts in a linear time! To be precise, it needs O(m) time for building a trie (where m is the total length of provided patterns) and O(n) time for searching (where n is the length of source text). The frequency of patterns in a text doesn’t affect this value. The best thing in that structure is, it may also be used in libpurple plugins.

For a contrast, the interim smiley parser implementation was not focused on performance at all. Its search time could be estimated as O(n * m * t), where n is the number of smileys, m is their length (assuming smileys are equal-length – in the real-world case the formula would be more complicated) and t is the text length. It’s that bad, mainly because every supported (not necessarily inserted into a text) smiley is parsed separately. The m * t part depends on strstr implementation, but for the better ones we could find another similarly bad instance of smileys configuration. With such bad complexity, it could even be exploited for a denial-of-service attack.

What’s next?

In 3.0.0 branch there were a lot of smiley- and image-related issues and I haven’t described all of them. But I hope I fixed all of them. You can look for details in our hg, in smiley-related commits between 302a7cb4c1ab and d598e7950c34. Now, I am focused on the permanent issue – the Windows version.

In Support of Instant Messaging Communications Freedom

February 16, 2014 02:40 AM by Ethan Blanton

As recent news events have driven home time after time, secure communications is a difficult yet important aspect of modern life. What “secure” may mean differs from person to person and from topic to topic, but certainly the typical person is somewhat uncomfortable with the idea that anyone — and in particular, large corporations or the government — might be eavesdropping on their casual communications. There are exceptions, and there are people who believe that only criminals should have something to hide, but for the moment let's assume that some measure of secrecy in private communications is warranted and/or desirable. This post is a commentary on some of the technologies that make it possible (or difficult or impossible, in many cases) to achieve secure communications on an instant messaging (IM) network.

I'm going to play a little bit fast and loose with terminology to help keep it understandable, but I also want it to be accurate. In particular, when I say that communication is secure, I mean private, specifically. For the purposes of this post, that effectively means encrypted and using some sort of authentication protocol to ensure that it's encrypted to the right person. The details are out of scope. Please contact me if you notice any particular discrepancies or incorrect statements, either arising from loose terminology or other errors.

First, where I'm coming from: I believe that essentially all communications should be secured, and I believe that that security should be very strong. By “very strong,“ I mean that it should be effectively impossible for any third party to eavesdrop without the acquiescence of a party participating in the communication. I have a fair amount of history in computer communications. I am a long time developer on the Pidgin instant messaging client and related software. I am on the board of directors for Instant Messaging Freedom, Inc., a non-profit organization “whose goal is to support free instant messaging software.” Instant messaging is important to me as a communication technology filling the gap between real-time spoken communication and e-mail.

Types of IM security

There are a number of ways in which an IM conversation can be "secured", and not all of them have the same properties. First, there is using a secure connection, such as SSL or TLS between your computer and your IM service's computer. Second, there is end-to-end security, such as Off-the-Record Messaging (OTR). Then there are additional, more complicated options that we won't discuss — but they basically break down to a combination of one or the other of the above, or one of the above with a different remote endpoint. (\textit{E.g.,} end-to-end security between you and a remote user's IM server to query her current status.)

Secure IM connections

A secure connection to the server protects your connection from eavesdropping on the local network and the path between you and your IM service (provided that it's done correctly), but it does not protect your conversation from the IM service itself and it does not tell you anything about whether the user you're chatting with is using a secure connection. Despite these weaknesses, a secure connection to the server is critically important, because it protects a large amount of private information. Things like buddy lists, status updates (online, offline, away, etc.) from your buddies, and your own status updates pass through this connection and are not necessarily targeted to any specific other user, which makes end-to-end encryption difficult.

This is the limit of security offered by most commercial IM services. There are a few exceptions, like AIM's encrypted IM, but often they have a similar effective security model — for example, the only proof that the encrypted channel you're talking on is actually terminated at the remote user (and not the IM service's servers) is a certificate signed by ... the IM service itself. (The details of why this is not sufficient are outside the scope of this post. Maybe I'll write some more on that later.)

End-to-end security

End-to-end security protects your communication from eavesdropping all the way to the other user. If some person or organization wants to read your messages, they have to do so at your computer or your interlocutor's computer. There are a small number of protocols that support this directly, but mostly not in a particularly useful way (see the discussion of AIM encrypted IM above).[1]

The usual solution for end-to-end encryption is a third-party protocol carried on top of the IM session. There are several such protocols, but by far the most popular is the previously-mentioned Off-the-Record. OTR provides end-to-end encryption for two-party conversations with authentication of the remote user and a variety of desirable encryption protocol characteristics. (It also provides repudiability for situations where that may be important.)

What end-to-end security cannot provide is protection for all of the stuff that's intended for a very broad audience.[2] Such data (things like away messages, online status, and buddy lists) is typically [3] handled by the service's servers on behalf of its users, in such a way that necessitates more-or-less trusting the servers with the information. (This is a reason not to put sensitive information in your status message!)

Secure connections with end-to-end security

Given the individual limitations of these two forms of data security, we arrive at the inescapable conclusion that, for instant messaging services, they are complementary rather than redundant or simply unrelated. Secure connections to the servers provide best-effort protection for group chats, administrative information, and metadata, while end-to-end security provides strong protection for conversation content.

The pros and cons of federation

Traditional IM networks are monolithic, walled gardens — if you want to chat with a user on the network, you get an account with the single service provider for that network. There have been limited exceptions to this over the years (e.g., MSN Live Messenger and Yahoo! Messenger offer(ed) some degree of interoperability), but for the most part, not only have networks been incompatible, there have been sole providers within those networks.

This kind of structure means that, while your metadata and administrative information are only ever managed by a single entity (the monolithic service provider), that service provider also sees all of the related metadata etc., and it's necessary some giant faceless corporation. That corporation stores your buddy lists, knows when you talk with whom, knows when you're away or idle, and all kinds of other behavioral information. Moreover, it has you in a lock — if you want to talk to your buddies, you have to use its services.

The alternative to the monolithic single-provider network is a federated service. In a federated service, multiple (possibly unrelated) service providers cooperate in a network tied together by a common protocol, allowing users of different service providers to interact with each other without worrying about whose service is provided by whom.

In a federated structure, you still have to trust your service provider with all of your metadata and administrative information, but you don't necessarily have to trust any of the other service providers in the network. In fact, in the general case, most of them don't even know you exist! Some portion of your data will necessarily be shared with the servers your friends and interlocutors are associated with, but you can scope that sharing to a greater or lesser degree. On the other side of the coin, you also have to place at least a small amount of trust in third-party providers with whom you have no specific relationship, if you want to talk to people on those providers' servers.

The quintessential federated network is XMPP, previously known as Jabber. XMPP is a widely federated network, wherein anyone can run an IM server and become a service provider for other users. Conversations between users work a lot like email; if I want to chat with you, I send a message to my server, it forwards it to your server, which forwards it to you. The return path is the same in reverse. Not only can anyone put up an XMPP server, but the protocol is entirely open and well-documented, so there are literally dozens of server software implementations and thousands of providers already in the network. Those providers range from large, commercial entities (such as Google's Google Talk, now known as Hangouts) to tiny servers serving just one user.

The huge benefit of federation is the freedom to choose your service provider. Moreover, to change that service provider. With an open federated network like XMPP, you can even be your own service provider if you so desire. That's a kind of communications freedom that no monolithic provider can ever provide, by definition.

It is my opinion that the federated structure is a superior solution, security, privacy, and freedom-wise, to old-style monolithic IM networks. The majority of your sensitive data (administrative information, complete buddy list, etc.) is kept and managed by only one entity, and is parceled out to third-party entities only as required to provide the services you specifically request. In the specific case of a closed group of users (such as a corporate or organizational server), it may be contained entirely. I am also a strong advocate of open standards which federated solutions tend to require (to make federation possible), and which XMPP certainly provides. The ability to pick up your data and move it to another service provider with limited (or nonexistent) loss of functionality is an extremely powerful argument for the freedom of a federated solution.

Today's best practices

The foregoing basically points to a simple best practices recommendation for IM freedom and security: use XMPP, find a server you trust, ensure that you're using TLS encryption, and employ an end-to-end security solution like OTR when it matters. (Unfortunately, without complete penetration of end-to-end security solutions, “when it matters” is the best we can do. Even then it can be hard to achieve!) Today, with the availability of a number of large XMPP service providers with federation and open registration (Google, DuckDuckGo, Jabber.org, or dozens of others), as well as many fine XMPP clients (including, of course Pidgin, finch, and Adium), getting an XMPP account and finding your friends is relatively painless. Many of them will already be available if you simply add gmail address as a buddy in your XMPP client.

Secure connections are not yet provided by all XMPP servers. Among servers that do provide secure connections to clients, not all provide secure connections to other servers. (If you recall the email-like communication model of XMPP, this means that your communicatiosn with users on other servers would not be secured between servers, even assuming you trust both servers.) The excellent xmpp.net directory of public XMPP servers provides ratings for client-to-server and server-to-server communications security; look for A-rated servers. If you set up your own server, xmpp.net also provides a security validator that can be used to ensure your personal server is up to snuff.

Improving the situation

Going forward, there are a number of efforts underway to further improve the already rather good connection security situation in the XMPP network. Notably, the XMPP manifesto is (documentation of) an effort to transition the entire federated XMPP network to secured connections by May 19, 2014.

The end-to-end security situation is still a little underdeveloped, in my opinion. OTR is great, but its protocol-independent nature leaves it with a level of integration that isn't as complete as it could be. I have some early-draft notes on desirable features for a new XMPP end-to-end encryption protocol, but a lot of work remains to be done on the topic — and much of it by people with stronger crypto chops than I have.

What you can do now

The takeaway from all of this? Ditch your AIM, MSN Live Messenger, Yahoo! Messenger, or whatever other IM services you're currently using, and get on board with a federated XMPP provider. Follow the recommendations in best practices, above. Use a public server or install your own, but do it sooner rather than later. Make sure you're using TLS (or SSL, if you have to) to protect your connections, and consider installing OTR.

Then, when you're done with that, start bringing your friends over. Talk to them about the benefits of freedom in IM services, describe the insecurity of communication on traditional commercial IM services, simply tell them you're not dealing with a closed IM service any more, or whatever. Point them at this article, if you think it will help. XMPP already has critical mass, it's simply a matter of expanding the borders.

If you have the necessary background, consider contacting me about working on an integrated end-to-end encryption and authentication solution for XMPP. Join the devel@conference.pidgin.im XMPP MUC and indicate your interest.

Footnotes

[1] SILC is an example of a service that provides native secure connections and native end-to-end encryption. Unfortunately, it is no longer a maintained codebase, it is not well-supported by IM clients (though libpurple, and thus Pidgin does support it), and it has a problematic federation model.

[2] “Cannot“ is a bit strong here, but I believe that, given the current state of the art of encryption protocols and mechanisms, it's true enough to use for the moment. Secure multiparty broadcasts with hundreds of recipients would be very expensive using standard techniques. The literature may (and probably will) have some answers to this problem in the future, but for now, I'll say cannot.

[3] I say typically here because I know of no non-local-network service that handles this any other way. Local network messaging (like Apple's Bonjour) has other solutions to this problem. Generally, however, you send your status updates to the server and it distributes them (possibly by way of other servers, in a federated protocol) to interested parties on your behalf.

Off-the-Record Messaging – the true privacy

October 10, 2013 03:09 PM by Tomasz Wasilczyk

Modern instant messengers claims about security and privacy. And users trusts them. But companies that delivers such services sometimes fails protecting their users privacy. The point is, that user doesn’t have any control over confidentiality of his private conversations – the service provider does it.

In most cases, this schema works pretty good: user A sends a private message via an encrypted connection to a service provider and the provider sends it to the user B (also using an encrypted connection). The problem occurs, when the intermediary company reveals such message – due to software failure or dishonest employee.

Encrypting the message by A seems to be the solution, but it creates another set of problems. To make user A able to encrypt a message for user B, he have to obtain a B’s public key. But how to obtain it in a secure way? What, if B loose his key, used to secure a long history of conversations? Also, to make user B sure, that the A sent the message, user A have to sign it. But what, if user A says something humiliating to user B in private, but the second one reveals the message to the public and use it against A? The message is signed by A, so there is a proof he wrote it.

And there comes the Off-the-Record Messaging protocol. It’s goal is to provide truly private conversations over any underlying IM protocol (such as xmpp, ICQ, Gadu-Gadu) in a easy to use manner. It ensures four aspects of privacy:

  • encryption, so nobody other than A or B can read their messages;
  • authentication of the other party, to make sure we are talking with the right person;
  • deniability, because messages does not have digital signatures that are checkable by a third party – the other side of conversation cannot prove, the messages he got are sent by you (but he’s still sure about it);
  • perfect forward secrecy – if someone get your private keys, he won’t be able to decrypt any past conversations.

OTR and Pidgin/libpurple

There is a plugin for Pidgin, implementing OTR messaging – pidgin-otr. Unfortunately, the user have to install it by himself, so it’s not as easy to use as in other messengers. I was asked to integrate it within the official release.

New conversation window with OTR controls

I decided to make this task a bit more challenging. Pidgin-otr is a Pidgin UI client plugin, so it’s not available for other libpurple clients. Some of them (Adium) implements OTR by itself, some (Finch) doesn’t offer such functionality. I decided to rewrite it as a pure libpurple plugin, to provide this excellent feature for all clients that use libpurple as a backend. Fortunately, pidgin-otr is pretty well designed, so I had only to alter things related directly to the UI.

There already was a similar attempt, named purple-otr. Its main problem was really poor UI – its author used pretty limited libpurple’s Request API to create dialogs, so he wasn’t able to clone all pidgin-otr’s functionality. My situation is way more comfortable, because I am a Pidgin developer (having direct impact on its code), working on a 3.0.0 version (which breaks API, so I don’t have to care about compatibility). That means, I was able to extend libpurple’s features to better fit OTR plugin needs. Some of libpurple API changes that made it possible:

  • End-to-End encryption providers API allows to present (implementation independently) conversation’s security state. Previously, pidgin-otr and similiar plugins placed their own controls in various parts of the conversation window, now it’s standardized.
  • Request API refactored with PurpleRequestCommonParameters, which makes this API easily extendable.
  • Using PurpleRequestCommonParameters for new features implementation, like an option to provide HTML decorated text descriptions, an option to alter the dialog icon, to make better control over the dialog buttons.
  • Adding a new window type: cancellable “please wait” dialog, with an optional progress bar.

These changes may not look crucial. In fact, without them the libpurple’s pidgin-otr port would look really poor or even awkward.

First step of buddy authentication

Second step of buddy authentication

At this point, all conversation-related code is ready and working fine, and it’s possible to use it conveniently. I haven’t implemented configuration dialogs yet (It’s the next task for me) nor adapted Finch for the new API (so it’s not possible yet to use it with OTR, but it’s not the hard part).

Unfortunately, there is no possibility to test it at the moment. The libpurple’s part is in its repository, but the new pidgin-otr is developed in its non-public branch in OTR authors repo.

101 HTTP implementations

August 26, 2013 10:47 AM by Tomasz Wasilczyk

Previous libpurple version suffers from poor HTTP implementation. Ordinary user won’t notice that, because plugins tries to fill the hole. However, when every single component that uses HTTP have to deal with the same issues, there must be some mess left.

Nearly a year ago, I’ve decided to put some effort here and implement new, flexible HTTP API. Now, I’ve came even deeper, replacing existing implementations with my new tool.

Replacing old HTTP API with the new one consisted of few stages. Firstly, I’ve replaced all purple_util_fetch_url occurrences with purple_http_get. That was the pretty easy step, because both functions does roughly the same thing: gets the URL and returns its contents. Then, it was the time for the tricky part: purple_util_fetch_url_request required building and parsing HTTP headers manually. Also, it leaves part of proxy handling on the caller in a weird way – he have to pass an full URL in request header, if proxy is on, or short (without hostname) otherwise.

Getting rid of purple_util_fetch_url* routines raised code quality pretty well, but the most difficult task was still undone. Some protocols had its own HTTP implementations – just imagine that amount of copy-paste. I’d like to share some examples:

  • oscar (ICQ, AIM) implemented it on it’s own to just perform simple GET request; the implementation was so horrible, that the author himself named the functions straight_to_hell, damn_you and struct pieceofcrap; fortunately, this code wasn’t used for a long time;
  • mxit: this could be implemented using old HTTP API, I have no idea, why author hadn’t done that;
  • msn: this one was somehow justified – there were no Keep-Alive connections in old API, so the author did it by himself to gain some performance; the bad thing is, he did it twice (for HTTP relays and SOAP handling);
  • xmpp also has two implementations, just not that obviously superfluous: the BOSH implementation was complex and still full of TODO’s; it took advantage of simple Keep-Alive request, but in really obfuscated way; rewriting of second implementation (out-of-band file transfers) was, by contrast, easy and pleasing task (especially, because I’ve done similar one within Gadu-Gadu protocol before);
  • yahoo protocol plugin was the record-holder with the value of four distinct HTTP implementations; I think, only in case of yahoo file transfers it wouldn’t be possible to do it using existing API.

There are still two remaining, but I’m not sure if I will refactor them at all. The first one is for Gadu-Gadu protocol – it’s included in libgadu library, so it’s not trivial to pull it out without violating its API. The second is for Novell GroupWise Instant Messenger – a closed service, available for companies. There are no public test servers, which I could try out. Also installation and configuration of its demo overwhelmed me (in fact, it’s buggy). I’m not sure if it’s worth the effort to do both cases above.

My next task will be Off-the-record plugin refinement and integration into the main tree.

Pidgin with Video finally for Windows

June 08, 2013 05:54 PM by Tomasz Wasilczyk

I just did another long desired feature: Video conferences support for Windows. There already were attempts to do it, but they were not finished nor even published – this one is already in Pidgin’s tree, so it’s a matter of time to get them released (just wait for 3.0.0). If you don’t want to wait, you can always grab a development build.

Video conference with Pidgin on Windows

I did a lot of tweaks both in Pidgin and its dependencies (GStreamer and related), but finally everything looks working and stable. The hard part, unlike in the previous Eion’s attempt, was the camera capture plugins for GStreamer – DirectShow and WinKS. The first one is not buildable in newer gst-plugins-bad releases (at least, for mingw), the second is buggy. I’ve chosen to work with WinKS: the main problem was, it had broken support for different capture resolutions (I guess, old cameras had no support for it, so they were not affected). Many hours of debugging resulted in a simple patch, that makes the resolution fixed. The other one existed in Pidgin itself: there were no possibility to select camera, because winks used different method of device enumeration, than Pidgin supported.

On the occasion of testing VV on Windows, I’ve came up with a simple idea: some users may do not want to show their faces in video conferences, but they could want to see others. So, I implemented a new, virtual device (both for Linux and Windows): Disabled. Depending on user’s choice, it displays black screen or random noise, like on TV. Simple, but useful.

Disabled camera preview

Testing on Windows platform showed some bugs, that were hardly noticeable before: hangs on video testing, displaying a video output in a separate window. Beyond them, I also fixed some memleaks and did other tiny fixes.

The source code is available directly from the hg repository and openSUSE Build Service project (for dependencies). If you just want to test the VV feature on Windows, you can just grab the offline installer.

As always, I’m waiting for any feedback, bug reports and comments.

Four Pidgin Summer of Code students

May 28, 2013 12:52 AM by Mark Doliner

Pidgin was awarded four students for this year’s Google Summer of Code. It was a difficult process to select just four students from the 34 great applications we received. These are the projects we finally chose:

We’re looking forward to seeing what they create! The coding period begins June 17 and ends September 23.

As always, thanks to everyone who applied. And remember, this is an open source project and you’re welcome to contribute even if you’re not participating in Summer of Code.

Password safe support, even for Windows OS

May 16, 2013 03:12 PM by Tomasz Wasilczyk

The Pidgin’s master password branch, which engaged me for the last two months is now ready to show to the public eye. Moreover, it depended on the other security-related task: taking care of Windows build – this one is also mostly done now. Today, I would like to present the current results of my work.

When I was just before start of work on master password branch, it was described as almost ready to merge, or even ready. In fact, it worked fine only for a small part of cases, where user sets up everything correctly and never mistakes while typing in a password for the configured keyring. The funny thing is, that the feature mentioned in the name of the branch, was never implemented (storing passwords internally, encrypted with a master password).

I must admit, that some part of the work done by GSoC’08 student was fine. I would say, it was a good proof-of-concept implementation. I just done the final one and made it close to be ready to ship. Handling all possible use cases (not only the ideal one) required almost twice more code than produced during SoC, also the small part of the code was left as-is (because of its good enough quality). The chart below presents some estimations about it, made only on newly added source files (keyring.c and specific keyring implementations).

Code refactored for master password branch

Doing things right required also a lot of fixes in the main development branch: ciphers reworking, fixing win32 build (there is also a special keyring for this OS), fixing UI components like request API and some minor 3.0.0 bugs. Some of them wasn’t too absorbing (preparing ciphers API for my needs taken just few days), and some were big enough to entitle them as separate task (win32 build taken about 1/4 of the total time spent).

Windows build

The 3.0.0 win32 build was broken for a year or two, so I had to fix it to be able to implement a keyring for this OS. Although, it was a separate security issue, because runtime dependencies, shipped with Windows version of Pidgin, were collected from sources that weren’t eager to update them.

I have managed to switch almost all win32 dependencies to pretty reliable openSUSE Build Service. I guess, some of them (like these related to WebKitGTK) would be hard to obtain from elsewhere than OBS. This service proved high responsibility for their releases, by fixing security issues in few hours after my requests (updating some libraries or applying patches with security fixes). Non security issues also were fixed in reasonable time. I would make a special thanks to Fridrich Strba, who assisted me with preparing OBS repository for Pidgin’s needs. Without his help, doing a pretty stable 3.0.0 build would be twice as hard.

Supported keyrings

Keyring is a safe, which holds passwords for accounts configured in Pidgin (or other libpurple client). There are few of them implemented, so the user may choose the one, which best fits his needs. If none of them are suitable, he could provide his own implementation.

Internal keyring in cleartext mode provides backward compatibility, where passwords are stored without any encryption, just as-is. This doesn’t need to be insecure, as long as the access to accounts.xml file is restricted.

Internal keyring in master password mode also stores them in accounts.xml file, but encrypted. The encryption key is derived from a master password, which have to be entered once per every client startup.

GNOME Keyring and KWallet moves this responsibility to a service started by operating system. Passwords are stored here encrypted using a master password, entered once (depending on configuration) per OS startup.

Secret Service (implemented by libsecret) is another OS keyring. Unfortunately, its plugin serves a lot of issues at the moment, so it’s not enabled yet.

Windows credential manager encrypts passwords using Windows user account data. Its security depends on system configuration – encryption does nothing, if user’s account isn’t even protected with password.

KeePass is a cross-platform keyring, which could be a good choice for people using the same Pidgin configuration files for multiple operating systems. Unfortunately, it doesn’t seems that KeePass could work with Pidgin out-of-the-box – it requires installing a plugin for KeePass to operate with various applications. At this moment, there is no implementation for it.

Encryption scheme for internal keyring

Password encryption implementation for internal keyring is really sensitive spot, so it had to be done very carefully. It’s important not to try doing own, fancy schemes, but to use approved methods. Even then, it’s really easy to screw things up.

Choosing a good blocks to build this feature is crucial, but if they turn out, to be weak, there have to be an option to change them. So, I made a possibility to choose encryption method, but I’ve implemented only one for now. I have chosen two algorithms: AES-256 for encryption and PBKDF2-SHA256 for deriving a key from the master password.

PBKDF2 algorithm have configurable amount of iterations (default is 10000), so user may to raise it, if he needs paranoid level of security, or decrease (but not below 1000) if his machine is too slow for the default value.

It’s important to use AES correctly, to eliminate various classes of potential attacks. I’ve implemented the following scheme:
ciphertext := [IV] ++ AES( [plaintext] ++ [min length padding] ++ [control string] ++ [pkcs7 padding] )
where:

  • ciphertext is base64 encoded, so it’s possible to store it in xml file.
  • IV is a random, 128bit vector, which makes every ciphertext different, even for the equal plaintexts. It’s stored with the AES output, to make it easier to maintain.
  • AES is used with 256bit keys in CBC mode.
  • plaintext is stored without NUL terminator – its length is determined by one of the paddings, depending of its length.
  • min length padding extends plaintext length to be not shorter than 50 characters, by appending NUL bytes to its end. This hides an information about a password being short or long.
  • control string is a constant string – allows verifying, if decryption was successful.
  • pkcs7 padding determines the length of the [plaintext] ++ [min length padding] ++ [control string] sequence.

Builds ready for testing

The source code is available directly from the hg repository, but I’ve prepared some packages for these of you, who just want to take a quick look:

I’m waiting for any feedback, bug reports and comments. I would also appreciate any linguistic corrections – you may review all of them by searching in pidgin.pot file for /keyring.c and /keyrings/ phrases.

Students: Apply to Pidgin Google Summer of Code now!

April 25, 2013 05:31 AM by Mark Doliner

The application period for applying to Google Summer of Code opened on Monday and we’ve already received a number of applications. The application period closes next Friday, May 3rd. Just 8 days left—don’t wait, submit your application soon!

Pidgin in Google Summer of Code 2013

April 09, 2013 07:21 AM by Mark Doliner

Google has accepted Pidgin into Google Summer of Code 2013. Woo-hoo! We’re looking forward to mentoring a few lucky students again this year.

For more information, read Google’s announcement, peruse our application template, and see our list of project ideas. The application period beings April 22nd—just two short weeks away!

We always encourage our users to brainstorm and share your ideas on improvements you would like to see in Pidgin, Finch, and libpurple. Feel free to share thoughts and ask questions on our devel mailing list.

Google donates Pidgin to improve its security

March 26, 2013 06:24 PM by Tomasz Wasilczyk

Some time ago, Google contacted me regarding security concerns related to Pidgin. After a long discussion, they decided to make a donation to Instant Messaging Freedom foundation, which was then able to sponsor some work related to Pidgin’s security improvements.

Therefore, I will periodically publish here news about work being done (of course, containing only the non-sensitive information).

The first task I took care of, was finishing and migrating master password branch. It was a Google Summer of Code project in 2008, but wasn’t finished. When I started working on this, it was pretty unstable, but now it’s slowly becoming polished and useable.

Master password branch provides Keyring support, which solves one of the most complained security issues about Pidgin – storing all passwords in plaintext. There will be an option to store passwords in system-provided safe (like GNOME Keyring, KWallet, or using Windows credentials to encrypt them) or still using the old, plaintext method. Supported keyrings are implemented as plugins, so every interested developer can provide his own method of storing such sensitive data. There will be even an option, to encrypt all passwords with one master password, supplied by user.

Development version of Pidgin with Keyring support can be obtained directly from its repository. Any comments, code reviews or tests would be appreciated.

Pidgin development builds with GG11 support are available now

February 07, 2013 06:04 PM by Tomasz Wasilczyk

I’ve just shared a page containing Pidgin development builds with GG11 support. Probably, we’ll have to wait for official release a little more, so I’ve prepared it to make testing of new features (described on this blog) easier.

Package is installed similarly, as official Pidgin from sources – the only difference is necessity of having auto-tools installed. Moreover, we need to keep in mind, that 3.0.0 branch have different dependencies, than 2.x.y. Unfortunately, Windows version is not available at this moment.

If there will be anyone interested, I will update these packages in case of interesting new functions or fixes. Please leave some feedback and inform me about potential issues related to Gadu-Gadu protocol plugin.

Pidgin and the Impending Shutdown of Windows Live Messenger

November 12, 2012 04:58 AM by John Bailey


So, Microsoft recently announced that they’ll terminate the Windows Live Messenger service in favor of Skype in early 2013.  We’ve been getting a number of questions about what this means for Pidgin.  Quite honestly, we don’t know.  At this point, all we know is that China will still be able to use Windows Live Messenger.  That leads us to believe that the servers providing MSNP service will remain active and maintained for some period of time after the announced shutdown, but it’s not clear whether or not that will be the case.  It’s also not clear if the servers supporting China’s continued use of WLM will be accessible to non-Chinese IP space.  Even further, it’s not clear if the recently-launched XMPP interface to the WLM network will remain functional.  We don’t support that yet though, as it requires some authentication magic we don’t implement.  Even if we implement support for the authentication this XMPP gateway requires, it could end up being a waste of time, as it could get shut down at any time, either before or after the rest of WLM.

And before anyone goes there, we can’t support Skype.  There is no documentation of the protocol available to us, nor is there code we can borrow from a cleanly reverse-engineered alternative implementation.  All that exists is SkypeKit, whose license agreement explicitly forbids its use in open-source software.  The license also forbids use in “server applications” which precludes doing something like wrapping a simple closed-source XMPP daemon around SkypeKit.  It is not currently possible to legally support Skype, so we won’t try.

The bottom line is we have no idea what the announcement means for Pidgin or any other alternative clients yet.  We’ll all just have to wait and see.

Pidgin becomes the first IM with new GG file transfers

November 07, 2012 06:56 PM by Tomasz Wasilczyk

I’ve just managed to create a fully working implementation of new file transfer method in GG11. That means, Pidgin is the first alternative IM with support for it. This also means, I’ve managed to fulfill the most expected promise from this year’s GSoC.

The new method is way more reliable, because it uses Gadu-Gadu servers for file transfers and is built using the HTTP protocol. Thus, there are no more issues related to externally (in)visible ports for people behind firewall or NAT. GG servers also takes care of legacy clients, so there is no need for implementing the old protocol.

Files are sent via the GG Drive service in this way (schematically): sender puts “ticket” on GG Drive – proposal for a file transfer. After the recipient’s acceptance, we can send the file to the GG server – when it’s completely transmitted, the other party can download it. All operations are made through HTTP protocol, so it should work even using proxy servers.

To be able to enjoy this new functionality, download development versions of Pidgin and libgadu from applicable repositories and compile by hand. Unfortunately, we have to wait for a final version to be released for reasons beyond my control: releasing libgadu with GG11 support and Pidgin 3.

Side note: GG file transfer support is one of the oldest tickets for Pidgin: #372.

The last stop before file transfer

October 20, 2012 07:31 PM by Tomasz Wasilczyk

I have just finished new http support module for libpurple. Old one was a bit poor, so it didn’t allowed well-designed file transfer implementation for Gadu-Gadu. So, I decided to put here some more work and do it well.

Before making a decision about own HTTP support, we (with Pidgin dev team) were trying to choose any library, which can do it for us. I had succeeded with initial libcurl integration, but it turned out, that it’s not possible to force this library to use all required libpurple callbacks. Such library have to use Pidgin’s proxy, ssl and dns implementation. Other libraries case is – unfortunately – much worse.

All obstacles before introducing file transfer support for GG are suppressed, so I can handle this now. Furthermore, I had succeeded with test implementation of sending a file – it’s unusable rigth now, but at least – it works.

 

Announcing our four Summer of Code students!

April 24, 2012 08:19 AM by Mark Doliner

We’re pleased to announce that we’ve accepted four students for this year’s Summer of Code!

  • Gadu-Gadu PRPL improvements by Tomasz Wasilczyk, mentored by Ethan Blanton
  • Plugin website by Nikhil Bafna, mentored by Kevin Stange
  • Usage stats collection by Sanket Agarwal, mentored by Eion Robb
  • libpurple on Android by Michael Zangl, mentored by Mark Doliner

It’s always difficult to narrow down so many great applications into just a handful, and we want to thank everyone who applied. The coding period runs from May 21 through August 24. If you want to follow the progress of the four students, they’ll be providing periodic status updates to our devel mailing list throughout the summer.

Read Google’s official announcement here.

Libpurple in GSoC 2012

March 27, 2012 05:28 PM by Jorge Villaseñor

Libpurple was accepted in the Google Summer of Code  this year 2012.

I urge every student reading this to apply for any of the projects accepted and if you like, apply to Libpurple.

We have a set of proposed ideas but you are encouraged to bring your own ideas since they will be fresher and will not compete with other people over the same project.

You can find libpurple's application page at Pidgin, Finch and libpurple.

Pidgin Accepted to 2012 Summer of Code!

March 18, 2012 02:11 AM by Mark Doliner

Good news, everyone! Google has accepted the Pidgin project‘s application to be a mentoring organization in this year’s Google Summer of Code. If you love programming and are looking for a chance to help an open source project, look no further.

How much can you accomplish in a single summer? Quite a lot. To give you an idea, here’s a list of some of our heftier projects of past years:

  • SSL certificate verification and management
  • Voice and video chat for XMPP
  • The Bonjour protocol plugin
  • The MySpace protocol plugin
  • The SIMPLE protocol plugin
  • Finch (command-line based IM client based on libpurple)

Details

  • Get inspired by our ideas list. But don’t limit yourself to those ideas—we love when students propose their own projects.
  • The application period starts March 26 and ends April 6th (full timeline)
  • Once the application period opens, apply here
  • We’re guessing we’ll request slots for 3 students this year.
  • If IM isn’t your thing but you still want to participate, check out the list of other great organizations

Major Changes Afoot

August 28, 2011 06:44 PM by John Bailey

Well, it's been about forever since I last bothered to post anything here. Since my last post, we've released several times, introducing and fixing a bunch of bugs. Now, however, we're shifting our focus to new development of a sort that we don't do often—compatibility breakage and big internal changes. This means that our main development effort is now in working toward Pidgin and libpurple 3.0.0. I'm going to try to explain some of the work going on for the benefit of anyone who reads my ramblings, so here goes.

Versioning

First of all, there seems to be some confusion about how Pidgin and libpurple version numbers work. I'd like to try to clear some of that up.

Pidgin, Finch, and libpurple use what's called “Semantic Versioning.” That is, each part of the version number has a particular meaning for users and developers. We chose this scheme to assist plugin developers in knowing when significant changes would require effort on their part to maintain compatibility with current Pidgin and/or libpurple versions. It also helps our users by letting them know when their existing plugins will stop working. So, how does this semantic versioning work? Let's look at the format of our version numbers and find out.

Pidgin, Finch, and libpurple version numbers have three components, separated by dots. At the time of this writing, 2.10.0 is the current version number. These components are called major (currently 2), minor (currently 10), and micro or patch (currently 0). Let's look at what each means.
  • Major: the major version doesn't change often. The last time we changed it was in 2007 with the release of Pidgin 2.0.0, and before that was in 2004 with 1.0.0 when we started using semantic versioning under our previous name. Whenever this number changes, we've made changes to Pidgin, Finch, or libpurple that break compatibility with every UI and plugin that currently exists. Usually this means that we've removed something from the API exposed to plugins and UI's, or that we've changed something about a function (its name, arguments, return type, or the header file it's in). Sometimes plugins or UI's can just be recompiled when this happens; other times they need maintenance to become compatible with the new release. Additionally, the major version never decreases. It will always increase when it changes.
  • Minor: the minor version changes more frequently than the major version, but generally less often than the micro or patch version. Whenever the minor number increases, we've added things to our API that do not break compatibility with existing plugins or UI's. A prime example of this is adding the voice and video support we added in 2.6.0. We added a bunch of stuff to the existing API, but didn't change anything that would cause a break in backward compatibility with existing plugins and UI's. When the minor version decreases (gets set back to 0), it means that the major version has changed, and the whole cycle starts over.
  • Micro or patch: The patch version changes with almost every release. When this version increases, it means that we haven't touched API at all; instead we've done nothing but fix bugs or add small features that don't affect compatibility with plugins or UI's. When this version decreases (gets set back to 0), it means that the minor version has changed.

Related to all this, and important only for plugin and UI developers (so skip this paragraph if you're not a developer!), is the behavior of the PURPLE_VERSION_CHECK macro. Many developers expect PURPLE_VERSION_CHECK(2, 5, 0) or similar to expand to a statement that evaluates to 1 or TRUE in the case of building against libpurple 3.0.0. This, however, is not the case. Because our major versions are incompatible with each other, we have intentionally written PURPLE_VERSION_CHECK to fail if the major version is not an exact match. We understand this can be confusing and inconvenient, and we sincerely apologize for that, but we're not going to change it.

What all this means is that if the first number (major version) changes, you're going to need to update your plugins when you upgrade Pidgin, Finch, and libpurple. If the second (minor version) or third (micro version) numbers change, it means you need to upgrade Pidgin, Finch, and libpurple, but your existing plugins will still work. No matter what the Pidgin, Finch, and libpurple version numbers are, you should always be using the newest version.

Structure (“struct”) Hiding

This paragraph is for those who aren't programmers. Feel free to skip it if you don't care about it. Pidgin is written in the C language. C has a type system, which means that if you declare a variable (imagine this as a box somewhere in your computer's memory) you can store only one type of data in it. A structure, or struct in C parlance, is a type made up of other types arranged sequentially. This is pretty easy to picture if you think of Lego blocks--stack a red block, a blue block, a green block, a yellow block, and a white block on top of each other and connect them together and you now have a structure made out of Lego blocks. It's pretty similar in C, except that you're telling the compiler to assemble something out of sequential boxes of memory instead of little plastic blocks.

Pidgin uses structs everywhere. We use them to represent things like your buddies, conversations, accounts, etc. Currently in Pidgin 2.10.0, most of the structs are in the public API—that is, anyone can directly access the members of the structs and do whatever they like. This is all fine and well, but it means that if our code changes such that a particular struct needs to grow significantly by the addition of new members, we can't always do that without breaking backward compatibility. (Yes, we included padding in a number of our structs, but we've burned through the padding in several of them, and although there are ways to work around it, I and a few other developers don't like them.) It also means that if we discover, for example, that switching the order of members in a struct allows the compiler to improve its optimizations or if we think a different order makes more logical sense for those of us reading and maintaining the code, we absolutely can't do this without breaking compatibility. We also can't rename members of structs for the same reason—it breaks compatibility with existing plugins, UI's, etc.

Because having these structs in the public API limits us so much, we're striving to hide as many of them as possible. By hiding, we mean that we're removing the struct definitions from the header files and moving them to the .c files, thus making them private. Plugin and UI authors will still be able to reference the “objects” with pointers, passing them to functions and operating on them with the appropriate sections of our public API, but no longer will the members of the structs be directly accessible outside of the .c files that define the functions that interact with them. For example, we have a PurpleRoomlist struct in libpurple/roomlist.c and libpurple/roomlist.h. For 3.0.0, the struct definition is in roomlist.c; thus the individual members are not directly accessible outside roomlist.c, even in other parts of libpurple. This “hiding” of the strcuts allows us significant internal flexibility in each file to modify the struct as we see fit.

Clean-Slate API Documentation

Because we generally change so much each time we increase our major version number, the API documentation can get a bit confusing if we keep doing @since tags and whatnot in our doxygen documentation. Our general overall feeling is that we prefer just starting with a clean slate at each new major version. This means that each time we do a new major version, all existing @since tags will disappear, any functions marked as @deprecated will be removed, renamed, or replaced as described in the @deprecated statement, and so on. We realize this decision may make some things more difficult for some plugin and UI authors, and we apologize for that, but our aim is to have overall cleaner documentation for everyone.

Merging of Old Projects

Over the years that we've participated in Google's Summer of Code program, we've accumulated a number of branches that have been sitting for quite some time. Most of these need some form of TLC and integration work. We want to try to incorporate at least some of them into 3.0.0 so we can finally benefit from the fruits of the students' labor. Most notably, we've been talking about merging the webkit integration branch into what will become 3.0.0. Eventually, this would allow the support of Adium's message styles, although it may not happen right away.

Another project of notable interest is some of the logging changes that went on in a previous Summer of Code project. One of our new Crazy Patch Writers took some of that work and made some progress on it; we don't know if this will make it for 3.0.0 or not yet, but it would be nice to have some of the features, such as non-blocking log writing.

Other Changes As Wanted

We may decide to make other changes since a major version change gives us the opportunity to break so much. There have been a number of ideas floated, ranging from supporting that XDG directory spec that I can't stand to doing away with the xml files in .purple and replacing them with something else (with what, in particular, has not seriously been discussed). There is a whole range of possibilities of things we could do for 3.0.0; it's just a matter of one of us wanting it and sitting down to write it.

So everyone should stay tuned, as we'll be making more and more changes over the course of the 3.0.0 development cycle. We don't yet know when we'll be releasing 3.0.0—this is another one of our famous “when it's ready, and not a minute before” time frames. We do know, however, that it will change a lot!

Tool to View DNS SRV Records for XMPP

August 04, 2011 07:49 AM by Mark Doliner

I wrote a crummy tool that displays the current DNS SRV records for a given XMPP domain. After writing it I discovered that Olark has already created something similar.

My tool attempts to do some really basic sanity checks and warns if it sees problems. If you think of something that it should check for but currently does not, please let me know! And please let me know if you notice any problems or have other suggestions.

No Summer of Code for Pidgin This Year

March 12, 2011 09:13 AM by Mark Doliner

Pidgin/Finch/libpurple/Instant Messaging Freedom, Inc. didn’t apply to be an organization in Google’s Summer of Code (GSoC) this year.

Why not? The short answer is that I didn’t feel we had enough developers willing to mentor students for it to be worthwhile.

The long answer is a little more complicated. I feel very lucky that Pidgin has been able to participate in GSoC for the previous 6 years. GSoC is an amazing program and it gives me a warm feeling that Google has consistently funded it, and I hope they continue long into the future. I think I’m a better person for having mentored some students and for having been our organization administrator a few years.

Pidgin has certainly benefited. Heck, Finch itself was a GSoC student project. And the developer went on to be an integral part of our development team. Other benefits: Our UPnP implementation for peer to peer connections. SIP/SIMPLE/STUN. A D-Bus interface. Performance improvements. MSN, ICQ and Yahoo! maintenance. The MySpace protocol plugin. Our certificate manager.

But mentoring students does take time. Five hours per week per student is a reasonable approximation. Some students need less guidance. Some students are so productive they need more help, having their code reviewed, getting feedback on implementation ideas, etc. Reading through and ranking all the student applications takes time. There’s also a mid-term and final evaluation survey that must be filled out. The time required for all this isn’t particularly unreasonable, but many of us developers have very little time to start with.

I tried to get a verbal commitment from some developers this year on whether they’d be able to mentor and the response was pretty limited. In general people expect to be pretty busy this year, and it didn’t seem appropriate for us to commit to GSoC when we may not be able to give our students the level of attention they deserve.

And so it is with a sad heart that I decided not to submit our application this year. It seems likely we’ll apply again in 2012. Also, please don’t get the idea that we don’t want contributions—we love patches and new contributors!

Current AIM Issues

February 24, 2011 01:50 AM by John Bailey

Over the last couple hours, we've had nearly everyone and his/her brother in #pidgin asking about connections to AIM causing a certificate prompt. The specific prompt is for bos.oscar.aol.com. The issue here appears to be that AOL has let the certificate expire. Because our certificate validation is more strict than some other applications, Pidgin users will get this prompt at every connection until AOL installs an updated certificate.

To resolve the prompt, you can make one of a couple choices. You can choose to trust that the connection is fine even though the certificate is expired and click Accept, or you can take the safe route and click Reject until AOL replaces the certificate.

To reiterate, this is not a Pidgin problem, but an AIM server problem.

Complex Transient Statuses for Quick Effect

February 05, 2011 03:36 AM by John Bailey


In an earlier post, I discussed how to take advantage of Pidgin's "saved status" feature. After seeing some recent confusion in #pidgin about our status features, I decided it would be a good idea to give a quick overview on how to take advantage of a feature I never use--creating statuses (including complex statuses) from the status selector on the buddy list window.

The status selector feels like it's been around forever. Prior to this, we had a rather horrible interface to "status" whereby you could either globally set all accounts to "away" or go to each individual account and configure a given status (away, do not disturb, vacation, etc.). All this was done via a single menu. For those of you who don't remember it, let me just say that it sucked. Someone (I think it was Sean Egan) threw that whole UI out the window and came up with the current status selector that reminds me a lot of the old Windows ICQ 99 client that had a pretty similar status selector. Ours is a bit more sophisticated, though.

The simple use of the status selector is fairly obvious--pick a status and type a message. In fact, this is what the majority of our users do when they change statuses. This is what is called a transient status. What most people don't know, however, is that you can create complex statuses, such as having your MSN account set to "Busy" while your AIM account is set to "Invisible." Let's give a quick example.

I have a number of accounts in Pidgin. Let's say I want to have my pidgin.im XMPP account set to "Available" but the rest of my accounts set to "Away" to create a simplistic scenario that's really easy for me to snag a screenshot of. To do this, I'll go to the status selector and select "New status..." like in this picture:


Then I'll see this window, where I've already gone and entered a title for the status. This is the name you'll see in the status selector and in the middle section of the status selector's menu. If I wanted a message to go with the away status, I'd enter it here now, as well.



Since I want most of my accounts to be away, I'll leave the "Status" selection as "Away." As you can see, I have my mouse pointer over "Use a different status for some accounts," which when expanded will allow me to set statuses for individual accounts, like so:



I found my pidgin.im account in the list already, so I'll check the box in the "Different" collumn. That brings me to a new window:



Now I want my pidgin.im account to be available, so I'll accept what I see. If I wanted a message here, I'd type it in the "Message:" box. When I'm happy with what I have, I'll click OK. Notice that the "Status" column in the previous window changes to reflect what you selected for the individual account.

Since I'm happy with what I have, I'll now click "Use", which applies the status. If I wanted to make this a saved status that is saved permanently, I could click the "Save" button or the "Save & Use" button if I wanted to also immediately apply it when saving.

That's all there is to it!

Fun at the GSoC Mentors Summit

November 06, 2010 06:21 AM by Elliott Sales de Andrade

Last weekend was the GSoC Mentors Summit. As a mentor for the Pidgin, Finch and libpurple project, I attended for the first time this year.

It was pretty interesting and a lot of fun, but I have to say I didn't really feel like much of a geek there! Everyone's either got an iPhone or a Droid, and they've all done awesome stuff. Can you say you've worked on WordPress, used by millions of websites, or Apache, serving even more websites, or RTEMS, running several space exploration instruments and other consumer products, or who knows what else?

Nevertheless, it wasn't like I was totally unknown. There were quite a few people who use Pidgin, even one or two using Finch. Oh, and some Mac users using Adium (using libpurple), too.

The flight was fine, but the weather was a bit disappointed. It would have been nicer to explore a bit more, but at least I got to eat a ton of chocolate. I'll try and write a longer re-cap later.

MSNP16 and SLP-rewrite merged

November 01, 2010 06:01 AM by Jorge Villaseñor

I have just pushed the revision that merges my MSNP16 and SLP branches to the main development branch in pidgin. I'm very happy to have this branches merged since they represent almost all the code I have been writing on the last year.

Yes I have started coding MSNP16 support almost a year ago and it took a lot of effort, reverse engineering, debugging Wireshark dumps and a lot of pidgin debug logs to get it working. That is a lot of time!

It is true that the MSNP16 code was almost complete when I started my SoC work but I though it would be better to start the SLP rewrite over the MSNP16 branch to be able to easily test both codes at the same time and try to get it in a better shape before merging it to i.p.p.

I know I have announced this merge like two weeks ago, but you know, I wanted this merge to be followed by a reasonable "beta" testing before being released and at that time it got that we had an security issue and needed to release 2.7.4. Once it was out, there were some ICQ issues that needed a quick release to fix that bugs, so we got a 2.7.5. Now I was able to merge and get a normal release cycle to get beta testers to find bugs in this new and nice code.

I hope this code will fix more issues than it brings up, specially the ones related to data transfer. Since most of the code on this area have changed due DirectConn and SLP-rewrite, I guesss it would be a good idea to review and close most of the related tickets since the traceback and debug output would be really useless now. Yei for smashing tickets!

I hope you all like 2.7.6 when it get released!

Death of a thousand tickets

October 26, 2010 05:06 AM by John Bailey

Well, by now it's obvious to the world that I kicked Pidgin 2.7.4 out of the nest last week. Although that release included some nice new features for ICQ users, lots of bug fixes, and some remote crash fixes, it's not without its share of problems. Those problems are producing a bunch of duplicate tickets for us to deal with, so I thought it might be a good idea to post about them here just in case anyone bothers to read my rambling.

The first problem is the AIM/ICQ chat bug. When using a multi-user chat on AIM or ICQ, no messages can be sent. You'll get an error stating your message was too long. This was an unintended side effect of merging the work of one of our Summer of Code students. Ivan accidentally removed a line of code that he shouldn't have. Yes, you read that right--a single line of code. Apparently it was pretty important! Ivan restored that line and things will be working as they should in 2.7.5 when we release it.

The second problem is an old one that's come back with a vengeance. This one is an ICQ issue where messages that contain
disappear because they're treated as HTML tags. This seems to be specific to combinations of Pidgin and other non-official clients. Pidgin recently gained some new ICQ protocol-level features related to formatted messages; some non-official clients, including older versions of Pidgin, don't handle it gracefully. At this point, we've done all we can do about it; the other clients will have to step up and make fixes to handle the messages correctly now.

The next problem is another wonderful ICQ encoding bug. I'm not sure if this one is a side effect of Ivan's work or not, but Ivan did do a lot of work on encoding problems on ICQ. We all thought things were improving, but apparently there are still a few odd cases (and a crapton of stupid ICQ clients that should be purged from existence) in which our handling of encodings just isn't quite right. We're aiming to have this fixed for 2.7.5, but it's not quite done yet.

Is it obvious I wish ICQ would just disappear immediately and permanently?

The last "big" problem that seems to be cropping up is a crash related to MSN file transfers. When using direct-connect file transfers, somewhere along the line we do something stupid internally that causes a crash. We thought we had it fixed for 2.7.4, but alas it still exists. We're looking into it and hope to get it fixed soon. In the meantime, it's pretty easy to prevent the crash--edit your MSN account and turn off the direct-connect file transfers. It's an option on the advanced tab of the account editor.

At any rate, since we know about these bugs already, please, please, please DON'T open new tickets about them!

Merge Plan for MSNP16 and SLP

September 21, 2010 06:56 AM by Jorge Villaseñor

As you know I have been working on a refactor for the SLP code on the msn-prpl for libpurple as part of my Summer of Code. Before I started this project I have been working on adding support for "Multiple Points Of Presence" which are part of the MSN Protocol 16 (MSNP16).

Since by the time I started the SoC the msnp16 code was almost complete, I started to refactor the SLP module over the msnp16 code. At some time in the process of the refactio I got an ugly crash from one of the features of MSNP16, the P2P version 2 which uses a different binary header for SLP transfers than the one used before. This bug was caused by some clients not paying attention to the capabilities we expose, we clearly say we don't support P2P V2. To avoid this crash I have disabled MSNP16 in the SLP branch.

I have been testing the SLP code for a while with the MSNP16 feature enabled and it looks stable to be merged. The crash has gone, there are some minor changes that must be done, specially UI stuff, the new SLP stack have no known bug.

I have updated the SLP branch with the latest changes from the MSNP16 branch, so this branch [1] have every work waiting to be merged to pidgin's main development branch. This was the first step to get it merge-ready. There have been some testing form some of our closest geeky-friends and now I think it's ready.

My plan is to merge the SLP code with the MSNP16 feature enabled the next week to im.pidgin.pidgin. I ask everyone interested in this features to test it before it get to main branch so we can have a softer merging process.

[1] im.pidgin.soc.2010.msn-tlc

New SLP stack working

September 20, 2010 08:24 AM by Jorge Villaseñor

Note: This post was wrote 2 months ago and never published, my bad.

After some bug squash, I finally got p2p transfers working. I have tested sending and receiving custom emoticons and display pictures, smallers than the SB limit and bigger than that. They just work ok.

I guess I need to test some file transfers before thinking my work is done, I also have some cleanup that I want to do but I guess I can declare that the new stack is working nice and is very close to be merge ready. I'm really glad to say so.

XMPP, meet Facebook...

August 11, 2010 12:58 AM by Elliott Sales de Andrade

So, it seems the Facebook has finally added support for logging in to their Chat via XMPP. This is, of course, awesome so people stop asking about it, but also terrible since it took them so darn long. That meant a lot of horrible hacks just to get it working in the interim. (No offence Eion, I just mean the scraping, not the code.)

This announcement comes straight from the developer blog. You need to have set up a username (which you can check here). Casey's blog has all the screenshots you need to figure it out. The only downside is it doesn't use encrypted streams.

The biggest problem, though? Having to organize a few hundred buddies into their correct contacts.

The Proxy Conundrum

June 26, 2010 03:37 AM by John Bailey

As many of you Yahoo users out there know, Pidgin 2.7.0 and 2.7.1 introduced a challenge for you. In certain proxied environments, many of you can't connect. Quite frankly, (mostly) it's my fault. Let's start by explaining what I've done.

In Pidgin 2.7.0 I made some seemingly innocent changes to the Yahoo protocol plugin that I thought would magically make some problems go away. These changes make us mimic the official Yahoo client better. Very early in the connection process (in fact, it's the first step), the official clients request a specific URL from a Yahoo web server. The server's response tells the client what messaging server to connect to. The client reads that address and connects to it, then goes along its merry way. So I made our Yahoo plugins do this, once I determined the correct URL's for both Yahoo and Yahoo Japan. Everything was great when I tested.

But there's my mistake--I tested it. I don't run a proxy server--I'm an IT nerd, but I'm not that much of an IT nerd. So, I allowed this to be released, not knowing there were problems in environments with restrictive proxies. It came back to bite me hard, as I backported this stuff to Adium's build of libpurple 2.6.something for one of their beta releases and several users started complaining about being unable to connect. Then we got a couple tickets of our own. At first, I was stumped, until a user who was willing to almost bend over backwards to assist us came along.

This user worked in an environment where two separate proxy servers were in use. One handled HTTP and HTTPS traffic and the other handled IM protocols. It took me quite a while to finally wrap my head around his problem, but once I did (with the help of my fellow developer Etan Reisner), I was able to start solving the problem. It turns out I actually had two problems on my hands, but at first it seemed like none of it was the fault of our code.

The first problem is that Pidgin allows configuring proxies in two places--the "Proxy" tab shown when editing an account and the "Proxy" tab in Preferences--but does not allow specifying what ports or services are handled by each proxy. This ordinarily isn't such a big issue, except that I went to the extreme of creating an option for Yahoo accounts to help some users with weird proxies that don't handle SSL at all. This option is used to determine whether to send an HTTPS request (like Yahoo uses for login) to the account's configured proxy or to the global proxy configured in Preferences. When I did this for 2.6.0, I created a new utility function for URL requests that allowed specifying the account the request was for. If the magic value NULL is passed to this function, the request will use the global proxy instead of the account proxy. This allows our proxy code to route the request to the appropriate proxy. Being the idiot I can be on occasion, I set all URL requests in the Yahoo plugins to use the account proxy server no matter what. This is where the user's problem came in--his IM proxy doesn't do HTTP at all. Once Etan and I agreed on how to fix it, I changed the SSL option I mentioned earlier to act on both HTTP and HTTPS requests. I supplied our user the test build. It still didn't work, but the debug log showed some interesting new info.

Again, it took some head-bashing to understand what was going on, and I still wasn't the one who figured it out. In the meantime, I decided to implement a quick hack to make users at least be able to connect temporarily while we figured out what was really broken in our proxy code. The quick hack was to implement a worst-case, last-resort connection to a previously known messaging server. Unfortunately, I could implement this only for Yahoo, not for Yahoo Japan, as the Japan network appears to no longer has a single canonical hostname by which to refer to the messaging servers. I am, however, positive that even on the regular Yahoo network, this hack will eventually break, which is why I resisted it in the first place. After a couple quick bugfixes brought to light by the user's further testing, we had a working fallback mechanism.

Once this was done, Daniel Atallah, one of our co-lead developers, realized that the second problem our helpful user had was our own proxy code's fault--for some stupid reason, we would never authenticate to an HTTP proxy if we were making an HTTP request on the standard HTTP port (80). Daniel implemented some quick fixes to make our URL requesting code behave better in these proxy situations, thus (we hope) finally solving this problem completely.

All these fixes will end up in Pidgin 2.7.2. We're not sure yet when we're going to release that, but we're hoping it's soon, as we have some fixes for other annoying bugs, such as the famous MSN direct connection crash.

Now, after reading all this, some of you might think, "Why don't they just use libproxy?" Truthfully, that's probably a good idea. There are two reasons we haven't done this yet. First, we looked at this over two years ago. At that time, we were led to believe libproxy didn't handle all the proxy types we did. We were corrected on that, but I, at least, missed that correction being posted. (Sorry, sometimes I'm blind, especially if something like that is buried in a flood of other ticket mails.) I suspect I'm not the only developer who did. Secondly, we're all creatures of habit. As the old saying goes, "if it ain't broke, don't fix it." Well, quite honestly, it's not broken for us as developers! Mainly that's because either we don't use proxies or any proxies in our way don't do any of the crazy stuff we ran into in this case. If it's not broken for us, we don't have much incentive to go hacking stuff up to support new libraries that solve problems we don't experience. I know that sounds like a lame excuse to a lot of you, and I can understand how it seems that way. But if we don't experience the problem, it's hard to test that we've fixed it, and we could in fact be making matters far worse for everyone else.

All this said, we probably should support libproxy. I've been thinking about how we could do this in a quick and effective manner, and so far I've come up with a few ideas that don't feel like they work very well. I think this is a perfect case where code should talk, or at least provide an example. So, I invite people who would like to see us support libproxy to write patches. Actively engage us on our development mailing list (devel@pidgin.im), talk with us in #pidgin on irc.freenode.net, or drop by our XMPP MUC (devel@conference.pidgin.im). Or, do some combination of the three. I can't promise we'll be the most valuable resources during the process, but we do at least try.

Now supporting libproxy is one thing. Actually being able to do useful stuff with the information libproxy gives us is another thing entirely. Our actual proxy connection code is, shall we say, a bit sub-par. Quite frankly, it sucks, even ignoring the crappy configuration portions. While supporting libproxy is nice and may solve a lot of our proxy problems, I guarantee it won't solve them all. Ideas and patches to fix this are greatly appreciated too.

I look forward to hearing from those of you inclined to help!

Coming soon, to a Pidgin near you...

May 21, 2010 09:43 PM by Elliott Sales de Andrade

Direct connections for MSN! What do I mean? Just take a look at the screenshot below.

What's that? You don't see it? Just take a look at how fast it's going. Yes, that's right; it's not a pitiful 10KiB/s, but about an order of magnitude faster.

It all started with this little patch on trac. But it required a lot of work to get it going. I must have made at least 60 commits just fixing things and getting it to cooperate with aMSN and the official client. I think it should be good now, but you can always disable it in the Account options. It even makes buddy icon and custom emoticon loading faster.

Anyway, I promised I would say something about 2.7.1, and here you go. Direct connections will be in 2.7.1. In fact, I'm just about to push all the revisions.

Branchifying Revisions in Monotone

May 21, 2010 08:43 PM by Elliott Sales de Andrade

Have you ever started committing things in Monotone and then realized "This should probably go in a separate branch"? It's somewhat difficult to fix that. You can add a new branch cert, but the old one won't go away.

I wrote a little script based on a mailing list message I saw. It removes the old branch cert, and adds the new one, moving forward along revisions. It even changes merges with "the rest" into propagate messages instead. There are limitations, though. It only works with linear history (but if you've been working alone, that shouldn't be a problem) and it requires mtn 0.45 (since it uses the binary cert format.) Of course, it goes without saying that if you've shared any of your changes with someone else, then this is pointless (or you'd have to change everyone's database.)

It's pretty short. Make sure those SQL statements don't break up when you copy it.

#!/bin/bash

set -e

DB=db.mtn
KEY='someone@domain.com'
START=0123456789abcdef0123456789abcdef01234567

OLD_BRANCH='com.domain.main'
NEW_BRANCH='com.domain.fancy-stuff'

prev=
curr=$START
while [ -n "$curr" ]; do
    echo "Changing revision $curr"

    mtn --db=$DB db execute \
      "DELETE FROM revision_certs WHERE hex(revision_id) = upper('$curr') AND name = 'branch';"
    mtn --db=$DB -k$KEY cert $curr branch $NEW_BRANCH

    parents=`mtn --db=$DB automate parents $curr`
    if [ `echo "$parents" | wc -l` -eq 2 ]; then
        # Fix a merge
        echo "Fixing merge for $curr"
        mtn --db=$DB db execute \
          "DELETE FROM revision_certs WHERE hex(revision_id) = upper('$curr') AND name = 'changelog';"
        other=`echo "$parents" | sed -e "/$prev/d"`
        mtn --db=$DB -k$KEY cert $curr changelog \
"propagate from branch '$OLD_BRANCH' (head $other)
            to branch '$NEW_BRANCH' (head $prev)"
    fi

    prev=$curr
    curr=`mtn --db=$DB automate children $curr`
done

Pidgin 2.7.0: Movin' on Up

May 14, 2010 02:40 AM by John Bailey

Well, I finally managed to kick Pidgin 2.7.0 out the door, as most of you are surely aware of by now. There are several important things to note about this release:
  • AIM and ICQ clientLogin issues have been resolved. You should be able to now use both clientLogin and SSL together.
  • Our internal libgadu has been upgraded from something from the stone age to something much, much more current. This solves the issues that (primarily) Windows users had with not being able to communicate with new (> 17,000,000) Gadu-Gadu numbers.
  • Our Yahoo and Yahoo Japan plugins now retrieve connect server information directly from Yahoo, just like the official clients do. This took me way too much time to implement, but I believe I've implemented a fairly stable and maintainable method to retrieve this information. During this process, I learned a lot about how Yahoo's authentication works and made some improvements there too that will make us look much more like the official client on the wire during authentication.
  • ICQ now has X-Status support! It took far longer than it should have--I originally wanted this in for 2.6.0 but it was nowhere near ready.
  • Minimum GTK+ and GLib requirements are now 2.10.0 and 2.12.0, respectively. This allowed us to drop some 3800 lines of code, much of which was the result of needing to bundle certain GTK+ widgets so Pidgin could function on platforms with ancient GTK+. This also makes use of many of glib's convenience functions much more convenient.
  • The Windows installers have changed to be less obnoxious with respect to GTK+. We no longer install a system-wide copy of GTK+ and instead install GTK+ locally to the Pidgin installation. This may disrupt some GTK+ theming. The other big thing is that now, the default installer will download GTK+ and optionally the debug symbols (so there's no longer a special debug installer needed for crash reports!). There's also an offline installer that, while almost double the size of the default installer, I prefer to use. This offline installer includes GTK+ and the debug symbols. I prefer to call it the everything-but-the-kitchen-sink package (the kitchen sink in this case being spell checking support, which still must be downloaded).
Of course, any release I had a major role in is bound to have problems. I built the Windows installers this time around, and did not know that I needed to do extra work to make the online installer function. Thus, many of you complained (and complained, and complained some more!) that the installer didn't work. Even after Daniel fixed this for me, we still saw complaints. If the online installer does not work for you, try the offline installer. I tested the offline installer and it worked for me. I forgot, however, to test the online installer. I was more interested in getting the release out in order to (I thought) minimize complaints.

Additionally, after I cut the tarballs for 2.7.0, several bug fixes rolled in that will be delayed until 2.7.1. Among these are a couple minor annoyance fixes for MSN and XMPP. And today fixes for more bugs rolled in, including two for Bonjour and a fix for the taskbar entry (window) flashing on Windows. I plan to push out Pidgin 2.7.1 in a couple weeks to resolve these, and hopefully more, problems.

In the mean time, enjoy!

Pidgin 2.7.0 Released!

May 13, 2010 05:36 AM by Elliott Sales de Andrade

So it seems like I only post about minor releases (as opposed to micro), but I promise that I'll definitely say something about 2.7.1.

Anyway, Pidgin 2.7.0 should be released pretty soon now. You may or may not know that the release versions are dictated by API/ABI requirements. A minor release means we've added API. I wouldn't say there are tons of new user-visible features in this release, but definitely some developer-related things.

Most importantly, we dropped support for old GTK+ and GLib, requiring 2.10.0 and 2.12.0 respectively. Visibly, that means we can't support Windows 98 anymore, but I think we can say that's an acceptable loss by now. For development purposes, though, that means a ton of code that can be removed. It means a lot less code that needs to be maintained. We also were able to upgrade some things like from EggTrayIcon to GtkStatusIcon, which is better at integrating into the notification area.

Don't worry; it wasn't all development changes. We've also got integrated Mood setting. This interfaces to ICQ and XMPP, so I don't know too much about it. More in my area, MSN gained support for file transfer previews, which show up in the request window and the conversation.

There are quite a few other changes that I don't really even know about, so you might want to check the ChangeLog.

Mentoring a student for GSoC

May 06, 2010 05:08 AM by Elliott Sales de Andrade

So last week the accepted proposals for GSoC were announced. Pidgin was given 4 slots.

First up, we have Ivan Komarov working on improving ICQ. I don't really use ICQ, so I can't say much, but if you do, hopefully there'll be a lot of great things coming out of it that will be useful to you. Next up is Adam Fowler working on a better chat log viewer. People really seem to complain about what we have now and at the very least, it's kind of slow once you have a lot of logs, so there should be room for some good improvements there. And then we have Gilles Bedel working on detachable libpurple sessions. I'm not sure how often I'd use it, but every once in a while, I feel like checking in on Pidgin from the laptop and it's a very painful process through SSH and D-Bus. This project should make that sort of thing much simpler. Finally, we have Jorge Villaseñor Salinas working on refactoring MSN.

So that last one is the one I'm mentoring. Personally, I've been wanting to re-write bits and pieces of the MSN prpl for a while now, but I've never had the time to get around to it. Jorge's already done some great work getting custom smiley support working for Pidgin 2.5.0. I'm sure he'll have no problem re-working the MSN prpl. I'm looking forward to getting a more structured and (hopefully) well-designed prpl which should make development a lot easier and quicker.

Plugin Authors: Save Your Users' Sanity!

April 30, 2010 05:46 AM by John Bailey

OK, maybe I'm being a bit sensationalist there, but it's my blog, so I'm allowed! Anyway, on to the subject matter.

Recently we've seen a few users in #pidgin (and I've received some emails too) with some strange issues that result from plugins doing things they really shouldn't but can anyway. I'm going to run down a small punch list of items for you, as plugin authors, to do to prevent your users having hard-to-diagnose issues.
  • Never use printf() in a plugin. This causes terminal disruption for Finch users and is a bad idea in general. It will probably behave unpredictably on Windows and/or in other UI's too. We provide a debugging API to avoid this kind of stuff. Instead of printf(), use purple_debug_info() or purple_debug_misc(). You can see the debug output (even during initial startup) if you run 'pidgin -d', even on Windows and even if your Pidgin is built without '--enable-debug'. For Finch, you can do 'finch -d 2> finch_debug.txt' to get your debug info.
  • Don't lie about UI requirements. If you use GTK+, you're a Pidgin plugin, not a "core" (or libpurple) plugin. If you use GNT (or if you're really brave and use ncurses directly), you're a Finch plugin. There are plenty of examples in the Pidgin and Purple Plugin Pack source trees that show you how to properly declare your UI dependency. The UI requirement is designed specifically so that plugins requiring a specific UI can't be loaded by, and thus cause problems for, other UI's. Sure, it's possible to make a UI-agnostic plugin that requires GTK+, but I guarantee it will cause at least one or two Finch users unnecessary problems.
  • Install your binaries to the correct place. If you're installing a Pidgin plugin, install to $prefix/lib/pidgin, not $prefix/lib/purple-2. If you're installing a Finch plugin, install to $prefix/lib/finch. This will prevent other UI's from needlessly trying to probe your plugin and is just general courtesy. (Note that this particular item doesn't apply to plugins for Pidgin on Windows at the moment, but may be important later.)
  • If your plugin is a protocol plugin, you absolutely are not allowed to touch UI-specific code at all. That means no GTK+, no GNT, no XUL, etc. If you need UI elements, use the request API libpurple provides. If the request API doesn't cut it, there are other approaches you can take, such as having two plugins, one of which does the actual protocol work and the other providing the UI. A better approach, though, is helping us identify where the request API is lacking so we can improve it. We know our API's aren't perfect, so when someone can show us a concrete reason why we're deficient, we will try to fix that deficiency.
Please check your plugins for these problems and help your users have a more pleasant experience with your plugins!

Time flies...

April 27, 2010 11:27 PM by John Bailey

It feels like it's been forever since I stepped up on the soapbox to say something. This time it's just a relatively short update.

As is very commonly known, Pidgin was accepted as a mentoring organization for the Google Summer of Code again this year. This year, Google has given us the privilege of mentoring four students. While less than we hoped for (we'd wanted five), we certainly can't complain about the projects we were able to accept.

First up is an MSN project manned by Jorge Villasenor Salinas, who some may recognize as having helped implement custom emoticon support for Pidgin 2.5.0. His project is to rewrite the SLP layer of the MSN plugin, simplify the other parts of the plugin, and overall make the plugin much easier to maintain and extend. I'm sure he'll be trying to squeeze some new features into his project if time allows.

Next up is one of my personal favorite projects for this year. Gilles Bedel will be working on detachable sessions. This is something a number of us have wanted for a long time, but just haven't quite gotten around to making it happen. The idea here is that we can have a daemon (service for the Windows-inclined among us) running in the background and connect to it with Pidgin or Finch at will. I feel that this is perhaps the most ambitious project this year, but the benefits it could reap are worth it.

Adam Fowler is going to rewrite the log viewer in Pidgin. This particular project will likely be the only project that visibly changes Pidgin this year. We've received a lot of complaints and suggestions over the years focusing on our log viewer. We know it's deficient in a lot of ways, and that's what Adam's project aims to resolve. I'm confident we'll get a lot of positive feedback about this project once it's complete.

And last, but not least, is Ivan Komarov's ICQ project. Ivan's aim is to beat our ICQ implementation into shape. As most of our non-US ICQ users can confirm, our interaction with the service could use some improvements. While Ivan has proposed a nice wide range of things to fix, even just a few of them being completed would make a huge difference for our international users.

So, from all of us developers, and we hope from our users too, good luck to our students!

AOL, AIM and Openness

April 02, 2010 07:31 AM by Mark Doliner

When I started working on Pidgin eight years ago (eight years‽ holy crap!) my main focus was the code used to connect to AIM and ICQ. The protocol is called ‘OSCAR,’ and it is a proprietary protocol created by AOL.

As IM protocols go, OSCAR is actually quite decent. Flexible, extensible, reasonably concise. Could maybe be simpler.

Background

AOL’s history with public access to their IM network has had its ups and downs.

  • In 1998 a few AOL employees released a GPLv2-licensed IM program called ‘TiK.’
  • Sometime in 1999 this project was abandoned by the AOL employees. Some non-AOL employees created a TiK project on SourceForge and continued development.
  • Things were good.
  • In 2001 AOL made changes to their protocol in an attempt to block unofficial clients. Some people believe these changes were made to block Jabber<–>AIM transports, and that Pidgin (named ‘Gaim’ at the time) got caught in the crossfire. Pidgin developers were able to keep the program working for the most part, and AOL relaxed their efforts to block unofficial clients.
  • Things were good again.
  • In 2008 AOL announced Open AIM. Open AIM was a lot of things: A set of SDKs for interfacing with AIM and the AIM servers. Documentation for the OSCAR protocol. A forum for users to ask questions.
  • Things were great… for a time.

Recent Changes

Sometime in January, February or March of this year AOL decided to shut down the Open AIM program. No more SDKs, no more protocol documentation, and no more developer outreach.

More specifically, the OSCAR documentation provided by AOL described a method of authentication (using a “startOSCARSession” API request) that required the use of a developer application key. The current Open AIM website implies that new keys cannot be created.

Now What?

Does this mean the use of startOSCARSession is deprecated? If so, what auth scheme should we use, instead? Should we go back to BUCP, the authentication scheme used by AIM 5.9? Should we try to reverse engineer the authentication scheme used by current version of AIM? Should we go back to masquerading as the official AIM client and stop politely identifying ourselves as ‘Pidgin’?

To quote the great Yoda, “the shroud of the dark side has fallen.”

On The Subject of Bugs, or Help Wanted and Needed!

February 01, 2010 04:29 AM by John Bailey

As bug master for Pidgin, I bear a lot of the responsibility for triaging bugs--getting them assigned to the appropriate developers, weeding out the invalid bugs, etc. I admit that lately I’ve not been doing such a good job at this. I’ve been ignoring Pidgin so I could relax and unplug somewhat after the stress of work. Thankfully, in my increased absence, my fellow developers, particularly Paul, Daniel, and Elliott, have been picking up the slack. All the while, however, not much has changed as far as actual ticket resolution goes.

As much as I hate quoting numbers, let’s take a look at some (as of 2010-01-31, 0500 UTC) to get an idea of scope:
  • Historical ticket count: 11270
  • Currently open tickets: 1818
  • Open bugs: 958
  • Open bugs awaiting response from reporter: 11
  • Bugs that can’t be fixed until we’re ready for 3.0.0: 19
Obviously, 958 open bugs is unacceptable. But to put those bugs in perspective, let’s look at how they break down:
  • AIM: 64
  • artwork: 9
  • Bonjour: 4
  • custom emoticons: 2
  • finch: 7
  • Gadu-Gadu: 19
  • Google Talk: 3
  • Groupwise: 7
  • ICQ: 47
  • IRC: 26
  • libpurple: 68
  • MSN: 70
  • MySpace: 17
  • pidgin: 198
  • plugins: 28
  • QQ: 23
  • Sametime: 13
  • SILC: 6
  • SIMPLE: 3
  • trac: 4
  • unclassified: 160
  • Voice and Video: 11
  • webpage: 3
  • winpidgin: 81
  • XMPP: 47
  • Yahoo!: 35
  • Zephyr: 3
Now, yes, on an IM service level, MSN has more open tickets than any other listed service. However, looking at the underlying protocol for these services, AIM and ICQ combined give the OSCAR protocol a 41-ticket lead. Google Talk is an XMPP service, but even combining Google Talk and XMPP numbers, we come up with 50 tickets, trailing MSN by 20. None of our other listed IM services share an underlying protocol with another. In fairness, the unclassified bugs really should be sorted into proper components, which would change these numbers, but at this time I don’t know by how much.

Now, looking at the IM services we support, many of them don’t have a current “maintainer” who is ultimately responsible for the plugin. Currently, Zephyr, Yahoo, SIMPLE, SILC, Sametime, MySpace, Groupwise, Bonjour, and QQ either have no “official” maintainer, have no one at all working on them, or haven’t had meaningful development activity in at least three months. Obviously, protocols without an active maintainer are not going to see much in the way of quick bug resolution.

We‘re often asked, “Why don’t protocols have maintainers?” or “Why don’t you spend time working on bugs?" or a hundred other questions about the lack of time spent working on Pidgin. The reasons for this are many:
  • We’re all volunteers. Not a single one of us gets paid to put any time or effort into Pidgin.
  • Not all of us use all the services we support. I’ll use myself as an example here. I use AIM, ICQ, Yahoo!, and XMPP regularly. I also have an MSN account that I couldn’t possibly care less about and a couple IRC accounts for the rare instance that I need to test something with the Purple Plugin Pack. I don’t use, nor do I have interest in using, any of the other services.
  • We all have families that we’d like to spend time with. Many of us have spouses/significant others, and a few of us also have children. Family is a demand on our time that should never be ignored.
  • All (or almost all) of us have jobs. Since, as I already mentioned, we’re volunteers, the jobs with which we earn our paychecks have to take priority.
  • We’re human too--we get burned out, experience stress, need to unwind after work, etc.
  • Not all of us know anything about a given protocol, nor can we be reasonably expected to. For example, I don’t know how the MSN protocol works, and I have no desire to learn about it. We don’t have to be experts in a given protocol if we don’t want to be. Also, the fact that MSN is allegedly the most-used protocol in the world is not a valid reason for anyone to expect us all to be experts on (or even to care about) the protocol.
  • Pidgin largely just works for us. If we don’t see a bug, we’re not going to be irritated by it and sit down to try to fix it.
So yes, we have been “neglecting” Pidgin to a certain extent. Our time is a precious resource, and in many cases we simply have better things to do or simply don’t have the time to sit down and attack bugs to get them closed. That’s not to say that our development process is coming to an end or that we’re abandoning Pidgin, though. We do, however, readily admit that we need fresh blood.

In that vein, I’m calling for you, the Pidgin community, to help us. Take a look at the open bugs and feature requests and submit patches to resolve them. Pidgin is a fairly complex beast, and there’s a lot of code to maintain, but it is manageable if people work on small changes at a time (instead of completely rearranging things unnecessarily). Additionally, Pidgin uses a distributed version control system, monotone. Contributors who want to seriously work on Pidgin can use this to their advantage by creating their own branches in monotone. When contributions are ready, those contributors can submit the changes from their branches either by attaching patches to tickets on trac or by asking a Pidgin developer to pull and review the changes. At that point a developer can commit or merge the changes and push them on to the central repo at mtn.pidgin.im.

The only problem with the model of a developer pulling changes from a contributor is that in order for someone to pull revisions, the contributor must be able to run mtn in server mode, which isn’t always possible or feasible. I’m trying to come up with some custom lua code to help in this regard, to do something similar to what git and hg can do with emailing patches, but that’s going to take some time, as I need to learn some more of the internals of monotone first. But none of this should stop anyone from trying to contribute! We will still accept single one-off patches or sets of patches and series of patches from longer-term contributors, and using monotone to develop against is going to produce patches that are the easiest for all of us to work with.

So, yes, in short, Help Wanted and Needed!

Voice & Video

January 20, 2010 02:13 AM by Elliott Sales de Andrade

Sooo, Pidgin supports voice and video now.

It's only on XMPP (aka Google Talk), and only on Linux, but hopefully that will change soon. I hear it's a real pain to get all the farsight/gstreamer dependencies compiled for Windows right now.

And here's hoping we can get MSN support soon, too. I'm not too sure how well farsight supports it though.

The Quest Never Ends

December 25, 2009 06:09 PM by John Bailey

In my last blog post, I alluded to our quest to make Pidgin perfect (in our own eyes, at least). This is a quest that by its very definition can never end, because as we near "perfection," there will always be something else that crops up to demonstrate we haven't reached perfection quite yet. This "quest" can have elements of many different forms. A few months ago, one of those forms was discussed on our development mailing list.

As a good many people are aware, Pidgin has long supported GTK+ and GLib 2.0.0. Several years ago when GTK+ 2.0.0 first came out, we underwent a nine-month rewrite of our user interface (UI) to move from the older GTK+ 1.2.10 (or newer) to the new GTK+ 2.0.0. Since that time, until 2.6.0, we've always maintained compatibility with GTK+ 2.0.0 through the use of conditional code that disabled certain features or UI elements that required newer versions of GTK+ or GLib than what was available at compile time. In some cases, we had conditional use of code to work around the lack of certain convenience functions present in newer versions of GTK+ or GLib. In still other cases, we actually carried (that is, distributed in our source tarballs and compiled where necessary) the source of several GTK+ widgets in order to make our UI work for users with older versions of GTK+. Over the years, this has become more cumbersome, to the point that for 2.6.x, it became impractical to maintain compatibility with GTK+ 2.0.0--when we realized this, we changed our requirements to GTK+ and GLib 2.4.0, which contained the features we needed.

The increased difficulty of supporting older libraries prompted me to bring a discussion up on our development list prior to the release of 2.6.0 asking for a vote, discussion, etc. on raising minimum GLib and GTK+ version requirements for Pidgin 2.7.0. This discussion has come up before and been shot down. This time, I put it to a vote and gave quite a while for votes to be cast. There were no "No" votes cast. The versions we voted on were GLib 2.12.0 and GTK+ 2.10.0. These versions allow us to support reasonably recent Linux and UNIX systems, as those GTK+ and GLib versions were 3 years old at the time of the vote, while also making our lives significantly easier since we no longer have to care about really old versions of GTK+ or GLib. Ideally, I would have liked to have GLib 2.14.0 and GTK+ 2.12.0 as the minimums, but I was trying to reach a middle ground that would avoid angering too many people.

This change in the minimum required GTK+ and GLib versions has a few consequences. Obviously, Linux and UNIX distributions that don't ship GTK+ 2.10.0 or newer won't be able to support Pidgin 2.7.0 when we release it. On these systems, users can, however, compile GTK+, GLib, and friends, then compile Pidgin. This is really a lot of work, but some users may be willing to go through it.

For our Windows users, there will be some major changes. When building and linking our releases, we'll be stepping up to GLib 2.18.0 and GTK+ 2.14.0. We already ship this version or newer in our installers, but now we'll actually be linking against these versions. The consequences of this are that Windows NT 4.0, Windows 98, and Windows ME will no longer be supported. These newer versions of GTK+ and GLib require features that just aren't present in those old operating systems. None of these operating systems have been commercially supported (by this I mean modern games, word processors, spreadsheets, etc.) for years, and even projects like Firefox have stopped supporting them.

Additionally, our Windows expert, Daniel, has made some changes to the Pidgin installer and to our crash report generation for 2.7.0:
  • Instead of installing GTK+ in a system-wide configuration, we will be changing to installing GTK+ local to Pidgin. This means that it should be harder for us to conflict with other GTK+ applications on Windows. This has long been requested, but we just finally got around to doing it.
  • Debug symbols can now be read from parallel copies of files. Normal installations of Pidgin ship "stripped" binaries--that is, there is no information useful for generating crash reports. Now, instead of replacing all the existing files, the installer will offer the option to install debug symbols. Selecting this option will install parallel, unstripped copies of every file with the extension ".dbgsym" to a special location.
  • The installer will allow choosing which Pidgin translations to install.
  • The installer will have "online" and "offline" variants. The "online" variant will include only Pidgin. GTK+ and debug symbols will be downloaded as needed. The "offline" variant will include both GTK+ and the debug symbols.
Also among the changes coming up for Pidgin 2.7.0 is the switch from the eggtrayicon widget we have carried in our source tarball for years to the GtkStatusIcon implementation that was added in GTK+ 2.10.0. There are still a few bugs to work out with this change, but I believe once we have those ironed out, our notification area icon will behave better for a number of users who have been experiencing difficulties.

Of course, these changes aren't the only ones that will make it into 2.7.0. I have my own plans for a few additions, plus I won't allow 2.7.0 to be released without a few other features being merged in. But so far, it looks like Pidgin 2.7.0 will make some long-awaited forward strides. Hopefully everyone enjoys it!

Also, to those who celebrate the holiday, Merry Christmas!

gtkdoc for trac, FINALLY!

November 28, 2009 03:54 PM by Gary Kramlich

This has been on my todo list forever, and I started tinkering with it the other day. But after a mega hackathon tonight, I have a pretty functional gtkdoc plugin from trac.

The idea of the plugin is to integrate gtkdoc books into trac's interface. This is accomplished, currently, by and iframe displaying the actual htlm output of gtkdoc, as well as integrating into the search engine in trac.

There's two new user permissions in this plugin, GTKDOC_VIEW and GTKDOC_ADMIN. GTKDOC_VIEW is obviously to view the api and well as search it, where admin allows the addition and removal of books.

As mentioned above the admin interface provides an interface for adding and removing books. Books are keyed on a name given to them at the time they're added, as well as an absolute path on the local filesystem of where the book actually is. This way, you can update the documentation and not have to touch a thing in trac.

At any rate, it's still pretty rough around the edges and lacking a ton of error checking. But feel free to grab it from our extras repo (hg clone http://hg.guifications.org/extras) and tinker with it. It's in trac/gtkdoc, use the normal "python setup.py bdist_egg" to build it.

The Quest for Perfection

November 17, 2009 04:37 PM by John Bailey

Recently I've been frustrated by the fact that we have a number of tickets open on Pidgin's Trac that deal with inadequacies in the preferences window. The biggest complaint is that in a number of configurations, the preferences window is too tall to fit on a screen. This has only recently become a problem with the advent of the so-called "netbook" with their nearly microscopic screens (seriously, how do people use those things when they have such tiny screens? It drives me absolutely nuts when I try to use one).

Prior to the netbook craze, we've always aimed for all our windows, dialogs, etc. to fit on an 800x600 screen. With the shorter and wider screens found on netbooks, 800x600 isn't realistic anymore. In that vein, I've started working on paring down Pidgin's preferences window to fit better on a netbook screen. Let's take a look at what I've done so far.

The first, and most obvious, change I've made is to move the tabs that were previously at the top of the preferences window to the side. I did this for two reasons--in my environment, I have a Browser tab because I don't use Windows or GNOME. This meant the tabs artificially forced my window to be wider than it strictly needed to be (our preference window "notebook," as it's called in GTK+ parlance, doesn't scroll or stack the tab row). Second, moving the tabs from the top to the left gains back some valuable pixels that help us fit on those really short screens. In retrospect, I didn't gain much--only about 20 pixels or so in my configuration--with the width of the actual pages of the notebook because of other changes I made to help the height. But every pixel of height counts to make us fit on short screens.

After that, I started picking on the Network tab. We have a lot of stuff that's just crammed into the Network tab. FIrst we have a bunch of stuff for automatically detecting your public IP address or optionally manually specifying one. Then we have some stuff for port forwarding. After that comes stuff for TURN relay servers, which is yet another method for traversing NAT "routers." Finally, we have the wonderful proxy server stuff. All this stuff being crammed on a single tab makes the dialog ridiculously tall and makes the Network tab tied with the Conversations tab for biggest overall height.

Clearly, a few changes were in order here. I made a minor change here that moved the port range spin buttons to be on the same "line" as the checkbox that enables and disables them. Beyond that, there were still a lot of changes that could be beneficial. For example, now that I've made the notebook tabs go down the left side of the window, that gives us a lot of room to better organize our preferences into tabs that make logical sense. Since I had this extra tab room to work with, it made sense to add a new tab to put the proxy configuration options on. While a proxy server is obviously a network configuration item, it's not always obvious to our users that the Network tab is the correct place to configure the global proxy settings.

Now, looking at the Smiley Themes tab, there's a crapton of wasted space there. Seemed to me like a perfect candidate to be the target of some moves. Over on the Interface tab, we have two theme selectors--one for buddy list themes and the other for status icon themes. I decided those were a better fit on a dedicated Themes tab, so I renamed the Smiley Themes tab to Themes and slapped those theme selectors in above the smiley theme selector. I also excised the sound theme selector from the Sounds tab and slapped it into the newly-renamed Themes tab. Elliott, another Pidgin developer, converted the smiley theme selector to be consistent with the other theme selectors. Now the Themes tab doesn't seem under-utilized, but it does still need a little work.

I also decided to reorder the tabs. I ordered them alphabetically, but intentionally left the Interface tab as our first tab. It seems to be convention that the "General" tab is the first tab in a notebook for a preferences window. In that vein, our Interface tab is our equivalent to a General tab, so I left it first to fit with convention. The order of the other tabs should be something logical. Sure, I could have grouped Browser, Network, and Proxy together, then grouped Conversations and Logging together, and later figured out where to stuff Sounds and Status/Idle, but an alphabetical arrangement is just as sensible considering that not all our tabs clearly separate into logical groups.

The next things to get rid of were the "Sound Method" section on the Sounds tab and the Auto-away section on the Status/Idle tab. Note here that I didn't actually remove any preferences; I simply removed a section and placed the relevant preferences in another section. The existing groupings seemed redundant and excessive. This seems better to me.

There, was, however, one thing I needed to give the axe to. Over on the Conversations tab, there was this annoying preference section called "Font" that contained two preferences designed to allow users to override the GTK+ theme settings, but only for the conversation history pane. This preference serves very little purpose anywhere but on Windows, so I removed it. I then made sure the Pidgin GTK+ Theme Control plugin could control the history area font. There were some objections about unconditionally removing this preference, so I compromised and made it available only on Windows, even though I fail to see why the plugin isn't sufficient to control this font.

In the end, I believe I accomplished my initial goal of making the preferences window fit on smaller screens, but I also managed to make it so the dialog still fits on an 800x600 screen. The new window measures 698 pixels wide and 492 pixels high in my configuration, which is a bit too big for 640x480, but will fit with plenty of room to spare on 800x600 and should fit pretty well on just about any netbook screen.

You can take a look at what the new window will look like with these pictures (sorry, but the order is backwards and I'm too lazy to fix it):










This will be in Pidgin 2.6.4 when we release it. Hopefully everyone enjoys the changes!

Google Summer of Code Mentor Summit 2009

October 29, 2009 09:37 AM by Mark Doliner

Last weekend Google hosted their 3rd annual mentor summit, following the end of their 4th annual summer of code. The mentor summit is when a few hundred mentors gather together and participate in an unconference style conference. I went for the Pidgin project, along with Gary Kramlich and Ethan Blanton.

The conference was super cool. I got to be humbled by talking to a whole bunch of really smart open source people. Here are my notes:

On One Laptop Per Child (“OLPC”)

I’ve been wondering for a while whether the OLPC program could actually make a difference. One session, led by Bryan Berry of Sugar Labs, makes me think that it can and already has. Bryan is the co-founder and CTO of OLE Nepal, an organization helping deploy OLPC in Nepal, and creator of Karma, a framework for creating interactive activities for the Sugar environment using javascript and html5.

Seeing demos of the exercises they’ve created and hearing his first hand stories was pretty incredible. At least some schools in Nepal teach by having the teacher recite something (e.g. “one plus two is three”), and all the students repeat it and memorize. But this often fails to teach the students why one plus two is three. In one example a student was asked “what is one plus two” and they replied with “three.” But the same student was not able to answer “what is two plus one.”

Children in third world countries generally want to learn–more so than children in the US. They realize that education can help them achieve something greater in life. And computers are interesting to them. Combine students, computers, and engaging lesson plans about math, geography, etc. and the students will have a more varied education and will learn better.

On Forking Open Source Projects

  • Forking helps keep people motivated. It increases competition, keeps developers on their toes.
  • A fork could be like a “research and development” branch. People work on crazy fun new features in the forked project, and the good stuff gets merged back into the original.
  • The smaller the project, the more willing the maintainer should be to give people access. There is a natural inclination to be protective of your project–it’s your code, your baby. But you must be willing to give up control for there to be forward progress. This reminds me of dictator governments like Cuba/Fidel Castro and North Korea/Kim Jong-Il. The dictator is afraid to relinquish control for fear of what might happen.
  • Benefits of a fork? Developers have more freedom to do what they want, which allows for innovation. The best project will survive–if developers want their project to survive then they must make decisions that benefit the community at large.
  • Downsides of a fork? Development effort is divided. Users might not know which project to use. Distributors may not know which package to distribute; distributing both means more work.

Miscellaneous

  • STUN – A protocol used to determine your public IP by asking a server on the “outside” Internet
  • TURN – A protocol used to proxy traffic through an intermediate server. Written with SIP in mind. Increases the likelihood of being able to establish a connection to another party, but it also introduces an additional hop, which leads to lagginess, which is bad for voice/video communication.
  • ICE – A protocol that describes a method for establishing a direct connection with another peer. Written with SIP in mind. It uses an exhaustive algorithm to try every possible IP address for yourself in the hopes that one will work. You construct a list of your host’s IP addresses plus your public IP address determined by using STUN. This information, along with a fallback TURN server, is sent to the other party, who begins attempting to connect.
  • OpenAFS is under active development, and is used by some very large organizations
  • I should change my alias for grep to enable the color option
  • I should read Zen and the Art of Motorcycle Maintenance

On Trolling (this session was half intended as a joke)

  • I should read the UNIX-HATERS Handbook
  • I should read the Sokal paper
  • “Linus==troll”
  • “Version con-trolling”
  • “We had this problem where people had to download our software and type ‘make'” –Marty Connor
  • Adding support for the old school Unix talk command to Pidgin could be a fun April Fools joke

Pidgin and the Incredible Shrinking (Finally!)--And Growing‽--About Box

October 04, 2009 12:02 AM by John Bailey

Have you ever taken a look at the text in Pidgin's About box? If not, do so now. Click the Help menu, then click "About." Be prepared to scroll. And scroll. And scroll some more. And again. Until finally you reach the end of the thing and see a crapton of build-related information that probably doesn't make a whole lot of sense. Along the way, you'll scroll through a list of our developers and crazy patch writers, then through a list of retired developers and retired crazy patch writers, then through translators and retired translators. The list of translators, especially, seems to go on forever. The amount of information in that box has grown nearly exponentially in the last couple years.

Get tired of scrolling through all that information? Me too. And apparently a bunch of other people--I've seen a number of complaints from people involved with several Linux distributions indicating that our about box has way too much information. Well, this morning, I set out to remedy that.

Remember that insanely long list of current and retired translators? Well, when we release Pidgin 2.7.0, it will no longer be in the About box. To excise this information from the About box, I created a new entry on the Help menu called "Translator Information" that pops up a new dialog similar to the About box. This new dialog lists all current and retired translators.

Once the translators were gone from the About box, the next largest chunk of text was the "Debugging Information" section all the way at the bottom. Considering how frequently we ask users to look at this information, it is nowhere near visible enough sitting at the bottom of the About box's scrollable area. This, too, became its own window, accessible at "Build Information" on the Help menu. (As an interesting side note, this is insanely long in code, too. In fact, it's more lines of code than the printed list of translators takes up in the new translator info dialog!)

Even losing all that text from the dialog wasn't good enough for me. Next I axed the list of developers, crazy patch writers, and retired developers and crazy patch writers. Now these lists are available by clicking "Developer Information" on the Help menu. Since I'd just added three items to the Help menu, it was time to toss a couple menu separators in there so that things look more organized and cleaner. Hey, our Help menu is kinda respectable now!

Now I looked at the about box again. It still felt too verbose to me. Granted, a lot of that information is important, but I started thinking about better ways to say the same things. For a great example, let's look at the first two blobs of text you see after the version info:
Pidgin is a graphical modular messaging client based on libpurple which is capable of connecting to AIM, MSN, Yahoo!, XMPP, ICQ, IRC, SILC, SIP/SIMPLE, Novell GroupWise, Lotus Sametime, Bonjour, Zephyr, MySpaceIM, Gadu-Gadu, and QQ all at once. It is written using GTK+.

You may modify and redistribute the program under the terms of the GPL (version 2 or later). A copy of the GPL is contained in the 'COPYING' file distributed with Pidgin. Pidgin is copyrighted by its contributors. See the 'COPYRIGHT' file for the complete list of contributors. We provide no warranty for this program.

All this information is hugely important. But it's too long. So I tried a surgical strike on the words. It took a few more tries than I care to admit, but I found that I liked this text better:

Pidgin is a messaging client based on libpurple which is capable of connecting to multiple messaging services at once. Pidgin is written in C using GTK+. Pidgin is released, and may be modified and redistributed, under the terms of the GPL version 2 (or later). A copy of the GPL is distributed with Pidgin. Pidgin is copyrighted by its contributors, a list of whom is also distributed with Pidgin. There is no warranty for Pidgin.
Obviously I took away some information, namely which IM services Pidgin can connect to. I'm not convinced that needs to be in the about box, considering the exact same text that is in Pidgin 2.6.2's About box is on Pidgin's website. So I carefully dissected the text and came up with a shorter text that gets all the important information across while cutting information most users aren't going to care about reading. I even managed to squeeze in that Pidgin is written in C, which was previously missing (it's a ridiculously frequently asked question).

Now I had something better. But I could still improve it. We had oversized text pointing to pidgin.im, the FAQ, the IRC channel, and our XMPP conference. I added a new, bold heading, "Helpful Resources," and added those items as indented, normal-sized text items under the heading. This was getting better. Now, one final improvement. We point out that help from other Pidgin users is available by emailing support@pidgin.im. I changed the heading a little so that it reads a bit better. I also changed "3rd party" to "third-party," which is the more correct form.

Something still seemed wrong about the About box, though. I experimented with the size, making it default to 450x450 pixels like the new dialogs I added. This seemed to help a lot. I'd like to make it a touch larger still, but I'm afraid that making it any larger will make it a tight fit on netbook screens. We're already getting complaints that some of our windows don't fit on these smaller screens, so I'm not exactly eager to cause more complaints.

At any rate, for me, the About box's text fits in just under two full "pages" of the scrollable area--that is, the last line of the scrollable area when scrolled all the way to the top is the first line when scrolled all the way to the bottom. There are some other minor tweaks I'm going to look at making that may make the area scroll even less.

In the end, my boredom caused me to take a look at something people have been complaining about for ages. Overall, I think what I've done is a good thing, but we won't really know for sure until we release Pidgin 2.7.0 (don't ask when--we have no idea yet!) and people see the new dialogs. I hope everyone likes what I did!

Pidgin 2.6.0--It's About Time

August 19, 2009 05:40 AM by John Bailey

Well, by now most people probably realize that we've released Pidgin 2.6.0. It feels like this has been in the works forever, particularly these last couple weeks.

First off, some statistics for this release:
  • 99 bullet points in the ChangeLog.
  • 221 tickets closed for the release (that is, 221 tickets that we believe are fixed or are patches that we accepted).
  • 2 major new features
  • More other new features than I care to count
For the new features:

Voice and Video support - Thanks to Mike Ruprecht and his Summer of Code project from 2008, libpurple now has a voice and video framework that can be used to add these features to our protocol plugins. Currently we support these features only on XMPP, but Mike is working on other protocols as I write this and hopes to have more protocols at least partially supported soon. The dependencies are a bit of a mess for the uninitiated, but unfortunately that's unavoidable. I'm hoping most distributions will be able to catch up with this soon and make it completely effortless for users, but this is a headache even for some distributions. The biggest setback thus far is we're currently not able to support these features on Windows--but we're working on it! Please be patient!

Theme support - Another Summer of Code project from 2008, this time by Justin Rodriguez, adds theming support to libpurple and Pidgin. This currently isn't very well documented at all, but themes are now supported for the buddy list, sounds, and status icons.

Yahoo users will notice a few changes. First and foremost, we split the Yahoo protocol plugin into two, one to handle the Yahoo JAPAN network and one to handle the rest of the world's Yahoo network. This has the side effect that if you happen to have the exact same account registered on both networks, you'll finally be able to use both accounts in Pidgin. It's also a lot more obvious to people looking to use their Yahoo JAPAN accounts in Pidgin. Sulabh Mahajan, another Summer of Code student from 2008, implemented a ton of new stuff for Yahoo and Yahoo JAPAN. Among the changes are the addition of SMS support. You can now send SMS messages by sending to "+<country code><phone number>". Sulabh also implemented peer-to-peer file transfers for Yahoo as well as adding MSN buddies to the buddy list of a Yahoo account. Unfortunately, proper support of adding MSN buddies isn't possible to do until 3.0.0 when we can make some major changes to the internal workings, but for now, if you want to add an MSN buddy to a Yahoo account, add them as "msn/foo@bar.tld". The "msn/" is the important part--this tells our Yahoo code to look across the MSN bridge to add the buddy.

On top of all this, our developers, crazy patch writers, and contributors have been pouring a ton of work into our XMPP support. Beyond the voice and video support, we've gained a service discovery ("disco" for those familiar with the term) browser plugin, support for BOSH (Bytestreams Over Synchronous HTTP), idle time reporting (XEP-0256), attention ("buzzing") support (XEP-0224), in-band bytestreams file transfer as a last-resort transfer method (XEP-0047), custom smiley support in small (less than 10 users) MUC's via the "bits of binary" extension, as well as updated support for buddy icons (User Avatar XEP-0084 v1.1). There have also been a ton of bug fixes and other enhancements. All this adds up to 29 bullet points in the changelog for XMPP alone, and even that is surely not 100% complete.

Other notable items include our new (optional) support for GNU libidn, allowing us to support UTF-8 domain names throughout all of libpurple, three new environment variables that can help in debugging (and thus possibly help some plugin authors as well), a new authentication mechanism for AIM implemented at AOL's request, the ability to receive voice clips and handwritten (ink) messages on MSN, and a crapton of fixes and enhancements in Pidgin. Even Finch got some love this time around, gaining a new TinyURL plugin and some important bug fixes.

Of course, I'd be neglecting important details if I didn't mention the security issue we fixed for this release, as well as the 2.5.9 release. CORE Security Technologies found a way to remotely crash a running Pidgin instance that was logged into an MSN account via two specially crafted messages. They were kind and responsible enough to inform us of this privately and provide us with a proof of concept script so we could fix the problem before they made it public. The release of Pidgin 2.5.9 was done in source form only, explicitly to provide distribution packagers with a fixed release in the event they preferred to avoid the behemoth release that is 2.6.0.

Of course, since I was heavily involved in the creation of the 2.6.0 release, we have some issues that we're going to need to follow up on shortly with a 2.6.1. I'm sorry for any inconvenience this causes anyone, but hopefully 2.6.1's release will make up for it by being what 2.6.0 should have been. At any rate, enjoy all the shiny new features!

GTK+ 3.0 for Pidgin

August 05, 2009 05:15 AM by Elliott Sales de Andrade

A little while ago, grim posted a link about preparations for GTK+ 3.0. I started a bit of work on it in a separate branch.

So far, it hasn't been too much trouble. I managed to fix single includes in the main branch, and all deprecated functions in libpurple core too. Finch was super-easy and required almost no work. What was left was the Pidgin UI.

The main Pidgin UI required a bit more work. Replacing GtkOptionMenu with GtkComboBox was pretty simple, just a couple preprocessor checks needed. Using GtkTooltip instead of GtkTooltips was also pretty painless, but there's still one matter left. The headline text that goes in the buddy list for mail notification used to do a hack with the tooltip to colour it correctly. At the moment, it ends up grey because I'm not sure how to get it working.

There are still a couple big changes left. I still really don't know how to write UI Manager stuff, and it's hard to find good examples. It only has to be done twice, for the buddy list and for the conversation window, but I'm not sure how much that really means. Fortunately, we are not heavily GNOME dependant, so that's probably the biggest change.

Why I work on open source

July 06, 2009 05:52 AM by Mark Doliner

I haven’t written about this before, have I?

I think the world is better off with free software. I don’t have anything against closed source or non-free software, I just think typical development processes for free software produce better products in the long run. They produce something that meets the needs of users better, with less fluff.

And I guess I feel like I can have a positive impact on open source software. Working on Pidgin is like my way of giving back to the authors of all the other free software that I use.

And I take a lot of pride in the code that I write. It is a reflection of who I am. If I write something that’s buggy then it makes me look bad. So you don’t need to try to talk me into fixing something that I wrote, because I care regardless. I care a lot more than you do, believe me. And it pains me when I don’t have time to fix my bugs.

Pidgin 2.5.8 and Other Ramblings

July 02, 2009 05:01 AM by John Bailey

Well, Casey already pointed out that we released Pidgin 2.5.8, but I thought I would expand on that a bit.

As everyone reading this knows already, Yahoo was broken for us briefly. We rushed out a 2.5.7 release to address the problem and after release discovered that we had a number of other problems. Among these were broken file transfers, broken buddy icons, etc.

We were receiving almost as many complaints about buddies never changing to show as offline as we got about the original Yahoo connection problem. Quite frankly, I got tired of it, so I started grabbing additional changes that we had committed to the upcoming 2.6.0 and applying them to 2.5.7. Eventually this yielded 2.5.8, which turned out to take a lot longer than I expected to get released.

As the ChangeLog indicates, we fixed a bunch of stuff, including an ICQ crash, an MSN crash for users with long buddy lists, a Yahoo crash introduced in 2.5.7, as well as receiving messages from the web version of MySpace IM and signing on to MySpace IM if you have an empty buddy list.

If you're not using Pidgin 2.5.8, upgrade!

We're also busily at work on Pidgin 2.6.0. In recent weeks we've come down to just a few things holding up the release. Among the blockers are the merging of some new features and fixing a new crash that we introduced by fixing ther crashes. At this rate, we might actually get 2.6.0 out sometime this year!

On an unrelated side note, I was recruited to participate in a charity fundraiser for the Muscular Distrophy Association. If anyone would like to help me reach my fundraising goal, I'd appreciate it.

Some Clarification on Yahoo! Issues

June 24, 2009 06:42 AM by John Bailey

From the recent activity on our support mailing list and in our IRC channel (#pidgin on irc.freenode.net), it's plain to see there is still a LOT of confusion about the recent troubles logging into Yahoo Messenger via Pidgin. I'm going to try to clear some of the confusion up. I apologize for the technical nature of this post. Feel free to skip ahead to the last paragraph of "Solving the Problem" below if you don't care what the problem is and just want to fix it. Or take the simple approach and just upgrade to Pidgin 2.5.7 and skip down to "But I already upgraded! It still doesn't work!" if you still have trouble.

The Problem

The specific problem that affected us is the authentication mechanism. Yahoo Messenger 6 used a spectacularly complicated password obfuscation method to "encrypt" the password as it was being sent over the wire to Yahoo's servers. Back in 2004 when we were preparing our 0.79 release, Cerulean Studios, the creators of the Trillian client, were kind enough to implement this authentication mechanism for us. As part of this change, we began to identify to the Yahoo servers as Yahoo Messenger 6. This was all fine and well.

The real problem came relatively recently. At some point in our recent history, which I honestly don't feel like tracking down now, we started identifying as Yahoo Messenger 7. At some point after that we again updated to identify as Yahoo Messenger 8. Both of these changes were related to file transfer and other feature enhancements. When we made these changes, however, we never updated our authentication code. This means we were claiming to speak Yahoo Protocol version 15 but we authenticated the same way protocol version 13 clients do. Even this worked for quite some time.

Yahoo began upgrading their servers at some point recently to phase out the old Yahoo Messenger 6 client. In effect, they want to force users of older software to update to the current client. Whether this is a good thing or not depends on your perspective. From Yahoo's point of view, I'm sure this is an excellent move, as it should make their server software simpler (speak one less protocol, one less auth scheme, etc.) and reduce their support load (fewer client versions to deal with). Where it became a problem for us is that at the same time, they started requiring protocol version 15 clients to speak the version 15 authentication scheme, which we never implemented. Since we still spoke version 13's authentication, this cut us off entirely.

[Note: Some users have discovered that not all of the Yahoo Messenger servers are rejecting the old authentication mechanism from Pidgin. The number of servers that still accept it appears to be shrinking, however. We advise upgrading to Pidgin 2.5.7 as soon as possible.]

Solving the Problem

A few months ago, a kind soul pointed out to us documentation he had assembled on how the current Yahoo Messenger client authenticated and communicated with the servers. It turns out that the convoluted authentication mechanism we had used for years was replaced with a significantly simpler one that uses a few simple HTTPS requests. The beauty here is the login data is encrypted in the same manner as your communications are encrypted whenever you purchase merchandise at a retailer like Amazon. This makes it significantly simpler for us, and probably allows Yahoo to simplify things on their end as well while increasing security during the login process for all their users.

As I mentioned in my previous post, two of our Summer of Code students, Sulabh Mahajan and Mike Ruprecht (keep an eye out for these names--they're responsible for chunks of Pidgin 2.6.0's changelog!), implemented this new authentication scheme. We eventually merged it into what will become Pidgin 2.6.0. This code has had some quality testing already, thanks to the guys over at Adium.

In response to the Yahoo server changes, I (quickly) yanked the most important changes for this new authentication scheme and slapped them onto Pidgin 2.5.6. After some testing and some other important bug fixes, we kicked Pidgin 2.5.7 out the door. In theory, upgrading to Pidgin 2.5.7 should make most people's Yahoo problems go away.

But I already upgraded! It still doesn't work!

Yeah, we have had a few people with this problem. Here are a few suggestions:
  • Change your Yahoo account's Pager Server field (located on the Advanced tab when editing the account) to 'scsa.msg.yahoo.com', especially if you previously read this document
  • If you're crashing when you connect your Yahoo account, again, set your Pager Server to 'scsa.msg.yahoo.com' and try to log in again. If you can't edit your account, you have two options:
    • Run Pidgin from a shell (command prompt). 'pidgin -n' will cause Pidgin to start up in an Offline status, thus allowing you to edit your accounts. Set the Pager Server as described above.
    • If you know the name of the server that you entered in the Pager Server field, you can change this manually. Locate your .purple directory (covered in the FAQ!). In this directory is a file called accounts.xml. Open this file with an editor (Wordpad on Windows; any text editor will do on Linux and other Unix-like systems) and find the server name you entered. Change this value and save the file. Start Pidgin and you should be fine.
  • If you upgraded by compiling Pidgin yourself, make sure you removed your distribution's existing libpurple package (for Ubuntu and Debian users this is libpurple0; for RedHat/Fedora users it should be libpurple). Then run 'sudo ldconfig' (or 'ldconfig' if you have a root shell already) and try again.
  • If you're behind a proxy server that proxies HTTP connections but not HTTPS connections, and you have to explicitly configure the proxy in Pidgin, you're probably not going to have any luck. Our current proxy code can't handle this kind of configuration. Sorry.
  • If you're on Windows and you get errors about a corrupted installer, clear your browser's cache, then restart your browser and try to download again.
But I can't upgrade!

If you really can't upgrade, then try setting the pager server to either 'scsc.msg.yahoo.com' or 'cs101.msg.mud.yahoo.com' and try again. But you really, really, really, really should upgrade. If you do eventually upgrade, you'll have to change this again, as I describe above.

I upgraded but now ______________ doesn't work!

We've had some reports that after upgrading there are some issues such as missing buddy icons and buddies never being shown as signed off. We are looking into these issues and hope to have them fixed for Pidgin 2.6.0. These issues didn't show up in my testing before the 2.5.7 release.

Tracking Yahoo Problems

Please don't open any new tickets about these issues or about not being able to log in. There are already open tickets to track them. Please also don't comment with "Me too!" or similar on these tickets--it just creates more noise for us to sort through when we're working on Pidgin. If you want to express a "Me too" on one of the tickets, click the arrow near the top of the ticket page that's pointing up. This will cast a vote for the ticket.

I hope this information is helpful to people.

The Great Yahoo Debacle

June 21, 2009 03:32 AM by John Bailey

By now it's no great secret that Yahoo has decided to break clients that use outdated authentication code. The result of that change is that Pidgin 2.5.6 and older no longer work unless you connect to one of a very few servers left that still speak the old authentication mechanism. We knew this change was upcoming, but all indications we saw pointed to us having almost another two months to push a release with updated authentication code.

All that aside, we've had some updated authentication code for a while now, and were originally waiting to include it in the Pidgin 2.6.0 release. However, since Yahoo made the authentication changes earlier than we expected, we had to react as quickly as possible to stop the flow of incoming complaints. As a result, we backported the Yahoo authentication changes so we could cut a Pidgin 2.5.7 release fairly quickly.

Pidgin 2.5.7 is now released and does work to connect to Yahoo.

Just for the record, I'd also like to point out to those who complained about waiting three days to get a working release that the last time Yahoo screwed with authentication, it took over a week to even get code to fix the problem, let alone prepare a release and actually get it out to users.

Additionally, we fixed the annoying MSN disconnect-on-block issue and changed AIM's behavior such that the "could not retrieve your buddy list" error appears only once per AIM account.

Enjoy the new release!

Random Monotone Hacks

May 29, 2009 01:26 AM by John Bailey

I imagine most people who bother to read this blog know that I am rather fond of Monotone, the distributed version control system (a.k.a. DVCS) that we use to maintain Pidgin's code. One of the most convenient features of monotone is that it's extensible via the Lua scripting language. Over our time using monotone, some of us have come across (or written!) some useful snippets of lua code that can be tossed into ~/.monotone/monotonerc to make things easier on us.

The first such snippet is pretty simple. I have three monotone keys that I use for various purposes. One is for Pidgin development, another is for my work on the Guifications project and the Purple Plugin Pack. The third is for my private projects. Instead of needing to remember to specify which key I want to use in specific circumstances, a small snippet of lua can take care of this for me (of course, anyone wanting to use this will need to edit branch patterns and key names appropriately):

function get_branch_key (branch)
d = { ["im.pidgin"]="rekkanoryo@pidgin.im",
["org.guifications"]="rekkanoryo@guifications.org",
["org.rekkanoryo"]="rekkanoryo@rekkanoryo.org"
}

for k, v in pairs(d) do
if string.find(branch, k) then
return v
end
end
end


Monotone also has the ability to cherry-pick revisions from one branch to transplant into another. (Of course, I recognize that other DVCS tools have this capability too.) To do this, ordinarily we would issue a command like mtn pluck some_revision_id within a working copy of the branch we want to transplant the revision to. In our experience, the default log messages for a pluck leave some to be desired. My fellow developer Sadrul wrote this cool bit to add a pluck-log command to Monotone.

The pluck-log command takes a series of individual revision ID's as arguments. For each revision ID in the list of arguments, the command will execute mtn pluck $REV to grab and apply the changes to the workspace. Additionally, the plucked revision's original changelog entry will be inserted into the changelog for the new commit on the current branch. This is particularly useful if you're creating a new release branch by branching from a previous tag, then grabbing individual revisions from your main development branch. This command became popular among us Pidgin developers while we were preparing the Pidgin 2.5.6 release.

Here's the code:

--[[
Pluck a revision with the log and authors filled in for the next commit log.

@author: Sadrul Habib Chowdhury (updated for mtn 0.43 by John Bailey)
--]]

-- pluck-log command
function pluck_log(...)
local revs = {...}
local result
local topsrcdir
local logfile
local log

-- mtn_automate() returns a pair of a boolean and a string; we don't really
-- care about the boolean here, but we need to do something with it.
result, topsrcdir = mtn_automate("get_workspace_root")
topsrcdir = string.gsub(topsrcdir, "\n", "")
logfile = io.open(topsrcdir .. "/_MTN/log", "r")
log = ""

if logfile then
log = logfile:read("*all")
logfile:close()
end

table.foreach(revs,
function (index, rev)
r, sel = mtn_automate("select", rev)

if r == false then return end

for rev in sel:gmatch("%S+") do
r, certs = mtn_automate("certs", rev)

certs:gsub("%s+key \"(.-)\"\n%s*signature \"(.-)\"\n%s*name \"(.-)\"\n%s*value \"(.-)\"\n%s*trust \"(.-)\"",
function(key, sig, name, value, trust)
if name == "changelog" then
log = log .. "*** Plucked rev " .. rev .. " (" .. key .. "):\n" .. value .. "\n"
end
end
)
execute("mtn", "pluck", "-r", rev)
end
end
)

logfile = io.open(topsrcdir .. "/_MTN/log", "w")
logfile:write(log)
logfile:close()
end

register_command("pluck-log", "REVISION1 [REVISION2 [...]]", "Pluck a revision with a good log",
"This plucks a list of revisions, each individually, and adds the changelog of each revision for the next commit log." ..
"\nEXAMPLE:\tmtn pluck-log h:im.pidgin.pidgin deadbeef\n",
"pluck_log")


The resulting log entry will look something like this:

*** Plucked rev 074c5aedf9bbc512331f0d3130f076190b290676 (rekkanoryo@pidgin.im):
Set the default pager host to scsa.msg.yahoo.com; this seems to be what the
official client uses.


Originally, Sadrul wrote this code for mtn 0.42 and earlier, which did not have the mtn automate get_workspace_root functionality. I updated the code to call mtn_automate("get_workspace_root") instead of finding the workspace's root with successive checks of parent directories. The revision ID printed in the log was also truncated to the first 8 digits. Ordinarily, this is fine; however, it's possible that in the future a new revision will be created that has the same first 8 digits and thus the short ID will be ambiguous. I wanted to avoid this situation, so I removed the truncation. I also rearranged a few things to make it easier for me to read. I also have to credit my fellow Pidgin developer Elliott with figuring out that I needed to kill the trailing newline in the string returned from mtn_automate("get_workspace_root").

Hopefully we're not the only ones who find this stuff useful. As I make frequent use of other useful monotone lua hacks, I'll probably post about them here too.

Pidgin 2.5.6

May 29, 2009 12:14 AM by John Bailey

Well, as most people have noticed by now, we recently released Pidgin 2.5.6. This release was simply a bug and security fix release. Hopefully it will hold everyone over until we can kick 2.6.0 out the door!

Linux Journal Readers' Choice Award

May 02, 2009 04:49 PM by John Bailey

Yesterday, Linux Journal issued a press release announcing the winners of its annual Readers' Choice Awards. It's an annual event that attracts a lot of attention in the magazine. In their effors to "take the pulse of the Linux community," they run this poll and announce the winners and grant honorable mentions to "strong contenders" in the pool of runners-up.

This year, it's been announced that Pidgin has won a Readers' Choice award for Favorite Communications Tool, a category that we're no stranger to winning--we won this same category last year as well. The landscape is similar as well. This year, we won with 42% of the votes, beating out honorable mention Skype, which received 18% of the votes. Last year, we also garnered 42% of the votes, with Skype and Kopete earning honorable mentions at 17.8% and 12.8% respectively.

Winning this award for the second year in a row reminds us that our work is quite well appreciated, something that is often easy to forget. It also reminds us that every once in a while we need to stand back and thank everyone involved in making Pidgin such a popular project. In this case, we need to thank the 2,000+ people who voted for Pidgin in these Readers' Choice Awards, as well as all our developers, Crazy Patch Writers, drive-by contributors who spot a simple bug and fix it, and our users. All of these people in our community come together to make Pidgin a success as a project, and while we'd be just as happy to work on Pidgin if we had only a few hundred users, we certainly appreciate all of the contributions to our success. Thanks!

And the Summer of Code is set!

April 21, 2009 06:32 AM by John Bailey

Well, Google finally announced the list of accepted students for this year's Summer of Code. You can read about our accepted proposals here. Of course, we're all very excited about the projects we've chosen for this year. We have two students returning to us this year, both with projects that will be hugely important moving forward. Those students are Eric Polino and Sulabh Mahajan.

Eric has chosen to work on GObjectification, which will move libpurple from using our own object system to taking advantage of the GObject capabilities of GLib, one of our existing dependencies. As a result of this project, a massive amount of libpurple infrastructure will change, allowing us to leverage a tried and true typing system and giving plugins an incredible amount of increased flexibility by allowing them to subclass objects and providing interfaces they can implement. This will, of course, have huge rammifications for everyone using libpurple--us, plugin authors, Adium's authors, and even the Instantbird authors. We're confident the result will be worth the effort, though.

Sulabh has volunteered to take on the Privacy Rewrite. What makes this such a daunting task is that the protocols we support provide a very wide range of privacy options, some of which are unique. For example, ICQ supports an "invisible" list and a "visible" list in addition to a "block" list and probably a number of other things that I'm forgetting. Sulabh will have to look at what all our protocols support and provide a foundation in libpurple for all the protocol plugins to provide the privacy options their protocol allows for. Additionally, he'll have to smooth over the differences between protocols to make things consistent.

In addition to these projects, we've accepted two proposals for native Windows user interfaces. Each project will take a different approach to achieve the same goal--providing a user interface that fits better into a Windows desktop than Pidgin currently does. I've already seen a few complaints that we accepted two projects for this. Quite frankly, I don't understand the complaints. We chose two projects for two reasons--first, we want to have the highest possible chance to better serve our Windows users, and second, it's entirely possible that a single Windows UI project won't be able to do enough to satisfy all our users. Having two Windows UIs will allow each to focus on whichever set of users it is more appropriate for.

We've also accepted projects that will make XMPP transports based on libpurple, allow libpurple to use telepathy protocol support in addition to or instead of our own, and using webkit for our message displays. Each of these projects is going to be challenging in its own right. Of all the projects, the XMPP transports will certainly be the least user-visible, as the transports will run on an XMPP server. Conversely, the most visible will probably be the webkit message view, as this will completely replace what we currently use to display messages. By using webkit, it's possible that we'll eventually be able to support Adium Message Styles (no, I'm not promising this support will happen, but it will be possible). The Telepathy plugin will provide some interesting functionality by allowing us to use nothing but telepathy for protocol support if the user so chooses.

There are other deeper details in these projects, but I've already wasted too much time obsessing over this post and discussing the larger projects in detail. I'll leave further discussion of the technical merits and details of projects to the students and mentors. Needless to say, this post could have gone on just about forever. ;)

Overall, this Summer of Code looks like it will be one of our best. Good luck to all our students!

It's that time again...

March 27, 2009 03:53 AM by John Bailey

Well, once again we've come to the student application phase of Google's Summer of Code program. We're several days into the process, and we've seen a marked decrease in our intake of student applications compared to prior years. Why is anybody's guess, but I'm personally hoping it means that potential applicants are spending more time polishing their applications before submitting.

Based on the applications we've seen so far, I'd like to publicize a few notes for applicants:
  • Provide a detailed timeline for your project. "I expect to complete work in 10 weeks" doesn't cut it here. Tell us what milestones you expect to achieve and how far along. You could instead estimate how long a specific goal will take to implement. Yes, your estimate could ultimately prove wrong, but that's not necessarily the end of the world. We really want to see that you have a concept of time management and prioritization.
  • Don't plagiarize. If we know you're plagiarizing, we'll invalidate your application. If you can't deliver an application without plagiarizing, you shouldn't be applying. We don't mind if you use some of the text on our ideas page to help with the abstract, but keep in mind that a serious application won't rely on simply copying and reformatting what we've already said.
  • Don't be afraid to come up with a unique idea! The ideas page are not the only ideas we'll entertain. We love to see well thought-out, original ideas, especially ones that make us wonder why no one proposed it before.
  • Make sure your proposal is for a specific project idea. A general "I want to work with Pidgin for the summer" is a sure ticket to the reject bin.
  • You don't have to write an encyclopedia for the application, but you do have to give us something to work with. There's an old adage that "less is more." Sometimes that's true. Sometimes the opposite is true. The point here is that you should be verbose enough to explain your idea, but don't ramble. I know this can be difficult to judge. Just re-read the application, make sure it makes sense, and make sure it doesn't drag on needlessly.
  • You have to apply using the SoC webapp, not our mailing lists. We have to have applications submitted via the SoC webapp in order to review them and have them in the running for our student slots. This is a Google thing, and it makes everyone's lives a lot easier.
  • The application isn't final until submissions close. We will provide feedback on your application if it needs work. You can modify your application to address our feedback until applications close. Use this to your advantage!
  • We don't yet know who will mentor any given project. We don't assign final mentors and backup mentors to projects until we have decided which projects we want to accept. Asking us about this is just wasting your time.
  • We expect you to treat the project as a full-time job. That means at least 35 hours per week. We want to make sure everyone gets the most out of Google's money, and this is one way to do that. Since you'll technically be a contract worker, we expect the contracted work (your project) to take priority.
  • We expect you to remain actively involved with us after the Summer of Code finishes. Quite frankly, if you're going to collect the checks and run, you're missing the entire point. The Summer of Code is intended to attract new contributors and get them involved in open source software, turning them into long-term contributors. Participating in the Summer of Code and then disappearing isn't serving that intention, and leaving us with code we have to maintain ourselves is quite rude. If that's your intention, please don't apply. Let the potential for acceptance go to someone who will stay with us.
Keep these in mind, and happy applying!

Saved Statuses for Fun and Profit

February 28, 2009 04:50 AM by John Bailey

From my limited experience with Pidgin users among my family and coworkers, I've discovered that a number of users don't know about Pidgin's saved status features and instead use only the transient statuses created directly in the status selector by typing a message. Some users don't even realize that they can select a previously used transient status from the middle section of the status selector's menu. As an attempt to spread deeper insight into Pidgin, I submit for your reading this saved status tutorial.

Overview of Saved Statuses

In Pidgin, a saved status gives you a considerable amount of flexibility. For example, if you use six accounts, three of which are personal and three of which are for work, you can use a saved status to have your work accounts away while your personal accounts are available and vice-versa. Or, if the text box you get from using the status selector irritates you, you can use saved statuses to get rid of it forever.

Creating a Simple Saved Status

Let's start with two basic saved statuses--one Available status and one Away status.

Click the status selector at the bottom of the buddy list window. You should see something like this:


Select the "New status..." entry. You should see something similar to this:


The fields here should be pretty obvious and self-explanatory. The title will be what you see in the status selector menu. Generally, I use a descriptive title, such as "Away - Sleeping" or "Available - no message," that will quickly identify the message's contents to me without needing to read the message. For this example, I'll use "Away - example."

For Status, obviously you want to choose one of the basic statuses listed--Available, Away, Invisible, Extended Away, Offline, Do Not Disturb, or Invisible. If you use only one account, your choices could be significantly different. For this example, we'll accept the default of Away.

in the Message box, type your status message. For my example Away status, I'll use the message "Away for the sake of being away."

Click Save. Alternatively, if you want to use this status now, click "Save & Use." You could also click "Use," but then the status would not be saved.

Now, repeat the process for an Available status. In this example, I'll have a title of "Available - no message", a status of Available, and a blank message.

Now, if you select "Saved statuses..." from the buddy list window, you should see something like this, but probably with a lot fewer statuses listed:


Creating a Complex Saved Status

A complex saved status is where the real power of saved statuses becomes obvious. Let's modify the existing away example we created. Select it in the saved status list and click "Modify..." Now click the expander just to the side of "Use a different status for some accounts" and observe the list of accounts that appears.

Put a checkbox in one of your accounts. For this example, I'll use my pidgin.im XMPP account. Now you see a much simpler new dialog:


For simplicity's sake in this example, I will choose "Extended away" as the status and use "test" as my message. After clicking OK, I return to the modify status dialog, to see something similar to this:


Now you can click "Save" or "Save & Use." This gives you essentially the simplest form of complex status--one in which only one account differs in status from the other accounts. Using this status will cause all but that one account to appear as away, and cause that one specific account to appear as extended away (note the clock vs. the circular sticky note).

Using a Saved Status

To use a saved status, you can select it in the status selector's menu if it's listed. If it's not listed, simply click "Saved statuses..." in the status selector to bring up the Saved Statuses dialog, then select the status you want and click "Use." The status should now be remembered in the "popular statuses" (middle) area of the status selector menu. Of course, with disuse or use of a large number of statuses, a given saved status can fall out of the saved statuses list, in which case you'll need to repeat the process of finding it again. If you use the statuses a lot, though, this won't be a problem.

Practical Applications

Saved statuses aren't for everyone, of course. They do provide a number of possible complex status combinations limited only by the number of accounts you have in Pidgin. They also let you get rid of the ugly text box for status messages for as long as you use saved statuses. They also offer a relatively quick setup time that allows you to create your status once and use it as many times as you want without ever having to think about how to configure the status again.

Enjoy playing with your newfound saved statuses!

MSN Issues

February 26, 2009 11:38 PM by John Bailey

Since last evening, we've had an influx of users into #pidgin, trac, and the support mailing list because they're not able to connect to MSN. Initially, we thought this was a server-side issue, because the error message we receive from the server makes it look that way. However, we've come to find that it is actually a minor protocol change in which the server now expects us to send a piece of data we don't send. We are working on a solution. There is no need to send further mail to the support list, open new tickets, or pour into #pidgin.

Again, we know what the problem is and are working to fix it.

Last updated: December 12, 2017 09:00 AM (All times are UTC.)

Powered by: Planet