Open Advice/Cross-Project Collaboration

Cross-Project Collaboration

by Henri Bergius

in: Open Advice

This text is available under the CC-BY-SA license. (see also: Open Advice/Info)

Contents 1 On projects and communities 2 The Midgard lesson 3 On libraries and frameworks 4 How to make collaboration work 5 about the author

There may be a whole new system where you’re defined more and more by who you are and not by what you own, by what you’ve created and shared, and what other people have then built on” – Former Xerox PARC director John Seely Brown in An Optimist’s Tour of the Future
(Mark Stevenson, 2010)

On projects and communities

Much of the free software world is split into tribes gathered around something called projects. There are major projects like GNOME, KDE or Drupal, and lots of smaller projects revolving around a single application or a library.

Actually, calling them projects is kind of silly.

In my mind, a project is a plan of effort towards an achievable aim, with a schedule that has start and end dates. So, for example GNOME 3.1 would be a project, but GNOME as whole is not. It is a community of individuals maintaining and creating a body of software through various smaller efforts, or projects.

Enough with pedantry. The problem with the concept of projects is that they end up keeping people apart, creating insular communities that often are reluctant or unable to collaborate with “the competition”. But in the end, all of these communities consist of individuals who write free software, and it is their choice whether this software can be used in different environments or not.

In the end we all want the software we created to be used by others. And even better, we want others to join in our efforts and build cool stuff on what we have created. That is, after all, what is in the heart of free software.

So why do we enact these walls around ourselves? Keeping an insulated community just fosters an us-versus-them mentality. The incompatibilities of different programming languages already do so much to keep us apart, why add to that?

The Midgard lesson

What I wish I had known when I started, in those optimistic dot-com days of the late 90s, is that in reality software efforts do not need to be isolated. With a bit of care we can share our software and ideas across communities, making both the communities and our software stronger and better.

When I started my free software career, it was a time of big projects. Netscape was open-sourced, the Apache Software Foundation was getting a form, and venture-funded efforts were going on everywhere. It felt like a norm to try and build your own community. This was the sure path to fame, fortune and building cool stuff.

So what we did was build our own web framework. Back then there were not that many of them, especially for the fledgling PHP language. PHP was not even the first choice for us, only picked after a long debate about using Scheme which our lead developer preferred. But PHP was gaining popularity, becoming the programming language of the web. And web was what we wanted to build.

At first, things looked very promising. Lots of developers flocked into our community and started contributing. There were even companies founded around Midgard. And the framework became more featureful, and more tighly coupled.

In hindsight, this was the mistake we made. We positioned Midgard to be something apart from PHP itself. Something that you would install separately, and build whole websites on top of. It was either our way or the highway.

With Midgard you would have to use our content repository interfaces for everything, as well as our user management and permissions model. You would have to use our templating system, and store much of your code into the repository instead of a file system.

This obviously did not sit too well with the wider PHP community. Our ideas were strange to them, and Midgard at the time was even distributed as a huge patch to the codebase, as PHP3 did not have loadable modules.

Many years have passed, and PHP’s popularity has waxed and waned. At the same time the Midgard community has been quite constant – a small, tightly knit group making progress in the long run, but apart from the wider PHP world.

We always wondered why we found it so hard to interact with the PHP world. Even some communities doing something completely different, like the GNOME desktop, seemed easier to approach. Only recently, after years of isolation, we realized the problem.

In a nutshell: frameworks keep us apart, while libraries allow us to share our code and experiences.

On libraries and frameworks

In the end, software is about automation, about building tools that people can use for solving problems or connecting with each other. With software, these tools have many layers in them. There are low-level services like an operating system, then there are libraries, frameworks and toolkits, and then there are actual applications.

Applications are always written for some particular usecase, and so between them there are very few opportunities for sharing code.

The much more appealing opportunity is on the libraries and frameworks level. A framework, if generic enough, can usually be utilized for building different sorts of software. And a library can be used to bring a particular piece of logic or connectivity anywhere. In my view, this is the layer where most programming should happen, with specific applications being just something that connects various libraries into a framework that then runs the actual app.

What is a library and what is a framework? People often use these terms interchangeably, but there is a useful rule of thumb to know which is which: a library is something that your code calls, while a framework is something that calls your code.

If you want your code to be used and improved upon, the best way to go about it is to maximize the number of potential users and contributors. With free software, this works by ensuring your code can be adapted to multiple different situations and environments.

In the end, what you want to do is to build a library. Libraries are cool.

How to make collaboration work

The hardest part is to get over the barrier of them-versus-us. The developers of the other community are hackers building free software, just like you. So just get over the question and start talking with them.

After you have the discussion going, here are some points that I have found important when actually implementing common ideas or libraries across project boundaries:

• Use permissive licensing and try to avoid copyright assignments or other requirements potential users would find onerous.
• ˆ Host the project on neutral ground. For web projects, Apache is quite a good home. For desktop projects, Freedesktop is probably the best option.
• ˆ Use technologies that do not impose too many constraints. Libraries should be quite low-level, or provide D-Bus APIs that can be used with any system.
• ˆ Avoid framework-specific dependencies. For example, KDE has found GeoClue hard to adopt because it uses GNOME-specific settings interfaces.
• ˆ Meet the other guys. If you are from the GNOME project, go to aKademy and give a talk, and if you are a KDE developer, go and talk at GUADEC. After you have shared a beer or two collaboration over IRC happens much more naturally.
• ˆ Finally, accept that not everybody will use your implementation. But if they at least implement the same ideas, then collaboration is still possible.

Good luck with breaking down the project boundaries! In most cases it works if your ideas are good and presented with an open mind. But even if you do not find a common ground, as long as your implementation solves the use case for you it has not been in vain. After all, delivering software, and delivering great user experience is what counts.

about the author

Henri Bergius is the founder of Midgard, a free software content repository. He has also been involved for a long time in making Linux desktops location-aware, and in the Maemo and MeeGo communities. He runs a small consultancy called Nemein, hacks in CoffeeScript and PHP, and spends much of his free time motorcycling through remote parts of the Eurasian continent. He lives in the cold Nordic city of Helsinki, Finland.