A pile of books, and a single dialog box better than all of them

I had a couple hours to kill downtown, so I wandered into the bookstore.  After futzing about with popular economics and romance novels, I decided to go check out the various Ubuntu books on offer.  I ended up looking through the index and sorting them into two piles.

Two piles of books at Borders

So, what’s the difference between the left and the right pile?  The books on the left, despite some of them being much thicker, make absolutely zero mention of Wine or running Windows applications.  Not even a cursory mention of how it’s possible, or a small paragraph suggesting users go look it up themselves – these books don’t even list Windows applications or Wine in the Index.

I’m sorry, but if your book is in the left pile, it’s crap.  At best, it’s written for an audience in some alternate reality. 90 percent of computer users run Windows; about half of all Ubuntu users have installed Wine, and an untold number more would like to if they could get it to work. And these books, on the left, pretend that they simply don’t exist.  It’s like making a tourist guide and forgetting to mention the majority of cabdrivers will kidnap you unless you tip in advance.

Concern for at least one Windows application is a use case so common that it could be fairly described as the main barrier for human beings running Ubuntu.  Maybe they don’t need to run their application since there’s a good Ubuntu equivalent.  Maybe their application works without fuss in Wine.  Maybe it doesn’t work at all.  It’s a question users have, and it’s a question that needs to be answered – even a short “no, don’t bother with Wine” is better than pretending this question doesn’t exist.

Fortunately, most users don’t read books, and they shouldn’t have to.  If we’re smart about it, we can convey the information missing from these books through the operating system itself.  I shouldn’t have to look in a book to find out what Wine is, how it may or may not work with Windows applications, and how to install it – that information should be given to me as soon as the computer can figure out I’m interested.  That is, as soon as I double click on a Windows application.

A dialog more valuable than any book (pen board draft)

This is just one dialog, but it can convey information more valuable than any book, to millions more people.  That kind of small change having a big impact is exactly the reason I get excited about developing for Ubuntu.

UPDATE: Apparently Hacking Ubuntu does mention Wine, it’s just that Wine and Windows applications aren’t in the index because the author didn’t write that part.

Ubuntu 9.04 is here, and work continues

Ubuntu 9.04 is here, and work continues

Ubuntu 9.04 was released just over a week ago.  A release represents the public distribution of six months of serious work, a chance to show the world what we’ve been doing and how good it is.  It seems like a time to celebrate.  For me, it’s just as busy as ever.

The very next day I spent the greater part of the day putting out the new Wine release for beta testing.  It’s become a bit of a routine of mine every other Friday – wake up, download the new tarball, test build it, update the package, push it to the Wine team PPA for building, redownload it from there and put it on the budget dedicated server.  Then I’ll go read the Digg article and see people post how Wine is getting better all the time.  Every now and again I’ll have to do something special – this release, for instance, included the new icons I found.

I’m still not done with them though.  Nothing’s ever done, of course, but some days you spend 4 hours going over redesigns with another volunteer who probably wouldn’t be there without you.  This past week has given me a few more of those days.

I volunteered to teach an online class for Ubuntu Open Week on Friday.  I teach in real life professionally, however the experience on a muted IRC channel is quite different.  There’s absolutely no continuous feedback; no students slightly nodding their head or twisting it in confusion – just empty channel.

At first I thought I wouldn’t have much to talk about, but after taking a few questions I found myself running out of time.  I didn’t even get to talk much about some of the games projects I’ve been working on – people always have many questions about Wine.  They are, of course, completely right to do so.  You can read the session here.

Anyway, it’s getting late so I’ll make this brief: I spent the past week also catching up on 5-a-day bugs; the Wine package now ranks very highly on the list of packages with good bug reports (upstream percentage).  I’ve got about 5 different major Wine package changes around the corner, and I’ve decided to move forward with Wine beta releases in Karmic even though I can’t be 100% sure that Wine 1.2 will be released on time.

Making the Wine icons beautiful

It started with a bug report.  The problem was the icon. Here is Wine’s current icon:

The Wine icon is old.  It’s ugly.  It looks like it was made for Windows 95 (it was).  All this is enough reason to change it to something, but what does a good icon actually look like?  The Tango project has an answer to this question.

A good icon is intuitive

They’ve defined a set of standards for icons.  Usability is improved when these standards are followed – a user can easily pick out the icon and, if a good metaphor is possible, have a reasonable guess as to what it means.  The two playing cards next to eachother hint as to what Applications->Games has inside of it, even before you read the text.

When a good metaphor isn’t possible, your best bet is to have an easy to recognize logo.  Since a “program for running Windows applications” doesn’t lend itself to any obvious metaphor, a logo suggesting the name Wine is our best bet.1

A good icon is easy to recognize

Enter the Wine glass.  To an English speaker, it’s very easy to create a mental connection between “wine glass” and “Wine program”.  If you don’t know what “Wine” is, at the very least you can recognize the wine glass as something rather unlike any other icon on the system.

We also benefit from using the same Wine icon everywhere the user interacts with Wine itself.  Applications->Wine should have the main Wine logo.  Browse C:\ Drive should have a folder icon with a Wine logo on it, since it’s both a folder and has a lot to do with Wine.

A good icon is consistent with the rest of the system

The Tango project gives simple standards for visibility and usability, however they also give helpful standards for design.  Icons are generally three dimensional and, importantly, from the same perspective with the same lighting.  It looks very off when you jam a 2D, backwards tilted, viewed and lit from the bottom wine glass onto a menu full of 3D, front-viewed, lit from the top objects.

Selecting the Wine glass

The bug report was helpful enough to include a link to a Tango-compliant Wine icon that someone had made over 3 years ago.  It looked like this:

I appreciated it, since it certainly was consistent with the other system icons, however something was off.  It looked like a wine glass, but a strange one.  Not being a drinker, my closest encounters with real life wine glasses all involved sparkling cider or rubbing my finger around the rim to make a humming sound.  I turned to Wikipedia.

As it turns out, some real life wine drinkers really care about what kind of glass they use.  I learned that for all these years we’ve been committing the horrible sin of using a white wine glass for red wine.  That’s why the tango icon looked different – red wine glasses are supposed to be fatter.  But they vary on this too: as I learned from the internet, burgundy-style glasses are a bit fatter, and bordeaux-style glasses are a bit thinner.

I also discovered a whole lot of websites use the same public domain wine glass clipart images:

My uncultured user expectation

So, that’s why the tango icon looked off.  It was violating my intuition of what a wine glass should look like – I hadn’t seen many fat glasses, and was expecting a thin wine glass.  The author of the icon was probably expecting wine to be in a burgundy glass.  Maybe he’s Burgundian.

Being an uncultured American prone to these kinds of things, I decided the typical user was more like me.  I figured the best compromise was an actual red wine glass, but not one as fat as the icon provided.  I sent off emails to the wine devel list and posted on web forums discussing the situation, hoping someone might appear out of nowhere to make the perfect icon for me.

YOSPOS, Bitch!

The most helpful answer came from the Something Awful “Your OS is a POS” forum.  It’s a forum designed for nonsense so users of the “Serious Hardware/Software Crap” forum can blow off some steam.  In practice it’s a much more casual place to talk about nerd stuff where you don’t have to be burdened by making an effortpost.2

Apparently, the work I was looking for was already done, and even put into an Ubuntu package.  Specifically, the ubuntustudio-icon-themes package, which has an icon for Wine that looks like this:

It was everything I wanted.  They even had sub icons made for the Wine-related programs like notepad.  After a quick consensus from the Wine-devel mailing list, I’ll be bundling the icons into the next beta package.  Most users won’t see the new look until the upcoming Wine 1.2 release makes its way into Ubuntu 9.10.  There’s a nice coincidence there, because a slick new look tends to increase users expectations about an application, and Wine has vastly improved since 1.0.

1: A Windows logo might seem like a good icon to use here, however that could create confusion as to what Wine is actually doing.  It’s very easy to think Wine might be opening some sort of virtual machine if you’ve never used it before.

2: Actually, I find myself reading/posting almost exclusively in YOSPOS now instead of the serious forum.  It’s just more enjoyable somehow.  The catchphrase “YOSPOS, bitch!” is, of course, repeated widely in users signatures.

Modeling Wine development and Linux adoption

A while back I found myself facing the question of when we could expect Wine to get substantially better.  Wine has been in development for 16 years, and while it has millions of users there are still a great deal who are unhappy with it.  This is completely understandable – Wine doesn’t yet work perfectly with all their applications, and they shouldn’t be happy until it does.

So, I decided to make a model of Wine development.  That way I could figure out if we were doing something wrong, or if it even mattered which bugs we were solving as long as we were working.

Modelling Wine development and user happiness

Let’s define the Wine project as a set of 10,000 or so bugs yet to be solved.  Maybe these represent API functions, or usability issues, or performance problems – whatever.  Some of these bugs may be harder to solve than others, so we’ll give each one a difficulty rating in terms of time.

Now let’s define an application as some subset of these bugs.  A working application is one that has all its bugs solved.  We can also give each bug a different relative probability of affecting an application – maybe bug x is 10 times more likely to affect an application than bug y.

A user is then defined as a set of applications he needs.  A “happy user” is one who has all his applications working.  Just like with the applications, we can assign relative probabilities to reflect the real world – World of Warcraft is 60 times more likely than CuteCatExploderPro.

Finally, we need some strategy for solving the bugs.  We’ll get to 100% bugs solved in the same amount of time regardless, but the order we do them in will matter.  I was able to come up with about a dozen or so strategies, such as “pick a random unhappy user and solve a random bug in one of his applications”.  I picked the most realistic of these, and told the simulation to alternate between them.

I then turned this all into a python script and combed it with the cairoplot chart generator, generating a picture like this:

Model of Wine development

Output of wine-model with 10,000 bugs, 500 users, and 500 applications simulated

You can download the script and play with it yourself – much of it is easily customizable

UPDATE: I’ve cleaned up this code again after a long time, find a newer version here on github.

Surprising things learned from the model:

  • The strategy we use – the order we tackle various bugs – really does matter.  Every strategy gets to the perfect 100% end after solving all the bugs, but some get you 10 times as many happy users when you’re only half done.  In practice, having far more users likely translates into extra developers and a much faster rate of development.
  • Varying the difficulty of individual bugs didn’t matter much.  The pictures came out pretty much the same
  • Prioritizing the last few bugs in apps that are almost done is one of the most productive ways to increase happy users – in the simulations I ran it even outperformed working on the most popular application.  Unfortunately in the real world it can sometimes be difficult to tell if an application “almost works” in Wine.
  • Similarly, “almost happy users” are the easiest to satisfy.  When we have many to choose from, picking one arbitrarily and ensuring he was happy before moving on to the next user significantly outperformed trying to satisfy almost happy users at random.
  • Instances of “collateral damage” – the fixing of one application causing another application to start working without any extra effort – are rather uncommon until most applications are almost working.  The wintrust API is needed by both Steam and iTunes, however when enough of wintrust was implemented to make Steam work there were still many unrelated bugs causing iTunes to remain broken.
  • Just about every reasonable way of generating bug difficulty, relative bug probability, applications, and users that I could think of lead to the same general picture: something that looks roughly linear for most of it before taking a very sharp upward turn near the very end.  In other words, the model tells us that we should expect to be pleasantly surprised.  At some point, Wine will get very good very fast.

So, when will that point of sudden very rapid growth begin in the real world?  No one can really know – even analyzing statistics like the growth of platinum ratings on AppDB would only give us information after the fact.

Modeling Linux adoption

We can make a similar model for Linux adoption itself.  Just imagine a few well-known barriers to entry, like driver support and compatibility with existing applications.  Now suppose that a particular barrier affects some large percentage of potential users.

If 80% of users can’t switch until they have working drivers, 80% can’t switch because of some existing Windows application, and 80% can’t switch because they haven’t heard of Ubuntu, then that means we can only expect to get 0.8% of the users.  This is, incidentally, about as many as we have.  As Joel on Software* puts it:

Think of these barriers as an obstacle course that people have to run before you can count them as your customers. If you start out with a field of 1000 runners, about half of them will trip on the tires; half of the survivors won’t be strong enough to jump the wall; half of those survivors will fall off the rope ladder into the mud, and so on, until only 1 or 2 people actually overcome all the hurdles.

So what’s our best strategy then?  Eliminating these barriers to entry – turning the rope ladder into an elevator – is the only way to get new users.  If we fix the driver problem, and the driver problem affects 80% of our would-be users, then we can quintuple our user base.  If we fix the Windows application problem, and make Wine work well, then we can quintuple usage again.

Full article by Joel on Software

Modelling Regressions

If you’re one of the 130,000 or so people who have been using Wine beta releases, then at some point you’ve probably run into a regression.  Version 1.1.12 ran TerroristExploderPro fairly well, so you upgraded to Wine 1.1.13 only to find that it functions worse than TerroristExploder Home Edition.  This is a regression.

Regressions are best caught quickly.  When we have a regression, it means some “improvement” we made is actually very wrong, and the longer we go without realizing there’s a problem the harder it is to figure out which change actually caused it.  This was the main reason why Wine first built an extensive Applications Database – not so users could find out if their application worked, but so developers could find out when it stopped working.

An example regression

Suppose we have an incomplete “stub” function in Wine for telling the time that just returns a blinking 12:00.  This could be good enough; rather than crashing outright when Photoshop asks for the time, instead the wrong time is printed in some log file the user never sees.

Now, suppose a Wine developer comes along and sees that some other application, Baby’s First Clock, doesn’t work because a blinking 12:00 isn’t good enough.  So that Wine developer tries to implement the time function.  He can’t, however, be exactly sure how to do it since Microsoft’s documentation is often inaccurate or even nonexistant.

So, he makes his best guess, and tells the function to return 6:00 PM when it’s dinnertime.  He tests Baby’s First Clock and it seems to work, so the patch gets committed and a new version of Wine is released touting advanced Baby’s First Clock support.  By doing this, however, we may have broke Photoshop: because it expected 24-hour time, Photoshop now crashes whenever it sees 6:00 PM instead of 1800.

Preventing regressions with an automated test suite

Preventing regressions has become an important part of Wine development, however we’re still pretty bad at it.  One approach that’s had partial success is to write a big automated test suite. In theory, if we had a magic way of running Photoshop every patch then regressions like this wouldn’t happen.  Currently, Wine’s test suite, while very large, is not very reflective of real-world applications; instead, individual functions are tested to make sure we do the same thing as Windows.

In theory, if we had one of these individual tests for every case for every function, Wine would never have any regressions.  In practice, we miss a few, and real world applications break every now and again.  Actually, that’s not quite true – Wine would still have regressions even with a complete test suite; some tests we expect to fail because we haven’t written the functions yet, but we might be accidentally relying on broken behavior when we write another function.  It’s like painting a wall light brown to match the ugly yellow carpet – when you finally replace it with something good you may find your wall looks as ugly as it sounds.

Regressions can also come from passing well made tests.  Performance regressions are the best example: at one point a certain direct3d shading function was “stubbed” out and did nothing.  Most gamers didn’t care, since all it meant was the color of the beards on the terrorists was a slightly duller shade of brown.  However, this stub behavior (correctly) caused a Direct3D test to fail, so the shading function was implemented and the test passed.  Unfortunately, the implementation was very slow and now instead of seeing 50 dull beards per second gamers were seeing 5 shiny beards per second, making the game an unplayable albeit beardy slideshow.

Wine’s test suite grows as Wine development progresses.  In many ways, having a “complete” test suite is almost the same thing as having a complete Wine – once we know exactly what to test for we also know how to write our implementation.

A test suite, however, can only get you so far.  Until we have a nearly complete test suite that can expose all the subtle problems in partially broken applications, we’re still going to run into periodic regressions.  For that, we need another tool – users need to actually play the shiny beard game.

Preventing regressions with stable releases

The most effective way to prevent regressions is done by all serious projects: the stable release process.  The process starts with a feature freeze: no new code features are done, and all those “implement this” regressions just can’t happen.  Baby’s First Clock stays broken, but Photoshop won’t break either.  We stay in this freeze until we’re confident enough that anything that ever worked in Wine still works in Wine – that is, we fix Photoshop.

That’s what Wine 1.0 was about: it was the first release that actually made that promise.  It took 15 years, but this made a certain amount of sense given how visibly incomplete Wine was – there was always some new feature about to go in, and so many applications were “almost” working.  A year and a half later, they’re still almost working.

Wine 1.0 came out shortly after Ubuntu Hardy (8.04), so I had to ship a Wine beta version and then later replace it with a stable release update.  As a distribution meant for human users, Ubuntu tries to only ship stable software in its releases; this way, end users don’t have to worry so much about regressions.  Attracting new users with features is important, but not scaring off our existing users is important too – that’s the kind of innovation that gets it right.

This need for stable releases is why Jaunty (9.04) will still have Wine 1.0.1, and it’s why I’ve been secretly lobbying for speeding up the next stable Wine release for over a year now.  “If Wine 1.2 can come out in the next six months,” I’d argue, “then we can ship it in the next Ubuntu and end users will be able to actually see some of these improvements you’ve been making.”  This is the same reason Mark Shuttleworth has been calling for more synchronous freezing of upstream projects – if everyone releases around the same predictable time, then we maximize the amount of new but stable software users actually get.

Well, it looks like I may finally get my way in the Karmic time frame.  Wine may actually make that 1.2 release happen, even if it’s not as “complete” as the project’s leader originally envisioned.  So look forward to Wine and Ubuntu being particularly powerful come 9.10, and I’ll do my part to help it along the way.

The kind of innovation that gets it right

There was an article that ended up on Slashdot last week:

“In the short term, GNOME’s gradualism seems sensible. It doesn’t confuse users, and it doesn’t accumulate bad reviews from writers with an aversion to change. But, in the long-term, it could very well mean continuing to be dragged down by support for legacy sub-systems. It means being reduced to an imitator rather than innovator.”

There is absolutely no reason why incremental change and innovation are incompatible.  Consider the new notifications system in Ubuntu 9.04, for instance – it took some extra effort to make sure we didn’t break old applications, but it did happen.

“By contrast, you could say that KDE has done what’s necessary and ripped the bandage off the scab. In the short term, the result has been a lot of screaming, but, in the long-term, it has done what was necessary to thrive.”

The “ripping the bandage” approach the author advocates is a lot like how my dad drives down the freeway; sure, he gets there about 5% quicker by aggressively tailgaiting and weaving in and out of traffic, but during 95% of the drive his passengers think he’s a total asshole.

While it’s quite possible KDE gets new features a bit sooner by dumping loads of bugs on their users, not having these kinds of bugs is an important feature too.  Maybe they’re good features, and Gnome decides to include them later using its slow, incremental changes.  That’s innovation too – the kind that gets it right.

Ripping bandages can be a good idea if you have no users

Ripping bandages can help speed up development, but it also really annoys users.  This means it’s perfectly ok when you don’t actually have users – early alpha software, for instance, shouldn’t even pretend to be nice.

Recognizing that your project has grown up can be hard.  Wine got very used to having no users; we made biweekly releases that were little more than CVS snapshots.  Before we knew it, we had a million users and frequent complaints that using the latest version was a bad idea.  We even built a database of which Wine versions worked with which applications to help users along.

Making real releases is what grown up projects do to respect their users

Eventually, however, we made a stable release.  90% of our users are on it now, even though it’s over a year old.  The current Wine betas can run a lot more programs, but it would be a real asshole move to trick millions of people into using one of these by falsely labeling it “stable”.  Without making a real effort to ensure that our next release is at least as good as the last, what we’re really doing is breaking software for millions of people.

Inaugural Post

Greetings everyone, this is my new blog. I know I already have thousands of readers because it’s syndicated on Planet Ubuntu. So was my old blog, on Livejournal, but I still neglected posting when life got busy.

Why blog?

I work on free software for the same reason I give blood. I enjoy helping others, making a positive change, and not feeling bad about drinking 3 cans of soda before stepping outside.

I help with free software even though I can’t be sure if it’s useful – sometimes you make things worse with buggy code, and sometimes your donated blood is biohazardous waste. Or maybe you save a life, have millions of happy users, and end up enabling billions of dollars in new economic productivity.

The world needs more free software and we can’t expect companies to make all of it. Some will free ride and enjoy the benefits without paying for development. Others will help write free software, but mostly to sell stuff like support contracts and servers; meanwhile, no one’s financing the magical piece of software that doesn’t need support and makes extra servers obsolete.

An old girlfriend once told me that I can write. I believed her, since she can read. By blogging I help tell my readers about how they can help make the world a better place (that is, by being more like me). Some of my readers will find this useful. Others will find it inspiring. Most will find it orange and brown. But that’s what Ubuntu is all about.

Why microblog?

Microblogging is like reading the Onion – 95% of the content is in the headline, so we might as well just have those. Gwibber makes it very easy to do – I can update to Twitter, identi.ca, and Facebook simultaneously. Real, actual (internet) people can then follow me on Twitter, or on identi.ca. They could even take my real, actual name and search for me on Facebook so we become real, actual eFriends.

OK that’s a bit of a stretch. I only accept friend requests from internet strangers when they’re women who want to marry me for citizenship. Then I make them post their undying love on my wall so I can make my ex girlfriend jealous.

That’s what microblogging is all about. I let the internet know how cool I am. Or, more likely, I haven’t done anything important at all and instead start doing something productive so I can at least tweet about that. If I can’t feel accountable enough to myself to get Ubuntu work done, I can at least feel like I owe something to a handful of internet strangers.

For that reason, I’m going to come out and commit, right here, to a seemingly modest two posts a week. I’ll partially write a few of them ahead of time, and aim to publish on Tuesdays and Thursdays. No one will notice when I finally go on a two week vacation to the Saint John’s Memorial Trauma Center. It’s work, but it’s work on something I love, and it’s time to get serious about it.