IanG on Tap

My upgrade to Mac OS X 10.4 ('Tiger') finally arrived yesterday. If you've read much of my writing before, it probably won't surprise you a great deal to know that for me, the most exciting aspect of Tiger is not end-user visible stuff like Dashboard or Spotlight, cool though they may be. No, to a dyed-in-the-wool graphics geek like me, the advances in Quartz, the OS X graphics architecture, are much more interesting.

In fact the feature I was most interested to see is one that isn't strictly an official feature of the OS. You have to install the developer tools to get to it, and run the 'Quartz Debugger' utility to switch it on. That feature is resolution independence.

Resolution independence is interesting to me for a number of reasons. First, my PC laptop has a 150 pixels per inch resolution, which is pushing the limits of how small you can make the pixels if you're running a system that can't scale its graphics. Secondly, I have a very long-running interest in all things graphical, and have been enthralled with high-quality scaling ever since I first used a system with fully-scalable anti-aliased fonts back in the late 1980s. (RISC OS, running on Acorn's 32-bit successor to the BBC Micro in case you were wondering; the OS that was the early proving ground for the ARM processor that these days is found in almost every mobile phone.)

Thirdly, as you may have noticed, I'm quite into Avalon, Microsoft's next generation UI technology. Resolution independence is a significant feature of Avalon, because Microsoft want to enable widespread use of high-dpi displays. And I can see from the preview releases I've been using that it's a great feature - I can't wait for Microsoft to ship it!

Fourthly, the reason I bought a Mac a few years back was to learn about OS X and its graphics architecture. And from what I learned, I didn't see how they were going to make it support resolution independence without hitting problems. I've been taken to task by a number of Apple enthusiasts for having said so in public in the past. The resulting debates were very interesting and, for me at least, informative, but did nothing to convince me that Quartz didn't present some significant barriers to implementing a scalable UI successfully.

So having an actual proper release of OS X with resolution independence support is rather exciting. Of course before I go any further, I should point out that YES I KNOW THIS FEATURE ISN'T DONE YET. A few people have been touting this as though it's finished and ready to go, but the reality is that this is a bit of a preview feature; it's not more ready to go than Microsoft's Avalon. It's in there mainly so that developers can find out whether their applications will behave properly when run at non-standard resolutions. The fact that you have to install developer tools and use a debugging aid to switch the feature on tells you that Apple don't think this is ready for prime time.

Which is just as well. It certainly doesn't look ready.

I ran the relevant program and told it to set the resolution to 150dpi. (I chose that because it's what my PC laptop display offers, and I intended to transfer the images to the PC, to see how they looked.) Actually it wouldn't let me use 150dpi - for some reason 149dpi was the closest it would let me go. The resolution chooser has a slider that goes in fixed increments, and although you can type in whatever number you like for the dpi, it seems to snap back to numbers corresponding to positions on the slide. So, 149dpi it was.

The first couple of applications I ran up were a bit of a disappointment. One was OS X's equivalent of Control Panel, the applet that configures various system settingss. For some reason all of the icons lost their individual pictures, and I was given an array of very fuzzy-looking identical images. Then I tried running a copy of DreamWeaver. It ran fine, but everything was very blurry. It was quite clear that all it was doing was drawing the thing offscreen at normal resolution and then doing bitmap interpolation to enlarge it. The results are pretty crummy. If you bought an expensive high dpi display in the hope of getting a clearer picture, you would be disappointed.

However, it occurred to me that these applications might have issues that made it hard for OS X to scale them. After all, this bitmap scaling behaviour is, as I understand it, exactly the same as the fallback behaviour Longhorn will use for pre-Avalon applications. If an application is written to a non-scalable API, then it's going to be hard to do anything other than bitmap scaling.

It occurred to me that Tiger might be treating applications differently according to which set of APIs they are using, e.g. Carbon vs Cocoa, and that I should try a few different apps. So next, I tried Safari. That was a lot better. The HTML rendering was all correctly scaled. However, everything else was still rather ropey - the buttons and other imagery around the edge was still using bitmap scaling. So you're left with the slightly bizarre experience of a window with crystal clear contents but very fuzzy edges.

At this point I decided to try out an application I wrote, because that way, I'd know how the application in question worked. (I'm not familiar with the inner workings of any of the applications I'd tried up to this point.) I decided to try out a very simple test application I wrote early on when I was experimenting with scalable drawing in OS X. Here's how it looks at its default resolution:

What you see there is a fairly ordinary Cocoa application. It has a bunch of standard OS X widgets on there. The slightly bizarre over-inflated looking thing at the top left was what I did after being exasperated by the fact that OS X doesn't appear to expose any kind of API for drawing things that look a bit like the built-in buttons. There is a themeing API, but as far as I can tell, it all harks back to the OS 9 days. So I wrote some Quartz 2D code to draw something that looks vaguely like a button. And by the way, the drawing scales as you enlarge the window, so you can make that blob any size you like, unlike the built-in buttons.

Here's what the same application looks like when run in high-dpi:

Let's start with the good stuff. Note that the fonts have scaled up nicely. They all look as crisp as you would hope, and you can see the shapes of the characters much more clearly than in the smaller version. Also notice that my little bit of Quartz 2D rendering at the top left has also scaled up properly - it is looking nice and crisp. (But as mentioned above, you could already scale that by just resizing the window on previous versions of OS X.)

Unfortunately, these good points rather show up the rest of it.

The heavily bitmap-based nature of today's OS X look has become very apparent. The button outlines are pretty poor - rather blurry and with pixels fairly clearly visible. This might not be so bad if it weren't for the crisp text and vector graphics right next to them showing how it should be done.

It doesn't have to be this way incidentally. One of the arguments I've seen repeated many times when discussing scalability is this idea that things like button imagery are intrinsically non-scalable, and that you just can't do these things without bitmaps. Sorry, but that just shows a lack of imagination. The following buttons are drawn using three Rectangle elements (with rounded corners obviously) plus a TextBlock in Avalon:

Each of the following blobs consists of just three Avalon Ellipse elements, with a variety of fill and stroke options.

These things seem to scale just fine. And if they were drawn by someone with artistic talent (as opposed to me) they'd probably look good too...

But the shoddiness of the imagery in OS X doesn't worry me too much. Doubtless higher resolution bitmaps will be drawn by someone at Apple if they ship an OS with high-dpi as a user-visible feature. What's much more worrying is the shabbiness elsewhere.

Look at the button towards the top of the window in the middle. See how its middle section is not correctly aligned with the end caps. And there's a more subtle version of the same problem going on with the the other buttons - if you look closely you'll see little vertical discontinuities in the same places. There's a similar subtle artifact on the option picker control at the bottom left. The slider control at the upper right also has alignment problems where the track meets its endcaps.

Also, look at the menus. There's some very odd stuff going on here - the menu bar at the top has lots of fine vertical lines in it. The drop down Help menu has a grid of such lines. Obviously I don't have access to the OS X source code, so I cannot be certain what is causing this, but I know what it looks like. To me it looks like the kind of trouble you get when tiny rounding errors in your drawing calculations are magnified to the point where you can see them.

To understand why there are gaps, it helps to understand how the menus are being drawn. Menus are all textured in OS X. In Tiger, the main menu has a sudden change in colour about half way up, presumably to suggest a shiny reflection like you get off the buttons. (Although to me it just looks like it's partially transparent, and the desktop background doesn't quite go to the top of the screen...) And the menus themselves have a subtle pin strip. OS X achieves this by drawing a repeating grid of the same bitmap over and over, to provide the texture. It looks to me that once you scale up the UI, it shows up minor inaccuracies in the placement and/or sizing of these bitmaps.

At standard resolution it looks OK, but crank up the dpi, and the cracks start to show.

These are typical of the symptoms you get when you take something that was never designed to be scaled, and try to scale it. I've said before that tackling the scalable UI is as much a cultural problem as a technical one. You pretty much have to test for scalabililty of graphics, because it tends not to work by accident. It's not enough to be provided with the tools to make a resolution-independent UI - you have to test to find out whether you're using them properly. In other words, unless resolution independence is part of the development culture, it won't work. This is why it's so important to have early access to the scaling technology.

So it's good that they put this tool into Tiger. But don't kid yourself that Tiger is ready for resolution independence. (My girlfriend suggested I should call this blog entry 'Easy, Tiger!') These screenshots show that even Apple's own widgets aren't ready for it. And Apple are probably not the only ones with these kinds of bugs in their drawing code...

However, I think Apple may choose to take a short cut. So I'm going to stick my neck out and make a prediction.

My Prediction

Apple will move to high-dpi support by going straight for displays that have double the resolution of their current offerings.

Apple have stuck steadfastly at the low end of the resolution market to date. They've even gone so far as to make the ridiculous claim that 100dpi is somehow the optimal resolution for a screen. Today this puts them at a disadvantage - I'm always disappointed by the poor resolution on Apple displays, and I've seen others comment about this problem. However, 200dpi displays are available today. And since Apple are in the unusual position of supplying the hardware, they are free to move people straight up to that resolution, bypassing all the tricky resolutions in between.

What about the problems I showed above? Most of them go away if you pick a scale factor of 2. Scaling up by integer factors is much easier than arbitrary scaling. The main reason for this is that anything that was aligned to a particular pixel at a scale of 1 has a good chance of still being aligned to a specific pixel at a scale of 2. And multiplying by integers tends to introduce fewer errors into your floating point calculations than multiplying by messy fractions.

Of course, there is still scope for errors where something works at scale 1 but not 2. But while arbitrary scaling tends to make even the tiniest little mistakes and rounding errors visible, integer scaling is a lot more forgiving.

So my guess is Apple won't bother to solve the whole problem. Why would they tackle the really rather hard problem of supporting arbitrary dpi when a much easier option is open to them?

Apple have already successfully positioned their displays as a premium technology. (I'm still not quite sure how. They all have distinctly average resolution, and if the four in this house are anything to go by, they have pretty lousy response times too. I suppose they've got 'big' covered though.) This means Apple are in a position to present a doubling of dpi as a distinctive advantage of choosing an Apple display. They will doubtless quietly ignore the fact that limitations in scaling in Quartz prevent them from using medium resolution displays effectively. It wouldn't surprise me if they also neglect to mention the fact that other companies have already been shipping displays with this resolution for years. Or has Steve Jobs' marketing style turned me cynical over the years? I still haven't recovered from that 100dpi comment!