IanG on Tap

There's been a certain amount of buzz around these new Apple 30 inch monitors.

Cyrus wonders about scaling:

"I'm hoping Tiger will have some facility for scaling up the UI so that you will actually be able to read and work with the regular screen widgets"

Actually, the pixel size on these things is the same as the pixel size on a normal screen, so screen features will remain the same size as before.

This makes me sad though.

I'm not sad about the vast pixel count, nor the fact that screen features remain the same size. Those are both good things. What saddens me is that it's looking unlikely that Tiger will have the scaling facility that Cyrus hopes for, and Macs will therefore be stuck with crappy 100 pixels/inch display resolutions for the foreseeable future.

Apple are always quick to put a positive spin on things. Far from trying to hide or quietly ignore the Mac's lack of progress when it comes to screen resolution, they draw attention to it, and even claim that it's a feature in the technology section on these monitors. I was astonished to read this on that page:

"Apple engineers have discovered the ideal resolution to display both sharp text and graphics -- a pixel density of about 100 pixels per inch (ppi). Other vendors may offer a larger monitor, but with less resolution, so you end up with fewer pixels, or a smaller monitor with a high resolution that causes eyestrain and headaches."

Huh? Since when did high resolution cause eye strain? Now that Apple have discovered that 100 pixels per inch is "the ideal resolution to display ... sharp text" does this mean we can expect a 100dpi laser printer from Apple, and we can throw out those crappy 600dpi ones that we hadn't even noticed were giving us eyestrain?

And before anyone tries to claim that high resolution is somehow only useful on paper, that's just not true. I own 4 flat panel screens of one kind or another. Two are about 100 pixels per inch (both Apple - one iMac, and one PowerBook). The others are about 130dpi and 150dpi. Apple ones look really clunky compared to the higher resolution ones. Text and graphics are less crisp. The only point at which graphics quality is any kind of an issue on the high res screens is when I'm dealing with bitmaps that are of sufficiently low resolution that they need to be enlarged. That's really not the fault of the display technology - it's a problem with the image source. High resolution images look superb.

What Apple Might Possibly Have Been Thinking

I can only assume that Apple left out some of their working here. I'm guessing that what they really mean is that if you crank up the resolution without increasing the number of pixels used to represent any particular feature, those features get smaller, and it's the small features that give you eye strain. But that's just a flaw in the design of the graphics system - an inability to scale the UI so that imagery has whatever physical dimensions you require. So what they should have said is this:

Apple engineers have discovered that they can't scale the user interfaces of OS X applications to cope with varying display resolutions -- a higher resolution display will therefore just make everything look smaller. Apple would like to avoid making the UI shrink, so you're stuck with a pixel density of about 100 pixels per inch (ppi). Other vendors may offer better resolution, but unfortunately, your Mac OS X applications will come out looking tiny, and there's nothing OS X can do about it.

Eyestrain is not caused by higher resolution displays. Eyestrain is caused by making on-screen features too small. If an OS cannot adjust the size-in-pixels of a feature to make it take up the same amount of physical space on screen regardless of resolution, that's a deficiency in the OS. Without that flaw, using a higher resolution display won't cause everything to get smaller; it will simply mean that your imagery looks crisper. We've already been through this with printers - your text and images don't suddenly shrink to half the size when you upgrade from a 300dpi printer to a 600dpi one.

Resolution Independence as Technology

Of course Mac OS X isn't alone here - there are other OSs which only work properly at one resolution. Windows doesn't do a whole lot better here, for example. To be fair, Windows has always let you change between 'Small Fonts' and 'Large Fonts'. (And more recently, Extra Large Fonts.) This is a slightly misnamed feature, because it doesn't just change the font size - it's really effectively setting the notional screen resolution. Some applications honour this and scale up their whole UI to match the chosen size. However, most don't. And in some cases you will end up with a bizarre mixture, where some features are enlarged, but some render to the same size-in-pixels regardless of the chosen font size. It's debatable whether this is any more useful than Mac OS X which doesn't even attempt to offer this feature.

I'm conscious that some Mac developers will be foaming at the mouth at this point, because I've sort of implied that the graphics architecture Mac OS X is not resolution independent. In fact, if you look at the Quartz 2D APIs, they all appear to be gloriously resolution-independent, and completely scalable. (And I'm talking about geometric scalability here of course, rather than 'loads of users' scalability.) So in theory, it should be a breeze to produce a resolution-independent version of Mac OS X - all Apple have to do is change the way they map coordinates in Quartz 2D onto screen coordinates - effectively they just need to apply a scaling transform to everything and they're done. Indeed, one Mac developer argued this quite extensively in a discussion in the comments on my article on graphical composition in Avalon. (Note that the forum system crops off comments after a certain number of levels - you'll need to click the View link of the last followup to see the next part of the conversation.)

If only it were that easy.

Now would be a good time to point out that Windows also has a resolution independent drawing API. In fact it has two. The original GDI32 (introduced with the first Windows NT) has always supported arbitrary affine transformations. In fact its feature set looks surprisingly similar to those of Quartz 2D - much more so than most people realise. GDI+, is the successor that shipped in 2001, and is redistributable to older versions of Windows. It leapfrogged Quartz 2D's functionality when it came out, and unsurprisingly, it has supported resolution-independent drawing from day one. So with both GDI32 and GDI+ it's perfectly possible to apply a scaling transform to everything you draw, which means that it's as easy to write resolution-independent code in GDI32 as it is in OS X.

So why don't Microsoft just automatically apply a scale factor appropriate to the screen resolution? The answer is the same as the reason Apple continue to ship 100dpi displays: it's not as simple as just scaling all the drawing.

At one point in the aforementioned discussion on my Avalon article, I challenged the Mac advocate to prove his claim that Mac OS X really does have scalable graphics. He produced a PDF here. I've excerpted a couple of bits of that PDF, rendered to bitmaps, scaling up by a factor of 4 during rendering:

Note how the image on the right, which shows part of the window's title bar, has scaled up properly - the text has been rendered correctly for the chosen scaling. But notice how the 'traffic light' icons on the left are all pixellated. (Also, if you examine the graduated fill closely, you'll see that it is not a continuous fill - it is actually pixellated. Although some PDF viewers perform interpolation on bitmap images, so you may find that hard to see if you look at the original PDF. But take the PDF apart with a drawing tool, and you'll see that the fill is really just a bitmap texture being used to fill the window title area.)

I then wrote a Mac OS X application of my own with a little more stuff on it to show some more problems. The PDF is here. Here's a button scaled to double the normal size:

Notice how the bottom of the button doesn't really line up with the end caps. There's also a slightly more subtle problem at the top - the thickness and positioning of the outline on the middle section appears to be different from the end caps.

This is typical of the problems you tend to get if you try to scale imagery that wasn't originally designed to be scaled, even though it was drawn with an API designed to support scalability. You'll see similar problems with similar examples from Windows. (And by the way, it is technically possible to get an .emf file (Enhanced MetaFile - a Windows vector graphics file) of the contents of a window in Windows, so you can then start trying to scale it.)

Having a drawing API that supports scalability isn't enough, it turns out.

(As an aside, I'm not convinced that Mac OS X's graphics architecture has the right ingredients. Quartz 2D has everything required for scalability, but it's just one source of imagery feeding into their desktop composition system. And as far as I've been able to tell, the Quartz composition engine is thoroughly pixel-oriented. The Mac advocate seemed to think this wasn't a problem, but didn't produce any convincing evidence (well, not convincing to me) to support that opinion; moreover, a fair amount of what he said seemed to contradict Apple's own documentation. As I'm not an OS X expert, it was hard to know what to make of it.)

Resolution Independence as Culture

There are two main reasons that UIs tend not to scale well even when they are drawn using scalable APIs like Quartz 2D, or GDI+. One is that most applications use a lot of small bitmaps in their UI. The other is that people rarely test for multiple resolutions, meaning that you tend to end up with little quirks such as in the button above when you try to scale. (Some Windows do get tested for multiple resolutions, but most developers don't bother. I'm not sure it happens at all on the Mac, since Apple seem to have been using a "one pixel resolution fits all" approach for as long as I can remember, so how on earth would you test? The Mac advocate in the discussion on my Avalon article forum didn't mention any way of doing this when I asked.)

These are essentially problems with the culture of application development, rather than technical problems. It is technically possible to build resolution-independent UIs today in both Windows and OS X, it's just that nobody does.

One of the goals for Avalon, the graphics subsystem in Longhorn, is to support high resolution displays. And they are certainly making it very easy to scale bits of your user interface. But the question is: is this enough? Building applications which don't look crummy when they're scaled is about more than just providing the necessary technology. It requires developers to do the right thing - to test at multiple resolutions, and to minimize reliance on bitmaps.