Silverlight and WriteableBitmap
I’ve just released a library to CodePlex that implements a feature missing from Silverlight: the ability to generate bitmaps from pixels in code at runtime. WPF provides this through its WritableBitmap class, but Silverlight 2 doesn’t have that. My SlDynamicBitmap project offers a solution.
I was inspired to create this by the recent reports of the Silverlight port of Quake. From what the author says, it’s clear that they must have devised some sort of mechanism for pushing raw pixels onto the screen, something Silverlight really doesn’t help you with. I’ve been toying for a while with an idea about how you might solve this, but hadn’t tried it before. Quakelight shows that it’s definitely possible, which encouraged me to try out my idea.
Incidentally, I have no idea how Quakelight solves this problem. It may well have a much more elegant solution than the ghastly hack I’m perpetrating here.
Why Would I Want This?
Before I show how it works, it’s worth reviewing why you would even want a Silverlight equivalent of WPF’s WriteableBitmap – what’s it for? It’s useful for when you want to generate pixels at runtime based on some kind of algorithm. Ray tracing is one classic example – the computer constructs an image one pixel at a time, and in order to display the results, you need some way of controlling every single pixel on a region of the screen, rather than using higher level primitives.
To illustrate this I’ve chosen another classic example: a Mandelbrot set fractal renderer.
In case you’re not familiar with the Mandelbrot set, it’s a fractal generated with a deceptively simple process. It’s a striking demonstration of how incredibly basic non-linear systems can exhibit behaviour that is very complex, and which shows some very surprising geometric scaling features.
But for my present purpose, the interesting thing about the Mandelbrot set is that you end up with a program that generates a lot of pixels and needs some way to get them on the screen. In WPF, this is the kind of scenario that WriteableBitmap is for. To enable this in Silverlight, I’ve written a new class called PngGenerator.
It’s Called What?
I’ve chosen to call the main type in this library PngGenerator, and not WriteableBitmap. This may seem like an odd choice, but there are two good reasons for it. First, I regard this code as a workaround for a missing feature, not a proper solution, so I’m hoping that Silverlight will eventually get a WriteableBitmap of its own. So it would have been unhelpful of me to use the same name. Second, the usage model has ended up being pretty different – it’s not really possible to write a custom class in Silverlight that’s a drop-in replacement for WPF’s WriteableBitmap. Only Microsoft can provide us with that – it would require support from the Silverlight plug-in itself to build an exact replica.
So that’s why it’s not called WriteableBitmap. Why PngGenerator? That’s because of the ghastly hack I used to make this work.
The Ghastly Hack
The PngGenerator works by converting an array of pixel colour values into a Stream of bytes that represent the image as a PNG file. (I chose PNG rather than JPEG because JPEG files typically use lossy compression, so you don’t get precisely the pixels you asked for. It has since been pointed out to me that JPEG does now support lossless images too, so I guess you could use either format. BMP would have been a whole lot easier as it’s a simpler format than either, but Silverlight only supports PNG and JPEG.)
If Silverlight had an equivalent of WPF’s PngBitmapEncoder class, this would have been pretty straightforward to write. Unfortunately Silverlight doesn’t contain any code to generate PNGs from pixels. So most of the code in this library is a PNG builder.
By the way, I would not recommend using this code to generate PNG files for general use. It does a couple of rather crufty things. It only generates the bare minimum contents required to be just enough of a legal PNG file for Silverlight to render it. And it also doesn’t compress the data. The ‘deflate’ compression format mandated by PNG lets you put in so-called ‘non-compressible’ blocks. You’re supposed to use this for any data that gets bigger when you ‘deflate’ it. Some data is just hard to compress, and the idea is that you mitigate this problem by storing that data verbatim. But I’m storing all the pixel data this way to provide the simplest possible path from pixel to screen.
So it’s a little too specialized to be a useful general-purpose PNG generator.
I’ve seen nastier ways of solving this by the way. I’ve seen people generate Silverlight content made up of hundreds of tiny rectangles in an attempt to render their own pixel data! This doesn’t work all that brilliantly – you get anti-aliasing artefacts, and it’s mind-bogglingly slow. Unpleasant though the technique I’m using is, it does at least produce good quality results, and is tolerably quick, although not as good as a native WriteableBitmap built into Silverlight could be.
Using the PngGenerator
To use the PngGenerator, you do this sort of thing:
PngGenerator pngGen = new PngGenerator(640, 480); Color[] m_pixelData = GenerateMyPixels(); pngGen.SetPixelColorData(m_pixelData); BitmapImage imgSource = new BitmapImage(); imgSource.SetSource(pngGen.CreateStream()); myImageElement.Source = imgSource;
That may seem slightly less direct than necessary. I’m contemplating adding a CreateSource() method that builds the BitmapImage for you, to make the code a couple of lines shorter. But the verbose approach required by the current version does have the benefit of showing the process relatively clearly.
Walking through it, we build a new PngGenerator, telling it the image dimensions we require – 640x480 in this case. Then we build an array of Color values indicating the pixel colours we want. (So GenerateMyPixels() here stands for whatever code you’ve got that generates pixel values.) We pass these pixels to the PngGenerator’s SetPixelColorData method. Then we construct an instance of Silverlight’s BitmapImage type, and call its SetSource method. (BTW, BitmapImage.SetSource is the Silverlight feature that makes this possible. It accepts any System.IO.Stream object, and expects it to contain either a PNG or a JPEG.) The PngGenerator’s CreateStream method returns a Stream that contains a PNG representing the pixels passed in to SetPixelColorData. Finally, we use the BitmapImage as the Source property of a Silverlight Image element, and the image appears.
If you want to change the image, you just call SetPixelColorData again, and then repeat the steps to build a Stream and a BitmapImage. Silverlight’s BitmapImage only loads the PNG once – it doesn’t expect it to change – so we need to build a new one each time round. However, you’re free to pass the same array into SetPixelColorData every time around, and internally, the PngGenerator reuses all the same buffers for building the stream, so the costs of building a new frame aren’t as bad as they might be.
(My first prototype did build everything from scratch each time round. It was about three times slower than the current implementation! Memory allocation and copying start to look expensive when you’re trying to push millions of pixels around several times a second.)
Silverlight and Multicore
Incidentally, the example also happens to be a clear demonstration of Silverlight’s ability to exploit multiple CPU cores on the client side. In fact the time I first ran into the absence of WriteableBitmap in Silverlight was when I sat down to write exactly this Mandelbrot example as an illustration of how to use Silverlight’s multi-threading capabilities. At the time I had to abandon the idea due to the lack of writeable bitmap support. But finally, 6 months down the line, I can use the demo!
The code that performs the calculations to generate the fractal image uses the thread pool to parallelize the work. Crude, but effective. On my quad core desktop, it’s really quick, despite the lack of any clever optimizations that some fractal generators use. And since I first started writing Mandelbrot rendering code on 8-bit micros with 2MHz processors back in the mid 1980s, which used to take about 3 hours to produce a puny 256x256 image, I can hardly believe how fast this runs on new hardware.
Missing Features
The code up on CodePlex is designated version 0.1 because it’s incomplete. It doesn’t support an alpha channel, i.e. no semi-transparent bitmaps. There’s no fundamental reason for this, it’s just that I’ve only had time to spend a couple of evenings on this so far. I plan to add this. I also want to add a mechanism for using raw byte arrays for pixel data instead of having to use Color – there are some performance issues that this will solve. And there are a couple of places where the code is not as efficient as it could be. Right now on my machine, if I generate 1024x768 bitmaps as fast as possible it only manages about 15-20 frames per second, and I believe I can improve on that. (Although the tests I’ve performed suggests this is never going to do better than 30fps for 1024x768 on current hardware unless Silverlight evolves to provide native support for this feature.)
Getting Started
The code is up on the SlDynamicBitmap project on CodePlex. You can download a ZIP containing both binary and source, or you can browse the source online. Enjoy!
[Update, 2008-11-07: Pete Brown pointed out to me that this has already been done... Joe Stegman has some code that enables similar stuff. However, as far as I can tell my library is faster. Modifying Joe's code to do a 1024x768 image, it doesn't seem to be able to do more than about 5 frames per second. That's the same speed I got with my code before I started reducing the amount of copying and memory allocation. So while my contribution isn't original, its a few times faster than the existing work. So I hope it's still useful.]