IanG on Tap

Silverlight 3 at PDC

Sat, 05 Sep 2009 15:42:01 GMT

I’m absurdly excited about PDC this year because for the first time, I’ll be speaking there! Rich Griffin and I will be talking about Silverlight 3. This is one of the day-long workshops that run the day before the full conference starts.

I’m pleased to be talking with someone who I’ve worked with as a fellow developer—Rich and I have collaborated on some of the many WPF and Silverlight projects we’ve worked on over the last few years. We both have a lot of experience solving real problems, and we’re looking forward to bringing that to the workshop—we will be talking about building real software.

PDC is just over a couple of months away, so we haven’t quite set our talks in stone yet. We’d both love to hear from you if you’re contemplating coming to our session and have ideas on what you’d like to see. That said, we obviously need to give people a fair idea of what to expect so they can plan now. So here’s what we’re currently thinking.

We see the day breaking down into four pieces, taking roughly a quarter of the day each: The Ideas of Silverlight; Connecting; Software Design; Working with Designers.

The Ideas of Silverlight

I want to start by saying what this will NOT be. We absolutely don’t want to do a laundry list of Silverlight features, or a straight forward walk through of the developer and the design tools. We believe that a sound understanding of a critical set of ideas at the heart of Silverlight is the secret to true proficiency.

So we plan to talk about some features that make Silverlight very different from most other UI technologies, and how they combine in ways you might not expect. In particular, we plan to describe the roles played by templates, and styles, the structure of controls, the surprising importance of data binding, and why something called MVVM is so important. We also want to talk about some of the practical issues around deployment, testing, and tools.

Connecting

Silverlight is for the internet, so our second section will be all about how to connect with the rest of the world. We plan to look at WCF, REST, and POX in some depth. We’ll discuss the tension between security and flexibility, and some practical resolutions. We’ll talk about RIA services too, although since that’s a topic that could easily fill a day on its own, we don’t intend to cover it in depth—our current thinking is that we want to make it clear what it’s for and when you would or wouldn’t use it, illustrating just enough of it that you’ll see its overall approach. (Although if you think we should dedicate a large chunk of the day to RIA services, we’d like to hear from you, and we’d like to know what you think we should drop to make space for it.)

Software Design

It’s all very well to look at the core ideas, features and technologies of Silverlight, both on the inside, and where your code meets the rest of the world, but how do you tie it all together in practice? We’ll look at strategies for structuring your application’s design. (That’s software design, as opposed to interaction design or visual design, by the way.) We expect to go into more detail on MVVM here. We’d also like to talk about where technologies such as MEF and Prism can fit in.

Working with Designers

Yes, PDC is for developers. So is this session. Building Silverlight applications that look great takes more than throwing a finished application over the wall to where the visual designers work and asking them to slap a layer of sparkle on at the last minute. Integrating good visual design into your application requires planning. Developers need to understand the process and get involved. The way to get the best results is for developers and designers to work together closely. We’ll talk about the steps you need to take as developers to make this possible. We’ll also show how Blend fits into the world of the developer, and we’ll offer tips for how it can make your life easier when creating Silverlight applications.

So that’s the current plan. I’ll be posting more as we refine our ideas, and most likely tweeting too—we might do a couple of polls to gauge the level of interest in the various topics, so now is the perfect time for feedback. We look forward to your suggestions!

(If you’re wondering about why I only just mentioned this, when it was all announced a few weeks ago, I came on board to speak just as I went off on an extended holiday, and now that I’m back, I’m feverishly trying to catch up with everything.)

Designing a WPF Timeline Control Part 1: Structure

Thu, 04 Jun 2009 02:38:22 GMT

For a change, I’m involved in an endeavour I can blog about. I’m working with Brian Randell, Matthew Adams, and Felix Corke on a project that involves building a kind of timeline control. This blog describes the thinking behind the control’s structure—WPF controls tend to end up looking very different from their equivalents in anything based on classic Win32 UI underpinnings such as Windows Forms, and I thought it might be interesting to explain how and why the differences come about.

We’re not quite ready to show the code for this yet, as it’s still a work in progress, but we’ll make the source available before too long.

Requirements

The requirements were a little nebulous, because we set out to build a control for fun, rather than for any particular application or customers. So one of the most important requirements was that it be interesting to build. Nonetheless, we had some ideas about what sorts of things we wanted to be able to do. For example we wanted a way to show files found via Windows Search (or, as I find it hard to stop calling it, Windows Desktop Search). Positioning and grouping files by dates along a timeline seemed like it might be an interesting way to visualize search results.

Here’s an early prototype we built to get a rough idea of how it might work with some real data. I’m jumping ahead here a little, but a picture makes it easier to understand what we had in mind:

The user can drag the whole timeline left or right to move around, and zoom in and out with the mouse. The items are all expanded in that example above. It doesn’t have to be that way though—here’s how it might look when the user has zoomed out a bit and only has one item expanded:

Clearly we have a bit of a problem with the captions on the timeline here, but you can see how this gives an impression of which bits of the timeline have a lot of content, and which bits are empty. Notice that in this example we also experimented with vertically stacked bars to indicate how much stuff was inside each group, to give a better idea of information density over time.

Now obviously this example looks horrible. Our goal at this stage was to try out some ideas to see if they would work before building something properly. As you’ll see in future blogs, it started looking a lot better once Felix got his hands on it.

Another feature we wanted was the ability to support multiple levels of detail—we envisaged the initial view as being a bunch of blobs on the timeline indicating files placed by date, which could be expanded to reveal details, possibly multiple levels of detail. So the examples above aggregate a bunch of files that appear in quick succession as a single blob, showing a list of all the files if you click on that blob, but we might want to go further, with some sort of details fly-out available on each of the items in that list.

We also wanted to avoid assuming much about the nature of the data source. We had discussed various ideas for what we might want to display along a timeline—I forget the complete list, but TV schedule information, appointment booking, gaming, and project management came up as possible application areas.

So our ‘spec’ was pretty vague (although I’ve had worse). We decided to start with the file system/search example, and see where it took us—more a voyage of exploration than a software project. Perhaps not the most efficient approach to software development, but if you’re building a UI component of a kind you’ve not attempted before, it’s not a bad idea to be at least a little chaotic—your first design is unlikely to be your best, so some space to experiment is useful.

But where to start? If you want a custom control to get the best out of WPF, it’s a good idea to try and align yourself with WPF’s philosophy. But what is that, exactly?

Composition is King

I’ve long thought that WPF’s single biggest benefit is its composition-based approach. Examine most of the built-in controls, and you’ll find that they do less work than you might think, deferring to other WPF components to get the job done. And these subordinate components are almost invariably multi-purpose—they show up in other WPF controls, and can be used where appropriate in your own custom controls.

Take the ListBox. Amongst the things it doesn’t know how to do are: size and position the list items; render list items; provide selection highlighting; provide its own appearance. For these tasks it defers to its items panel, its item template, its item containers, and its control template respectively, and you can plug in your own objects for any of these, to customize the control. Of these, only the item container type (ListBoxItem) is specific to the ListBox, and even that shares a lot in common with other list controls’ items containers.

A single monolithic black box of a control might be in keeping with long-standing traditions of the Windows 3rd party control market, but it’s not the WPF way.

Areas of Responsibility

The WPF way is for each area of responsibility to be owned by a separate type. This is not a new idea, of course, but it’s one that seems to be less well represented in the UI layer than elsewhere.

When developing new UI element types, we design them to be combined in the ways we need them to work, and with luck, it will be possible to combine them usefully in ways we did not first anticipate. In fact that second part often requires more than luck—types are rarely reusable in practice until you’ve tried to use them in multiple scenarios and have fixed the problems that made them less useful than you had hoped. We haven’t done that yet for these controls, so this is probably not yet the definitive design.

So, what are the areas of responsibility we think exist for our timeline UI? Here’s a breakdown, along with a suggestion of what kind of object might fit the bill:

Generation of UI elements from a data source (ItemsControl?)
Visual representation of the ‘line’ in the timeline control (ControlTemplate?)
Control to provide panning/scrolling (ScrollViewer?)
Positioning of UI elements along a line (Panel?)
Generated element type to be positioned on the line (some sort of ContentControl or Expander?)
Visual representation of an item on the line (ControlTemplate?)
Expanded view of content of generated element (DataTemplate?)

You’ll notice that we have borrowed shamelessly from the pattern established by WPF’s various items controls. We can produce a very similar-looking list for ListBox:

Derives from ItemsControl, enabling it to generate items from a data source
Being a control it has a ControlTemplate to define its appearance
Relies on ScrollViewer to provide a scrollable area
Uses a VirtualizingStackPanel to position its elements vertically
Generates a ListBoxItem (which is a ContentControl) for each item in the list
ListBoxItem is a control, so it also has a template
ItemTemplate or implicit data template used to render items in the list

If WPF already has a pattern that meets your needs, it’s usually a good idea to follow suit. Anyone already familiar with WPF will find your code easier to follow, and it also means you’re on a path likely to lead somewhere useful. But the degree of similarity here raises an obvious question: could we get away without creating any custom elements at all?

No Custom Elements?

It’s surprisingly common for WPF applications to be able to cobble together various WPF elements to achieve effects that would have required a custom control in Windows Forms. Could we do that here?

As an early experiment we built an application that queried Windows Search for all the bitmaps on a system, used LINQ to group them by month, and dumped the results into an ItemsControl to produce a timeline. To keep it simple we just used a StackPanel to get the items in order, but not yet positioned precisely according to date. The ItemTemplate contained a list of the files, using an Image element to render the actual image file. This list was initially collapsed, but could be expanded by clicking a button.

It rapidly became obvious that this wasn’t going to fly from a performance perspective. Even with the image lists initially collapsed, mere invisibility wasn’t enough to prevent the Image elements from being created. This meant that when the application started, it would attempt to load every single bitmap found by Windows Search. We let it run for a while to enjoy the spectacle of a single process consuming over 4GB of memory—it’s entertaining to pretend you have a justification for owning a 64-bit system—before giving up and killing it off.

The naive approach clearly wasn’t going to work, but we could make a small change to reduce the memory footprint: we decided to take the image loading out of WPF’s Image element’s hands. Instead of binding the Image element’s Source directly to the bitmap path, we wrote some code to load the image. This gave us the option to specify a size for the loaded image—since we were only showing thumbnails, we told WPF to decode the image to a much smaller size, just 40x30 pixels. This way it was at least conceivable that all the images we were trying to load might fit in memory. This ‘improved’ matters in the sense that the application progressed from not working at all to working unusably slowly.

It was starting to look like some form of virtualization might be necessary.

(This is of course all utterly predictable. Less subtle minds than your own might even conclude that we were idiots for bothering to try this naive approach. But I’ve learned that it’s wise to start your performance investigations with the parts you can predict. That way you get to find out which of your predictions were wrong. Plus, you might get to observe things going wrong in a slightly different fashion from the one you expected, which can offer useful insights. Indeed, besides the rather obvious expected memory consumption problems, we observed some CPU-bound performance issues in the image handling in some places that surprised us, and which uncovered an opportunity to significantly speed up our eventual load time performance which might have been harder to spot at a later stage. I’ll cover this in a later blog.)

A couple of options stood out for improving the bitmap handling. We could get slightly cleverer with the expansion of the per-group list: rather than merely collapsing the list until the user expanded it, we could arrange not to create the list at all until needed—that should avoid the premature (and rather expensive) loading of every bitmap on the system. Or we could use a virtualizing panel for the main timeline items control, so that the groups’ UIs didn’t get instantiated at all until they appeared on screen. We tried both.

Either of these on their own produced a profound improvement—the application went from “Fail slowly” to “Sub-second load” performance. However, using just one or the other wasn’t enough to get that “sub-second” quite “sub-” enough—we were seeing a half second UI freeze between the Windows Search results coming back and the results being displayed. Only with both techniques applied was the performance satisfactory.

So we need at least one custom element: our panel needs to virtualize, and the only built-in virtualizing panel is the VirtualizingStackPanel. That doesn’t give us the positioning we want: we need to be able to select a position based on some date. Only Canvas and Grid make that sort of control possible, and neither of those can virtualize. So we need our own panel. We provisionally chose to call this a VirtualizingOffsetPanel, for want of a better name.

We also decided to build a custom control to perform the expansion. It is technically possible to get the deferred list realization needed with an Expander: the trick is to use a ContentTemplate rather than just making our image list the child of the Expander. However, we’d be relying on a side effect of an undocumented Expander behaviour. I’ve gone down a similar path before with tree view item expansion, and didn’t enjoy it. So we decided to make our own expander (which is a pretty simple thing to construct) in order to give us control over what actions occur and in which order when an item is expanded. And since this control was going to be instantiated for each item on the timeline, we decided to designate this as the item container for our timeline control. And since a custom item container requires a custom items control, we have a reason for a third custom element, the TimelineControl itself.

So it’s now looking like we’re going to write three custom elements: a TimelineControl, a TimelineGroupItem, and a VirtualizingOffsetPanel. Yes, for that second one, the name TimelineControlItem might have looked more consistent, but TimelineGroupItem felt like a more accurate description of what it represented. So far, it still feels that way, so we’re still calling it that.

Aside: To Group or Not to Group

If you’re familiar with the capabilities of WPF data binding, you might be thinking: hold on, why aren’t you using CollectionView.GroupDescriptions and a GroupStyle? Well we tried that, but it seems that grouping in ItemsControls has the side-effect of disabling virtualization. And since we need virtualization to get acceptable load performance, that’s a non-starter.

In any case, this didn’t seem like a great loss. After a fiddling around with Windows Search for a while, LINQ started to look like an easier and more flexible way to manage grouping.

Prototyping Matters

Trying to build a working example with no custom elements at all was an important exercise. We had outlined a rough design of what we thought we probably wanted before we started. This up-front design was by no means ‘big’, but we got something wrong nonetheless. We originally thought we’d want not just the timeline and group item controls, but also an individual timeline item control. But in working through the prototype, no clear need for such a control emerged—it seems that having established a need for the TimelineControl, TimelineGroupItem, and VirtualizingOffsetPanel items, a combination of content controls and data templates was enough to finish off the job.

In future blog entries I’ll talk about how we went about building each of these custom elements.

Oslo WPF and Silverlight Demos

Thu, 30 Apr 2009 15:49:57 GMT

This week I've been in Oslo, teaching a combined Silverlight and WPF course. For those of you who attended, here are the demos for download. And for those of you who didn't attend, well, you can download the demos too if you like.

Silverlight and WriteableBitmap

Fri, 07 Nov 2008 12:19:55 GMT

I’ve just released a library to CodePlex that implements a feature missing from Silverlight: the ability to generate bitmaps from pixels in code at runtime. WPF provides this through its WritableBitmap class, but Silverlight 2 doesn’t have that. My SlDynamicBitmap project offers a solution.

I was inspired to create this by the recent reports of the Silverlight port of Quake. From what the author says, it’s clear that they must have devised some sort of mechanism for pushing raw pixels onto the screen, something Silverlight really doesn’t help you with. I’ve been toying for a while with an idea about how you might solve this, but hadn’t tried it before. Quakelight shows that it’s definitely possible, which encouraged me to try out my idea.

Incidentally, I have no idea how Quakelight solves this problem. It may well have a much more elegant solution than the ghastly hack I’m perpetrating here.

Why Would I Want This?

Before I show how it works, it’s worth reviewing why you would even want a Silverlight equivalent of WPF’s WriteableBitmap – what’s it for? It’s useful for when you want to generate pixels at runtime based on some kind of algorithm. Ray tracing is one classic example – the computer constructs an image one pixel at a time, and in order to display the results, you need some way of controlling every single pixel on a region of the screen, rather than using higher level primitives.

To illustrate this I’ve chosen another classic example: a Mandelbrot set fractal renderer.

In case you’re not familiar with the Mandelbrot set, it’s a fractal generated with a deceptively simple process. It’s a striking demonstration of how incredibly basic non-linear systems can exhibit behaviour that is very complex, and which shows some very surprising geometric scaling features.

But for my present purpose, the interesting thing about the Mandelbrot set is that you end up with a program that generates a lot of pixels and needs some way to get them on the screen. In WPF, this is the kind of scenario that WriteableBitmap is for. To enable this in Silverlight, I’ve written a new class called PngGenerator.

It’s Called What?

I’ve chosen to call the main type in this library PngGenerator, and not WriteableBitmap. This may seem like an odd choice, but there are two good reasons for it. First, I regard this code as a workaround for a missing feature, not a proper solution, so I’m hoping that Silverlight will eventually get a WriteableBitmap of its own. So it would have been unhelpful of me to use the same name. Second, the usage model has ended up being pretty different – it’s not really possible to write a custom class in Silverlight that’s a drop-in replacement for WPF’s WriteableBitmap. Only Microsoft can provide us with that – it would require support from the Silverlight plug-in itself to build an exact replica.

So that’s why it’s not called WriteableBitmap. Why PngGenerator? That’s because of the ghastly hack I used to make this work.

The Ghastly Hack

The PngGenerator works by converting an array of pixel colour values into a Stream of bytes that represent the image as a PNG file. (I chose PNG rather than JPEG because JPEG files typically use lossy compression, so you don’t get precisely the pixels you asked for. It has since been pointed out to me that JPEG does now support lossless images too, so I guess you could use either format. BMP would have been a whole lot easier as it’s a simpler format than either, but Silverlight only supports PNG and JPEG.)

If Silverlight had an equivalent of WPF’s PngBitmapEncoder class, this would have been pretty straightforward to write. Unfortunately Silverlight doesn’t contain any code to generate PNGs from pixels. So most of the code in this library is a PNG builder.

By the way, I would not recommend using this code to generate PNG files for general use. It does a couple of rather crufty things. It only generates the bare minimum contents required to be just enough of a legal PNG file for Silverlight to render it. And it also doesn’t compress the data. The ‘deflate’ compression format mandated by PNG lets you put in so-called ‘non-compressible’ blocks. You’re supposed to use this for any data that gets bigger when you ‘deflate’ it. Some data is just hard to compress, and the idea is that you mitigate this problem by storing that data verbatim. But I’m storing all the pixel data this way to provide the simplest possible path from pixel to screen.

So it’s a little too specialized to be a useful general-purpose PNG generator.

I’ve seen nastier ways of solving this by the way. I’ve seen people generate Silverlight content made up of hundreds of tiny rectangles in an attempt to render their own pixel data! This doesn’t work all that brilliantly – you get anti-aliasing artefacts, and it’s mind-bogglingly slow. Unpleasant though the technique I’m using is, it does at least produce good quality results, and is tolerably quick, although not as good as a native WriteableBitmap built into Silverlight could be.

Using the PngGenerator

To use the PngGenerator, you do this sort of thing:

PngGenerator pngGen = new PngGenerator(640, 480);
Color[] m_pixelData = GenerateMyPixels();
pngGen.SetPixelColorData(m_pixelData);
BitmapImage imgSource = new BitmapImage();
imgSource.SetSource(pngGen.CreateStream());
myImageElement.Source = imgSource;

That may seem slightly less direct than necessary. I’m contemplating adding a CreateSource() method that builds the BitmapImage for you, to make the code a couple of lines shorter. But the verbose approach required by the current version does have the benefit of showing the process relatively clearly.

Walking through it, we build a new PngGenerator, telling it the image dimensions we require – 640x480 in this case. Then we build an array of Color values indicating the pixel colours we want. (So GenerateMyPixels() here stands for whatever code you’ve got that generates pixel values.) We pass these pixels to the PngGenerator’s SetPixelColorData method. Then we construct an instance of Silverlight’s BitmapImage type, and call its SetSource method. (BTW, BitmapImage.SetSource is the Silverlight feature that makes this possible. It accepts any System.IO.Stream object, and expects it to contain either a PNG or a JPEG.) The PngGenerator’s CreateStream method returns a Stream that contains a PNG representing the pixels passed in to SetPixelColorData. Finally, we use the BitmapImage as the Source property of a Silverlight Image element, and the image appears.

If you want to change the image, you just call SetPixelColorData again, and then repeat the steps to build a Stream and a BitmapImage. Silverlight’s BitmapImage only loads the PNG once – it doesn’t expect it to change – so we need to build a new one each time round. However, you’re free to pass the same array into SetPixelColorData every time around, and internally, the PngGenerator reuses all the same buffers for building the stream, so the costs of building a new frame aren’t as bad as they might be.

(My first prototype did build everything from scratch each time round. It was about three times slower than the current implementation! Memory allocation and copying start to look expensive when you’re trying to push millions of pixels around several times a second.)

Silverlight and Multicore

Incidentally, the example also happens to be a clear demonstration of Silverlight’s ability to exploit multiple CPU cores on the client side. In fact the time I first ran into the absence of WriteableBitmap in Silverlight was when I sat down to write exactly this Mandelbrot example as an illustration of how to use Silverlight’s multi-threading capabilities. At the time I had to abandon the idea due to the lack of writeable bitmap support. But finally, 6 months down the line, I can use the demo!

The code that performs the calculations to generate the fractal image uses the thread pool to parallelize the work. Crude, but effective. On my quad core desktop, it’s really quick, despite the lack of any clever optimizations that some fractal generators use. And since I first started writing Mandelbrot rendering code on 8-bit micros with 2MHz processors back in the mid 1980s, which used to take about 3 hours to produce a puny 256x256 image, I can hardly believe how fast this runs on new hardware.

Missing Features

The code up on CodePlex is designated version 0.1 because it’s incomplete. It doesn’t support an alpha channel, i.e. no semi-transparent bitmaps. There’s no fundamental reason for this, it’s just that I’ve only had time to spend a couple of evenings on this so far. I plan to add this. I also want to add a mechanism for using raw byte arrays for pixel data instead of having to use Color – there are some performance issues that this will solve. And there are a couple of places where the code is not as efficient as it could be. Right now on my machine, if I generate 1024x768 bitmaps as fast as possible it only manages about 15-20 frames per second, and I believe I can improve on that. (Although the tests I’ve performed suggests this is never going to do better than 30fps for 1024x768 on current hardware unless Silverlight evolves to provide native support for this feature.)

Getting Started

The code is up on the SlDynamicBitmap project on CodePlex. You can download a ZIP containing both binary and source, or you can browse the source online. Enjoy!

[Update, 2008-11-07: Pete Brown pointed out to me that this has already been done... Joe Stegman has some code that enables similar stuff. However, as far as I can tell my library is faster. Modifying Joe's code to do a 1024x768 image, it doesn't seem to be able to do more than about 5 frames per second. That's the same speed I got with my code before I started reducing the amount of copying and memory allocation. So while my contribution isn't original, its a few times faster than the existing work. So I hope it's still useful.]

Oslo WPF and Silverlight Demos

Fri, 17 Oct 2008 15:07:43 GMT

For those of you who saw me in Oslo this week, here are the Silverlight and WPF demos for download.

WPF Course in London

Mon, 15 Sep 2008 16:57:54 GMT

Shameless marketing plug alert!

I’ll be teaching the Pluralsight WPF Fundamentals course in London later this month. (It runs from 29th September to 2nd October, so I guess that’s later this month and also early next month.) The course is for software developers, and offers detailed information on how WPF works, and how to use it effectively.

A Year of Online Training Included

If you attend, you’ll also get access to an online version for a year – this includes recordings of all the talk content and demos. While Pluralsight’s Online Training service is still in beta, this is one of the “select Pluralsight instructor-led courses” that will get you access to the online WPF material today without having to be on our private beta program.

This is why you’ll be asked to sign up for an account if you register for the WPF training online – I dislike it when sites make you create an account just to buy something, but there is a good reason in this case: the login gets you to the online content that’s included as part of the course. And we think the online material makes a great addition – there’s a lot of content crammed into the course, so we hope you’ll find it useful to be able to go back and review it at your leisure.

LINQ Aggregation and Bounding Boxes

Wed, 30 Jul 2008 18:56:27 GMT

I recently worked on a project that included code to calculate the bounding box of a collection of rectangles. The code did more or less what you might expect – it iterated over the collection, and any time a new rectangle wasn’t contained by the current bounds, it enlarged the bounds to fit.

There was nothing wrong with this, but I knew of a cute trick that could do the job with a fraction of the code. I wasn’t being paid to replace working code with cute tricks, so I left it as it was – when it comes to production code I’m not a fan of the “If it ain’t broke, let me take a crack at it” school of coding. So I thought I’d illustrate the idea here instead.

Here’s a one-line implementation of the same logic, using the System.Windows.Rect type in WPF. (Note that this is a very simple value type, not a full UI element. It’s just 4 numbers.)

public static Rect GetBounds(IEnumerable<Rect> rects)
{
    return rects.Aggregate((currBounds, nextRect) => Rect.Union(currBounds, nextRect));
}

That’s pleasingly succinct, although possibly a little obscure if you’re not accustomed to this style of coding. Actually we can go one more notch on the obscure-and-succinct scale. This comes to the same thing:

public static Rect GetBounds(IEnumerable<Rect> rects)
{
    return rects.Aggregate(Rect.Union);
}

LINQ’s standard Aggregate method implements a mechanism often known as fold in functional programming languages. It’s a pretty simple idea: apply some sort of cumulative operation across all items in a list to build up some sort running total. In this example, LINQ’s uses the first item in the list as the starting point. (Alternatively, you can provide an explicit start value.) It passes that in to the function supplied, along with the second item. The result becomes the new running total, which is fed again into the function supplied, this time along with the third item...and so on. Each time round the aggregating function we provide gets the running total as its first argument, and the next item in the list as its second.

It’s easier to show what it does than to describe it. Here’s what that code would do if applied to an array containing five Rects:

Rect.Union(Rect.Union(Rect.Union(Rect.Union(rs[0], rs[1]), rs[2]), rs[3]), rs[4])

Or if you prefer sequential imperative code, you might like to picture it as:

Rect currBounds = rs[0];
currBounds = Rect.Union(currBounds, rs[1]);
currBounds = Rect.Union(currBounds, rs[2]);
currBounds = Rect.Union(currBounds, rs[3]);
currBounds = Rect.Union(currBounds, rs[4]);

As you can see, it’s not particularly magical. It’s nothing you couldn’t implement with a straightforward loop. (Or with tedious duplication, as here.) But it’s an idea that crops up often enough that it’s rather handy to have as general purpose function – you get to specify what you’d like to use as the cumulative operation, and LINQ does the rest for you.

Aggregate can be used to implement some common operations. For example, you can use it to sum together all the numbers in a list:

public static int Sum(IEnumerable<int> numbers)
{
    return numbers.Aggregate((currTotal, nextInt) => currTotal + nextInt);
}

Or you could use it to find the highest number in a list:

public static int Max(IEnumerable<int> numbers)
{
    return numbers.Aggregate((highest, nextInt) => Math.Max(highest, nextInt));
}

or again, since this just forwards the arguments directly to a function, we can skip the lambda and use the more terse form:

public static int Max(IEnumerable<int> numbers)
{
    return numbers.Aggregate(Math.Max);
}

Of course, you don’t need to write these two examples, because LINQ already supplies Sum and Max extension methods for you. But conceptually at least, they’re just a handy shortcut to a specialized form of aggregation.

Getting the total bounding box of all of the rectangles in a list is not so different from finding the highest number – it’s just a slightly different kind of maximum.

Not a particularly astounding idea, admittedly. I just thought it was a neat trick, and wanted to share it.

Visual State in Silverlight Control Templates

Tue, 10 Jun 2008 20:38:46 GMT

Silverlight 2 introduces a mechanism for customizing the dynamic aspects of a control’s appearance. This new model appeared for the first time in the recent beta 2 release, so I thought I’d take a crack at explaining it.

The Role of Visual States

In both Silverlight and WPF, a control’s appearance can be customized by providing a ‘template’. Templates are typically written in Xaml, and all the usual visual elements are available. But it’s usually not enough just to define the static structure of a control’s visuals.

Most controls change their appearance when the user interacts with them. We expect elements to signal their ability to respond to input with subtle visual shifts when the mouse moves over them, for example. And this is where Silverlight’s ‘visual states’ come in. You can associate animations with visual state changes.

(If you know WPF you’ll be familiar with triggers, which enable exactly this sort of connection between interaction and visuals. In an omission some WPF veterans find surprising, Silverlight doesn’t support triggers. Visual states are what we use instead.)

States and State Groups

To work with Silverlight’s visual state system, we must get to grips with states and state groups.

A control might have many facets, each with its own visual representation. For example, a checkbox may be checked, or unchecked. And it may also be focused or unfocused. The checked/unchecked state is wholly independent of the focused/unfocused state: the keyboard focus can move into and out of the control without changing the checked state. But not all states can be freely combined – obviously a checkbox cannot be both checked and unchecked (although it might be neither – there’s a third, ‘indeterminate’ state, but that’s mutually exclusive with both checked and unchecked).

To clarify how states may combine, states are partitioned into state groups. For example, CheckBox defines three state groups:

Group Name	States
CommonStates	Normal, MouseOver, Pressed, Disabled
CheckStates	Checked, Unchecked, Indeterminate
FocusStates	Focused, ContentFocused, Unfocused

So if we ask “What is a CheckBox’s state?” the answer comes in three parts: a state from the CommonStates state group, one from CheckStates, and one from FocusStates. A control effectively maintains multiple state machines, one for each state group. Here are some examples:

For the CheckStates state group, all three checkboxes here are in the Checked state. With respect to the FocusStates state group, the first checkbox is in the Focused state, while the second and third are in the Unfocused state. Finally, as far as the CommonStates state group is concerned, the first and third checkboxes are in the Normal state, while the second is in the MouseOver state. (The mouse cursor isn’t shown here, but you can see that the control has changed appearance.)

A control will advertise the states it offers and the group to which each state belongs with the TemplateVisualState attribute, as you can see here:

[TemplateVisualState(Name = "ContentFocused", GroupName = "FocusStates")]
[TemplateVisualState(Name = "MouseOver", GroupName = "CommonStates")]
[TemplateVisualState(Name = "Focused", GroupName = "FocusStates")]
[TemplateVisualState(Name = "Checked", GroupName = "CheckStates")]
[TemplateVisualState(Name = "Unchecked", GroupName = "CheckStates")]
[TemplateVisualState(Name = "Indeterminate", GroupName = "CheckStates")]
[TemplateVisualState(Name = "Pressed", GroupName = "CommonStates")]
[TemplateVisualState(Name = "Disabled", GroupName = "CommonStates")]
[TemplateVisualState(Name = "Unfocused", GroupName = "FocusStates")]
[TemplateVisualState(Name = "Normal", GroupName = "CommonStates")]
public class CheckBox : ToggleButton ...

We now know that a control will maintain a simple state machine for each state group. So what? To do anything useful with this, we must use the VisualStateManager class.

VisualStateManager

The VisualStateManager class lets us connect animations to state changes. It defines an attached property we can use to establish these connections in Xaml.

So that we can see this in action, let’s start with a simple template for a CheckBox:

<CheckBox Content="Check!" IsChecked="True">
  <CheckBox.Template>
    <ControlTemplate>
      <Grid>
        <Grid.ColumnDefinitions>
          <ColumnDefinition Width="19" />
          <ColumnDefinition />
        </Grid.ColumnDefinitions>

        <vsm:VisualStateManager.VisualStateGroups>
        </vsm:VisualStateManager.VisualStateGroups>

        <Rectangle x:Name="tickBox"
          Stroke="Blue" Width="12" Height="12" HorizontalAlignment="Left"
          RadiusX="2" RadiusY="2" Fill="Aqua" />

        <Path x:Name="tick" Opacity="0"
              Width="12" Height="12" HorizontalAlignment="Left"
              Stroke="Black" StrokeThickness="2"
              Data="M3,6 L5,9 9,2" />

        <ContentPresenter Grid.Column="1" />
      </Grid>
    </ControlTemplate>
  </CheckBox.Template>
</CheckBox>

This defines some pretty basic visuals – just a simple box and tick, plus a placeholder. It looks like this:

Note that although I’ve got a Path element in the shape of a tick, it’s currently invisible. Later I will supply a couple of animations to make the tick visible and invisible.

Notice that the example above already contains an empty <vsm:VisualStateManager.VisualStateGroups> element – it’s just after the grid’s column definitions. I’ve been working with Xaml for half a decade now, but for anyone who hasn’t had that pleasure, I suppose it might not be totally obvious that this element sets a property on the containing Grid! In general, Xaml elements of the form <Xxx.Yyy> denote a property. The part after the ‘.’ is the property name, so that tells us that this is the VisualStateGroups property. And the part before the ‘.’ is the name of the class that defines the property – so here we can see this property is defined by the VisualStateManager class. Again, if you’re not familiar with Xaml, it may seem a bit funky that I can set property defined by VisualStateManager on Grid, a completely unrelated class. This is an example of a Xaml feature called attached properties– they’re a way of bolting new properties onto existing types.

For some reason the VisualStateManager class isn’t in the Silverlight Xaml namespace, so we’ve had to qualify it with a namespace prefix. (I’ve chosen “vsm:” in this example, short for VisualStateManager.) For this to work, I need to have defined that prefix somewhere in my Xaml. It’s not shown in the above snippet, but I put it at the root element, alongside the other namespace declarations:

xmlns:vsm="clr-namespace:System.Windows;assembly=System.Windows"

Normally you’d let Blend do all this for you. The June preview of Expression Blend 2.5 provides new features for working with the VisualStateManager. But my goal here is to explain the underlying mechanisms, so I’ll steer clear of Blend for now.

Note: in the Silverlight 2 beta 2 preview bits, the Visual Studio Xaml editor reports an error on the line with this property. It says: “The attachable property ‘VisualStateGroups’ was not found in type ‘VisualStateManager’.” But it’s lying – the attachable property is present, and although you get what looks like an error in Visual Studio. But you’ll find the project will happily build and run despite this ‘error’, and if you close the Xaml editor, the error goes away. It’s not really an error. That’s beta software for you.

So what do we put inside this property? We can provide an animation to run for each state. On top of that, we can optionally get fancy, and provide animations that handle the transition between a specific pair of states. Let’s start with the simpler case.

Per-State Animations

The simplest way to use the visual state manager is to define an animation to run when a particular state is entered. The following example is about as simple as it gets:

<vsm:VisualStateManager.VisualStateGroups>
  <vsm:VisualStateGroup x:Name="CommonStates">
    <vsm:VisualState x:Name="MouseOver">
      <Storyboard>
        <ColorAnimation Storyboard.TargetName="tickBox"
            Storyboard.TargetProperty="(Rectangle.Fill).
                                (SolidColorBrush.Color)"
            To="PaleGreen" Duration="0:0:0.5" />
      </Storyboard>
    </vsm:VisualState>
  </vsm:VisualStateGroup>
</vsm:VisualStateManager.VisualStateGroups>

Notice that I am required to say first of all which state group I’m interested in, with a VisualStateGroup element. In this case, I’m looking at the CommonStates group. And inside the group, I provide a VisualState element for each state I wish to animate. To keep things simple, I’ve just handled one state here: MouseOver.

With this markup in place, when the CommonStates state group enters the MouseOver state (i.e. when the user moves the mouse over the control), the Storyboard I’ve supplied here will run. This particular animation changes the checkbox’s fill to pale green.

Since I’ve only handled one state, the checkbox will get stuck in that state – I’ve not yet provided any animations to put it back how it was. And I’m also missing some other states, so to complete the job, I should add the following inside that VisualStateGroup after the first VisualState:

<vsm:VisualState x:Name="Pressed">
  <Storyboard>
    <ColorAnimation Storyboard.TargetName="tickBox"
                    Storyboard.TargetProperty="(Rectangle.Fill).
                    (SolidColorBrush.Color)"
                    To="Navy" Duration="0:0:0.5" />
  </Storyboard>
</vsm:VisualState>
<vsm:VisualState x:Name="Disabled">
  <Storyboard>
    <ColorAnimation Storyboard.TargetName="tickBox"
                    Storyboard.TargetProperty="(Rectangle.Fill).
                    (SolidColorBrush.Color)"
                    To="Gray" Duration="0:0:0.5" />
  </Storyboard>
</vsm:VisualState>
<vsm:VisualState x:Name="Normal">
  <Storyboard>
    <ColorAnimation Storyboard.TargetName="tickBox"
                    Storyboard.TargetProperty="(Rectangle.Fill).
                    (SolidColorBrush.Color)" Duration="0:0:0.5" />
  </Storyboard>
</vsm:VisualState>

Note that the last of these doesn’t specify the To attribute for the animation. As is usual for animations in Silverlight and WPF, this means ‘animate back to the base value’, which is exactly what you’d hope for when returning to the Normal state.

I’m going to use a similar but cruder trick to handle the checkbox state. The following goes after the first VisualStateGroup element:

<vsm:VisualStateGroup x:Name="CheckStates">
  <vsm:VisualState x:Name="Checked">
    <Storyboard>
      <DoubleAnimation Storyboard.TargetName="tick" 
                       Storyboard.TargetProperty="Opacity"
                       To="1" Duration="0" />
    </Storyboard>
  </vsm:VisualState>
  <vsm:VisualState x:Name="Unchecked" />
</vsm:VisualStateGroup>

As before, this declares that I want to handle state changes for a particular state group – the CheckStates group this time. And I’ve provided VisualState elements to tell Silverlight what to do when entering either the Checked or Unchecked states.

The Checked storyboard is slightly odd: it has a Duration of 0. This is a case of trigger envy – in WPF I’d just write a simple trigger that says when IsChecked is true, the ‘tick’ element should be visible. But Silverlight doesn’t have triggers, so instead I’ve had to use an animation that runs instantaneously to get the same effect.

The Unchecked handling is even stranger: I’ve provided an empty VisualState element. The significance of this is that I’m telling Silverlight that I care about this state, but that I have no new animations to run. And the important, non-obvious point is that when a new VisualState element comes into play, Silverlight deactivates any animations that were previously in effect from other VisualState elements for that state group.

And even though my Checked animation has a duration of 0, it remains active until we enter the Unchecked state. Animations in Silverlight (and WPF) usually continue to have an effect even after they complete – if you animate something from red to green, it will remain green once the animation finishes. (You can change that if you like – this is controlled with the animation’s FillBehavior property.) So in this case, the Checked animation completes instantly, but remains active, holding the ‘tick’ element’s Opacity property at 1 until the animation is deactivated. That deactivation happens when we enter the Unchecked state.

So it’s quite common to see empty VisualState elements. They’re usually present to deactivate any other animations for the state group.

Note that the example so far is incomplete. I’ve not handled the Indeterminate state in the CheckStates group. And I’ve also not handled the FocusStates state group at all. Nor do I plan to – I’m just trying to show the model here, and to fill in the gaps would mean more of the same.

(If you want to see a fuller example, I’ve put focus state handling into the version available for download: SilverlightVisualStateDemo.zip. If you want to run the application online, it’s up at http://www.interact-sw.co.uk/slapps/visualstate/ Note that these versions also include the feature described in the next section.)

Transition Animations

Whereas the VisualState element we’ve just seen defines the animation to use when entering a particular state, a transition animation is more specific: you can define the animation to run when transitioning between a particular pair of states.

Note that transition animations do not replace the VisualState animations. A transition animation runs before the VisualState animation. For example, suppose you’ve defined a transition animation for the transition from the Pressed to Normal states in the CommonStates group. Silverlight will first run this transition animation, and once that has finished, it goes on to run the state animation.

You might think that you could use transition animations as a replacement for state animations – why not provide only a transition, and leave out the final state animation? That turns out not to work: Silverlight requires you to specify the duration of your transition, and it will deactivate the transition animation at the end. And if you leave off the duration, the duration becomes 0 at which point the animation effectively never runs at all.

This constraint ensures that no matter how a control entered a state, and no matter which transition animation ran as a result, it always ends up looking the same when it’s in that state, once all the animations have played out.

To put it another way, Silverlight will always eventually run the animation for the target state, as specified in the relevant VisualState, but before it runs that you get to insert something specific to the preceding state. This gives you just enough power to ensure that every possible state change looks coherent, while minimizing the scope for chaos.

Here’s a simple and slightly contrived example:

<vsm:VisualStateGroup x:Name="CommonStates">
  <vsm:VisualStateGroup.Transitions>
    <vsm:VisualTransition From="Pressed" To="Normal" Duration="0:0:0.5">
      <Storyboard>
        <ColorAnimation Storyboard.TargetName="tickBox"
                      Storyboard.TargetProperty="(Rectangle.Fill).
                      (SolidColorBrush.Color)"
                      To="Red" Duration="0:0:0.5" />
      </Storyboard>
    </vsm:VisualTransition>
  </vsm:VisualStateGroup.Transitions>
  <vsm:VisualState x:Name="MouseOver">
    ... as before ...

This has a VisualStateGroup for the CommonStates group as before, but I’ve added a Transitions property. Inside this I’ve added a VisualTransition defining an extra animation to be run prior to the animation for the Normal state if we are coming into that state from the Pressed state.

So if the user presses the button down on the CheckBox, and then moves away before letting go, the check box will briefly go red thanks to this transition animation, and then the Normal state animation will run, returning the control back to its standard appearance.

Note that you are allowed to leave off either the To, or From attributes from a VisualTransition. This turns it into a kind of wildcard transition. With just a From attribute, the transition will run any time moving from the named state. With just a To attribute, it will run any time moving to the named state.

You may wonder what is the point of a transition animation with just a To attribute. What does that offer that I can’t do more easily with a normal VisualState animation? Well there is one subtle but useful difference. VisualState animations run any time a state is entered, even when the control first appears. Transition animations do not. Consider our checkbox example – you might want the tick to fade in and out rather than coming and going abruptly. You could get that effect by adding a non-zero duration to the VisualState animation for the Checked state shown earlier. However, if the checkbox happened to be checked when it first appeared, the animation would run then too, which might not be what you want. By contrast, a transition animation with To="Checked" will not run when the control first appears, but will run any time the control enters the Checked state from some other state.

You can even leave off both To and From, in which case the transition will be inserted before every state change for that state group.

Designer Support

In summary, you get to define animations that run when the control enters a particular state, and you may also define specialized transition animations that run just before the main animation. It’s certainly different from WPF’s trigger mechanisms, and while it lets you achieve a lot of the same goals, it’s not quite as flexible – you only get to write state animations for the visual states the control author has decided to offer, whereas triggers can use any property.

However, there’s one thing visual states do better than triggers: designer support. That’s not to say that WPF triggers aren’t supported by Blend – it does in fact support them. But you have to work harder with triggers. The benefit of the visual state model is that Blend can see exactly what states a control offers, and it can help a visual designer fill in all the necessary pieces. By contrast, triggers are so flexible that it’s difficult for a design tool to know where to start.

If you’re happy cranking out Xaml by hand, you’ll probably miss triggers. But if you live in Blend, visual states offer useful benefits.

Learn More about Silverlight (Gratuitous Plug)

If you want to learn more about Silverlight, Pluralsight offers a couple of courses, for which I’m a co-author and instructor. There’s our main Silverlight course, recently updated for the Beta 2 release. And we also offer a dual feature Silverlight/ASP.NET Ajax course, running for more hours in the day and for more days, in which we cover both Silverlight and also ASP.NET Ajax.

Applied WPF Demos from London

Tue, 13 May 2008 15:25:15 GMT

My apologies to everyone who attended my WPF course in London last month - it looks like I forgot to upload the demos. I'm afraid I confused myself by putting the wrong title on the previous week's demos...until today the Silverlight demos from the previous week had a title claiming to be WPF demos. Oops.

So here, at least are the WPF demos from the WPF course I ran last month.

Deep Zoom at the BBC

Sat, 10 May 2008 14:45:50 GMT

The BBC has a Silverlight deep zoom app as part of the web coverage of the ‘Radio 1 Big Weekend’ event. (“Europe’s biggest free ticketed event” apparently.) If you live in the UK and have watched BBC television at all this week, you’ve probably already seen the adverts for the event itself.

The deep zoom app provides a flavour of the event for people who can’t turn up in person – photos of the event will be uploaded throughout the weekend. (And I guess people who did go could try and find themselves in the crowd shots next week.)

From a technical perspective, it’s nothing you won’t already have seen if you’ve looked at the Hard Rock Cafe’s memorabilia site – indeed, the BBC app is somewhat simpler. The main difference is that the BBC’s photo collection that will grow as the event unfolds – not quite live of course, but a bit less static than the Hard Rock application. (And more raw for it of course, but that’s probably in keeping with the nature of the event.)

Deep Zoom and Photo Collections

What I find interesting is that I like this way of browsing photos much more than I thought I would. Deep zoom is definitely one of the more conspicuously fun features of Silverlight 2, but as it’s ultimately a bit of a one-trick pony, I thought I’d be pretty much done with it after playing with the Hard Rock demo.

But it turns out still to be growing on me long after the initial ‘ooh’ factor. I think it’s a surprisingly natural way to present photographs.

I just went to flickr, and coincidentally enough the first picture it showed me on the front page was a shot from a concert. It’s a good picture, but wow the user interface feels flat! Having got used to the deep zoom style of photo browsing, flickr now feels positively broken.

So I want deep zoom for my online photo collections please!