IanG on Tap

LINQ Aggregation and Bounding Boxes

Wed, 30 Jul 2008 18:56:27 GMT

I recently worked on a project that included code to calculate the bounding box of a collection of rectangles. The code did more or less what you might expect – it iterated over the collection, and any time a new rectangle wasn’t contained by the current bounds, it enlarged the bounds to fit.

There was nothing wrong with this, but I knew of a cute trick that could do the job with a fraction of the code. I wasn’t being paid to replace working code with cute tricks, so I left it as it was – when it comes to production code I’m not a fan of the “If it ain’t broke, let me take a crack at it” school of coding. So I thought I’d illustrate the idea here instead.

Here’s a one-line implementation of the same logic, using the System.Windows.Rect type in WPF. (Note that this is a very simple value type, not a full UI element. It’s just 4 numbers.)

public static Rect GetBounds(IEnumerable<Rect> rects)
{
    return rects.Aggregate((currBounds, nextRect) => Rect.Union(currBounds, nextRect));
}

That’s pleasingly succinct, although possibly a little obscure if you’re not accustomed to this style of coding. Actually we can go one more notch on the obscure-and-succinct scale. This comes to the same thing:

public static Rect GetBounds(IEnumerable<Rect> rects)
{
    return rects.Aggregate(Rect.Union);
}

LINQ’s standard Aggregate method implements a mechanism often known as fold in functional programming languages. It’s a pretty simple idea: apply some sort of cumulative operation across all items in a list to build up some sort running total. In this example, LINQ’s uses the first item in the list as the starting point. (Alternatively, you can provide an explicit start value.) It passes that in to the function supplied, along with the second item. The result becomes the new running total, which is fed again into the function supplied, this time along with the third item...and so on. Each time round the aggregating function we provide gets the running total as its first argument, and the next item in the list as its second.

It’s easier to show what it does than to describe it. Here’s what that code would do if applied to an array containing five Rects:

Rect.Union(Rect.Union(Rect.Union(Rect.Union(rs[0], rs[1]), rs[2]), rs[3]), rs[4])

Or if you prefer sequential imperative code, you might like to picture it as:

Rect currBounds = rs[0];
currBounds = Rect.Union(currBounds, rs[1]);
currBounds = Rect.Union(currBounds, rs[2]);
currBounds = Rect.Union(currBounds, rs[3]);
currBounds = Rect.Union(currBounds, rs[4]);

As you can see, it’s not particularly magical. It’s nothing you couldn’t implement with a straightforward loop. (Or with tedious duplication, as here.) But it’s an idea that crops up often enough that it’s rather handy to have as general purpose function – you get to specify what you’d like to use as the cumulative operation, and LINQ does the rest for you.

Aggregate can be used to implement some common operations. For example, you can use it to sum together all the numbers in a list:

public static int Sum(IEnumerable<int> numbers)
{
    return numbers.Aggregate((currTotal, nextInt) => currTotal + nextInt);
}

Or you could use it to find the highest number in a list:

public static int Max(IEnumerable<int> numbers)
{
    return numbers.Aggregate((highest, nextInt) => Math.Max(highest, nextInt));
}

or again, since this just forwards the arguments directly to a function, we can skip the lambda and use the more terse form:

public static int Max(IEnumerable<int> numbers)
{
    return numbers.Aggregate(Math.Max);
}

Of course, you don’t need to write these two examples, because LINQ already supplies Sum and Max extension methods for you. But conceptually at least, they’re just a handy shortcut to a specialized form of aggregation.

Getting the total bounding box of all of the rectangles in a list is not so different from finding the highest number – it’s just a slightly different kind of maximum.

Not a particularly astounding idea, admittedly. I just thought it was a neat trick, and wanted to share it.

Visual State in Silverlight Control Templates

Tue, 10 Jun 2008 20:38:46 GMT

Silverlight 2 introduces a mechanism for customizing the dynamic aspects of a control’s appearance. This new model appeared for the first time in the recent beta 2 release, so I thought I’d take a crack at explaining it.

The Role of Visual States

In both Silverlight and WPF, a control’s appearance can be customized by providing a ‘template’. Templates are typically written in Xaml, and all the usual visual elements are available. But it’s usually not enough just to define the static structure of a control’s visuals.

Most controls change their appearance when the user interacts with them. We expect elements to signal their ability to respond to input with subtle visual shifts when the mouse moves over them, for example. And this is where Silverlight’s ‘visual states’ come in. You can associate animations with visual state changes.

(If you know WPF you’ll be familiar with triggers, which enable exactly this sort of connection between interaction and visuals. In an omission some WPF veterans find surprising, Silverlight doesn’t support triggers. Visual states are what we use instead.)

States and State Groups

To work with Silverlight’s visual state system, we must get to grips with states and state groups.

A control might have many facets, each with its own visual representation. For example, a checkbox may be checked, or unchecked. And it may also be focused or unfocused. The checked/unchecked state is wholly independent of the focused/unfocused state: the keyboard focus can move into and out of the control without changing the checked state. But not all states can be freely combined – obviously a checkbox cannot be both checked and unchecked (although it might be neither – there’s a third, ‘indeterminate’ state, but that’s mutually exclusive with both checked and unchecked).

To clarify how states may combine, states are partitioned into state groups. For example, CheckBox defines three state groups:

Group Name	States
CommonStates	Normal, MouseOver, Pressed, Disabled
CheckStates	Checked, Unchecked, Indeterminate
FocusStates	Focused, ContentFocused, Unfocused

So if we ask “What is a CheckBox’s state?” the answer comes in three parts: a state from the CommonStates state group, one from CheckStates, and one from FocusStates. A control effectively maintains multiple state machines, one for each state group. Here are some examples:

For the CheckStates state group, all three checkboxes here are in the Checked state. With respect to the FocusStates state group, the first checkbox is in the Focused state, while the second and third are in the Unfocused state. Finally, as far as the CommonStates state group is concerned, the first and third checkboxes are in the Normal state, while the second is in the MouseOver state. (The mouse cursor isn’t shown here, but you can see that the control has changed appearance.)

A control will advertise the states it offers and the group to which each state belongs with the TemplateVisualState attribute, as you can see here:

[TemplateVisualState(Name = "ContentFocused", GroupName = "FocusStates")]
[TemplateVisualState(Name = "MouseOver", GroupName = "CommonStates")]
[TemplateVisualState(Name = "Focused", GroupName = "FocusStates")]
[TemplateVisualState(Name = "Checked", GroupName = "CheckStates")]
[TemplateVisualState(Name = "Unchecked", GroupName = "CheckStates")]
[TemplateVisualState(Name = "Indeterminate", GroupName = "CheckStates")]
[TemplateVisualState(Name = "Pressed", GroupName = "CommonStates")]
[TemplateVisualState(Name = "Disabled", GroupName = "CommonStates")]
[TemplateVisualState(Name = "Unfocused", GroupName = "FocusStates")]
[TemplateVisualState(Name = "Normal", GroupName = "CommonStates")]
public class CheckBox : ToggleButton ...

We now know that a control will maintain a simple state machine for each state group. So what? To do anything useful with this, we must use the VisualStateManager class.

VisualStateManager

The VisualStateManager class lets us connect animations to state changes. It defines an attached property we can use to establish these connections in Xaml.

So that we can see this in action, let’s start with a simple template for a CheckBox:

<CheckBox Content="Check!" IsChecked="True">
  <CheckBox.Template>
    <ControlTemplate>
      <Grid>
        <Grid.ColumnDefinitions>
          <ColumnDefinition Width="19" />
          <ColumnDefinition />
        </Grid.ColumnDefinitions>

        <vsm:VisualStateManager.VisualStateGroups>
        </vsm:VisualStateManager.VisualStateGroups>

        <Rectangle x:Name="tickBox"
          Stroke="Blue" Width="12" Height="12" HorizontalAlignment="Left"
          RadiusX="2" RadiusY="2" Fill="Aqua" />

        <Path x:Name="tick" Opacity="0"
              Width="12" Height="12" HorizontalAlignment="Left"
              Stroke="Black" StrokeThickness="2"
              Data="M3,6 L5,9 9,2" />

        <ContentPresenter Grid.Column="1" />
      </Grid>
    </ControlTemplate>
  </CheckBox.Template>
</CheckBox>

This defines some pretty basic visuals – just a simple box and tick, plus a placeholder. It looks like this:

Note that although I’ve got a Path element in the shape of a tick, it’s currently invisible. Later I will supply a couple of animations to make the tick visible and invisible.

Notice that the example above already contains an empty <vsm:VisualStateManager.VisualStateGroups> element – it’s just after the grid’s column definitions. I’ve been working with Xaml for half a decade now, but for anyone who hasn’t had that pleasure, I suppose it might not be totally obvious that this element sets a property on the containing Grid! In general, Xaml elements of the form <Xxx.Yyy> denote a property. The part after the ‘.’ is the property name, so that tells us that this is the VisualStateGroups property. And the part before the ‘.’ is the name of the class that defines the property – so here we can see this property is defined by the VisualStateManager class. Again, if you’re not familiar with Xaml, it may seem a bit funky that I can set property defined by VisualStateManager on Grid, a completely unrelated class. This is an example of a Xaml feature called attached properties– they’re a way of bolting new properties onto existing types.

For some reason the VisualStateManager class isn’t in the Silverlight Xaml namespace, so we’ve had to qualify it with a namespace prefix. (I’ve chosen “vsm:” in this example, short for VisualStateManager.) For this to work, I need to have defined that prefix somewhere in my Xaml. It’s not shown in the above snippet, but I put it at the root element, alongside the other namespace declarations:

xmlns:vsm="clr-namespace:System.Windows;assembly=System.Windows"

Normally you’d let Blend do all this for you. The June preview of Expression Blend 2.5 provides new features for working with the VisualStateManager. But my goal here is to explain the underlying mechanisms, so I’ll steer clear of Blend for now.

Note: in the Silverlight 2 beta 2 preview bits, the Visual Studio Xaml editor reports an error on the line with this property. It says: “The attachable property ‘VisualStateGroups’ was not found in type ‘VisualStateManager’.” But it’s lying – the attachable property is present, and although you get what looks like an error in Visual Studio. But you’ll find the project will happily build and run despite this ‘error’, and if you close the Xaml editor, the error goes away. It’s not really an error. That’s beta software for you.

So what do we put inside this property? We can provide an animation to run for each state. On top of that, we can optionally get fancy, and provide animations that handle the transition between a specific pair of states. Let’s start with the simpler case.

Per-State Animations

The simplest way to use the visual state manager is to define an animation to run when a particular state is entered. The following example is about as simple as it gets:

<vsm:VisualStateManager.VisualStateGroups>
  <vsm:VisualStateGroup x:Name="CommonStates">
    <vsm:VisualState x:Name="MouseOver">
      <Storyboard>
        <ColorAnimation Storyboard.TargetName="tickBox"
            Storyboard.TargetProperty="(Rectangle.Fill).
                                (SolidColorBrush.Color)"
            To="PaleGreen" Duration="0:0:0.5" />
      </Storyboard>
    </vsm:VisualState>
  </vsm:VisualStateGroup>
</vsm:VisualStateManager.VisualStateGroups>

Notice that I am required to say first of all which state group I’m interested in, with a VisualStateGroup element. In this case, I’m looking at the CommonStates group. And inside the group, I provide a VisualState element for each state I wish to animate. To keep things simple, I’ve just handled one state here: MouseOver.

With this markup in place, when the CommonStates state group enters the MouseOver state (i.e. when the user moves the mouse over the control), the Storyboard I’ve supplied here will run. This particular animation changes the checkbox’s fill to pale green.

Since I’ve only handled one state, the checkbox will get stuck in that state – I’ve not yet provided any animations to put it back how it was. And I’m also missing some other states, so to complete the job, I should add the following inside that VisualStateGroup after the first VisualState:

<vsm:VisualState x:Name="Pressed">
  <Storyboard>
    <ColorAnimation Storyboard.TargetName="tickBox"
                    Storyboard.TargetProperty="(Rectangle.Fill).
                    (SolidColorBrush.Color)"
                    To="Navy" Duration="0:0:0.5" />
  </Storyboard>
</vsm:VisualState>
<vsm:VisualState x:Name="Disabled">
  <Storyboard>
    <ColorAnimation Storyboard.TargetName="tickBox"
                    Storyboard.TargetProperty="(Rectangle.Fill).
                    (SolidColorBrush.Color)"
                    To="Gray" Duration="0:0:0.5" />
  </Storyboard>
</vsm:VisualState>
<vsm:VisualState x:Name="Normal">
  <Storyboard>
    <ColorAnimation Storyboard.TargetName="tickBox"
                    Storyboard.TargetProperty="(Rectangle.Fill).
                    (SolidColorBrush.Color)" Duration="0:0:0.5" />
  </Storyboard>
</vsm:VisualState>

Note that the last of these doesn’t specify the To attribute for the animation. As is usual for animations in Silverlight and WPF, this means ‘animate back to the base value’, which is exactly what you’d hope for when returning to the Normal state.

I’m going to use a similar but cruder trick to handle the checkbox state. The following goes after the first VisualStateGroup element:

<vsm:VisualStateGroup x:Name="CheckStates">
  <vsm:VisualState x:Name="Checked">
    <Storyboard>
      <DoubleAnimation Storyboard.TargetName="tick" 
                       Storyboard.TargetProperty="Opacity"
                       To="1" Duration="0" />
    </Storyboard>
  </vsm:VisualState>
  <vsm:VisualState x:Name="Unchecked" />
</vsm:VisualStateGroup>

As before, this declares that I want to handle state changes for a particular state group – the CheckStates group this time. And I’ve provided VisualState elements to tell Silverlight what to do when entering either the Checked or Unchecked states.

The Checked storyboard is slightly odd: it has a Duration of 0. This is a case of trigger envy – in WPF I’d just write a simple trigger that says when IsChecked is true, the ‘tick’ element should be visible. But Silverlight doesn’t have triggers, so instead I’ve had to use an animation that runs instantaneously to get the same effect.

The Unchecked handling is even stranger: I’ve provided an empty VisualState element. The significance of this is that I’m telling Silverlight that I care about this state, but that I have no new animations to run. And the important, non-obvious point is that when a new VisualState element comes into play, Silverlight deactivates any animations that were previously in effect from other VisualState elements for that state group.

And even though my Checked animation has a duration of 0, it remains active until we enter the Unchecked state. Animations in Silverlight (and WPF) usually continue to have an effect even after they complete – if you animate something from red to green, it will remain green once the animation finishes. (You can change that if you like – this is controlled with the animation’s FillBehavior property.) So in this case, the Checked animation completes instantly, but remains active, holding the ‘tick’ element’s Opacity property at 1 until the animation is deactivated. That deactivation happens when we enter the Unchecked state.

So it’s quite common to see empty VisualState elements. They’re usually present to deactivate any other animations for the state group.

Note that the example so far is incomplete. I’ve not handled the Indeterminate state in the CheckStates group. And I’ve also not handled the FocusStates state group at all. Nor do I plan to – I’m just trying to show the model here, and to fill in the gaps would mean more of the same.

(If you want to see a fuller example, I’ve put focus state handling into the version available for download: SilverlightVisualStateDemo.zip. If you want to run the application online, it’s up at http://www.interact-sw.co.uk/slapps/visualstate/ Note that these versions also include the feature described in the next section.)

Transition Animations

Whereas the VisualState element we’ve just seen defines the animation to use when entering a particular state, a transition animation is more specific: you can define the animation to run when transitioning between a particular pair of states.

Note that transition animations do not replace the VisualState animations. A transition animation runs before the VisualState animation. For example, suppose you’ve defined a transition animation for the transition from the Pressed to Normal states in the CommonStates group. Silverlight will first run this transition animation, and once that has finished, it goes on to run the state animation.

You might think that you could use transition animations as a replacement for state animations – why not provide only a transition, and leave out the final state animation? That turns out not to work: Silverlight requires you to specify the duration of your transition, and it will deactivate the transition animation at the end. And if you leave off the duration, the duration becomes 0 at which point the animation effectively never runs at all.

This constraint ensures that no matter how a control entered a state, and no matter which transition animation ran as a result, it always ends up looking the same when it’s in that state, once all the animations have played out.

To put it another way, Silverlight will always eventually run the animation for the target state, as specified in the relevant VisualState, but before it runs that you get to insert something specific to the preceding state. This gives you just enough power to ensure that every possible state change looks coherent, while minimizing the scope for chaos.

Here’s a simple and slightly contrived example:

<vsm:VisualStateGroup x:Name="CommonStates">
  <vsm:VisualStateGroup.Transitions>
    <vsm:VisualTransition From="Pressed" To="Normal" Duration="0:0:0.5">
      <Storyboard>
        <ColorAnimation Storyboard.TargetName="tickBox"
                      Storyboard.TargetProperty="(Rectangle.Fill).
                      (SolidColorBrush.Color)"
                      To="Red" Duration="0:0:0.5" />
      </Storyboard>
    </vsm:VisualTransition>
  </vsm:VisualStateGroup.Transitions>
  <vsm:VisualState x:Name="MouseOver">
    ... as before ...

This has a VisualStateGroup for the CommonStates group as before, but I’ve added a Transitions property. Inside this I’ve added a VisualTransition defining an extra animation to be run prior to the animation for the Normal state if we are coming into that state from the Pressed state.

So if the user presses the button down on the CheckBox, and then moves away before letting go, the check box will briefly go red thanks to this transition animation, and then the Normal state animation will run, returning the control back to its standard appearance.

Note that you are allowed to leave off either the To, or From attributes from a VisualTransition. This turns it into a kind of wildcard transition. With just a From attribute, the transition will run any time moving from the named state. With just a To attribute, it will run any time moving to the named state.

You may wonder what is the point of a transition animation with just a To attribute. What does that offer that I can’t do more easily with a normal VisualState animation? Well there is one subtle but useful difference. VisualState animations run any time a state is entered, even when the control first appears. Transition animations do not. Consider our checkbox example – you might want the tick to fade in and out rather than coming and going abruptly. You could get that effect by adding a non-zero duration to the VisualState animation for the Checked state shown earlier. However, if the checkbox happened to be checked when it first appeared, the animation would run then too, which might not be what you want. By contrast, a transition animation with To="Checked" will not run when the control first appears, but will run any time the control enters the Checked state from some other state.

You can even leave off both To and From, in which case the transition will be inserted before every state change for that state group.

Designer Support

In summary, you get to define animations that run when the control enters a particular state, and you may also define specialized transition animations that run just before the main animation. It’s certainly different from WPF’s trigger mechanisms, and while it lets you achieve a lot of the same goals, it’s not quite as flexible – you only get to write state animations for the visual states the control author has decided to offer, whereas triggers can use any property.

However, there’s one thing visual states do better than triggers: designer support. That’s not to say that WPF triggers aren’t supported by Blend – it does in fact support them. But you have to work harder with triggers. The benefit of the visual state model is that Blend can see exactly what states a control offers, and it can help a visual designer fill in all the necessary pieces. By contrast, triggers are so flexible that it’s difficult for a design tool to know where to start.

If you’re happy cranking out Xaml by hand, you’ll probably miss triggers. But if you live in Blend, visual states offer useful benefits.

Learn More about Silverlight (Gratuitous Plug)

If you want to learn more about Silverlight, Pluralsight offers a couple of courses, for which I’m a co-author and instructor. There’s our main Silverlight course, recently updated for the Beta 2 release. And we also offer a dual feature Silverlight/ASP.NET Ajax course, running for more hours in the day and for more days, in which we cover both Silverlight and also ASP.NET Ajax.

Applied WPF Demos from London

Tue, 13 May 2008 15:25:15 GMT

My apologies to everyone who attended my WPF course in London last month - it looks like I forgot to upload the demos. I'm afraid I confused myself by putting the wrong title on the previous week's demos...until today the Silverlight demos from the previous week had a title claiming to be WPF demos. Oops.

So here, at least are the WPF demos from the WPF course I ran last month.

Deep Zoom at the BBC

Sat, 10 May 2008 14:45:50 GMT

The BBC has a Silverlight deep zoom app as part of the web coverage of the ‘Radio 1 Big Weekend’ event. (“Europe’s biggest free ticketed event” apparently.) If you live in the UK and have watched BBC television at all this week, you’ve probably already seen the adverts for the event itself.

The deep zoom app provides a flavour of the event for people who can’t turn up in person – photos of the event will be uploaded throughout the weekend. (And I guess people who did go could try and find themselves in the crowd shots next week.)

From a technical perspective, it’s nothing you won’t already have seen if you’ve looked at the Hard Rock Cafe’s memorabilia site – indeed, the BBC app is somewhat simpler. The main difference is that the BBC’s photo collection that will grow as the event unfolds – not quite live of course, but a bit less static than the Hard Rock application. (And more raw for it of course, but that’s probably in keeping with the nature of the event.)

Deep Zoom and Photo Collections

What I find interesting is that I like this way of browsing photos much more than I thought I would. Deep zoom is definitely one of the more conspicuously fun features of Silverlight 2, but as it’s ultimately a bit of a one-trick pony, I thought I’d be pretty much done with it after playing with the Hard Rock demo.

But it turns out still to be growing on me long after the initial ‘ooh’ factor. I think it’s a surprisingly natural way to present photographs.

I just went to flickr, and coincidentally enough the first picture it showed me on the front page was a shot from a concert. It’s a good picture, but wow the user interface feels flat! Having got used to the deep zoom style of photo browsing, flickr now feels positively broken.

So I want deep zoom for my online photo collections please!

Nulls and Lifting Member Access

Sun, 13 Apr 2008 17:30:31 GMT

Nulls can complicate code. One of the lesser known C# language features, the ?? operator, can sometimes help. This operator lets you provide an alternate value to be used in the event that an expression evaluates to null. For example:

string displayName = elem.Name ?? "[Unknown]";

If elem.Name evaluates to anything other than null, then displayName will be elem.Name. But if elem.Name is null, displayName will become "[Unknown]". To get the same result in C# 1 requires the cryptic but useful ternary operator (aka conditional operator) (‘Confusing C family language newbies since 1978’). The result is longer, and requires you to write out the expression twice:

string displayName = elem.Name == null ? "[Unknown]" : elem.Name;

In my experience, most C# developers don’t seem to know about ??. In my Applied WPF course, one of the labs uses it in passing, and I invariably get people either asking me what it does, or occasionally telling me it’s a typo!

People usually like it once they get it, because it’s a clear improvement for this sort of example. So why isn’t it better known? I have a theory: I think it’s less useful than it seems. It only solves one specific example of the ‘nulls are a pain’ class of problems. Most of the time, I want something slightly more powerful than ??.

What I’d Like

While I frequently have to deal with values that might be null, I’ve found that in the non-null case I usually want to work with a member of the object, rather than the object itself. For example, consider this very simple singly-linked list node:

public class Node
{
    public Node Next;
    public string Value;
}

Suppose you have a variable of type Node which may be null, and that you want to return the Value if the variable’s non-null, and return null otherwise. You can’t use the ?? operator here – you have to revert to the good old ternary operator:

Node n = GetNode();
string value = n == null ? null : n.Value;

This is tolerable, but it’s a case that comes up often enough that I can see the value in having a special syntax to handle it. Sadly, the ?? operator is not that syntax – it only seems to cover a small minority of the cases.

A different solution suggests itself when you look at another feature added in C# 2: lifting for nullable types.

Lifting

Lifting in C# is a convenient way of working with values that might be null: rather than crashing when you try and use a null value, the nullness just ends up propagating. For example, C# supports lifting when adding numbers together:

public static int? Add(int? x, int? y)
{
    return x + y;
}

If you call this function with integers, it will add them as expected. But you can pass in null for either argument, in which case the result will be null. So we say that the + operator is lifted. (See Eric Lippert’s description of lifting in C# for more information.)

Not all operators are lifted in C#. The arithmetic ones are, so you can perform numeric calculations with nullable types. But many are not. Member access is not lifted, for example. If it were, I’d be able to replace this:

string value = n == null ? null : n.Value;

with this:

string value = n.Value;

If ‘.’ were lifted, value would end up as null if either n itself were null, or the Value member of the object referred to by n were null.

This seems like a much more succinct way to deal with the problem, but of course that’s not what ‘.’ does in practice. Morever, we cannot overload ‘.’ in C#, so on the face of it, it seems that only the C# compiler team would be able to make this work. However, we can achieve something close by getting the relevant code into expression tree form, and then modifying the tree.

Expression Tree Rewriting

C# can convert an expression into tree of objects. This means we can change how an expression works by simply changing the tree. We could use that to modify the behaviour of ‘.’ to add lifting.

(This idea is pretty similar to LISP’s macro system – code rewriting expressions or generating new ones. There’s a significant difference though: LISP macros run at compile time. As far as I can tell, we don’t have that option with expression trees in C# 3. However, even though we are obliged to wait until runtime to perform the tree rewriting, we do get to compile the result into IL. So after the initial overhead, the performance should be identical to normally compiled code.)

One slight snag is that expression trees are immutable. So you can’t just change them – you end up having to build a new tree. Fortunately, Microsoft provides an example class in the documentation that helps you do this: ExpressionVisitor. This walks every node in an expression tree, and lets you build an edited copy. And it’s smart about unchanged subtrees – it just reuses those rather than copying them. (That’s one of the advantages of immutability – it’s safe to reuse objects like that because you know they’re not going to change.)

The following class derives from the ExpressionVisitor example, and overrides VisitMemberAccess to change the way member access works:

public class NullLiftModifier : ExpressionVisitor
{
  // The method that kicks off the tree walk is
  // protected, so we need to define a public
  // method that provides access to it.
  public Expression Modify(Expression expression)
  {
    return Visit(expression);
  }


  // Change how '.' performs member access.
  protected override Expression VisitMemberAccess(
               MemberExpression originalExpression)
  {
    MemberExpression memberAccessExpression =
     (MemberExpression) base.VisitMemberAccess(originalExpression);


    Expression nullTest = Expression.Equal(
        memberAccessExpression.Expression,
        Expression.Constant(null,
          memberAccessExpression.Expression.Type));
    return Expression.Condition(
        nullTest,
        Expression.Constant(null, memberAccessExpression.Type),
        memberAccessExpression);
  }
}

The call to the base class builds the expression we’re about to rewrite (possibly recursively rewriting some of its sub expressions.) We then generate some extra expressions to test the target object reference for null, and then to return either a null constant or the original expression, depending on whether the target reference is null. Or to put it in code, it changes this:

obj.Member

to this:

obj == null ? null : obj.Member

For ease of use, we can wrap this rewriter in a helper function that takes an expression, rewrites it, and then compiles the result:

static Func<T, TResult> LiftMemberAccessToNull<T, TResult>(
    Expression<Func<T, TResult>> expr)
{
    NullLiftModifier modifier = new NullLiftModifier();
    var x = (Expression<Func<T, TResult>>) modifier.Modify(expr);
    return x.Compile();
}

The compilation here will happen at runtime – Compile turns the expression tree into IL in a dynamically generated method, and returns a delegate pointing to that method.

To test this, I wrote the following, which uses the Node class introduced earlier:

var get0 = LiftMemberAccessToNull((Node f) => f.Value);
var get1 = LiftMemberAccessToNull((Node f) => f.Next.Value);
var get2 = LiftMemberAccessToNull((Node f) => f.Next.Next.Value);
var get3 = LiftMemberAccessToNull((Node f) => f.Next.Next.Next.Value);

So we have four methods, which pick out the value of the current node, the next node, the node after that, and finally the node after that. I’m using the ordinary member access syntax, but I’m calling the code I wrote earlier to add lifting to the ‘.’ operator. To test these methods, I wrote the following (which, just for fun/added confusion, uses the ?? operator to help print out the results):

Node f1 = new Node { Value = "One" };
Node f2 = new Node { Value = "Two" };
Node f3 = new Node { Value = "Three" };
f1.Next = f2; f2.Next = f3;

Node[] foos = { f1, f2, f3, null };
foreach (Node f in foos)
{
  Console.WriteLine("Value: "                + (get0(f) ?? "null"));
  Console.WriteLine("Next.Value: "           + (get1(f) ?? "null"));
  Console.WriteLine("Next.Next.Value: "      + (get2(f) ?? "null"));
  Console.WriteLine("Next.Next.Next.Value: " + (get3(f) ?? "null"));

  Console.WriteLine();
}

This builds a list with three nodes, and iterates through this list, trying all four get methods on each of the nodes.

Normally, you’d expect this to throw a null reference exception. If you have a reference ‘f’ to the first node, the expression f.Next.Next.Next.Value is looking for the fourth node (i.e. three along from the first). There are only three nodes, so that final Next will evaluate to null. Normally you’d expect the attempt to access Value to throw an exception. But we’ve run these expressions through my lifting rewriter, so instead, we see this:

Value: One
Next.Value: Two
Next.Next.Value: Three
Next.Next.Next.Value: null

Value: Two
Next.Value: Three
Next.Next.Value: null
Next.Next.Next.Value: null

Value: Three
Next.Value: null
Next.Next.Value: null
Next.Next.Next.Value: null

Value: null
Next.Value: null
Next.Next.Value: null
Next.Next.Next.Value: null

This illustrates that the lifting is working as desired. This is consistent with the + operator: adding null to a number gives you null; likewise here, when I try to access a member via a null reference I get back null.

Limitations

One big problem with this is that it’s pretty clunky. I have to wrap up the member access in a lambda, and then pass it to my function to get it rewritten and compiled. If I want to make sure I perform that step just the first time my code runs, and yet I still want my lambdas to have access to local variables in scope, the code’s going to get even uglier. So while it might be a neat trick to illustrate the idea, I don’t see myself using this in production code.

This is a shame, because the idea of rewriting expressions is a pretty powerful one. Lifting member access is only one example of what you can do with this technique. It would be great if there were a clean syntax for this sort of idea.

(Also, beware that I’ve not tested this in any serious way – it’s only intended to illustrate an idea. It’s probably full of subtle flaws as well as the more glaring issues. I suspect it doesn’t play nicely with Nullable<T>, for example. Please don’t use it as is in your production code! That said, I offer this code under the MIT license, so feel free to do what you will with it within the terms of the license. Just consider yourself warned. You can grab the source from here: http://www.interact-sw.co.uk/downloads/LiftMemberAccess.zip)

A Monadic Footnote

When I first started to look at this, I had been considering a slightly different approach. The idea of a nullable value (whether it’s a Nullable<T> or just an ordinary reference) is pretty similar to Haskell’s Maybe type. I can use Maybe to write a Haskell equivalent to the Add function I wrote in C# above:

addMaybe :: Maybe Integer -> Maybe Integer -> Maybe Integer
addMaybe x y =
  do
    xVal <- x
    yVal <- y
    return (xVal + yVal)

This is slightly long-winded, because in Haskell, the addition operator is not suitably lifted – it doesn’t recognize Maybe types. (I could arrange for this lifting to happen in Haskell by plugging the code above into the right place and doing some other extra work, incidentally. But this entry’s long enough already.) So I had to write code to explicitly extract the values. A slightly closer C# version might look like this:

// Note: doesn't work properly!
public static int? AddNullable(int? x, int? y)
{
    int xVal = x.Value;
    int yVal = y.Value;
    return xVal + yVal;
}

However, this C# doesn’t work, because we’ve lost the lifting – it’ll throw an exception if you pass in null for either argument. But the Haskell version works fine, as we can see from this interactive GHCi session:

*Main> addMaybe (Just 18) (Just 24)
Just 42
*Main> addMaybe (Just 123) Nothing
Nothing
*Main> addMaybe Nothing (Just 345)
Nothing
*Main> addMaybe Nothing Nothing
Nothing

(Note, ‘Just’ is a constructor for a Maybe instance. So (Just 18) is roughly similar to new Nullable<int>(18) in C#. And Nothing constructs a Maybe that contains no value – it’s more or less like null.)

This works because Maybe belongs to Haskell’s Monad type class. The syntax involving ‘do’ and ‘<-’ is available on monadic types, and in the case of the Maybe monad, the <- operator provides lifted access to the value inside the Maybe – if either x or y turn out to contain Nothing then that whole ‘do’ block will evaluate to Nothing. (See this allegedly ‘gentle’ introduction to monads for more information on monads in Haskell.)

I had been thinking of going down a similar path as a result of reading Wes Dyer’s excellent explanation of monads. He points out that LINQ uses monadic constructs – the standard SelectMany query operator is essentially the monadic ‘bind’ operator by another name. He even implements a C# version of Maybe, and I could perhaps use that to rewrite my C# version roughly like this:

var r = from xVal in x.ToMaybe()
        from yVal in y.ToMaybe()
        select x + y;

Not only does this follow a fairly similar pattern to the Haskell version, it would also offer the same lifting behaviour – it would work correctly in the face of Nothing (null) values.

So it occurred to me that I could use this technique to perform the member access lifting I was looking for. Here’s a class that tries to do this by providing some extension methods that effectively define the monadic ‘bind’ operator for references:

public static class NullableExtensions
{
  public static V Select<T, V>(this T maybeT, Func<T, V> s)
  {
    if (maybeT == null) { return default(V); }
    return s(maybeT);
  }

  public static V SelectMany<T, U, V>(this T maybeT,
                          Func<T, U> k, Func<T, U, V> s)
  {
    if (maybeT == null) { return default(V); }

    U maybeBoundResult = k(maybeT);
    if (maybeBoundResult == null) { return default(V); }
    return s(maybeT, maybeBoundResult);
  }
}

The SelectMany implementation here just bails out early if it hits a null, and returns null without attempting to evaluate anything that follows. (And Select does much the same.) Note: I’ve used the phrase ‘maybe’ here to indicate potentially empty values – I’m not actually using Wes’s Maybe implementation. This particular code snippet stands alone.

Again, I would recommend NOT using this in production code – it effectively adds standard LINQ query operators to System.Object, which is probably not a sane thing to do in a real program. But with just that code (I don’t need any of the code from earlier) I can now write this sort of thing:

int? two = 2;
int? one = 1;
int? none = null;

var twoPlusTwo = from x in two
                 from y in two
                 select x + y;
Console.WriteLine(twoPlusTwo);

var onePlusNothing = from x in one
                     from y in none
                     select x + y;
Console.WriteLine(onePlusNothing);

var nothingPlusTwo = from x in none
                     from y in two
                     select x + y;
Console.WriteLine(nothingPlusTwo);

var onePlusTwoPlusOne = from x in one
                        from y in two
                        from z in one
                        select x + y + z;
Console.WriteLine(onePlusTwoPlusOne);

This prints out results for the calculations that have non-null results, and blank lines for the others. (Console.WriteLine just prints an empty line when given a null.) So lifted use of nullable types is working. But we can also use this to deal with member access through potentially null references.

The simplest example of working with member access wouldn’t work if I had only provided the SelectMany method. I added Select as well was so it would work even if there were only one from clause. That lets me write this sort of thing:

foreach (var item in doc.SelectNodes("//item").Cast<XmlNode>())
{
    var title = from attr in item.Attributes["title"]
                select attr.Value;

    Console.WriteLine(title);
}

This provides a solution of sorts to the member access problem we started with. I’ve run into this particular example on many occasions – I’ve used an XPath query against an XmlDocument and need to deal with the fact that some attributes may be missing on some of the elements that came back. (And yes, I could have written a more selective XPath query in this particular case. But when you’re dealing with lots of optional attributes, that gets tedious.) If the ‘title’ attribute is missing on a particular item, the code won’t hit an exception on the attr.Value expression, because my implementation of Select chooses not to evaluate the select clause at all if attr is null. (And the presence of SelectMany means the same will be true if I use multiple from clauses.)

So it’s a similar result to the expression-tree-based lifting of the ‘.’ operator: I get to write code that attempts to access members via references that may be null without needing an explicit null check. However, the resulting code looks pretty arcane, so again, I don’t think I’d use this in production code. In practice, the problem in this particular example is better solved in C# with an ordinary helper function.

Conclusion

I’ve used some tricks here to implement a couple of variations on my original theme: trying to avoid writing explicit tests for null references when accessing member variables, by lifting member access over null. Unfortunately, neither solution is pretty, so I consider this experiment an interesting failure. And perhaps that’s to be expected – I’m trying to subvert a fundamental feature of the language that was never designed to be modified. If C# had wanted to make it easy, either ‘.’ would have been overloadable, or the compiler would support compile-time hygienic macros.

Would I want to see either of those things in C#? I’m not sure I would – I’ve suffered too much ‘clever’ C++ that tries to redefine language behaviours. To be fair, a lot of the horror there was down to how the clunky macro system in C++ is entirely unrelated to the rest of the language. However, some of the horror was with better integrated features, such as the way you can in fact overload member access in C++.

There’s a lot to be said for a language where code does what it appears to do. It’s normal to spend more time reading code than writing it, and if you need to question fundamentals such as “is that thing that looks like field access really accessing a field?” every time you look at a line of code, that’s going to slow you down massively, and increase the chances of misunderstanding the code.

It reminds me of a troubleshooting consulting engagement I had years ago. A memory leak turned out to be the result of a couple of ‘clever’ behaviours interacting badly. I still remember the growing expression of disbelief on the developer’s face as I stepped into what looked like a single, straightforward statement, but which turned out to involve no fewer than seven implicit function calls! He had no idea it was doing all that, and he’d written it!

If I were going to have a syntax that means “get this thing from that object, unless we have a null reference, in which case return null”, I think I’d prefer a new, specialized syntax over a new behaviour for an established syntax.

So I think I’ll file these under “neat trick; DO NOT USE!”

Applied Silverlight Demos from London

Thu, 03 Apr 2008 17:41:14 GMT

Thanks to everyone who attended the Applied Silverlight course in London this week. As promised, here are the demos built during the week.

LINQ Range Variable Oddness

Sat, 29 Mar 2008 14:57:22 GMT

Jon Skeet recently observed that range variables in LINQ expressions are not like other variables. A range variable is one like foobar in the following:

var query = from foobar in source
            where foobar > 20
            select foobar;

Jon points out that the analogy people often draw – the iteration variable in a foreach loop – isn’t that close an analogy. I thought it would be interesting to come up with a concrete illustration of the differences.

Before I start, let me introduce a type:

List<Func<int>>

That’s a list of functions. Each function in the list takes no arguments, and returns an integer. Here’s a method that walks through such a list, invokes each function in turn, and prints the result:

static void PrintValues(List<Func<int>> funcList)
{
    foreach (Func<int> func in funcList)
    {
        Console.WriteLine(func());
    }
}

I’m going to use this to illustrate some differences between LINQ query expression range variables and foreach iteration variables.

Capturing Variables

In order to take a look at these variables, I’m going to capture them in lambdas, so we can poke them about a bit and see what happens. This example iterates through the numbers 1 to 10, building a function that captures the iteration variable each time round:

var source = Enumerable.Range(1, 10);

List<Func<int>> capturedForeachIterationVariables =
    new List<Func<int>>();
foreach (int i in source)
{
    capturedForeachIterationVariables.Add(() => i);
}

That lambda (the “() => i” expression) denotes a function that takes no arguments (hence the empty ()) and returns the value of i. And the i in scope here is the iteration variable of the for each loop.

We can print out the results using the method I showed earlier:

PrintValues(capturedForeachIterationVariables);

And here’s the outcome:

This illustrates that there is just the one iteration variable. Our list contains 10 functions, all of which captured that one same iteration variable. That’s why they printed out the final value.

Per Loop Scope

With a slight modification, we can give each lambda its own private variable to capture:

List<Func<int>> capturedForeachLocals =
    new List<Func<int>>();
foreach (int i in source)
{
    int iLocal = i;
    capturedForeachLocals.Add(() => iLocal);
}

While i remains in scope for the entire execution of the loop, iLocal goes out of scope at the end of each iteration. Consequently, each time round the loop, we create a lambda that captures a different iLocal. So when we print the results, we see something different:

Capturing a Range Variable

What about range variables? Here’s the rough equivalent

var query = from i in source
            select ((Func<int>) (() => i));

List<Func<int>> capturedRangeVariables =
    query.ToList();

The syntax is a little messy here because C# is unable to infer the lambda type. Remember, lambda expressions don’t have an intrinsic type, so the compiler needs more information. In the previous two examples, the compiler was able to infer what was required from the signature of the List<T>.Add method to which we were passing the lambda. I can’t work out a neat way of giving the compiler what it needs here, so I’ve settled for a rather ugly cast. (I suppose I could also have used a variation on the Funcify function from that recent blog, but it doesn’t look much better to me.)

Anyway, here are the results:

So the behaviour is not like the foreach iteration variable – it’s more like the case where we made a copy of the iteration variable. That’s a better analogy, but we can dig a little further.

Modifying the Range Variable

Expressions in LINQ query expressions can sometimes have side-effects. Whether that’s allowed depends on what flavour of LINQ you’re using. We’re using LINQ to Objects here, and that permits it. Here’s a modified version of the previous query expression:

var query = from i in source
            select ((Func<int>) (() => ++i)); 

List<Func<int>> capturedRangeVariables =
    query.ToList();

This increments the captured range variable. Now that won’t interfere with the overall progress of the iteration, because as we already know, the lambda here seems to be working with a copy of i rather than the ‘real’ i. So if we print it out once, we get what you’d probably expect:

And if we print it out the list a second time, as you may or may not have been expecting everything moves up by one:

(Note, I’m only evaluating the query once – that happens when I call ToList. And then I’m printing out the resulting list multiple times, and the numbers go up by one each time. If I were to re-evaluate the query, that’d be different. It would build a brand new set of functions that would start back from 2 again.)

We can represent this as a foreach:

foreach (int i in source)
{
    int iLocal = i;
    capturedForeachLocals.Add(() => ++iLocal);
}

The behaviour is the same. Again, this suggests that LINQ query expression range variables work more like a copy of a foreach iteration variable than the iteration variable itself. (Incidentally, if you were to try and use i++ in this foreach loop, it wouldn’t even compile. C# prevents you from modifying the iteration variable – another way in which iteration variables are different from range variables.)

But we can go further still.

Multiple References to the Range Variable

LINQ query expressions can contain many clauses, each of which will often want to access the range variable. This means I can build a query that attempts to modify the range variable in one clause before going on to use it again in a later clause:

var query = from i in source
            where ++i < 5
            select ((Func<int>) (() => i));

List<Func<int>> capturedRangeVariables =
    query.ToList();

On the face of it, this has a where clause that increments the range variable, and then only lets through items where the result is less than 5. So we’d expect only to see three items in the output. But what do we expect those output values to be? It would be reasonable expect to that the i in the select clause will have the incremented value – after all, it looks like the same variable, i, that got incremented in the line above. And yet the output is as follows:

1
2
3

That is, the i captured in the select clause does not seem to have been affected by the increment operator applied to the i in the where clause. Clearly the ++ has had an effect – we’ve only got three numbers out, and we would have expected four if that ++ were not present.

So it’s as though the two clauses are working on different variables, despite being in the same scope and using the same name. The foreach equivalent looks like this:

foreach (int i in source)
{
    int iLocal1 = i;
    if (++iLocal1 < 5)
    {
        int iLocal2 = i;
        capturedForeachLocals.Add(() => iLocal2);
    }
}

Phantom Variables

So query expression range variables clearly are quite different from foreach iteration variables. Each clause in a query expression that refers to the range variable gets its own local variable, so while it may look like one variable, it’s not. And as Jon points out the range variable never really exists in the compiled result – once a query expression is compiled, the only tangible variables in the code are the local copies. And you can see why by manually expanding any of these query expressions out into their equivalent method call forms: you’ll see that the range variable vanishes as part of this transformation.

So the range variable is really just a label that directs the transformation of a query expression into code at compile time. Once it has done this job, it vanishes.

C# 3.0 Lambdas and Type Inference

Mon, 17 Mar 2008 16:07:32 GMT

Daniel Cazzulino recently wrote a blog entry whose main focus was on building pipelines using iterators in C#. Towards the end he showed a slightly irritating problem in C# 3.0. He wanted to write this:

var transformer = x => new { Original = x, Normalized = x.ToLower() };

However, the C# compiler complains because it doesn’t have enough information to infer the type of the transformer variable. The problem it reports is “Cannot assign lambda expression to an implicitly-typed local variable”.

Daniel doesn’t present a working solution to this particular problem – he ends up structuring his program differently to avoid the issue entirely. But in his discussion of this problem, he proposes something that he describes as ugly, and which, as he points out, doesn’t work anyway:

Func<string, {string Original, string Normalized}> transformer =
   x => new { Original = x, Normalized = x.ToLower() };

This is a direct approach to the problem described in the compiler error message. Can’t assign the expression to an implicitly-typed variable? OK, let’s make the variable explicitly typed. Unfortunately, you can’t specify the type because the expression involves an anonymous type. And that’s the thing about anonymous types: they don’t have names.

Daniel has made up a plausible syntax for denoting unnamed types: the first part in {} shows a possible solution. But that was just a hypothetical suggestion on his part – in fact C# 3.0 doesn’t provide a way to denote anonymous types.

But there is a solution. We carry on using an implicitly-typed variable, and fix the expression instead. The error says we can’t use a lambda expression. OK – let’s use something else. And as you’ll see, we don’t even have to get rid of the lambda. We can just wrap it in something.

Why Can’t I Use a Lambda?

To decide what to use in place of the raw lambda expression, we need to understand why C# isn’t letting us use a lambda here. The problem is that lambda expressions don’t have an intrinsic type. They’re more adaptable than that – a given lambda expression could mean several different things. The most straightforward demonstration of this is that the exact same expression could evaluate to either a delegate or an expression tree.

The compiler uses the context to determine the type. If you’re trying to assign or pass the lambda to something expecting a delegate, it’ll turn into a delegate. But if the target expects an expression tree, you’ll get one of those instead.

In short, a lambda’s expression’s evaluated type is determined by the context in which it’s used.

This gives us a problem with var: var says that its type is determined by the expression with which it is initialized. So when var meets a lambda, it’s like the two are stood by a doorway, each politely saying “No, after you”. (Apologies to my international readers – I suspect that might be a slightly over-British analogy...)

Something needs to decide the type. But the obvious approaches (ordinary explicitly-typed variables, or casts) end up hitting the problem Daniel hit: you can’t denote an anonymous type, so how do you specify the type? But it turns out that you don’t need to specify the type completely. All you really need to do is provide just enough information to let the C# compiler proceed.

Providing Just Enough Type Information

We can narrow down the C# compiler’s options without fully specifying the type. The trick is to go via a function call. Consider this apparently pointless function:

public static Func<T, TResult> Funcify<T, TResult>(Func<T, TResult> f)
{ return f; }

What use would that be, you may wonder? It takes any single-parameter function and returns that function without doing anything with or to it!

However, this provides the C# compiler with a little more information. If I pass a lambda as a parameter to Funcify, I’ve closed down an option: the C# compiler knows that whatever it produces needs to be a Func – it no longer has the option to produce an expression tree. This might be enough to disambiguate matters. Unfortunately, it’s not quite enough to fix our example, as we’ll see when we try it:

var transformer = Funcify(
      x => new { Original = x, Normalized = x.ToLower() });

This gets rid of the previous error, but replaces it with a new one:

“The type arguments for method 'Funcit.Program.Funcify<T,TResult>(System.Func<T,TResult>)' cannot be inferred from the usage. Try specifying the type arguments explicitly.”

But that’s progress. Honest! The compiler is no longer telling us that we can’t do this with var. It’s now telling us that it doesn’t have quite enough information.

The problem here is that it doesn’t know what ‘x’ is. And how could it? Any type with a ToLower method would do. The intent here is for it to be a string, so we just need to express that:

var transformer = Funcify(
      (string x) => new { Original = x, Normalized = x.ToLower() });

And now we’re good. The compiler is happy, because we gave just enough information to pin things down and it was able to infer the rest.

Mumble Types

I’ve seen in some C# team members’ blogs a suggestion for something they call ‘mumble types’. If I’ve understood them correctly, the idea is that you can provide a type specification, but you get to ‘mumble’ (i.e. be unclear) about parts of the type specification. This would give us an alternative solution:

Func<string, ?> transformer =
      x => new { Original = x, Normalized = x.ToLower() };

This doesn’t work today, but I would prefer this solution if it were available. It does exactly what I did – it lets me specify just enough to let the compiler’s type inference finish the job – but it doesn’t require the questionable hack of my Funcify function. And I think it’s clearer – with my example, it’s not particularly obvious that I was trying to fix some parts of the type specification while leaving other parts to be inferred. But with this ‘mumble type’ approach, it’s a lot more obvious that I’m saying ‘I want it to be a Func, and I want the input to be a string, but I want the compiler to infer the Func’s return type’. Also, this approach avoids the clutter of specifying the string type in the lambda parameter list itself, so I find it somewhat more aesthetically pleasing.

The use of the ‘?’ is based on what I’ve seen in C# team members’ blogs in the past year or two. However, here’s a different syntax to consider:

Func<string, var> transformer =
      x => new { Original = x, Normalized = x.ToLower() };

We already use the keyword var to say ‘the compiler should deduce this type’ for variable declarations. So I think it would be consistent to use the same keyword to mean the same thing in partially specified generic types too. That said, the ‘?’ has the benefit of standing out. And it seems reminiscent of Haskell’s use of ‘_’ to mean ‘whatever’. So I could see either approach, but I think I slightly prefer the idea of using var.

Gratuitous Plug for Training in London

And finally, some advertising. I’m teaching a couple of training courses for Pluralsight in London soon. I’ll be running the Silverlight course that Fritz Onion and I co-author at Old Street from 31st March to 3rd April. And I’ll be teaching our WPF course at the same location the following week – from 7th to 10th April.

Silverlight 2 Beta 1 Cross Domain Bug

Sun, 16 Mar 2008 15:37:54 GMT

I recently ran into what appears to be a bug in Silverlight 2 Beta 1’s handling of cross-domain web service access when using a clientaccesspolicy.xml file. I’m hoping this post might save a few other people the time it took me to work out what was going on here.

Here’s the executive summary: if the web service exposes resources whose URIs contain semicolons, you will not be able to access those resources cross-domain if you’re using clientaccesspolicy.xml. The workaround is to use crossdomain.xml instead.

Now for the more detailed version.

In case you’re not familiar with cross-domain web service access, here’s the basic idea. By default, a web browser won’t let client-side code go connecting to any old web site. Client-side code is allowed to make requests against the web site from which it was originally downloaded, and it should only have access to other sites if those sites opt in.

In pure AJAX sites, this is often achieved using a faintly smelly hack where web services return runnable script rather than simple data. Flash introduced a somewhat more formal mechanism by which a web site can declare that it’s happy to be accessed by client-side code from other domains. Silverlight now supports this feature as of v2 beta 1.

Here’s an example. If your web site offers a resource called /crossdomain.xml containing this:

<!DOCTYPE cross-domain-policy SYSTEM
   "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
  <allow-access-from domain="*" />
</cross-domain-policy>

the site is declaring that it is open to anyone.

This policy is enforced on the client side by whatever runtime is performing the cross-domain network activity. Both Flash and Silverlight 2 beta 1 will look for this file when code in these runtimes attempts to access a web site from another domain. (This doesn’t help you from JavaScript, because today, no browser script engine supports this mechanism.)

One problem with this is that it’s a bit of a blunt instrument. You can be selective about who is allowed to access your site – you don’t have to use a wildcard. But you can’t be selective about what they can access. E.g., I might like to say that only content beneath a particular URI should be accessible to client code that originated from external domains.

Selective Cross-Domain Access: clientaccesspolicy.xml

As well as supporting Flash’s mechanism, Silverlight 2 beta 1 also introduces a more selective form of cross-domain policy. Before looking for a Flash-style crossdomain.xml resource, Silverlight will look for a clientaccesspolicy.xml file. This has a slightly more complex structure that enables you to constrain what a client can do. Here’s an example:

<access-policy>
  <cross-domain-access>
    <policy>
      <allow-from>
        <domain uri="*"/>
      </allow-from>
      <grant-to>
        <resource path="/extsvc/" include-subpaths="true"/>
      </grant-to>
    </policy>
  </cross-domain-access>
</access-policy>

This says that clients from any external domain are granted cross-domain access, but it limits it to resources that start with /extsvc/.

This is a little more flexible than the Flash approach – it lets you make only selected parts of your site available. With crossdomain.xml, you would need to expose services under a distinct domain name to get this selectivity. Obviously that’s not rocket science – it’s certainly possible to present various services under various different domain names. So Silverlight’s approach doesn’t exactly enable anything that was impossible under Flash. On the other hand, this is a whole lot more convenient for people whose web hosting arrangements might not make it so easy to crank up new domain names for each set of services requiring distinct client access policies. And editing a text file is usually a more lightweight process than partitioning a web site’s services across multiple domains.

Semicolons Not Welcome in Beta 1

Unfortunately, there seems to be a problem in beta 1 of Silverlight. If the service you wish to access has a semicolon in its URI, Silverlight won’t allow your client-side code to access it, no matter what the clientaccesspolicy.xml file says. This is unfortunate if you’re using a service written the way advocated in RESTful Web Services, which recommends delimiting order-independent parts of a URL with semicolons. And the service I was using did exactly that. It took me a while to work out that it was specifically the semicolons that were causing problems, hence this post.

This is presumably just a bug in the code that works out whether any particular URI is permitted by the policy. Since the simpler crossdomain.xml grants all-or-nothing access, it never has to examine the URI, so it doesn’t have this problem. That’s why the simpler but less flexible crossdomain.xml offers a workaround here.

Learn More about Silverlight 2

If you want to learn more about Silverlight, I’ll be teaching the Silverlight course that Fritz Onion and I co-author for Pluralsight in London in a couple of weeks. The 4 day course will be running at Old Street from 31st March to 3rd April. (And I’ll be teaching our WPF course at the same location the following week.)

DevWeek 2008 Cross Platform Silverlight Demos

Wed, 12 Mar 2008 18:40:14 GMT

I just finished the Cross Platform .NET on Silverlight talk at DevWeek. Demos can be downloaded from http://www.interact-sw.co.uk/downloads/DevWeek2008XPlatDemos.zip

I'm all done at DevWeek for this year. But if you want to hear more about Silverlight, I'll be teaching Pluralsight's Applied Silverlight course in London later this month - running from 31st March. (And the following week I'll be teaching our Applied WPF course, also in London.)