Running with Code Like with scissors, only more dangerous

28Mar/110

A Recent Discovery: IronJS

Posted by Rob Paveza

Since Microsoft first announced Managed JScript and shortly thereafter announced its demise (more on that here), I’ve been chomping at the bit for an implementation of JavaScript on the DLR.  Actually, I’ve probably been hoping for it for far longer, because JavaScript is near and dear to my heart.  If not for Brinkster offering free ASP-based web hosting back when I was just a young’un, I may never have discovered server-side programming, and may never have ended up where I am today. 

IronJS is a free, open-source implementation of ECMAScript v3 on the DLR based predominantly on F#.  It’s even the right kind of free, currently licensed under the Apache License version 2 (not a copyleft license).  Check out this picture of its testbed application running:

image

As a learning exercise, I intend to figure out a good way to shoehorn this into my Battle.net (1.0) chat client, JinxBot.  It actually shouldn’t be terribly difficult, but while I considered adding it as a plugin, I think I’d like it to be part of the core application instead. 

I’ll more than likely be covering IronJS in several parts in my blog in the coming weeks (possibly months, since I’m getting married on May 1).  But these are topics that I intend to cover:

  • Hosting IronJS in a C# application, loading, and executing JavaScript code
  • Sharing objects between script and .NET
  • Isolating script from .NET (preventing script from having run of the whole runtime)
  • Isolating script from script (running multiple environments in one application, similar to how web pages in browsers like IE and Chrome all run from one instance of the script engine but have different contexts so they have different sets of objects)
  • Performance optimizations
  • Dynamically generating isolation layers
  • Other considerations

Stay tuned!  In the meantime, check out IronJS on GitHub and get it running on your machine!

22Mar/111

Worries over the Future of the .NET Framework

Posted by Rob Paveza

Over the last few years, as exciting as some of the things have been that have come out that make a developer’s time more and more powerful, particularly in the .NET space – LINQ, dynamic programming, lambdas – I’ve become a bit concerned over the evolution of the .NET Framework’s type system.  It seems to me like some things that should be obvious but are not are being left aside.  This post will explore some of the programming issues that I have seen and am predicting.

Automatic Type Conversion for Delegates

Consider a Predicate<T>.  A Predicate<T> delegate accepts a single parameter of type T and returns a Boolean value, true or false, indicating whether that item matched some condition.

List<int> numbers = new List<int> { 1, 2, 5, -20, 10, -5, -3, 0 };
Predicate<int> isPositive = n => n > 0;
List<int> positives = numbers.FindAll(isPositive);


In the code here, I’ve used a lambda expression to indicate that the method generated for the lambda expression should be assignable to a type of Predicate<int>.  Fortunately, C# doesn’t allow you to assign a lambda to an implicit variable (a var), but consider that I could have also written:

List<int> positives = numbers.Where(n => n > 0).ToList();

Note how the lambda expression is exactly the same, and the two pieces of code would produce identical auto-generated methods, yet I couldn’t use numbers.Where(isPositive) because Where takes a Func<T, bool>, and not a Predicate<T>.

So what’s wrong with this?  Hopefully, you can see that it would be reasonable to say that a Func<T, bool> is a Predicate<T>, and that a Predicate<T> is a Func<T, bool>.  That is, delegates in this way have a dual-direction inheritance relationship.

It gets even more interesting when you bring Actions into this.  Actions are delegates that perform some work, accepting 0 or more parameters, but have no return value.  That is, they return void.  Because C# does not allow “Void” to be a type parameter, it is impossible to create an Action<Void>, or an Action<int, Void>.  But, technically speaking, you could have such a construction.

Think about what a delegate really does.  A delegate is the information the compiler needs in order to:

  1. Push all of the parameters onto the stack in the correct order.
  2. Let the callee (the method being called) do its work.
  3. The callee cleans up its own parameters from the stack.
  4. The callee pushes its return value onto the stack, if there is one.
  5. The calling method knows, based on the signature, whether there is a return value on the stack, so it knows the state of the stack.

An Action<T> is really just a specialization of Func<T, Void>.  The two ought to be compatible, and we should stop having so many delegate types polluting the framework and Base Class Library.

Making this Work without Modifying the CLR

Delegate types are defined by IL as types as well.  It may be possible to shoehorn this feature in without modifying the CLR by:

  • Add cast overloads for all delegates to implicitly cast to a Func type that supports its method signature.
  • Add cast overloads for all delegates to be implicitly cast-to from a Func type that supports its method signature.

If I could define something like this in C#, it might look like this:

public delegate bool Predicate<T>(T item)
{
    public static operator implicit Func<T, bool>(Predicate<T> original)
    {
        Func<T, bool> result = _item => original(_item);
        return result;
    }

    public static operator implicit Predicate<T>(Func<T, bool> original)
    {
        Predicate<T> result = _item => original(_item);
        return result;
    }
};

Note that this method uses closures, which would preserve all existing object relationships and not break any object relationship semantics normally required by delegate types.

Numeric Interface for All Primitive Types, Integer Interface for Integral Primitives

Primitive types are the basic structures supported by the CLR:

  • byte/sbyte
  • short/ushort
  • int/uint
  • long/ulong
  • float / double
  • char

String is also included, but that’s a class and heap-allocated, so it’s not really important for the purposes of this discussion.

There are a lot of common operations that you can do with these types.  You can always add them, subtract them, divide them (dividing a char is a weird thing to consider, but is possible).  Actually, if you want to know the position of a given letter in the alphabet, you can just subtract ‘A’ or ‘a’ from it. 

But when it comes to something like a generic type parameter, you can’t say something like:

public class NumberAggregator<T> 
    where T : INumeric
{
    private T _sum = default(T);
    private List<T> _numbers = new List<T>();
    public void Add(T item) 
    { 
        _numbers.Add(item);
        _sum += item;
    }
    // rest of the class
}

This is not possible.  First, there’s no such interface as INumeric, but more importantly, even though the intent of this class is to be able to sum any single size of number, the restriction can be supported by the underlying IL code.  The IL add instruction is a single instruction that does not take into regard the operands (see CIL Instruction Reference, ECMA-335 Partition III page 33). 

You can do a lot of cool things with numbers, particularly integers, that you can’t do with normal types.  You can add, subtract, divide, multiple, bitwise and/or/xor, shift, and invert (I think that covers all of them).  But the generic type system is too limited to allow you to do that.

Type Parameter Restrictions

A long while ago, I talked about the limitation of generics in that they may not support certain types as type parameter constraints, in that article specifically, enumeration types.  Enumeration types would give you similar benefits that I noted in the previous section, about the operations you can perform on numeric types that you generally can’t perform on other types.

While generics in .NET lack some of the power of C++ templates (namely, the ability to create a template that accepts a value, not just a type, as a parameter), they’re still quite good, and they solved a big performance problem in the 1.x CLR of boxing and unboxing primitive types.  However, I still believe that the CLR is limited in a way it does not need to be.  I don’t want to repeat myself, but this particular component is a big one – and a lot of the problems that issue causes could be solved with #2 above.

The .NET Library is Not Unified

Product owners who develop the core CLR need to come together and bridge the gap between incompatible interfaces.  The ISynchronizeInvoke interface was developed and part of the CLR version 1.0.  It was known as the standard way to allow cross-thread communication, most commonly for instances in which a background thread needed to post updates to the main UI thread of a Windows Forms application.

When WPF was released as part of .NET 3.0, it included a completely new model, called Dispatcher, that achieved the exact same purpose.  There was only one problem: Dispatcher doesn’t implement ISynchronizeInvoke, which means that any libraries written for previous versions of .NET, if they had code that needed to synchronize with a user interface, had an incompatible interface.

Writing an adapter for this is not an issue; making a component written for ISynchronizeInvoke work for Dispatcher is fairly trivial.  But this isn’t the only situation in which it’s happened; you have it happening with LINQ to SQL and Entity Framework (a travesty that LINQ to SQL is as closed as it is, because it could have been Entity Framework without all of the enterprisey things that EF has).  You have it within ASP.NET.  In WCF.  How about Silverlight being a reproduction of but completely incompatible implementation of WPF?

Final Thoughts

I still love programming on .NET.  It allows me to spin up applications so quickly and conveniently that I don’t imagine that I’d like to change any time soon.  But on the other hand, sometimes the decisions that the language and platform designers make baffle me. 

So – to my friends who got this far hoping that I’m making my announcement of switching to PHP, Java, or Python – sorry.  You’ll have to wait for another day.  But, hey, Mads, Anders, if we could get some of these changes in – that’d be great!

Filed under: Uncategorized 1 Comment
15Mar/110

The Microsoft Reactive Extensions

Posted by Rob Paveza

The honest truth is that I’m having difficulty establishing exactly what they could be used for, but they’re still really cool.  The Microsoft Reactive Extensions for the .NET Framework are the dual of LINQ: whereas LINQ operates over objects, or you might say pulls objects out of collections, the Reactive Extensions (Rx) handles push notifications.  It is the ultimate generalization of events and event handling within .NET.

Getting There

First, let’s consider the normal interfaces for IEnumerable:

interface IEnumerable<T>
{
    IEnumerator<T> GetEnumerator();
}

interface IEnumerator<T> : IDisposable
{
    T Current { get; }  // throws exception at end of enumeration
    bool MoveNext();
}

These interfaces (okay, really, the non-generic IEnumerable interface, but let’s not split hairs) are the foundation of the foreach C# keyword (and the For Each… In in Visual Basic).  A foreach can also be written, roughly, as:

foreach (string str in myListOfStrings)
    Console.WriteLine(str);
// rewritten:
using (IEnumerator<string> enumStr = myListOfStrings.GetEnumerator())
{
    while (enumStr.MoveNext())
    {
        Console.WriteLine(enumStr.Current);
    }
}

Keep this example in mind for later, because we’ll revisit how this can be used in Rx programming.

Dualism

Dualism is something of a mathematical concept, and I don’t want to get into it because I don’t completely understand it myself, but most nerdy people reading my blog will probably appreciate an example from particle physics.  Consider a proton: its physical dual is the antiproton (because when they meet they annhilate each other.  It’s not an electron, because while they have opposite charge, they have substantially different mass).

The core of Rx is the dual of IEnumerable.  That is, IObservable<T> and IObserver<T>.  But let’s deconstruct these piece by piece.  Let’s start at IEnumerator<T>:

interface IObserver<T>
{
    // T Current { get; }
    // That method looks like: T get_Current();
    void OnNext(T next);
    // Current throws an exception if MoveNext() previously returned false, so:
    void OnError(Exception error);

    // bool MoveNext() 
    // returns true while Current is populated, false when we reach the end, so:
    void OnDone();
}

You can see that, whereas everything in IEnumerator<T> pulled data, now we’ve transitioned into pushing data.  But the observer isn’t really the cool part; rather, it’s the subject that’s cool:

interface IObservable<T>
{
    // GetEnumerator() returned an object; here we pass one in
    // We still needed to capture the disposable functionality, so we return IDisposable
    IDisposable Subscribe(IObserver<T> observer);
}

Now, if you want to see the specifics about how these were constructed, you can check out the Expert-to-Expert video on Channel 9.  I’ve included some high-level notes, but they’re not really as deep as you can get with these guys.

Creating a Subject

Creating a subject is a bit of a challenge; subjects are event-driven, and those are generally kind of difficult to think about because the fit usually only into one of two buckets: user interaction and system I/O.  For sake of example, I’ve created a simple Windows Forms project to start with, that has a couple observable Buttons (the class is called ObservableButton, go figure), and an observer, which is the containing form.  You can download the starter project, which requires Visual Studio 2010 and the Rx Framework.

Subjects can be anything, though, and the power you can glean from these is amazing.  For the Red Bull NASCAR team, I created a server for a Twitter feed aggregator using Rx.  It started as reading a socket into HTTP data, then into chunked HTTP data, then into JSON packets, then into POCO objects that were then re-serialized and sent over the wire to N Flash clients.  As you can imagine, network programming, social programming, or any other kind of programming where an event is coming in unpredictably is a great candidate for this.  Why?

Let’s look at the use case I just listed.  As Twitter’s live stream service sends data over the wire, I need to parse it and send it to a lot of listening sockets.  But I don’t want to just say “Oh I just got the data, let me send it out again” – that would possibly slow down processing on other threads, because I might have to wait – my socket might already be in the process of sending data and so it’s in an invalid state to send further data.  If I had tied a server socket directly to the “I’m ready to send” signal directly, I would have been in trouble.  Rather, I had a utility (an Observer) that aggregated incoming messages until all server sockets were ready to send, at which point it would push those updated messages to the server sockets.

Let’s look at the sample program:

image

This isn’t really anything spectacular.  I could have done that with regular event handlers.

Aggregating Subjects

The magic of Rx, from my perspective, lies with what you can do with subjects.  I’m no longer initializing my constructor to require two lines – I’m merging the two buttons into one observable sequence:

        public Form1()
        {
            InitializeComponent();

            observableButton1.Merge(observableButton2).Subscribe(this);
        }

 

The result is identical – the events get handled and all is good.

Modifying Sequences

Now I’m going to change the class definition slightly:

    public partial class Form1 : Form, IObserver<Timestamped<string>>
    {
        public Form1()
        {
            InitializeComponent();

            observableButton1.Merge(observableButton2).Timestamp().Subscribe(this);
        }

        public void OnNext(Timestamped<string> value)
        {
            this.textBox1.Text += value.Timestamp.ToString("hh:mm tt   ") + value.Value + Environment.NewLine;
        }

        public void OnError(Exception error)
        {
            this.textBox1.Text += "Exception caught: " + Environment.NewLine + error.ToString() + Environment.NewLine;
        }

        public void OnCompleted()
        {
            this.textBox1.Text += "Sequence completed." + Environment.NewLine;
        }
    }

Note that by adding in the .Timestamp() call, I’ve transformed the observable to sequence of strings to be an observable sequence of timestamped strings.  That’s pretty cool, right?

This is even cooler: the Delay() method:

observableButton1.Merge(observableButton2).Timestamp()
                .Delay(new TimeSpan(0, 0, 1)).ObserveOn(this).Subscribe(this);

The ObserveOn method accepts a Windows Forms control, a Dispatcher (for WPF), or other scheduler implementation that can be used to synchronize the delay.  If I didn’t include it, the delayed merge would be called on a different thread, and we’d get an InvalidOperationException (because you can’t update a window on a thread other than the thread that created it). 

Do you want to avoid repetition?

            observableButton1.Merge(observableButton2).Timestamp()
                .DistinctUntilChanged(ts => ts.Value).Subscribe(this);

This produced output that only emitted one message, no matter how many times I clicked the same button, until I clicked the other button.

So, What Can We Do?

Well, right now it doesn’t seem like there’s a lot of tooling for Rx.  There’s a community wiki around the framework, though, and I think that we can eventually see a lot of good use.

Some ideas:

  • Develop a way to completely repeat ASP.NET requests.  Treat IIS as an IObservable<AspNetRequest>, where AspNetRequest contains all the state data that would otherwise populate these tools, which would immensely help with debugging.  Imagine when your tester only needs to record a series of test cases once, and otherwise is just testing for UI errors.
  • Wrap event-oriented APIs for simplified logging and replaying.  (In JinxBot, an event-oriented chat API named for my cat, I always wanted to capture all the events of the core API and be able to replay them via a subclass, which would have allowed pixel-perfect replay of a chat session).
  • Handle periodic data services like Twitter, SMS, email, or others in a clean and efficient way.

I’d like to see this take off, but it’s a very different way of looking at programming than what most .NET developers are used to.  Enjoy it, take a look, and let’s build it up!