Running with Code Like with scissors, only more dangerous

28Jan/120

A Usability Problem

Posted by Rob Paveza

I don't use Firefox very often. Generally I'm in Chrome or IE. But, I keep Firefox on my taskbar, because I do enough web development that it's prudent for browser compatibility testing.

Every now and then I get this dialog:
Firefox Add-ons Dialog

Note that when I'm not taking a screenshot, that first item has a blue highlight instead of gray.

What's the problem?

Well, first of all, it's a popup. It has its own window, which in my mind, means that Firefox thinks I ought to do something about it. The only action that appears to be there is the "Find Updates" button; but of course, that doesn't find updates to the plugin I just installed, which is the purpose of the dialog in the first place, isn't it?

No. In the screenshot, the Java Console 6.0.27 is the newly-installed plugin because I just installed Java (with MonoTouch and MonoDevelop).

Mozilla, this dialog sucks, and has sucked since I saw it back in the early days of Firefox. Take a lesson from IE, and if you want to show me a notification, put it in a bar. Make it say, "The 'Java Console' add-on has been newly installed." Or, "4 new add-ons have been installed; click here to see this list."

At the heart of this problem, though, is a cultural difference; I strongly believe that the *nix culture is reflected in this user experience. It was as if someone said, "OK, well, we'll run firefox --check-plugin-updates | firefox-ui and the result was piped to a new window, because it had nowhere better to go.

That's not to say that command-line piping to the UI is a bad thing (though I believe that it's better to make the programmatic interface between two subsystems, well, programmatic, not text), but if someone had actually spent some time designing the user interaction for the "New plugins installed" use-case, they would not have decided to simply reuse the "Show all installed plugins" UI.

17Jan/120

A very fast, random text string generator in C#, Part One

Posted by Rob Paveza

This is about half of a type I designed a few years ago in response to my boss's assertion that I couldn't improve on the speed of random string generators. This is a pretty handy type to have around when you want to generate a character string of a fixed length. It so happens that 6 characters can fit a "base 36" number very well into just over 231 bits (it's just a little bit bigger than int.MaxValue and so it stores the value as a uint).

What is included below is a first step. I want to show how to refactor it for improved performance and where certain design scenarios might be better; for example, repeatedly calling the CreateNew method results in repeatedly creating new RNGCryptoServiceProviders. Although I believe this class is a bit better about seeds as it incorporates hardware-dependent data, it isn't necessarily cryptographically strong. You can add some strength by passing in a single RNG to each subsequent call, for example.

This sample also only goes one way: from number to string. Next time, we'll show the reverse operation.

Note: an important inclusion:

using System.Security.Cryptography;
    /// <summary>
    /// Represents a number that may be transitioned to a six-digit alphanumeric (base-36) representation.  This type is not CLS-compliant.
    /// </summary>
    [CLSCompliant(false)]
    public struct AlphaNumericNumber
    {
        private const uint MAX_VALUE = 2176782336; // = RADIX ^ 6
        private const int RADIX = 36; // 10 digits + 26 alphabetics
        private uint m_val;

        /// <summary>
        /// Creates a new <see>AlphaNumericNumber</see> with the specified value.
        /// </summary>
        /// <param name="value">The value with which to assign the number.</param>
        public AlphaNumericNumber(uint value)
        {
            if (value > MAX_VALUE)
                value %= MAX_VALUE;
            m_val = value;
        }

        /// <summary>
        /// Creates a new, random <see>AlphaNumericNumber</see>.
        /// </summary>
        /// <returns>A randomly-chosen <see>AlphaNumericNumber</see>.</returns>
        public static AlphaNumericNumber CreateNew()
        {
            byte[] container = new byte[4];
            using (RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider())
            {
                rng.GetNonZeroBytes(container);
            }

            return new AlphaNumericNumber(BitConverter.ToUInt32(container, 0));
        }

        /// <inheritdoc />
        public override string ToString()
        {
            char[] result = new char[6];
            uint msd = RADIX * RADIX * RADIX * RADIX * RADIX;

            uint currentDigit = msd;
            uint accumulator = m_val;
            unchecked
            {
                for (int charIndex = 0; charIndex < result.Length; charIndex++)
                {
                    uint currentDigitValue = accumulator / currentDigit;
                    accumulator -= (currentDigitValue * currentDigit);
                    if (currentDigitValue <= 9)
                    {
                        result[charIndex] = (char)('0' + currentDigitValue);
                    }
                    else
                    {
                        currentDigitValue -= 10; // adjust for 'A' equaling 10.
                        result[charIndex] = (char)('A' + currentDigitValue);
                    }

                    currentDigit /= RADIX;
                }
            }

            return new string(result);
        }

        /// <summary>
        /// Gets the value contained by this <see>AlphaNumericNumber</see>.
        /// </summary>
        public uint Value
        {
            get { return m_val; }
        }
    }
10Jan/120

Please, don’t ever use the “Remove Unused Namespaces” feature

Posted by Rob Paveza

Seriously, I don't know why this feature exists or who thought it would be a good idea. The time it takes to compile even the most complicated C# file is trivial.

On the other hand, when I try to access the .Take() method in System.Linq, or the Encoding class in System.Text, or suddenly I need a new List<T> from System.Collections.Generic - if you have used a tool to remove unused namespaces from your C# code file - you have made me wonder why my editor isn't detecting it.

There's a reason that many default namespaces are included with a default C# code file. Leave them there! Maintenance programmers will love you forever.

9Jan/121

Revealing Prototype Pattern: Pros and Cons

Posted by Rob Paveza

A short while ago, I wrote a post generally saying good things about the Revealing Prototype Pattern but mostly focused tearing down the other part that was presented with it, namely the way that variables were declared in a chain separated by the comma operator. This post will discuss some of the pros and cons of using this pattern, and give some examples about when you should or shouldn't use it.

Advantages

As Dan notes in his post, this features a significant improvement over straight prototype assignment (assignment of an object literal to a prototype), in that private visibility is supported. And because you're still assigning an object to the prototype, you are able to take advantage of prototypal inheritance. (Next time: How prototypal inheritance really works and how it can make your head explode).

Except for chaining his variable declarations, I have to admit Dan had me pretty well sold on the Revealing Prototype Pattern. It's very elegant; it provides good protection to its inner variables, and it uses a syntax we've been seeing more and more of ever since jQuery became a popular platform.

Unfortunately, it has some nasty drawbacks.

Disadvantages

To be fair, Dan lists some of the disadvantages about this pattern; however, he doesn't quite list them as such, and I think he was possibly unaware of some of their implications:

There's something interesting that happens with variables though, especially if you plan on creating more than one Calculator object in a page. Looking at the public functions you'll see that the 'this' keyword is used to access the currNumberCtl and eqCtl variables defined in the constructor. This works great since the caller of the public functions will be the Calculator object instance which of course has the two variables defined. However, when one of the public functions calls a private function such as setVal(), the context of 'this' changes and you'll no longer have access to the two variables.

The first time I read through that I glossed over the problems; I didn't quite understand the issue until I wrote some code. So let's do that - we'll implement the Java StringTokenizer class:

function StringTokenizer(srcString, delim)
{
    if (typeof srcString === 'undefined')
        throw new ReferenceError("Parameter 0 'srcString' is required.");
    if (typeof srcString !== 'string')
        srcString = srcString.toString();
    if (typeof delim !== 'string')
        delim = ' ';
    
    if (!(this instanceof StringTokenizer))    // enforce constructor usage
        return new StringTokenizer(srcString, delim);
        
    this.sourceString = srcString;
    this.delimiter = delim;
}
StringTokenizer.prototype = (function()
{
    var that = this;
    
    var _tokens = that.sourceString.split(that.delimiter);
    var _index = 0;
    
    var _countTokens = function() { return _tokens.length; };
    var _hasMoreTokens = function() { return _index < _tokens.length; };
    var _nextToken = function()
    {
        if (!_hasMoreTokens())
            return false;
        
        var next = _tokens[_index];
        _index += 1;
        return next;
    };
    var _reset = function() { _index = 0; };
    
    var resultPrototype = 
    {
        countTokens: _countTokens,
        hasMoreTokens: _hasMoreTokens,
        nextToken: _nextToken,
        reset: _reset
    };
    return resultPrototype;
})();

If you've ever written a jQuery plugin, you'll probably recognize what I did with the prototype assignment function; when writing jQuery plugins, it's common to close over the current instance of the jQuery object by assigning var that = $(this); so that you can write event-handler functions without losing access to the overall context. Unfortunately, what I did in this case is wrong; you may already see why.

var that = this;

In this context, this is a reference to the global object, not to the instance of the object - even though the prototype is being set. This is a generalization of what Dan said. Rewriting it to overcome it results in information leaking:

function StringTokenizer(srcString, delim)
{
    if (typeof srcString === 'undefined')
        throw new ReferenceError("Parameter 0 'srcString' is required.");
    if (typeof srcString !== 'string')
        srcString = srcString.toString();
    if (typeof delim !== 'string')
        delim = ' ';
    
    if (!(this instanceof StringTokenizer))    // enforce constructor usage
        return new StringTokenizer(srcString, delim);
        
    this.sourceString = srcString;
    this.delimiter = delim;
    this.tokens = srcString.split(delim);
    this.index = 0;
}
StringTokenizer.prototype = (function()
{
    var _countTokens = function() { return this.tokens.length; };
    var _hasMoreTokens = function() { return this.index < this.tokens.length; };
    var _nextToken = function()
    {
        if (!this.hasMoreTokens())
            return false;
        
        var next = this.tokens[this.index];
        this.index += 1;
        return next;
    };
    var _reset = function() { this.index = 0; };
    
    var resultPrototype = 
    {
        countTokens: _countTokens,
        hasMoreTokens: _hasMoreTokens,
        nextToken: _nextToken,
        reset: _reset
    };
    return resultPrototype;
})();

The code works correctly; but you can see that we have to make public all of the state variables we'll use in the constructor. (The alternatives are to either initialize the state variables in each function, where they would still be public; or to create an init function, which would still cause the variables to be public AND would require the user to know to call the init function before calling anything else).

Dan also indicated that you needed a workaround for private functions:

There are a few tricks that can be used to deal with this, but to work around the context change I simply pass “this” from the public functions into the private functions.

Personally, I prefer to try to avoid things one might call clever or tricky, because that's code for "so complex you can't understand it". But even in the case where you have a public function, you'll still get an error if you don't reference it via a public function call. This error is nonintuitive and could otherwise make you go on a bug hunt for a long time. Consider this change to the above code:

    var _hasMoreTokens = function() { return this.index < this.tokens.length; };
    var _nextToken = function()
    {
        if (!_hasMoreTokens())   // changed from:   if (!this.hasMoreTokens())
            return false;
        
        var next = this.tokens[this.index];
        this.index += 1;
        return next;
    };

Simply removing the 'this' reference in the caller is enough to cause 'this' to go out-of-scope in the _hasMoreTokens function. This is completely unintuitive behavior for developers who grew up in the classical inheritance model.

Alternatives

I wouldn't want to give you all of these options without giving you an alternative. The alternative I present here is one in which the entire object is populated in the constructor:

"use strict";
function StringTokenizer(srcString, delim)
{
    if (typeof srcString === 'undefined')
        throw new ReferenceError("Parameter 0 'srcString' is required.");
    if (typeof srcString !== 'string')
        srcString = srcString.toString();
    if (typeof delim !== 'string')
        delim = ' ';
    
    if (!(this instanceof StringTokenizer))    // enforce constructor usage
        return new StringTokenizer(srcString, delim);
        
    if (typeof Object.defineProperty !== 'undefined')
    {
        Object.defineProperty(this, 'sourceString', { value: srcString });
        Object.defineProperty(this, 'delimiter', { value: delim });
    }
    else
    {
        this.sourceString = srcString;
        this.delimiter = delim;
    }
    
    var _tokens = this.sourceString.split(this.delimiter);
    var _index = 0;
    
    var _countTokens = function() { return _tokens.length; };
    var _hasMoreTokens = function() { return _index < _tokens.length; };
    var _nextToken = function()
    {
        if (!_hasMoreTokens())
            return false;
        
        var next = _tokens[_index];
        _index += 1;
        return next;
    };
    var _reset = function() { _index = 0; };
    
    if (typeof Object.defineProperty !== 'undefined')
    {
        Object.defineProperty(this, 'countTokens', { value: _countTokens });
        Object.defineProperty(this, 'hasMoreTokens', { value: _hasMoreTokens });
        Object.defineProperty(this, 'nextToken', { value: _nextToken });
        Object.defineProperty(this, 'reset', { value: _reset });
    }
    else
    {
        this.countTokens = _countTokens;
        this.hasMoreTokens = _hasMoreTokens;
        this.nextToken = _nextToken;
        this.reset = _reset;
    }
}

The advantage of a structure like this one is that you always have access to this. (Note that this example is unnecessarily large because I've taken the additional step of protecting the properties with Object.defineProperty where it is supported). You always have access to private variables and you always have access to the state. The unfortunate side effect of this strategy is that it doesn't take advantage of prototypal inheritance (it's not that you can't do it with this strategy - more of that coming in the future) and that the entire private and public states (including functions) are closed-over, so you use more memory. Although, one may ask: is that really such a big deal in THIS sample?

Usage Considerations

The Revealing Prototype Pattern can be a good pattern to follow if you're less concerned with maintaining data integrity and state. You have to be careful with access non-public data and functions with it, but it's pretty elegant; and if you're working on a lot of objects, you have the opportunity to save on some memory usage by delegating the function definitions into the prototype rather than the specific object definition. It falls short, though, when trying to emulate classical data structures and enforce protection mechanisms. As such, it can require complicated or clever tricks to work around its shortcomings, which can ultimately lead to overly-complex or difficult-to-maintain code.

Like most patterns, your mileage may vary.

6Jan/120

Defining Variables in JavaScript

Posted by Rob Paveza

I've lately been reviewing different patterns and practices recently, and after reading his article about the Revealing Prototype Pattern, I wanted to take some time to analyze Dan Wahlin's approach to defining variables. It's hard to believe I found some common ground with Douglas Crockford, but as they say about broken clocks.... [Addendum: apparently, since JSlint says to 'combine with previous var statement,' I don't agree with Crockford.] Anyway, to begin, this post is inspired by Dan Wahlin's presentation of the Revealing Prototype Pattern; I noticed what I thought was a curious way for him to define his private variables, and looking back through his blog posts he discussed it in the original blog post of the series, Techniques, Strategies, and Patterns for Structuring JavaScript Code. For the most part, I like what Dan has to say, but I'm going to have to disagree when it comes to defining variables.

The Proposition

As Dan points out, this is the standard way of defining variables in JavaScript:

var eqCtl;
var currNumberCtl;
var operator;
var operatorSet = false;
var equalsPressed = false;
var lastNumber = null;

He advocates trading that for this:

var eqCtl,
    currNumberCtl,
    operator,
    operatorSet = false,
    equalsPressed = false,
    lastNumber = null;

It saves on 20 keystrokes, and he claims improved readability. Now, I disagree with Crockford's argument that, because JavaScript hoists variables to the top of the function, that you should always declare the variable there. I believe that, whenever possible, you should try to maximize locality of a variable. This is a principle discussed in Steve McConnell's Code Complete; the reasoning behind maximization of locality is that the human brain can only comprehend so much at once. (This is, of course, another argument in favor of many, simple, and small subroutines). By delaying the declaration of a variable until it needs to be used, we are able to better-comprehend the meaning of the variable and how its use affects and relates to the rest of the program. As such, I believe that one of the premises for moving these declarations into a combined var statement - specifically, to reflect the hoisting - is a poor rationale.

Let's carry on.

Similarities to Other Elements

In The Prototype Pattern, Dan demonstrates the use of a JavaScript object literal in assignment to the Calculator prototype, so that any object created using the Calculator constructor would inherit all of those properties:

Calculator.prototype = {
    add: function (x, y) {
        return x + y;
    },
    subtract: function (x, y) {
        return x - y;
    },
    multiply: function (x, y) {
        return x * y;
    },
    // ...
};

The important thing to note here is that we are simply defining an object literal; we are not writing procedural code, and that comma is not an operator! It is an important part of the JavaScript grammar, to be sure, but the comma here does not have the same semantic meaning as the comma we saw before. This subtle difference may lead to coding errors, in which someone who uses comma syntax with both will mistakenly believe they are declaring an object literal and use colons to separate the identifier from the value; or that they are declaring variables and use assignment syntax to separate the property from its value.

Comma Operator Considered Harmful

It surprises me to learn that JSlint advocates combining var declarations. Crockford's The Elements of JavaScript Style, Part 1 indicates that he isn't terribly fond of it either:

The comma operator was borrowed, like much of JavaScript's syntax, from C. The comma operator takes two values and returns the second one. Its presence in the language definition tends to mask certain coding errors, so compilers tend to be blind to some mistakes. It is best to avoid the comma operator, and use the semicolon statement separator instead.

Whichever way Crockford prefers it, I think what we need to remember is just because you CAN do something does not mean you SHOULD.

Let's consider Dan's full body of JavaScript from the Revealing Prototype Pattern. I'm going to shrink it a little bit, to emphasize the changes I'll make; and I'm removing any of his comments.

var Calculator = function (cn, eq) {
    this.currNumberCtl = cn;
    this.eqCtl = eq;
};

Calculator.prototype = function () {
    var operator = null,
        operatorSet = false,
        equalsPressed = false,
        lastNumber = null,
        add = function (x, y) { return x + y; },
        subtract = function (x, y) { return x - y; },
        multiply = function (x, y) { return x * y; },
        // I'm going to do something evil here.
        divide = function (x, y) {
            if (y == 0) {
                alert("Can't divide by 0");
            }
            return x / y;
        },
        setVal = function (val, thisObj) { thisObj.currNumberCtl.innerHTML = val; },
        setEquation = function (val, thisObj) { thisObj.eqCtl.innerHTML = val; },
        clearNumbers = function () {
            lastNumber = null;
            equalsPressed = operatorSet = false;
            setVal('0',this);
            setEquation('',this);
        },
        setOperator = function (newOperator) {
            if (newOperator == '=') {
                equalsPressed = true;
                calculate(this);
                setEquation('',this);
                return;
            }
            if (!equalsPressed) calculate(this);
            equalsPressed = false;
            operator = newOperator;
            operatorSet = true;
            lastNumber = parseFloat(this.currNumberCtl.innerHTML);
            var eqText = (this.eqCtl.innerHTML == '') ?
                lastNumber + ' ' + operator + ' ' :
                this.eqCtl.innerHTML + ' ' + operator + ' ';
            setEquation(eqText,this);
        },
        numberClick = function (e) {
            var button = (e.target) ? e.target : e.srcElement;
            if (operatorSet == true || 
                this.currNumberCtl.innerHTML == '0') {
                setVal('', this);
                operatorSet = false;
            }
            setVal(this.currNumberCtl.innerHTML + button.innerHTML, this);
            setEquation(this.eqCtl.innerHTML + button.innerHTML, this);
        },
        calculate = function (thisObj) {
            if (!operator || lastNumber == null) return;
            var displayedNumber = parseFloat(thisObj.currNumberCtl.innerHTML),
                newVal = 0;
            switch (operator) {
                case '+':
                    newVal = add(lastNumber, displayedNumber);
                    break;
                case '-':
                    newVal = subtract(lastNumber, displayedNumber);
                    break;
                case '*':
                    newVal = multiply(lastNumber, displayedNumber);
                    break;
                case '/':
                    newVal = divide(lastNumber, displayedNumber);
                    break;
            }
            setVal(newVal, thisObj);
            lastNumber = newVal;
        };
    return {
        numberClick: numberClick,
        setOperator: setOperator,
        clearNumbers: clearNumbers
    };
} ();

Note my comment: "I'm going to do something evil here." Here goes:
console.log('Hello, world')

Do you see what happened here? Let me put it in context.

        subtract = function (x, y) { return x - y; },
        multiply = function (x, y) { return x * y; },
        console.log('Hello, world')
        divide = function (x, y) {
            if (y == 0) {
                alert("Can't divide by 0");
            }
            return x / y;
        },

JavaScript semicolon insertion blew away the var when I inserted any valid statement or expression. In fact, I could have simply put 'Hello, world!' or 5 there, on its own line, and because the next line is a valid statement that stands on its own, JavaScript semicolon insertion blew away the var. As such, divide, setVal, setEquation, clearNumbers, setOperator, numberClick, and calculate were all just elevated to global scope, possibly blowing away existing variables and leaking a whole bunch of information with them. This could happen in any instance in which someone mistakenly types a semicolon (let's be honest - JavaScript is a semicolon-terminated language; it will happen somewhat frequently), or if they forget to put a comma at the end of a line.

As such, joining variable declarations together by using the comma operator is inherently an unsafe operation. You might think of it as a run-on sentence; it's not a good thing to do in English, so why would it be good to do in JavaScript or any other programming language?

And if that's not reason enough, here's another: declaring a variable is a statement. You are stating to the compiler, "I am declaring this variable to be a variable." Use the var statement to make a statement, and use the comma operator to indicate that there are operations. (Specifically, the one and only place I can think of in which a comma operator would be appropriate is if you need a single for-loop with multiple iterator variables, e.g., for (var i = 0, j = myArray.length - 1; i = 0; i++, j--). Of course, I don't want to say you should never use it, or else I'd be like Crockford with his dogmatic "I have never seen a piece of code that was not improved by refactoring it to remove the continue statement," which is patently silly.

But, beware the comma. He is correct in that it is easy to mark programming errors with commas. If you're going to declare a variable, do everyone a favor and declare it, using var, make it feel special by giving it its own line and declaration and semicolon. It will help in maintenance down the line.

2Jan/120

Facebook Security Concerns

Posted by Rob Paveza

I logged into Facebook today and I saw a new person under "People You May Know."  This person is someone I know from refereeing hockey here in Arizona.  It turns out that I only have one person on my friends list who I know happens to be an ice hockey referee, someone I know from not only refereeing but also playing hockey, more than 10 years ago.  When I clicked on Jeff's profile (the new person Facebook suggested), I saw he only had two friends - his is a very new profile!

Jeff and I haven't ever emailed back and forth directly.  I've sent a couple email blasts, and he probably has as well.  But that's the extent of it.

My best conclusion is that Jeff allowed the Facebook friend-finder application to have access to his email.  Because I'm on the same refereeing email distribution list as Jeff, I can only assume that Facebook has looked at his email and decided to inform me that I might know Jeff.

I do appreciate the flexibility of Facebook's find-a-friend tool.  But for it to be telling me that Jeff might be my friend based on data he provided seems to be a mild form of information leaking.

I'm only hoping that he provided his information to the Facebook friend finder tool.  I never did.  And if he didn't, well, now I'm concerned that Google and Facebook have been sharing that kind of information anyway...

6Aug/111

Joining the Jurassic Team

Posted by Rob Paveza

Paul Bartrum graciously just accepted me as a contributor to Jurassic, an open-source JavaScript interpreter written in C#.  I mentioned that I had recently discovered IronJS, but at the time at least, there was no object model – no way of cleanly interfacing .NET code with the script runtime, and frankly, I don’t know F# well enough to meaningfully contribute to that project.  (Please don’t hate on me, functional language junkies).

Jurassic is an exceptionally easy runtime to interface with.  You have to create a script engine, assign it some global variables or functions if you like, and then tell it to execute script.  Very, very easy.  In fact, it took me about an hour to drum up a fairly comprehensive script plugin for JinxBot, which is able to respond to all of BN#’s events (except the clan ones). 

I have several goals and some things I’ve already contributed to the Jurassic source:

  • If you’ve exposed a class to JavaScript using it so far, you’ve probably seen that it’s incredibly easy to expose a method to JavaScript – you decorate it with the [JSFunction] attribute.  Unfortunately, there hadn’t been an analogue for properties, until a commit a couple of days ago.  I added a [JSProperty] attribute, which exposes these properties to JavaScript as accessors.  They are enumerable by default (which means they appear in a for (var … in) loop), and may be opted-in to be configurable.  You can see this functionality in the latest pull, and see it in action in the ScriptEngineTests test class.
  • I also noted that Paul had enabled exposing CLR objects to JavaScript, at least in the source code repository.  In my eyes, this was a security issue, because it could expose the reflection system to the security sandbox.  To account for this, I added the ScriptEngine.EnableExposedClrTypes property, which is false by default.  Attempting to set a global value to a static or instance type while this is false will result in a JavaScriptException being thrown.

Some things that I’ve already started working on and would like to see accomplished:

  • Create a console-based REPL loop.
  • Create a standard library to allow for sandboxed lower-level I/O.  To account for this, I’m planning to add an import(package, [assembly = ‘Jurassic.Library’], [alias]) function to the global object.  For example, import(‘Jurassic.Library.Io’); would import the I/O library, which would enable construction of objects like new io.ByteStream()
  • Create events on ScriptEngine that would allow the engine’s host to intercept, handle, and debug scripting errors.  The biggest use-case I see here is, for example, the debugger statement.  The goal I have here is to be able to create your own debugger, instead of having to launch the system JScript debugger, especially since the JScript debugger is ignorant of the scope and other objects within Jurassic’s engine.
  • Another event I’d like to be able to add is something like EngineAppearsToCycle.  Someone made the point that you could go into the console and type while (true) console.log(‘’) and crash your browser.  The compiler could be refactored somewhat to enable this to be handled.

Eventually, I think it would be cool to refactor the compilation engine to function more like V8 – specifically with its metaclasses.  I’m not particularly adept at building a lexer or parser, but I think Paul has already done a great job, and the engine is still quite fast.  Consequently, this is more of a long-term because-I-can goal than we-need-to-do-it goal, because I honestly don’t think it needs to be done.

I’m excited to be a part of the Jurassic team.  Go check out the project – it’s incredibly easy to add JavaScript scripting to your .NET project!

Filed under: Uncategorized 1 Comment
28Mar/110

A Recent Discovery: IronJS

Posted by Rob Paveza

Since Microsoft first announced Managed JScript and shortly thereafter announced its demise (more on that here), I’ve been chomping at the bit for an implementation of JavaScript on the DLR.  Actually, I’ve probably been hoping for it for far longer, because JavaScript is near and dear to my heart.  If not for Brinkster offering free ASP-based web hosting back when I was just a young’un, I may never have discovered server-side programming, and may never have ended up where I am today. 

IronJS is a free, open-source implementation of ECMAScript v3 on the DLR based predominantly on F#.  It’s even the right kind of free, currently licensed under the Apache License version 2 (not a copyleft license).  Check out this picture of its testbed application running:

image

As a learning exercise, I intend to figure out a good way to shoehorn this into my Battle.net (1.0) chat client, JinxBot.  It actually shouldn’t be terribly difficult, but while I considered adding it as a plugin, I think I’d like it to be part of the core application instead. 

I’ll more than likely be covering IronJS in several parts in my blog in the coming weeks (possibly months, since I’m getting married on May 1).  But these are topics that I intend to cover:

  • Hosting IronJS in a C# application, loading, and executing JavaScript code
  • Sharing objects between script and .NET
  • Isolating script from .NET (preventing script from having run of the whole runtime)
  • Isolating script from script (running multiple environments in one application, similar to how web pages in browsers like IE and Chrome all run from one instance of the script engine but have different contexts so they have different sets of objects)
  • Performance optimizations
  • Dynamically generating isolation layers
  • Other considerations

Stay tuned!  In the meantime, check out IronJS on GitHub and get it running on your machine!

22Mar/111

Worries over the Future of the .NET Framework

Posted by Rob Paveza

Over the last few years, as exciting as some of the things have been that have come out that make a developer’s time more and more powerful, particularly in the .NET space – LINQ, dynamic programming, lambdas – I’ve become a bit concerned over the evolution of the .NET Framework’s type system.  It seems to me like some things that should be obvious but are not are being left aside.  This post will explore some of the programming issues that I have seen and am predicting.

Automatic Type Conversion for Delegates

Consider a Predicate<T>.  A Predicate<T> delegate accepts a single parameter of type T and returns a Boolean value, true or false, indicating whether that item matched some condition.

List<int> numbers = new List<int> { 1, 2, 5, -20, 10, -5, -3, 0 };
Predicate<int> isPositive = n => n > 0;
List<int> positives = numbers.FindAll(isPositive);


In the code here, I’ve used a lambda expression to indicate that the method generated for the lambda expression should be assignable to a type of Predicate<int>.  Fortunately, C# doesn’t allow you to assign a lambda to an implicit variable (a var), but consider that I could have also written:

List<int> positives = numbers.Where(n => n > 0).ToList();

Note how the lambda expression is exactly the same, and the two pieces of code would produce identical auto-generated methods, yet I couldn’t use numbers.Where(isPositive) because Where takes a Func<T, bool>, and not a Predicate<T>.

So what’s wrong with this?  Hopefully, you can see that it would be reasonable to say that a Func<T, bool> is a Predicate<T>, and that a Predicate<T> is a Func<T, bool>.  That is, delegates in this way have a dual-direction inheritance relationship.

It gets even more interesting when you bring Actions into this.  Actions are delegates that perform some work, accepting 0 or more parameters, but have no return value.  That is, they return void.  Because C# does not allow “Void” to be a type parameter, it is impossible to create an Action<Void>, or an Action<int, Void>.  But, technically speaking, you could have such a construction.

Think about what a delegate really does.  A delegate is the information the compiler needs in order to:

  1. Push all of the parameters onto the stack in the correct order.
  2. Let the callee (the method being called) do its work.
  3. The callee cleans up its own parameters from the stack.
  4. The callee pushes its return value onto the stack, if there is one.
  5. The calling method knows, based on the signature, whether there is a return value on the stack, so it knows the state of the stack.

An Action<T> is really just a specialization of Func<T, Void>.  The two ought to be compatible, and we should stop having so many delegate types polluting the framework and Base Class Library.

Making this Work without Modifying the CLR

Delegate types are defined by IL as types as well.  It may be possible to shoehorn this feature in without modifying the CLR by:

  • Add cast overloads for all delegates to implicitly cast to a Func type that supports its method signature.
  • Add cast overloads for all delegates to be implicitly cast-to from a Func type that supports its method signature.

If I could define something like this in C#, it might look like this:

public delegate bool Predicate<T>(T item)
{
    public static operator implicit Func<T, bool>(Predicate<T> original)
    {
        Func<T, bool> result = _item => original(_item);
        return result;
    }

    public static operator implicit Predicate<T>(Func<T, bool> original)
    {
        Predicate<T> result = _item => original(_item);
        return result;
    }
};

Note that this method uses closures, which would preserve all existing object relationships and not break any object relationship semantics normally required by delegate types.

Numeric Interface for All Primitive Types, Integer Interface for Integral Primitives

Primitive types are the basic structures supported by the CLR:

  • byte/sbyte
  • short/ushort
  • int/uint
  • long/ulong
  • float / double
  • char

String is also included, but that’s a class and heap-allocated, so it’s not really important for the purposes of this discussion.

There are a lot of common operations that you can do with these types.  You can always add them, subtract them, divide them (dividing a char is a weird thing to consider, but is possible).  Actually, if you want to know the position of a given letter in the alphabet, you can just subtract ‘A’ or ‘a’ from it. 

But when it comes to something like a generic type parameter, you can’t say something like:

public class NumberAggregator<T> 
    where T : INumeric
{
    private T _sum = default(T);
    private List<T> _numbers = new List<T>();
    public void Add(T item) 
    { 
        _numbers.Add(item);
        _sum += item;
    }
    // rest of the class
}

This is not possible.  First, there’s no such interface as INumeric, but more importantly, even though the intent of this class is to be able to sum any single size of number, the restriction can be supported by the underlying IL code.  The IL add instruction is a single instruction that does not take into regard the operands (see CIL Instruction Reference, ECMA-335 Partition III page 33). 

You can do a lot of cool things with numbers, particularly integers, that you can’t do with normal types.  You can add, subtract, divide, multiple, bitwise and/or/xor, shift, and invert (I think that covers all of them).  But the generic type system is too limited to allow you to do that.

Type Parameter Restrictions

A long while ago, I talked about the limitation of generics in that they may not support certain types as type parameter constraints, in that article specifically, enumeration types.  Enumeration types would give you similar benefits that I noted in the previous section, about the operations you can perform on numeric types that you generally can’t perform on other types.

While generics in .NET lack some of the power of C++ templates (namely, the ability to create a template that accepts a value, not just a type, as a parameter), they’re still quite good, and they solved a big performance problem in the 1.x CLR of boxing and unboxing primitive types.  However, I still believe that the CLR is limited in a way it does not need to be.  I don’t want to repeat myself, but this particular component is a big one – and a lot of the problems that issue causes could be solved with #2 above.

The .NET Library is Not Unified

Product owners who develop the core CLR need to come together and bridge the gap between incompatible interfaces.  The ISynchronizeInvoke interface was developed and part of the CLR version 1.0.  It was known as the standard way to allow cross-thread communication, most commonly for instances in which a background thread needed to post updates to the main UI thread of a Windows Forms application.

When WPF was released as part of .NET 3.0, it included a completely new model, called Dispatcher, that achieved the exact same purpose.  There was only one problem: Dispatcher doesn’t implement ISynchronizeInvoke, which means that any libraries written for previous versions of .NET, if they had code that needed to synchronize with a user interface, had an incompatible interface.

Writing an adapter for this is not an issue; making a component written for ISynchronizeInvoke work for Dispatcher is fairly trivial.  But this isn’t the only situation in which it’s happened; you have it happening with LINQ to SQL and Entity Framework (a travesty that LINQ to SQL is as closed as it is, because it could have been Entity Framework without all of the enterprisey things that EF has).  You have it within ASP.NET.  In WCF.  How about Silverlight being a reproduction of but completely incompatible implementation of WPF?

Final Thoughts

I still love programming on .NET.  It allows me to spin up applications so quickly and conveniently that I don’t imagine that I’d like to change any time soon.  But on the other hand, sometimes the decisions that the language and platform designers make baffle me. 

So – to my friends who got this far hoping that I’m making my announcement of switching to PHP, Java, or Python – sorry.  You’ll have to wait for another day.  But, hey, Mads, Anders, if we could get some of these changes in – that’d be great!

Filed under: Uncategorized 1 Comment
15Mar/110

The Microsoft Reactive Extensions

Posted by Rob Paveza

The honest truth is that I’m having difficulty establishing exactly what they could be used for, but they’re still really cool.  The Microsoft Reactive Extensions for the .NET Framework are the dual of LINQ: whereas LINQ operates over objects, or you might say pulls objects out of collections, the Reactive Extensions (Rx) handles push notifications.  It is the ultimate generalization of events and event handling within .NET.

Getting There

First, let’s consider the normal interfaces for IEnumerable:

interface IEnumerable<T>
{
    IEnumerator<T> GetEnumerator();
}

interface IEnumerator<T> : IDisposable
{
    T Current { get; }  // throws exception at end of enumeration
    bool MoveNext();
}

These interfaces (okay, really, the non-generic IEnumerable interface, but let’s not split hairs) are the foundation of the foreach C# keyword (and the For Each… In in Visual Basic).  A foreach can also be written, roughly, as:

foreach (string str in myListOfStrings)
    Console.WriteLine(str);
// rewritten:
using (IEnumerator<string> enumStr = myListOfStrings.GetEnumerator())
{
    while (enumStr.MoveNext())
    {
        Console.WriteLine(enumStr.Current);
    }
}

Keep this example in mind for later, because we’ll revisit how this can be used in Rx programming.

Dualism

Dualism is something of a mathematical concept, and I don’t want to get into it because I don’t completely understand it myself, but most nerdy people reading my blog will probably appreciate an example from particle physics.  Consider a proton: its physical dual is the antiproton (because when they meet they annhilate each other.  It’s not an electron, because while they have opposite charge, they have substantially different mass).

The core of Rx is the dual of IEnumerable.  That is, IObservable<T> and IObserver<T>.  But let’s deconstruct these piece by piece.  Let’s start at IEnumerator<T>:

interface IObserver<T>
{
    // T Current { get; }
    // That method looks like: T get_Current();
    void OnNext(T next);
    // Current throws an exception if MoveNext() previously returned false, so:
    void OnError(Exception error);

    // bool MoveNext() 
    // returns true while Current is populated, false when we reach the end, so:
    void OnDone();
}

You can see that, whereas everything in IEnumerator<T> pulled data, now we’ve transitioned into pushing data.  But the observer isn’t really the cool part; rather, it’s the subject that’s cool:

interface IObservable<T>
{
    // GetEnumerator() returned an object; here we pass one in
    // We still needed to capture the disposable functionality, so we return IDisposable
    IDisposable Subscribe(IObserver<T> observer);
}

Now, if you want to see the specifics about how these were constructed, you can check out the Expert-to-Expert video on Channel 9.  I’ve included some high-level notes, but they’re not really as deep as you can get with these guys.

Creating a Subject

Creating a subject is a bit of a challenge; subjects are event-driven, and those are generally kind of difficult to think about because the fit usually only into one of two buckets: user interaction and system I/O.  For sake of example, I’ve created a simple Windows Forms project to start with, that has a couple observable Buttons (the class is called ObservableButton, go figure), and an observer, which is the containing form.  You can download the starter project, which requires Visual Studio 2010 and the Rx Framework.

Subjects can be anything, though, and the power you can glean from these is amazing.  For the Red Bull NASCAR team, I created a server for a Twitter feed aggregator using Rx.  It started as reading a socket into HTTP data, then into chunked HTTP data, then into JSON packets, then into POCO objects that were then re-serialized and sent over the wire to N Flash clients.  As you can imagine, network programming, social programming, or any other kind of programming where an event is coming in unpredictably is a great candidate for this.  Why?

Let’s look at the use case I just listed.  As Twitter’s live stream service sends data over the wire, I need to parse it and send it to a lot of listening sockets.  But I don’t want to just say “Oh I just got the data, let me send it out again” – that would possibly slow down processing on other threads, because I might have to wait – my socket might already be in the process of sending data and so it’s in an invalid state to send further data.  If I had tied a server socket directly to the “I’m ready to send” signal directly, I would have been in trouble.  Rather, I had a utility (an Observer) that aggregated incoming messages until all server sockets were ready to send, at which point it would push those updated messages to the server sockets.

Let’s look at the sample program:

image

This isn’t really anything spectacular.  I could have done that with regular event handlers.

Aggregating Subjects

The magic of Rx, from my perspective, lies with what you can do with subjects.  I’m no longer initializing my constructor to require two lines – I’m merging the two buttons into one observable sequence:

        public Form1()
        {
            InitializeComponent();

            observableButton1.Merge(observableButton2).Subscribe(this);
        }

 

The result is identical – the events get handled and all is good.

Modifying Sequences

Now I’m going to change the class definition slightly:

    public partial class Form1 : Form, IObserver<Timestamped<string>>
    {
        public Form1()
        {
            InitializeComponent();

            observableButton1.Merge(observableButton2).Timestamp().Subscribe(this);
        }

        public void OnNext(Timestamped<string> value)
        {
            this.textBox1.Text += value.Timestamp.ToString("hh:mm tt   ") + value.Value + Environment.NewLine;
        }

        public void OnError(Exception error)
        {
            this.textBox1.Text += "Exception caught: " + Environment.NewLine + error.ToString() + Environment.NewLine;
        }

        public void OnCompleted()
        {
            this.textBox1.Text += "Sequence completed." + Environment.NewLine;
        }
    }

Note that by adding in the .Timestamp() call, I’ve transformed the observable to sequence of strings to be an observable sequence of timestamped strings.  That’s pretty cool, right?

This is even cooler: the Delay() method:

observableButton1.Merge(observableButton2).Timestamp()
                .Delay(new TimeSpan(0, 0, 1)).ObserveOn(this).Subscribe(this);

The ObserveOn method accepts a Windows Forms control, a Dispatcher (for WPF), or other scheduler implementation that can be used to synchronize the delay.  If I didn’t include it, the delayed merge would be called on a different thread, and we’d get an InvalidOperationException (because you can’t update a window on a thread other than the thread that created it). 

Do you want to avoid repetition?

            observableButton1.Merge(observableButton2).Timestamp()
                .DistinctUntilChanged(ts => ts.Value).Subscribe(this);

This produced output that only emitted one message, no matter how many times I clicked the same button, until I clicked the other button.

So, What Can We Do?

Well, right now it doesn’t seem like there’s a lot of tooling for Rx.  There’s a community wiki around the framework, though, and I think that we can eventually see a lot of good use.

Some ideas:

  • Develop a way to completely repeat ASP.NET requests.  Treat IIS as an IObservable<AspNetRequest>, where AspNetRequest contains all the state data that would otherwise populate these tools, which would immensely help with debugging.  Imagine when your tester only needs to record a series of test cases once, and otherwise is just testing for UI errors.
  • Wrap event-oriented APIs for simplified logging and replaying.  (In JinxBot, an event-oriented chat API named for my cat, I always wanted to capture all the events of the core API and be able to replay them via a subclass, which would have allowed pixel-perfect replay of a chat session).
  • Handle periodic data services like Twitter, SMS, email, or others in a clean and efficient way.

I’d like to see this take off, but it’s a very different way of looking at programming than what most .NET developers are used to.  Enjoy it, take a look, and let’s build it up!