Running with Code Like with scissors, only more dangerous


In a disconnected world, robust code is crucial

Posted by Rob Paveza

Probably 99.99% of HTML applications and websites are served over HTTP exclusively. (I'm referring here to HTTP as a transport protocol, not HTTP vs.HTTPS, for example, and I realize that HTTP is an application-layer protocol according to OSI; but developers generally treat it as an abstraction for "the network"). As anybody who has done web programming knows, HTTP is a stateless protocol; that is, it's based on a request-response model, and in general, one request has no knowledge of previous requests. This has posed some challenges for web developers over the years, and some brilliant abstractions of state on top of the statelessness have been devised.

The hard part now, though, isn't to deal with statelessness. It's dealing with the request-and-response model.

All network communication is inherently request-and-response. There are some applications that utilize full-duplex communications to get around that (think of chat software), but for the most part, that isn't really available behind the firewall. Web sockets are still yet to be standardized (and there are some questions about long-term compatibility with WebSocket-ignorant proxies). And typically, corporate firewalls say no to outbound connections except on ports 80 or 443. Some applications (think Meebo) have been able to get around this limitation by cleverly using long-timeout delays on AJAX requests. The client makes a request to the server, and the server either responds immediately (if an event is in queue) or holds the request for 30-90 seconds to see if an event comes in. I even did this once myself with good success, although I never took that app into production. (There was also some question about the total # of clients an ASP.NET server could sustain whilst holding threads in that way).

In many respects, Windows developers haven't had to deal with this. We could issue synchronous requests, and the UI would stand still for a second, and either it would work or it would fail. But usability concerns over this process, as well as issues with high network latency (imagine pressing the "Submit" button and having to wait 20 seconds while your app freezes - by then, I've force-closed the app) have seen platform providers decree that asynchrony is the only way to go.

HTML isn't the only application provider dealing with this limitation. Adobe Flash has long had an asynchronous-communication-only model, Microsoft Silverlight has also carried on this principle; of course, these two applications have lived predominantly in browsers, where a hanging UI probably means interfering with other apps as well as the one making the request. Interestingly, WinRT - the Windows 8 developer framework - is also going to mandate an asynchronous model, following in the Silverlight-based foodsteps blazed by Windows Phone 7.

So as we trek out into the world of asynchrony, well, we have a whole mess of questions to deal with now:

  • If there's an error, does it show up in the calling method or in the callback method? Does it even show up?
  • Does a network (transport-level) error surface differently than an application error? What if the server returned an HTTP 403 Forbidden response?
  • What are all of the different kinds of errors that can crop up? Do I need to handle SocketException or is that going to be abstracted to something more meaningful to my application?
  • What do I do if a network error comes up? Do I assume that I'm offline, panic, and quit? What if my application only makes sense "online"?
  • Do I surface an error to the customer? Silently fail? I might generally fail silently if I'm a background process, but then again, what if it's an important one? What if the customer thought he was saving his draft while all along it was offline, and then the customer closes the browser?
  • During the async operation, should I show the user a timeout spinner or something to that effect?
  • How should I design my async operations? For example, consider a Save operation. Should I capture all of my state at once and send it off, and let the user immediately keep working? Should I make the user wait until saving completes? Should I even use Save, or automatically save whenever something changes?
  • If I use auto-save, how do I handle undo? What if I want to undo between sessions? Is there a way to go back if the hosting application crashes? (Worst case scenario: the user accidentally hit Select All, Delete and then the browser crashed after the auto-save).

This merely scratches the surface of the kinds of questions we'll need to begin asking ourselves. It doesn't even deal with the difficulties of programming asynchronously, which C# 5 is going to deal with extraordinarily, but many developers will be unable to take advantage of these updates. For example, suppose I have a widget on my page that monitors the status of a long-running server-based process. I need to have JavaScript on my page that monitors that process and updates my widget accordingly. Should I:

  • Write a singleton object? This might be easier and afford strong member protection, but I can only have one widget, unless I somehow differentiate between them and multiplex, which can become hairy quickly.
  • Should the monitoring function accept a callback, or should it be event-based, so that multiple subscribers can listen? (Maybe an event-based model offers some interesting ways to deal with the complexities of a singleton?)
  • Should the widget manipulate the view directly, or should I write separate code that handles the view based on the state of the object (or objects)?

The list goes on.

We're moving faster and faster into an asychronous world. It is already happening, and we as developers need to be prepared to handle these difficulties. We also need to understand how to communicate these kinds of questions to our business analysts, our supervisors, and our customers. We need to be able to equip ourselves to ask the right questions of our customers, so that when it's time to make a decision, we have the information we need.


Model-View-Controller and Assembly Dependencies

Posted by Rob

The title of this blog post might be a slight misnomer because it isn't dealing with MVC directly.  It's dealing with something a little more high level (oddly enough).

I'm working on a personal project when I have time.  I haven't played around with in a while and there's still a fairly active user community around it (not to mention that Starcraft 2 is going to be released one day).  On top of everything else, I learned so much about all kinds of topics (design patterns, reflection, UI design) when I was working on clients in college that I still think it's valuable.  My latest project is giving me the opportunity to leverage some new additions to .NET 3.5 -- the System.AddIns namespace -- which will be a first for me, and it's the first time I'm using Model-View-Controller to design the client.  The primary motivator in using MVC is that I want to also build up an AJAX-powered rich web interface, and so the use of MVC allows me to incorporate multiple controller listeners to the model's events, to which I can hook up a web service.  Finally, I want to just make clean software.  MVC seems to be a particularly good pattern for this project, so I'm going to run with it.

In addition to the fact that it makes sense to separate out the model from the view, I've run into a little snag.  It's taken me three days to work out the problem in my head, but I think I'm finally at a point where I can be satisfied with the solution.

One of the great things about .NET is its rich extensibility model exposed via attributes.  By allowing developers to create their own metadata to be used in assemblies, we can create contracts in such a way that the extension code can tell us about itself.  Now, of course, we can create contracts in C++, too, but arguably it's not as clean.  Where a plugin system for a C++ library might require the code to provide us with a naked function, one for C# or another .NET language might be exposed via an assembly-level attribute:

// C++
__declspec(dllexport) extern "C" PPLUGIN_FACTORY GetPluginFactory();

// C#
[assembly: Plugin(typeof(MyPluginFactory))]

These are, of course, very similar, and they're used in a similar way.  The .NET app will know to look for that PluginAttribute on the assembly, and the C++ loader will know to look for an exported function called GetPluginFactory.  But I digress.

Among other things, I'd like in my app that the model classes be decorated with View-defined attributes.  For example, I plan on incorporating some of the features from Shiny Design into my app, and one of the things I'd like to do is create an adapter class to create a proxy for events.  Essentially, it's annoying to me to have to generate code that handles the configuration of events; specifically, each event type can have a color associated with it.  I'd decorate my event named "UserJoined" with [Name("User Joined Channel")] and [DefaultValue("Yellow", typeof(ColorConverter))].  It should know enough to enumerate the events, pull out the special data, and save to XML.  Then at runtime, access them via a special dictionary.

My dilemma was what to do about the dependencies.  Ultimately all of this will be open-source; I don't want there to be dependencies between my view classes and my model classes.  But how could I release my model classes without any dependency on my view classes if I wanted to have these attributes?

Then, tonight I realized: conditional compilation constants.

   2:  [Name("User Joined Channel")]
   3:  [DefaultValue("Yellow", typeof(ColorConverter))]
   4:  #endif
   5:  public event UserEventHandler UserJoined;



Thoughts on Generics and Best Practices – Building Cohesive Software

Posted by Rob

In general, I don't know whether I internalized this as a best practice because it makes sense to me, or because I read it somewhere.  But I have this notion that there are only certain places in which it is appropriate to expose a generic member to the outside world, such as when the item is clearly generic in scope.  For example, if I wanted to implement a FIFO list to get the behavior of a queue but the performance characteristics of a list, I might call my class ListQueue<T>.  As a library class, it makes sense to have this as a generic type.  But what about when it would save programmer effort only?  I don't think that this is a valid reason to expose the class to the outside world.

Between my last two projects, I have about a dozen classes that look like this:

   1:      [ConfigurationCollection(typeof(SerialPortConfigurationElement), AddItemName = "SerialPort", CollectionType = ConfigurationElementCollectionType.BasicMap)]
   2:      public class SerialPortConfigurationElementCollection : ConfigurationElementCollection, IEnumerable<SerialPortConfigurationElement>
   3:      {
   4:          protected override ConfigurationElement CreateNewElement()
   5:          {
   6:              return new SerialPortConfigurationElement();
   7:          }
   9:          protected override object GetElementKey(ConfigurationElement element)
  10:          {
  11:              return ((SerialPortConfigurationElement)element).PortName; 
  12:          }
  14:          public override ConfigurationElementCollectionType CollectionType
  15:          {
  16:              get
  17:              {
  18:                  return ConfigurationElementCollectionType.BasicMap;
  19:              }
  20:          }
  22:          public SerialPortConfigurationElement this[int index]
  23:          {
  24:              get { return BaseGet(index) as SerialPortConfigurationElement; }
  25:          }
  27:          public SerialPortConfigurationElement this[string index]
  28:          {
  29:              get { return BaseGet(index) as SerialPortConfigurationElement; }
  30:          }
  32:          #region IEnumerable<SerialPortConfigurationElement> Members
  34:          public new IEnumerator<SerialPortConfigurationElement> GetEnumerator()
  35:          {
  36:              return new LightWrapperEnumerator<SerialPortConfigurationElement>(base.GetEnumerator());
  37:          }
  39:          #endregion
  40:      }

This class enables the following type of configuration settings:

   1:      <SerialPorts>
   2:          <SerialPort PortName="COM6" BaudRate="9600" DataBits="8" Parity="None" StopBits="One" />
   3:          <SerialPort PortName="COM8" /> <!-- Only the PortName is required.  The others have defaults. -->
   4:      </SerialPorts>

This is all well and good.  However, as I look at my class, I see that it would be fairly easy to rip out a lot of that functionality and make it a generic class.  LightWrapperEnumerator<T> is already something like that - it's an internal-only class that basically wraps an IEnumerator into an IEnumerator<T>.  That's fairly easy to hide from the outside world, though; I only expose it to calling code using IEnumerator<T> and nobody is the wiser unless they call GetType() on it, which is unlikely.  I realized as I built the second duplicate of this class today involving a Motor configuration element, that I could rip out the guts and implement everything EXCEPT GetElementKey and have it out there:

   1:  abstract class ChildlessConfigurationElementCollection<T> : ConfigurationElementCollection 
   2:      where T : ConfigurationElement, new()
   3:  {
   4:      protected override ConfigurationElement CreateNewElement()
   5:      {
   6:          return new T();
   7:      }
   9:      public override ConfigurationElementCollectionType CollectionType
  10:      {
  11:          get
  12:          {
  13:              return ConfigurationElementCollectionType.BasicMap;
  14:          }
  15:      }
  17:      public T this[int index]
  18:      {
  19:          get { return BaseGet(index) as T; }
  20:      }
  22:      public T this[string index]
  23:      {
  24:          get { return BaseGet(index) as T; }
  25:      }
  27:      #region IEnumerable<SerialPortConfigurationElement> Members
  29:      public new IEnumerator<T> GetEnumerator()
  30:      {
  31:          return new LightWrapperEnumerator<T>(base.GetEnumerator());
  32:      }
  34:      #endregion
  35:  }

Implementing the serial port collection then, would be VERY easy:

   1:  [ConfigurationCollection(typeof(SerialPortConfigurationElement), AddItemName = "SerialPort", CollectionType = ConfigurationElementCollectionType.BasicMap)]
   2:  class SerialPortConfigurationElementCollection : ChildlessConfigurationElementCollection<SerialPortConfigurationElement>
   3:  {
   4:      protected override object GetElementKey(ConfigurationElement element)
   5:      {
   6:          return ((SerialPortConfigurationElement)element).PortName;
   7:      }
   8:  }

However, aside from the fact that they have the same "programmer functionality," there's not much motivating me to do anything else.

Hopefully, as you're looking at these two examples, you're saying "that doesn't look correct."  From an object model perspective, it doesn't make much sense, even in the abstract world (away from objects), to have even a concept of a "Childless Configuration Element Collection."  What does it mean, anyway?  Would it make sense for me to cast a configuration property object to a ChildlessConfigurationElementCollection of T?  I... don't think so.

I've been reading Code Complete 2nd Ed. lately.  I can't remember whether I've blogged about my respect for Steve McConnell before, but this guy is simply an amazing contributor to our craft.  In it, he talks at length about a software quality attribute that we'd never really discussed in school: cohesion.

The Wikipedia article gives it a great treatment, so I won't really delve into the depths of what cohesion means and how it can be measured (buy the book!).  As I look at this type of code, though, I can only see some kind of cross between procedural and logical cohesion - not exactly excellent in the scale.  The real bottom line is that these all operate on different data - how are we to treat them the same?

Incidentally, the way that C++ implements templates (not C++/CLI generics) would make this not be so terrible.  I could define this template class and it would actually generate the C++ code (Java actually would do this in an adequate way, as well).  Because the code is generated at compile-time, I don't need to expose the metadata about the template class, and so I'm saved in that regard.  Unfortunately, if I were to define this class in C# and need to access it in an external assembly, I'd need to mark the base class as public, and so everything else in the world can inherit from or create references to variables of type ChildlessConfigurationElementCollection<T>.  It doesn't make sense to have this type exposed; it only makes sense at the higher level. 

So I am left with the decision: do I duplicate code or expose bad, incomplete objects to the world?  I'll duplicate code, thank you very much.


Exceptional Exception Handling: Taking Exception with the Community

Posted by Rob

I could have gone on with the title for a while longer.

Frequently when I'm working on an application, it occurs to me that the user doesn't care about the technical reasons behind an exceptional condition.  Typically, the user simply cares that something went wrong; or, sometimes, the user doesn't care, especially if you can make the program recover gracefully.

Since the only .NET-sanctioned way to handle errors is through structured exception handling, and since I love SEH, I don't want to be too hard on it.  But the truth is, there are certainly instances where "On Error Resume Next" would be nice.

Consider the following constructor:

   1:          public ConnectionBase(string server, int port)
   2:          {
   3:              m_server = server;
   4:              m_port = port;
   5:              try
   6:              {
   7:                  m_ipep = new IPEndPoint(Dns.GetHostEntry(server).AddressList[0], port);
   8:              }
   9:              catch (SocketException) { }
  10:          }

This class essentially encapsulates a TcpClient.  Now, what I do here in the constructor is attempt to resolve the host address.  In the event that there is an error, for whatever reason, Dns.GetHostEntry will raise a SocketException, typically telling me that the URI could not be found.

Now, the general wisdom about SEH is to not just "eat" an exception; rather, we prefer to do something worthwhile with it.  But in this case, I haven't invalidated the state of my class, I haven't invalidated the state of the system, or done anything to harm my class.  In fact, if the m_ipep (an IPEndPoint object) isn't initialized when Connect() is called, it attempts to do it as well:

   1:          public virtual bool Connect()
   2:          {
   3:              // ...
   4:              if (m_ipep == null)
   5:              {
   6:                  try
   7:                  {
   8:                      m_ipep = new IPEndPoint(Dns.GetHostEntry(m_server).AddressList[0], m_port);
   9:                  }
  10:                  catch (SocketException se)
  11:                  {
  12:                      OnError(string.Format("Your computer was unable to resolve hostname {0}.  If necessary, add an entry to %SystemRoot%\\system32\\drivers\\etc\\hosts, or flush your DNS resolver cache, and try again.",
  13:                          m_server), se);
  14:                      return false;
  15:                  }
  16:              }
  17:              //...
  18:              return true;
  19:          }

This method correctly bubbles an error to the using classes in the event that the IP end point could not be located.  You might ask, though - what is the advantage of using early resolution?  That is, I could drop the exception code altogether.

Well, if someone is debugging their code using my library and has first-chance exception handling enabled, it is easy enough to spot where a parameter might be causing issues.

Once the class is initialized, someone using a debugger can see the internals of my class, including the resolved end point.  This may help them identify the remote end point's IP address; perhaps that particular server is not functional.

There are other issues with the "best practices" as well -- for instance, they are sometimes at odds with each other.  The .NET Framework Guidelines say not to overuse catch, but also to not to just catch a general exception.  Consider:

   1:              try
   2:              {
   3:                  // create an instance of this variable to make sure that the data provider can be loaded.
   4:                  DataProvider provider = DataProvider.Instance;
   5:              }
   6:              catch (TypeLoadException tle)
   7:              {
   8:                  throw new ConfigurationErrorsException(
   9:                      string.Format("Type '{0}' was not a valid type according to the runtime.", keyProviderType), tle);
  10:              }
  11:              catch (TargetInvocationException tie)
  12:              {
  13:                  throw new ConfigurationErrorsException(
  14:                      string.Format("An exception was raised in the constructor of type '{0}'.  More information may be available in the contained exception.", keyProviderType), tie.InnerException);
  15:              }
  16:              catch (MethodAccessException mae)
  17:              {
  18:                  throw new ConfigurationErrorsException(
  19:                      string.Format("Security policy prevented access to the default public constructor of type '{0}'.", keyProviderType), mae);
  20:              }
  21:              catch (MissingMethodException mme)
  22:              {
  23:                  throw new ConfigurationErrorsException(
  24:                      string.Format("No default public constructor existed on the type '{0}'.  Change the application configuration so that the 'dataProvider' property of the 'dataConfiguration' section points to a type with a default public constructor and inherits from the '{1}' type.", keyProviderType, typeof(KeyProvider)), mme);
  25:              }
  26:              catch (MemberAccessException memae)
  27:              {
  28:                  throw new ConfigurationErrorsException(
  29:                      string.Format("Could not create an instance of the abstract type '{0}'.  Change the application configuration so that the 'dataProvider' property of the 'dataConfiguration' section does not indicate an abstract type.", keyProviderType), memae);
  30:              }

In the code above, I have legitimate reasons for catching these - retrieving the DataProvider.Instance property is documented to raise these exceptions, which are in turn raised by Activator.CreateInstance.  The real problem with this kind of situation is that the outer calling code needs to be shielded, and so ultimately, the calling code handles only a ConfigurationErrorsException.  This in turn is handled and logged to the event log before terminating the Windows Service that hosts this code.

You can read more in the .NET Developer's Guide - Exception Handling article on MSDN.  Just bear in mind - you probably want different exception handling strategies if you're writing a user interface vs. library code vs. data access code.  At the end of the day, it's how much bad stuff you want to show your users, how much bad stuff you can handle internally to your application, and how much bad stuff you're willing to let your application produce because you over-caught exceptions.


A Reminder of Why We Like Object Orientation

Posted by Rob

I love Simple Machines Forum.  It's a fantastic community software package, it's open-source, it's pretty fast, free, and it's got boatloads of features.  It's so great, in fact, that I'm basing one of my personal projects on it.

SMF is written in PHP (I'm running into too many acronyms already - this is, after all, a post about OOP).  Being that PHP is a scripting language, and among other things strong with string processing, it seems pretty natural that it performs the parsing of UBBC (Universal Bulletin Board Code) itself, without moving out to a library function or something like that.  (I guess that's a superfluous statement anyway since there isn't a library function). 

BBCode is some of the coolest stuff around - good way to prevent worrying about things like script attacks and other evil things people can do to other people in a community environment.  It looks something like this:

[b][i]This is important[/i][/b] - [url=]Check out my blog[/url] and my latest info.

If I look at what SMF produces, I'll see the following HTML:

<b><i>This is important</i></b> - <a href="" target="_blank">Check out my blog</a> and my latest info.

Anyway, I've been trying to come up with an interesting way to implement this in C#.  My latest idea was to implement a stack tracking each BBCode tag and an implementation that allowed the combination of tags.  For instance, I'd like the [b] tag to render as a <span> with the CSS font-weight: bold; value.  If I have an [i] tag nested within it, I'd like the single <span> tag to have both of the CSS values. 

I was thinking that this might be kind of an annoying to do this; it would require a lot of classes - one for each tag - and while I thought it might be cool to implement this way for other reasons (configuration files, for example) - I was somewhat concerned.

I was also interested in taking a look at how the PHP works - particularly, I'd like to see the parameters and things like that.  My first guess was that it was in Sources/subs.php, and I was right.  They were kind enough to include a list of subroutines in the file in comments at the top - and sure enough, I found parse_bbc!

So I copied the function out into another file - it turned out the single function is 50kb.  One function.

Looking further into the function, I found an 81-line comment block explaining, basically, how they used associative arrays to create a series of objects.  Here's an example:

$codes = array(
        'tag' => 'abbr',
        'type' => 'unparsed_equals',
        'before' => '<abbr title="$1">',
        'after' => '</abbr>',
        'quoted' => 'optional',
        'disabled_after' => ' ($1)',
        'tag' => 'acronym',
        'type' => 'unparsed_equals',
        'before' => '<acronym title="$1">',
        'after' => '</acronym>',
        'quoted' => 'optional',
        'disabled_after' => ' ($1)',
        'tag' => 'anchor',
        'type' => 'unparsed_equals',
        'test' => '[#]?([A-Za-z][A-Za-z0-9_\-]*)\]',
        'before' => '<span id="post_$1" />',
        'after' => '',
        'tag' => 'b',
        'before' => '<b>',
        'after' => '</b>',

As you can see, each "object" has a series of properties.  Each object also has a type (lack of the "type" attribute would probably indicate a type as well).  But I could do it easily enough with C#, maybe even using object initializers (item = new UnparsedEqualsBBCode() { Tag = "abbr", Before="<abbr title=\"$1\">", After="</abbr>", Quoted = QuoteRequirements.Optional, DisabledAfter=" ($1)" };).  Undoubtedly there are other ways to do this too. 

It's just scary.  Looking through the function tonight has reminded me why I like my object-oriented programming and type safety.  Adding items to this - or changing it - seems like a maintenance nightmare.