Real-Time Web Apps Made Easy with WebSockets in .NET 4.5

In the world of browser-based development, interoperability is king. Unfortunately, interoperability can be at the expense of performance.

With support for bi-directional, full-duplex messaging simply out of the reach of the HTTP protocol, real time messaging support in browser-based applications can be severely limited. Fortunately, standards groups including the W3C and IETF have been hard at work over the last few years on a standard specification called the WebSocket protocol which aims to bring these much needed capabilities to the masses.

As with most industry standards, the WebSocket protocol has seen a significant amount of volatility over the last couple of years, but the good news is that the dust has settled making this a great time to start considering how WebSockets fits into your solution architectures.

Why WebSockets?

Before I explain what the WebSocket protocol is and how it works, let me talk about some of the challenges that it aims to solve.

Today's applications demand real-time, low-latency messaging. Whether you are building a web, mobile or composite application, users expect to be able to interact with data as close to real-time as possible while minimizing the impact to their overall user experience.

The key to enabling real-time, immersive experiences is to connect as closely as possible to the source of the events of interest so that when an event takes place, your application, service, or user experience is notified as quickly as possible.

This isn't that big of a problem for applications and services within the enterprise that can leverage socket-based messaging like TCP or UDP, but more and more applications are being built for the web and devices that rely entirely on the Internet for connectivity. Even enterprise applications today are becoming more and more dependent on services hosted by external vendors/partners and commercial cloud providers bringing hybrid solutions into the mainstream.

There are many applications you likely interact with today that require this type of connectivity and yet deliver real-time or near-real-time user experiences. If you've interacted with Twitter or Facebook, you've experienced near-real-time streams of activity constantly being updated all over the web, on your mobile device or browser without needing to hit a refresh button.

Periodic Polling via XHR

One common approach for delivering near-real-time messaging is commonly known as AJAX or XML Http Request (XHR). This works by polling an endpoint at a given interval and returning data when it is available. Since there are no page refreshes or post backs happening, this gives the user the illusion of a dedicated connection, but the reality is that XHR is both latent and consumes resources and bandwidth with each subsequent poll.

Long Polling

An alternative to XHR is long polling. With long polling, the server/endpoint holds on to the HTTP request until there is data/payload available and then returns to the client. The client then follows up with another request and waits until new data or event/payload is available.

While both of these approaches have powered a vast number of innovative solutions, they aren't without liabilities.

In addition to the high latency of both models, the programming model for each isn't the most intuitive. If you've worked with AJAX/XHR or long polling, you may have gotten the feeling that there's a bit of “magic” happening under the hood because there is. In addition, scalability can become a problem as the number of clients increase. In addition, consuming endpoints that reside outside of application's domain requires clever hacks like JSONP or the use of non-interoperable adapters.

The WebSocket Protocol

The WebSocket protocol addresses many if not most of these issues by enabling fast, secure, two-way communication between a client and a remote host without relying on multiple HTTP connections. WebSockets support full-duplex, bi-directional messaging, which is great for real-time, low-latency messaging scenarios.

Best of all, WebSockets is fully interoperable and cross-platform at the browser level, natively supporting ports 80/443 as well as cross-domain connections.

There are two parts to the WebSocket protocol: client and server. The W3C owns the JavaScript client API and the IETF Hypertext Bidirectional (HyBi) Working Group is responsible for guidelines governing how the server-side protocol should be implemented.

Browser vendors like Google, Microsoft and Mozilla natively support the WebSocket protocol by implementing the client API directly in their browsers. This means that if Chrome or IE support WebSockets, the API is native to the browser and you can start programming against WebSocket endpoints right away.

On the server side, platform vendors implement the IETF specification providing middleware tooling that enables you to expose your back-end services over the WebSocket protocol.

How it Works

As shown in Figure 3, when a WebSocket-enabled browser/client establishes a connection to a WebSocket endpoint, it conducts a handshake over port 80/443 and requests an upgrade to the WebSocket protocol. If the server supports the WebSocket protocol, **and **the protocol version of the client and server match, the web server will accept the upgrade request and upgrade the request. From this point on, the client and server have a direct, socket connection and can freely exchange messages.




Let's take a look at what the HTTP request and response from step 2 above looks like:

GET ws://localhost/TweetStreamService/TwitterStreamService.svc
Connection:Upgrade
Host:localhost:80
Origin:null
Sec-WebSocket-Key:GleUPijQdfEAxSyUGqNugw==
Sec-WebSocket-Version:13
Upgrade:websocket

The first thing you'll notice is that this is just an HTTP GET request. However, the URI is using a protocol scheme of ws. All WebSocket endpoints must be addressed in this manner. Everything after the prefix is that of a typical URI, including the server and resource path. In this case, I'm addressing a WCF service that implements the WebSocket protocol, but this is just one implementation option we'll explore along with others.

The Origin request header is optional and can be used to demarcate the domain in which the communication will take place. Using this header, the server can decide to only serve requests originating from a given domain and reject all others. It is in this way that cross-domain connections are implicitly supported but can be constrained as needed.

Next, notice that the Connection request header is set to Upgrade. This is to indicate that the client is requesting an upgrade to the WebSocket protocol, specified in the Upgrade header if the server supports it. The Sec-WebSocket-Key header consists of a hash which prevents an intermediary from impersonating the server (more on this in a bit).

Finally, the Sec-WebSocket-Version header indicates the protocol version which the client supports, which maps to the hybi server reference, in this case IETF RFC 6455.

If the server supports WebSockets and the versions are compatible, the server will respond as follows:

HTTP/1.1 101 Switching Protocols
Connection:Upgrade
Sec-WebSocket-Accept:tR1jreg0cIPCg0/jekUcTubHV5M=
Upgrade:websocket

Notice the first line in the response along with the Connection and Upgrade response headers. This indicates that the HTTP request has been upgraded to WebSockets and from this point forward, the client and server can exchange messages in a fully duplex bi-directional manner!

As I mentioned, the Sec-WebSocket-Key request header is used to prevent malicious scripts from fooling a WebSocket server into accepting non-WebSocket payloads. The way this works is that the server takes the value of the Sec-WebSocket-Key and concatenates it with a key, computes an SHA1 hash and returns that value to the client in the Sec-WebSocket-Accept response header. As a result, only the original WebSocket server that accepted the upgrade request can communicate with the client.

WebSockets Today

It is very important that the client version and server match (Sec-WebSocket-Version), and this wasn't always easy to guarantee in the early days when the protocol was being standardized. Imagine writing and shipping an application written for Google Chrome only to find that a month later, the client API was revised. If the server vendor didn't update their stack to the new specification in a timely manner, or the browser vendor lagged in updating their API behind the IETF standard, you could be stuck in Sec-WebSocket-Version version hell.

Fortunately, most of the standardization dust has settled and both the client and server standards are much more stable so you are unlikely to encounter the same pitfalls early WebSocket pioneers encountered. Even better, some server implementations like Socket.IO and WebSocket.IO are kind enough to support older client API versions and silently bridge the versions for you.

The latest version of the IETF server standard is RFC 6455. This is expected to be the final standard. On the client side, the latest W3C specification is Editor's Draft 26 June 2012. You can find links to both of these specifications in the sidebars.

IETF RFC 6455 requires that the value of the Sec-WebSocket-Version is set to 13. A W3C browser that is compatible with RFC 6455 will automatically include this version number. Although the W3C specification continues to be refined, many browser vendors and server stack providers have invested in RFC 6455.

Microsoft has invested significantly in the WebSocket protocol, both by contributing time and talent to the standards bodies and including support for WebSockets on the Microsoft platform including browser, client and server.

On the server side, ASP.NET and WCF 4.5 supports the latest version, RFC 6455, bringing the power and flexibility of WebSockets to the server-side developer's fingertips and providing the option to use whichever programming model you are most comfortable with.

Let's take a look at what the current state of support is on both the client and the server.

Browser Support

As of this writing, all major browsers including IE 10, Chrome 16, FireFox 11 and Safari support RFC 6455. The easiest way to ensure that your browser supports WebSockets is to use a JavaScript library called Modernizr which allows you to verify support for several emerging browser features (see the “Bullet Proof your Web Apps with Modernizr” sidebar for more information).

In addition, you can attempt to create a WebSocket connection and inspect the request/response headers using a tool like Fiddler or F12 developer tools. Figure 4 shows a test I conducted with Google Chrome. I'm consuming a WebSocket service that I'll cover in detail shortly, but as you can see, the service supports protocol version 13 which indicates that RFC 6455 is supported.


Server Support on the Microsoft Platform

As I've shared, WebSockets support is a feature in .NET 4.5, which has not been released as of this writing. However, that should not stop you from starting to experiment with WebSockets today.

To take advantage of WebSockets in ASP.NET and/or WCF, you'll need the .NET 4.5 Release Candidate (RC) which I recommend acquiring with Visual Studio 2012 RC. These pre-release bits provide a great opportunity to get your hands on the bits early and the RC is very stable.

In addition, WebSockets is only supported on Windows 8 and Windows Server 2012. This means that you can develop on Windows 7 or earlier OS versions, but your ASP.NET or WCF WebSocket services will only work on Windows 8 or Windows Server 2012.

Paul Batum, Program Manager on the WebSockets team at Microsoft, has a great post that walks through the steps for configuring Windows 8/Server 2012 for WebSockets so I won't repeat them. See Paul's post here: http://www.paulbatum.com/2011/09/getting-started-with-websockets-in.html.

While unfortunate, believe it or not, this is not a ploy by Microsoft to get developers to migrate to the latest OS. The problem is with the way that HTTP.sys works in Windows 7/Server 2008/R2 and earlier OS versions. Rather than issue a patch that could result in large regressions for existing customers that rely on the current behavior of HTTP.sys, the newest versions of Windows include a modification to HTTP.sys that enables WebSockets.

If you are using the Release Preview of Windows 8, I encourage you to install Visual Studio 2012 RC to get the most out of WebSockets today.

One Common Core, Multiple Choices

Support for WebSockets is provided via the System.Net.WebSockets namespace which provides powerful low-level APIs for leveraging WebSockets natively if you so choose. This namespace is common to both ASP.NET and WCF so regardless of your preference, you share a common base which ensures a consistent API.

While you are free to start programming right away against System.Net.WebSockets, Microsoft has developed some higher level abstractions for ASP.NET and WCF that make it very easy to get up and running via the Microsoft.WebSockets namespace, which is available via NuGet, and I will focus on these types in this article.

When you install the NuGet package, an assembly called Microsoft.WebSockets.dll (the latest version is 0.2.3 as of this writing) will be added to your project which contains two higher level classes for working with ASP.NET or WCF.

ASP.NET 4.5 WebSocketHandler

The WebSocketHandler class provides common methods that you'll work with when programming WebSockets. These methods wrap the lower-level APIs in System.Net.WebSockets.

Fundamentally, you'll work with three main methods when programming WebSockets. Take a look at the snippet below which represents a simple, yet fully functional WebSocket service:

public class ServerEventHandler : WebSocketHandler
{
    public override void OnOpen()
    {
        base.Send("You connected to a WebSocket!"));
    }
    
    public override void OnMessage(string message)
    {
        // Echo message
        base.OnMessage(message);
    }
    
    public override void OnClose()
    {
        // Free resources, close connections, etc.
        base.OnClose();
    }
}

As you can see, with ASP.NET you simply inherit from WebSocketHandler and override the OnOpen method which fires when a client establishes a WebSocket connection to your service. At this point, you can do nothing or simply send back a friendly greeting to indicate that the connection was successful using the Send method.

When your service receives a message, the OnMessage method is called and the payload of the message is provided as a parameter. WebSockets support text, XML and binary payloads. Here you would do some work, blocking or asynchronous, and send a response back to the client.

The OnClose method provides an opportunity to clean up resources such as closing connections, etc. It fires when the client explicitly sends a close request or automatically when a user closes a browser tab or the browser itself.

Simple Eventing with ASP.NET WebSocketHandler

Let's take a look at an example of using the WebSocketHandler to build a simple demo. While the canonical WebSocket example is a chat application (search the web to find a plethora of examples), I want to distill the power of WebSockets by showing you a simple event-driven sample that simply echoes the date in time at a given interval.

I have an HTML5 page consisting of some JavaScript references. The implementation of the form consists of an input button, a paragraph element and an unordered list element:

<body>
    <h2>Simple Event Sample</h2>
    <input type="button" id="connect" value="Connect" class="btn" />
    <p id="Status"></p>
    <ul id="messages"/>
</body>

Next, take a look at the JavaScript in Listing 2. I'm using some simple JQuery to wire up the simple HTML. First, I use Modernizr to detect if the browser supports WebSockets. This is very important for the reasons previously covered.

Listing 2: WebSocket Eventing script

$(function () {
    "use strict";
    
    if (Modernizr.websockets) {
        $("#Status").append("WebSockets is supported. Click the Connect button.");
    }
    
    if (!window.WebSocket && window.MozWebSocket) {
        window.WebSocket = window.MozWebSocket;
    }
    
    $('#connect').click(function () {
        var count;
        var connection;
        
        var host = "ws://localhost/SimpleEventingHandler/SimpleEventingService.ashx";
        //var host = "ws://localhost/SimpleEventingService/SimpleEventingService.svc";
        
        connection = new WebSocket(host);
        
        connection.onopen = function () {
            $(".btn").css("color", "green");
        }
        
        connection.onmessage = function (message) {
            var data = window.JSON.parse(message.data);
            $("<li/>").html(data).appendTo($('#messages'));
        };
    });
});

If the browser supports WebSockets, I just append a string indicating so to the paragraph element. From there, when I click on the Connect button, I initialize the host variable to the URI for my WebSocket service:

var host = "ws://localhost/SimpleEventingHandler/SimpleEventingService.ashx";

Next, since WebSockets is supported, I instantiate a native WebSocket class and set the instance to the connection variable:

connection = new WebSocket(host);

The WebSocket class is native to the browser, meaning that if the browser supports the W3C WebSocket API, the class will be available to you.

From there, I call onopen on the WebSocket instance and if the call is successful, I turn my button's text green to provide a visual indication that I've connected:

connection.onopen = function () {
    $(".btn").css("color", "green");
}

At this point, I'm waiting for messages. As soon as a message arrives, I'll convert to a string and append to my unordered list:

connection.onmessage = function (message) {
    var data = window.JSON.parse(message.data);

    $("<li/>").html(data).appendTo($('#messages'));
};

Now let's take a look at the ASP.NET implementation of the server using the WebSocketHandler that is available in Microsoft.WebSocket.dll.

Listing 3 and 4 provide the full code sample which consists of an implementation of an IHTTPHandler which will host the server as an .ashx and a WebSocketHandler which contains the implementation of the service.

Listing 3: WebSocket Eventing IHTTPHandler

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using Microsoft.Web.WebSockets;

namespace SimpleEventingHandler
{
    public class SimpleEventingService : IHttpHandler
    {
        public void ProcessRequest(HttpContext context)
        {
            if (context.IsWebSocketRequest)
            {
                context.AcceptWebSocketRequest(new SimpleEventingWebSocketHandler());
            }
        }
        
        public bool IsReusable
        {
            get
            {
                return false;
            }
        }
    }
}

First, as shown in Listing 3, I implement a simple IHTTPHandler which defines two methods, ProcessRequest and IsReusable. ProcessRequest is called when the HTTP request arrives at the handler and is passed to my code via an instance of HttpContext:

public void ProcessRequest(HttpContext context)
{
    if (context.IsWebSocketRequest)
    {
        context.AcceptWebSocketRequest(new SimpleEventingWebSocketHandler());
    }
}

If the request is a WebSocket request, I call AcceptWebSocketRequest to transition to the WebSocket protocol and pass an instance of a WebSocketHandler implementation which serves the initial and future requests.

As shown in Listing 4, the code for the WebSocketHandler is very simple. Upon establishing the client connection, a JSON message is returned along with the date and time every second:

public override void OnOpen()
{
    JavaScriptSerializer serializer = new JavaScriptSerializer();

    base.Send(serializer.Serialize("Connected!"));

    while (true)
    {
        base.Send(serializer.Serialize("Time now is: " + DateTime.Now));
        System.Threading.Thread.Sleep(1000);
    }
}

Listing 4: WebSocket Eventing with WebSocketHandler

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.Web.WebSockets;
using System.Web.Script.Serialization;

namespace SimpleEventingHandler
{
    public class SimpleEventingWebSocketHandler : WebSocketHandler
    {
        public SimpleEventingWebSocketHandler()
        {
        }
        
        public override void OnOpen()
        {
            JavaScriptSerializer serializer = new JavaScriptSerializer();
            base.Send(serializer.Serialize("Connected!"));
            
            while (true)
            {
                base.Send(serializer.Serialize("Time now is: " + DateTime.Now));
                System.Threading.Thread.Sleep(1000);
            }
        }
    }
}

Figure 4 shows the sample running in IE 10, which natively supports IETF RFC 6455.

While this is a very simple example, it shows the power of WebSockets in a simple way. Each second represents an event that the client cares about, and each time a second ticks, a message is sent to the browser over the same socket connection negating the need to poll at the client. Of course, this could be a stock ticker, a Twitter stream, or a price change from a pricing engine. The possibilities are endless.

WCF 4.5 WebSocketService

As I mentioned earlier, the Microsoft.WebSockets.dll also contains support for WCF. Since both the ASP.NET and WCF implementations are based on the same System.Net.WebSockets, the APIs are very similar.

As shown in Listing 5, after implementing the WebSocketService class, I simply override the OnOpen method and the implementation is otherwise identical:

public class SimpleEventingService : Microsoft.ServiceModel.WebSockets.WebSocketService
{
    public override void OnOpen()
    {
        JavaScriptSerializer serializer = new JavaScriptSerializer();
    
        base.Send(serializer.Serialize("Connected!"));
        
        while (true)
        {
            base.Send(serializer.Serialize("Time     now is: " + DateTime.Now));
            System.Threading.Thread.Sleep(1000);
        }
    }

Listing 5: Hosting WCF in IIS with WebSocketHost

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.ServiceModel;
using System.ServiceModel.Activation;
using Microsoft.ServiceModel.WebSockets;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.Threading.Tasks;

namespace Microsoft.ServiceModel.WebSockets.Util
{
    public class WebSocketServiceHostFactory : ServiceHostFactory
    {
        protected override ServiceHost CreateServiceHost(Type serviceType, Uri[] baseAddresses)
        {
            var host = new WebSocketHost(serviceType, new ServiceThrottlingBehavior {
                MaxConcurrentSessions = int.MaxValue,
                MaxConcurrentCalls = 20 
            }, baseAddresses);
            
            var binding = WebSocketHost.CreateWebSocketBinding(https: false, sendBufferSize: 2048, receiveBufferSize: 2048);
            binding.SendTimeout = TimeSpan.FromMilliseconds(500);
            binding.OpenTimeout = TimeSpan.FromDays(1);
            host.AddWebSocketEndpoint(binding);
            
            return host;
        }
    }
}

To host the WebSocketService, all of the options for hosting WCF apply. You can self-host, host as an NT Service or host in IIS/Windows Server AppFabric/AppFabric 1.1 for Windows Server. The Microsoft.WebSocket.dll includes a WebSocketHost which simplifies the process (I recommend using AppFabric 1.1 for Windows Server, a free extension to IIS as a host when possible as it provides a number of benefits around management and hosting of WCF services).

To host a WCF WebSocket service in IIS, you must create a service host factory as shown in Figure 5.

![Figure 5: Simple event sample in IE 10.](https://codemag.com/Article/Image/1210051/Garibay Figure 5.png)

Within the ServiceHost class, you use WebSocketHost to create a binding that is compatible with WebSockets. The CreateWebSocketBinding method on the WebSocketHost builds up a custom binding consisting of a ByteStreamMessageEncodingBindingElement and an HttpTransportBindingElement which results in a simple channel stack for supporting WebSockets natively from browser clients:

var host = new WebSocketHost(serviceType, new ServiceThrottlingBehavior { MaxConcurrentSessions = int.MaxValue, MaxConcurrentCalls = 20 }, baseAddresses);

var binding = WebSocketHost.CreateWebSocketBinding(https: false, sendBufferSize: 2048, receiveBufferSize: 2048);
binding.SendTimeout = TimeSpan.FromMilliseconds(500);
binding.OpenTimeout = TimeSpan.FromDays(1);
host.AddWebSocketEndpoint(binding);

Since we're using a custom factory, be sure to add a reference to the factory in your .svc file:

<%@ ServiceHost Language="C#" Debug="true" Service="Service.SimpleEventingService" CodeBehind="SimpleEventingService.svc.cs" Factory="Microsoft.ServiceModel.WebSockets.Util.WebSocketServiceHostFactory" %>

Now, go back to the client-side JavaScript we reviewed in the first sample, and change the host variable from:

var host = "ws://localhost/SimpleEventingHandler/SimpleEventingService.ashx";

To:

var host = "ws://localhost/SimpleEventingService/SimpleEventingService.svc";

If you refresh (or restart) the HTML page in Listing 1, the client will work in exactly the same way as when it was using the HTTP hander with ASP.NET.

Listing 1: HTML5 WebSocket Eventing sample

<!DOCTYPE html>
<html>
<head>
    <!--<meta http-equiv="X-UA-Compatible" content="IE=edge" />-->
    <title>Simple WebSocket Eventing with ASP.NET and WCF 4.5</title>
    <link rel="stylesheet" type="text/css" href="Styles/style.css"/>
    <script src="Scripts/jquery-1.7.1.min.js"></script>
    <script src="Scripts/jquery.cookie.js" type="text/javascript"></script>
    <script src="Scripts/json2.min.js" type="text/javascript"></script>
    <script src="Scripts/modernizr-2.0.6-development-only.js" type="text/javascript"></script>
    <script src="Scripts/script.js" type="text/javascript"></script>
</head>
<body>
    <h2>Simple Event Sample</h2>
    <input type="button" id="connect" value="Connect" class="btn" />
    <p id="Status"></p>
    <ul id="messages"/>
</body>
</html>

WCF NetHttpBinding

While support for WebSockets in browser clients is very powerful, this isn't the only scenario where WebSockets is useful. The WCF NetHttpBinding brings the flexibility of WebSockets to the WCF programming model for duplexing.

By using the WebSocket protocol, the reach of WCF services is greatly extended due to the firewall-friendly nature of the WebSocket protocol and Microsoft reports that performance of the NetHttpBinding is close to that of NetTcp.

The beauty of the WCF programming model is that taking advantage of a new channel is as simple as swapping out the binding, so if you are using duplexing today, I encourage you to consider NetHttpBinding.

While the focus of this article is on WebSockets for the browser, my friend and fellow CSD MVP, Damir Dobric, has a great blog post that shows you how to take advantage of the NetHttpBinding in non-browser scenarios: http://developers.de/blogs/damir_dobric/archive/2011/11/26/wcf-duplex-via-websocket.aspx.

In addition, Paul Batum has published a great sample that demonstrates how to consume a WebSocket service from Metro style clients in Windows 8. Please see the sidebar “Native WebSockets and Windows 8” for more information.

Building a Streaming Twitter Client with the WCF WebSocketService

In the previous example, I showed you how to expose a WebSocket service to a browser using both ASP.NET and WCF. We simulated the “real-time” aspect of the service by sending the date and time every second. Obviously, this sample isn't very useful, but it does demonstrate a very simple way of exposing real-time events to the client.

Now, let's take a look at a more realistic scenario. Someone recently said that Twitter demos have become the “hello world” of the modern web, which speaks to how pervasive Twitter has become. What makes Twitter interesting for a WebSocket application is that it not only epitomizes a real-time app, but also exposes an API that allows you to interact with it in a variety of fun and interesting ways.

I'm not going to turn this article into a primer on Twitter development as there are plenty of resources out there to help get you started. It is worth noting, however, that Twitter supports two APIs for consuming events: The REST Search API and the Streaming API. Both APIs are useful and provide varying features, and for this sample, I'm going to use the REST Search API to demonstrate one approach for providing real-time updates to your browser clients.

The sidebar includes a link to the Twitter docs for the REST Search API, but to summarize, the API is exposed via a URI with varying query string parameters. For example, the following URI would be used to search all updates that include the term “AZ”, with a maximum of 5 status updates per page and including any images or other media resources if available:

http://search.twitter.com/search.json?q=AZ&;rpp=5&include_entities=true

If you paste the above URL into your browser and are signed in to Twitter, you will get a JSON result resembling something like:

{
    "completed_in":0.12,
    "max_id":219118633002078208,
    "max_id_str":"219118633002078208",
    "next_page":"?page=2&max_id=219118633002078208&q=mtb&rpp=5&include_entities=1",
    "page":1,
    "query":"mtb",
    "refresh_url":"?since_id=219118633002078208&q=mtb&include_entities=1",
    "results":[
        {
            "created_at":"Sat, 30 Jun 2012 17:22:07 +0000",
            "entities":{
                "hashtags":[],
                "urls":[
                    {
                        "url":"http:\/\/t.co\/Kr2ZC3IZ",
                        "expanded_url":"http:\/\/youtu.be\/MDaSZR7siwo?a",
                        "display_url":"youtu.be\/MDaSZR7siwo?a",
                        "indices":[36,56]
                    }],
                "user_mentions":[
                    {
                        "screen_name":"YouTube",
                        "name":"YouTube",
                        "id":10228272,
                        "id_str":"10228272",
                        "indices":...

From here you would serialize the results into an array of literals and if you wanted to page ahead, or see if there are any more tweets that match your parameters, you could call the REST API in some kind of a loop, or leverage XHR, both of which will definitely work.

The opportunity is to push some of that messaging, looping, logic to the back end where it belongs (at least in my opinion) and this is where WebSockets comes in. By wrapping the APIs in a service, the client can focus on presentation and validation and not have to worry about tying up browser threads, managing polling intervals, etc., providing for a much cleaner separation of concerns.

For the service, you could use any server-side technology that supports WebSockets (or even build your own server by implementing RFC 6455) and in the steps that follow, I'll show you how to write a WCF service that leverages the Microsoft.WebSockets WebSocketService helper along with a great tool by fellow MVP Joe Mayo called Linq2Twitter, available on NuGet, which is exactly what you think it is: a Linq client library that handles all the heavy lifting of authenticating and interacting with the REST APIs. (You can learn more about Linq2Twitter by referring to the sidebar.)

Building the StatusStreamService with WebSocketService

Since I've already covered the fundamentals of how WebSocketService works, let's jump right in.

Listing 6 provides the full implementation of the StatusStreamService which, along with all the other code I reference in this article, is available for download on code-magazine.com. If you've read this far, the code should be pretty familiar as it includes the OnOpen method and a Send method that I demonstrated in the previous WCF and ASP.NET examples.

Listing 6: StatusStreamService with WebSocketService

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;
using System.Web.Script.Serialization;
using LinqToTwitter;
using Microsoft.ServiceModel.WebSockets;
using TwitterStreamService;

namespace StatusStreamService
{
    public class StatusStreamService : WebSocketService
    {
        string hashTag = "#devconnections";
        int pollingInterval = 5;
        
        JavaScriptSerializer serializer = new JavaScriptSerializer();
        
        public override void OnOpen()
        {
            ulong lastStatusId = 0;
            StreamManager mgr = new StreamManager();
            
            while (true)
            {
                SimpleStatus simpleStatus = mgr.GetSimpleStatus(hashTag);
                
                if (simpleStatus != null)
                {
                    if (simpleStatus.StatusId != lastStatusId.ToString())
                    {
                        Send(simpleStatus);
                        lastStatusId = ulong.Parse(simpleStatus.StatusId);
                    }
                }
                
                System.Threading.Thread.Sleep(pollingInterval * 1000);
            }
        }
        
        private void Send(SimpleStatus simpleStatus)
        {
            var message = serializer.Serialize(simpleStatus);
            base.Send(message);
        }
        
        public override void OnMessage(string message)
        {
            base.OnMessage(message);
        }
        
        protected override void OnClose()
        {
            base.OnClose();
        }
    }
}

I've kept the service implementation pretty light, delegating the interaction with the Twitter API (via Linq2Twitter) to a StreamManager class that I'll review shortly.

The StatusStreamService issues calls to the REST API given a hash tag (which for the purposes of this sample is hardcoded) in a loop, given a polling interval. Since there are API quotas that Twitter will hold you to in order to ensure that they can scale and ward off denial of service attacks, you want to be mindful and throttle your request accordingly. One of the problems I've had with the REST Search API is that it is quite possible for duplicate tweets to come back. Even though you've already sent the status to the browser, if the tweet is in the search window, it will come back in the results again and again and there's really no way I've found to reliably avoid this. The solution is a simple hack which caches the last StatusId, which is a unique identifier for each tweet and compares it with the latest retrieved tweet. If they match, I simply ignore the tweet; otherwise I send the status to the client.

That's really all there is to it, and I could have just as easily opted for the ASP.NET WebSocketHandler which would resemble the same approach.

As you can see in Listing 7, the StreamManager exposes a method called GetSimpleStatus which returns an instance of SimpleStatus which is custom DTO I created to simplify the JSON that is returned as I am really only interested in a few fields:

while (true)
{
    SimpleStatus simpleStatus = mgr.GetSimpleStatus(hashTag);
    //Dedupuping hack
}

Listing 7: StreamManager and Linq2Twitter

public SimpleStatus GetSimpleStatus(string hashTag)
{
    var auth = GetAuthorizer();
    
    TwitterContext ctx = new TwitterContext(auth, "https://api.twitter.com/1/";, "https://search.twitter.com/";);
    var tweets = from tweet in ctx.Search 
        where tweet.Type == SearchType.Search && 
        tweet.Hashtag == hashTag && 
        tweet.ResultType == ResultType.Recent 
        select tweet;
        
    List<SimpleStatus> stream = new List<SimpleStatus>();
    
    foreach (Search result in tweets)
    {
        stream.Add(new SimpleStatus {
            UserId = result.Results[0].FromUserID.ToString(),
            User = result.Results[0].FromUser,
            ProfileUrl = result.Results[0].ProfileImageUrl,
            Status = result.Results[0].Text,
            StatusId = result.MaxID.ToString(),
            MediaUrl= result.Results[0].Entities.MediaMentions.Count !=0 ? result.Results[0].Entities.MediaMentions[0].MediaUrl:""  
        });
    }
    
    return stream[0];
}

I serialize the SimpleStatus DTO to JSON to make it easy for the browser to work with, focusing only on the properties that I need to reduce bandwidth and complexity:

public class SimpleStatus
{
    public string User;
    public string UserId;
    public string StatusId;
    public string Status;
    public string ProfileUrl;
    public string MediaUrl;

The GetSimpleStatus method, shown in full in Listing 6, starts with a call to a private method I wrote for encapsulating the authentication goo so that it is reusable.

Linq2Twitter makes this very simple by providing an abstraction of the authorization token with the SingleUserAuthorizer class and once I have an instance of it, I can pass it into a new TwitterContext, which provides most of the API knobs I need to assemble my request:

var auth = GetAuthorizer();

TwitterContext ctx = new TwitterContext(auth, "https://api.twitter.com/1/";, "https://search.twitter.com/";);
var tweets = from tweet in ctx.Search 
    where tweet.Type == SearchType.Search && 
    tweet.Hashtag == hashTag && 
    tweet.ResultType ==  ResultType.Recent
    select tweet;

Each of the properties are well documented on the Twitter docs, but in a nutshell, I'm telling Linq2Twitter to prepare a REST API call using the Search API and include any tweets that have the hashtag that is passed in from the WebSocketService code.

From there, I simply enumerate the projection, grab the top tweet and return it to the StatusStreamService.

The StatusStreamingClient

The client is very similar to the one I demonstrated earlier, but I've added some bells and whistles (borrowed from designers far more talented at Web UI than me) to show what is possible by combining the modern user experience features of HTML5 and JQuery with the power of WebSockets.

The structure of the status streaming client is simple. As shown in Listing 8, I'm leveraging just a few HTML elements as placeholders for some assets that will be added via CSS. Each of the divs below act as a container, and when my StatusStreamService has a tweet to share, the status will appear in the div element with the class name of “quote”:

<body>
    <div id="wrapper">
        <div id="content">
            <div id="quotebox">
                <p class="quotemark leftquote">&#8216;&#8216;</p>
                <div class="quote"></div>
                <p class="quotemark rightquote">&#8217;&#8217;</p>
            </div>
        </div>
    </div>
</body>

Listing 8: StatusStreamClient

<!DOCTYPE html>
<html>
<head>
    <!-- <meta http-equiv="X-UA-Compatible" content="IE=edge" />-->
    <title>Twitter Status Streaming with WCF 4.5 WebSockets</title>
    <link rel="stylesheet" type="text/css" href="Styles/style.css"/>
    <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js";></script>
    <script type="text/javascript" src="Scripts/script.js"></script>
</head>
<body>
    <div id="wrapper">
        <div id="content">
            <div id="quotebox">
                <p class="quotemark leftquote">&#8216;&#8216;</p>
                <div class="quote"></div>
                <p class="quotemark rightquote">&#8217;&#8217;</p>
            </div>
        </div>
    </div>
</body>
</html>

As with the previous simple eventing sample, the client is wired up via JavaScript using the native WebSocket support in IE 10, Chrome 13, etc.

$(document).ready(function()
{
    var connection;
    var host = "ws://localhost/StatusStreamService/StatusStreamService.svc";

    connection = new WebSocket(host);

    connection.onmessage = function (message) {
        var simpleStatus = window.JSON.parse(message.data);
        startAnimation(simpleStatus.User, simpleStatus.Status);
    }
});

Since I chose WCF for this implementation, I've hosted the service in IIS/AppFabric using the same custom factory I demonstrated in the initial WCF WebSocketService example above, and the host variable is accordingly set to the URI of the service in IIS of my dev machine.

Once the handshake is complete and the connection is established, I'm ready to start receiving messages. Each message will be a JSON serialized instance of SimpleStatus which I parse into a literal and pass it into a JavaScript function called “startAnimation”.

This is where a lot of the magic happens, including nice animation and fade effects of each status as it arrives on the browser. I'd like to give credit to Marco Kuiper (marcofolio.net) who provided the initial design theme and Tory Douglas, Principal Consultant at Neudesic, who helped me bend the JQuery to my will. I won't cover the details of the rest of the JavaScript/JQuery here, but the full code is available in the samples download so please feel free to dig in.

If I browse to the Default.html page, within seconds, I start seeing status updates for the given hashtag (in this case #winrt) as shown in Figure 6.


As you can see, WebSockets brings a whole new way of providing real-time messaging to your browser-based applications and as I've demonstrated, has broad support in browsers like IE 10, Chrome 13, etc.

There is a ton more that you can do with WebSockets such as stream binary data, XML, as well as send messages from the client to the browser, of course. The Microsoft.WebSockets namespace makes it easy to get up and running, and if you ever need more control over your implementation, you can always dig deeper into the System.Net.WebSockets library which is used by both WebSocketHandler and WebSocketService under the hood.

Bonus: Fun with WebSockets and Node.js

So you may be thinking, “I want the WebSocket server sauce, but I'm not yet running Windows 8 or won't be for a while.”

One of the great things about working with standards-based technology is that even though it takes time for the standardization dust to settle sometimes, a well-defined, public specification makes it possible for anyone, be it a corporation or an individual, to implement the specification. Just as Microsoft has done in .NET 4.5, there are a number of other options available for building solutions with WebSockets regardless of platform. In fact, there are some really smart guys at Microsoft building a framework called SingalR that enables you to build real-time browser apps that will automatically use XHR, long polling and WebSockets depending on the capabilities of the client (browser) and the service you are consuming.

Node.js has recently gained tremendous popularity among web developers for a number of reasons. In addition to brining JavaScript, the world's most popular scripting language, to the server side, it is very well suited for implementing a WebSocket server due to its de-facto asynchronous programming model, which is highly optimized for IO operations.

Node.js is built on a single threaded, asynchronous programming model which leverages OS resources for completing IO operations. You could write an entire book on the topic of Node.js alone, and I've provided some good references in the sidebar if you'd like to learn more about Node.js.

For the purposes of this article, I want to show you what an alternate implementation in Node.js looks like and how, regardless of the back-end, the client API, courtesy of the W3C, is always consistent.

To do so, I'd like to take the StatusStreamingService sample further by applying the same concepts, but using Node.js as a WebSocket server and switching out the Twitter Search API with the Streaming API. In addition, I'll bring in any photos that are posted with a query term/hashtag along with the corresponding status updates to create a live stream of Polaroids.

Exploring PolaroidStream with Node.js and WebSocket.IO

One of the reasons Node.js has surged in popularity, in addition to the characteristics that will whet any nerd's appetite, is the incredibly rich community of developers that are building libraries (called modules in Node.js parlance) for Node. You can go out to the nodejs.org website and GitHub and find thousands of modules for building MVC-style web apps to providing low-level messaging support.

One of my favorite Node.js modules is Socket.IO. Written by Guillermo Rauch, Socket.IO provides a very flexible way of taking advantage of XHR, long polling and WebSockets in your applications. The module provides both a server-side and client-side API for building robust back-end services that greatly simplify delivering real-time messaging within your solutions.

Guillermo has also written a module that is fully compatible with the W3C WebSockets API called WebSocket.IO that allows you to build back-end services on WebSockets just as we have with ASP.NET and WCF 4.5.

In order to further showcase the strength of WebSockets, I want to briefly compare and contrast the Twitter REST Search API with the Twitter Streaming API. As we saw in the StatusStreamingService/Client sample, the biggest benefit that WebSockets provides is that it enables true, real-time communication by upgrading an HTTP/S request to a direct connection without resorting to polling patterns and thus allowing you to push event aggregation to the server.

On the server side, we still need a way to aggregate the events, and perhaps in a perfect world, all endpoints are connected via WebSockets, but until then, middleware is the key. In the StatusStreamingService sample, we leveraged polling in a while loop to make a REST API call every five seconds. Some polls result in a status being returned while others might not.

You could hook into these events of interest in a number of ways. One approach might be to have some other middleware component poll the REST API and publish each tweet to a broker to which our StatusStreamService subscribes. Another approach might be Twitter notifying the StatusStreamService that it has new tweets and only then, issuing a call to the Search API.

All of these approaches have their strengths and weaknesses depending on goals such as throughput, latency, scalability, etc.

One option we haven't discussed in detail is streaming. Streaming implementations vary depending on the transport protocol, but Twitter leverages HTTP to support streaming. By taking advantage of HTTP chunking, as a Twitter API subscriber, you can issue an initial request for tweets that match some filter and Twitter will hold on to that connection as long as there are tweets to send. From an HTTP perspective, this is actually one request that consists of several parts, or chunks.

This is very powerful, because unlike the REST Search API, which by its very nature is somewhat latent, the Streaming API is essentially a fire hose, delivering each tweet in a series of chunks as close to the same time that the Twitter servers receive each tweet.

Figure 7 demonstrates how the Streaming API works.


The Twitter Streaming API supports three types of streams as defined on dev.twitter.com:

• Public streams: Streams of the public data flowing through Twitter. Suitable for following specific users or topics, and data mining.
• User streams: Single-user streams, containing roughly all of the data corresponding with a single user's view of Twitter.
• Site streams: The multi-user version of user streams. Site streams are intended for servers which must connect to Twitter on behalf of many users.

Now, let's have a little fun with WebSockets and Node.js to see how all of this fits together.

Node Server

My goal with the service is to grab as many tweets as are available given a hash tag or filter and also bring back any photos that were posted with the tweet.

The URI below shows what a request on the Public stream for all tweets with the term MTB and twitpic:

https://stream.twitter.com/1/statuses/filter.json?track=MTB&;track=twitpic -u USERNAME:PASSWORD

To get a sense of how busy this stream is, you can test the request by using a tool like Fiddler or cURL. Figure 8 shows the result of issuing the request above with cURL:


If it was possible to show an animated GIF in this magazine, you would see constant scrolling action as each tweet is firehosed to the client. This is a good example of where having a broker like Azure Service Bus or RabbitMQ might make sense so that you don't bring down your client.

With our URL tested and ready to go, let's take a look at the Node.js WebSocket server I wrote in Listing 9.

Listing 9: PolaroidStreaming with Node.js and WebSocket.IO

var ws = require('websocket.io'), server = ws.listen(8080), http = require('https');

var sockets =[];

server.on('connection', function (socket) {

    console.log('Connected to client');

    sockets.push(socket);

    var options = {
        host: 'stream.twitter.com',
        port: 443,
        path: "/1/statuses/filter.json?track=VAZ303%20twitpic&track=VAZ303%20photo&track=devconnections%20twitpic&track=devconnections%20photo",
        method: 'GET',
        headers:{
            "Authorization":"Basic (CREDENTIALS)"
        }
    };

    var req = http.request(options, function(res) {
        res.setEncoding('utf8');
        console.log('STATUS: ' + res.statusCode);
        console.log('HEADERS: ' + JSON.stringify(res.headers));
        
        res.on('data', function (chunk) {
            console.log('BODY: ' + chunk);
            var s;
            for(i=0;i<sockets.length;i++)
            {
                sockets[i].send(chunk);
            }
        });
    });
    
    req.on('error', function(e) {
        console.log('Problem with request: ' + e.message);
    });
    
    req.end();
    
    socket.on('message', function (data) {
        console.log('Message received:', data);
    });
    
    socket.on('close', function () {
        // Clean up goes here
        console.log('Socket closed!');
    });
});

The first thing I do is register the WebSocket.IO and HTTP modules with Node.js:

var ws = require('websocket.io'), server = ws.listen(8080), http = require('https');

The HTTPS module is required to support connecting to the Twitter Streaming API which requires SSL.

When the server starts and establishes a WebSocket connection with the browser, Node prepares the request to Twitter using the same URL as above by initializing the options variable:

server.on('connection', function (socket) {

    var options = {
        host: 'stream.twitter.com',
        port: 443,
        path: "/1/statuses/filter.json?track=VAZ303%20twitpic&track=VAZ303%20photo&track=devconnections%20twitpic&track=devconnections%20photo",
        method: 'GET',
        headers:{
            "Authorization":"Basic (CREDS)"
        }
    };
    // (More Code)

Next, it makes the request to the Streaming API, passing in the options variable along with a callback that handles the response (res). From there, I set the encoding on the response to utf8 and log the HTTP response code and the headers:

var req = http.request(options, function(res) {
    res.setEncoding('utf8');
    console.log('STATUS: ' + res.statusCode);
    console.log('HEADERS: ' + JSON.stringify(res.headers));

Next, for each chunk in the response, I check to see how many browsers have established a WebSocket connection to me and multicast each chunk to each browser:

res.on('data', function (chunk) {
    console.log('BODY: ' + chunk);
    var s;
    for(i=0;i<sockets.length;i++)
    {
        sockets[i].send(chunk);
    }
});

That's all there is to it. With just a few lines of JavaScript, we have a full blown WebSockets server that leverages the Twitter Streaming API.

To start your Node server, simply open a command prompt and type:

node server.js

As shown in Figure 9, the node.js server running WebSocket.IO is up and running.


HTML5 Client

With the service done, let's review the client.

The client takes advantage of some nice HTML5 features such as CSS3 rotate and box shadow. Again, I tip my hat to marcofolio.net for providing the CSS and JavaScript that served as the foundation for this client sample.

Listing 10 shows the Default.html page, and once again, we're using HTML only for the structure:

<body>
    <input id="buttonAddPolaroid" value="Connect" type="button" class="btn" />
    <div id="polaroidContainer"/>
</body>

Listing 10: Polaroid stream client

<!DOCTYPE HTML>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html;charset=iso-8859-1"/>
    <title>Demo 4: WebSocket Eventing with Node.js</title>
    <link rel="stylesheet" type="text/css" href="css/style.css"/>
    <script src="Scripts/jquery-1.7.1.min.js"></script>
    <script src="Scripts/jquery-ui.min.js" type="text/javascript"></script>
    <script src="Scripts/modernizr-2.0.6-development-only.js" type="text/javascript"></script>
    <script src="Scripts/script.js" type="text/javascript"></script>
</head>
<body>
    <input id="buttonAddPolaroid" value="Connect" type="button" class="btn" />
    <div id="polaroidContainer"/>
</body>
</html>

The button is there to allow me to establish the connection to the Node.js WebSocket service and again, I'm using a div element as a container that will hold the images and tweet status for every tweet that is received.

The idea is to provide a user experience similar to sitting at a coffee table and going through a shoe box of Polaroid pictures. You grab a stack and start tossing them on the table as you go.

The entire implementation is provided in the download on code-magazine.com, and for the purposes of this sample, I'll highlight the code for connecting to and processing each chunk that is returned by the Node.js server.

As with the previous samples, the first thing I want to do is ensure that my browser supports WebSockets:

$(document).ready(function () {
    if (Modernizr.webjsockets) {
        $("#messages").append("WebSockets is supported!");
    }

If it does, then I can confidently click the Connect button to establish a connection to the WebsSocket server. I've set a host variable to the URL for my node.js server and created a connection:

//wire up button event
$('#buttonAddPolaroid').click(function () {
    var count;
    var connection;
    
    var host = "ws://localhost:8080/";
    
    connection = new WebSocket(host);

If the connection succeeds, I change the color of the button text to green to provide an indication that all is well:

connection.onopen = function () {
    $(".btn").css("color", "green");
};

From there, I wait for the first message and convert it into a literal so I can easily check the fields to determine if the tweet has an image:

connection.onmessage = function (message) {
    tweet = window.JSON.parse(message.data);
    var imgUrl;
    
    if(tweet.entities.urls.length !==0){
        imgUrl= GetImgUrl(tweet,"twitpic");
    }
    else
    {
        // Naive assumption that image service must be native twitter.
        imgUrl= GetImgUrl(tweet,"twitter");
    }


Since each image service handles URLs different, I have a helper that figures out the render URL which I'll use next. Of course, to truly cover the breadth of possible images, I'd want to support more than just TwitPic and Twitter's photo service.

I have another helper method called addNewPolaroid that takes the image's URL and corresponding status from the tweet and renders it to the div tag:

addNewPolaroid(imgUrl, 'Status from ' + tweet.user.screen_name, '@' + tweet.user.screen_name + ':' + tweet.text);

The implementation of the addNewPolaroid function is pretty straightforward:

$('#polaroidContainer').append('<div class="polaroid"><img src="' + src + '" alt="' + alt + '" /><p>' + title + '</p></div>');

I'm using JQuery to simply append an image and the text from the tweet. However, from there, CSS formats the image and text into what resembles a really cool Polaroid picture and from there some more JQuery is applied to support the ability to select any Polaroid and bring it into focus as shown in Figure 8.

Conclusion

As you can see, the WebSocket protocol opens up a ton of new opportunities for web, mobile and enterprise developers. While the focus of this article has been on programming WebSockets for the browser, WebSockets is just as applicable in native application development be it WPF, Metro or composite services.

WebSockets is an evolution for web applications because it improves upon contemporary approaches for delivering near-real-time messaging such as XHR-style polling or long polling, enabling true, bi-directional duplex communication between the client and server.

What makes WebSockets so compelling is the reach, both in terms of browser support and inherent firewall friendliness. Developers and organizations hosting HTTP/S services on a DMZ or hosting provider can apply the same approach to hosting WebSocket-enabled services.

As you saw in the examples, writing a WebSocket service is very straightforward thanks to the investments Microsoft has made in the System.Net.WebSockets namespace and corresponding APIs provided in ASP.NET and WCF. You also saw that with most of the standards churn over, WebSockets support can now be enjoyed across all modern browsers including IE 10, Chrome 13, Safari and FireFox 11.

While taking advantage of WebSockets is a .NET 4.5 and Windows 8/Server 2012 only feature, it's a great time to dive into these pre-release technologies and start building apps and services with WebSockets. If you're not ready to upgrade to the latest version of Windows just yet, there are a number of options available for experimentation, including Node.js.

As with any new, emerging technology, this fledgling new protocol is not immune to controversy. Some web developers posit that WebSockets is not “webby” at all because it only leverages HTTP/S initially and from there is really a TCP-like socket, which might be a liability from a scalability and state management perspective. As with all technologies, we must understand the benefits and liabilities of each of our tools and choose the appropriate tool for the job.

Personally, I believe that for real-time apps, gaming, B2B and composite apps, WebSockets offers a great alternative to traditional methods for exchanging real-time information, particularly on the Web, delivering a simpler programming model, lower latency and a variety of implementation options.

I hope this article and corresponding samples will help jump start your discovery of WebSockets today.