In my article in the November/December 2011 issue of CODE Magazine, I shared my projections for how important technologies like Azure Service Bus would be for realizing the integration imperative introduced with hybrid-cloud computing. Fast-forward a little over two years later and the cloud is now a $160B industry.
Today, I have few clients who don't rely on externally hosted SaaS (Software as a Service) providers or haven't moved certain key business workloads to the cloud. Ensuring that those assets truly become extensions of the company's enterprise, as opposed to one-off silos, and enabling the exposure of rich messaging endpoints across on-premise and cloud-hosted business assets has become a true integration imperative.
Why Middleware as a Service?
There's a vast array of on-premise middleware platforms that focus on traditional SOA, Enterprise Service Bus (ESB), and Enterprise Application Integration (EAI) composition scenarios within the enterprise. Robust, proven, and comprehensive offerings abound from key vendors such as Microsoft, Oracle, IBM, WS02, Mule, Neudesic, and TIBCO.
Many of those vendors have evolved their platforms to enable hybrid messaging scenarios. For example, both Neuron ESB and BizTalk Server provide first-class support for Azure Service Bus, enabling an extension of their on-premises messaging fabrics to Windows Azure. (You can learn more about Neuron ESB's Azure Service Bus adapter support here: http://bit.ly/1gwt0Kk.)
In addition, a new breed of messaging and middleware Platform-as-a-Service (PaaS) products have started to surface over the last 24 months to address a new set of challenges and scenarios that typically fall out of reach of traditional middleware product offerings. For example, if you are a SaaS ISV and need to stitch together various APIs to provide a feature, the notion of big iron on-premise makes little sense when your product is hosted in the cloud. Or, consider a small business that needs to integrate with suppliers and distribution chains to push modest messaging traffic but can't afford the costs inherent to licensing and managing an on-premise middleware platform.
To address this new kind of customer, more and more PaaS middleware products are popping up ranging from PC manufacturing giant Dell entering the PaaS middleware space with its Boomi offering, to SnapLogic, a relatively new company with traditional data roots. Even TIBCO, the company that started it all, has announced its foray into the PaaS space with its TIBCO Cloud Bus.
So what has Microsoft been up to in this relatively new space? Well, the BizTalk team has been very busy cranking out the latest version of its flagship middleware server product and investing in new capabilities that harness the power of the Windows Azure platform, both on-premises and in the cloud.
Now in its seventh release, BizTalk Server has established itself as the most widely deployed middleware platform in the world with over 10,000 customers worldwide and growing. In addition to productivity and performance improvements, as well as updated support for the latest versions of Windows Server, SQL Server, and Visual Studio, BizTalk Server 2013 introduces a number of cloud-friendly capabilities.
Support for the WCF NetMessaging binding enables the ability to send and receive messages over Windows Azure Service Bus brokered messaging, enabling modern hybrid messaging scenarios with the tried-and-true on-premise workhorse. In addition, Microsoft introduced BizTalk Server 2013 on Windows Azure VM, supporting full blown BizTalk Server workloads hosted on Windows Azure's latest IaaS model.
While these additions to the BizTalk family are useful indeed, they do little to address the growing need for a lighter-weight solution to middleware in a cloud and device era. Windows Azure BizTalk Services (WABS) is specifically designed for customers and companies that either don't need an on-premise solution or for which more comprehensive on-premise middleware is not practical for their business model.
Does WABS Replace BizTalk Server?
A common question I get from clients and one that is asked at conferences is whether Windows Azure BizTalk Services replaces BizTalk Server. The answer is simple: no. As I mentioned previously, Windows Azure BizTalk Services is designed for SaaS vendors, ISVs, and small companies that don't need the full spectrum of middleware workloads and associated costs. However, even if you are an existing BizTalk Server customer, just as WABS extends BizTalk Server, you'll learn how WABS is built on the same solid foundation providing new advanced messaging capabilities that reach beyond the scope of WABS itself.
That said, a more interesting question might be what the future for application integration at Microsoft looks like. It's clear that most ISVs (from small startups to large vendors like Amazon, SalesForce, and MuleSoft) are embracing a Platform as a Service (PaaS) model that allows them to ship at a far more frequent cadence and at a significantly lower cost than traditional middleware. Gartner Research coins this new generation of cloud services iPaaS (the “i” stands for Integration) and while the goal of WABS is not replacing BizTalk Server today, it's clear that Microsoft is investing heavily in this space with the advent of WABS.
Understanding WABS Workloads
WABS is comprised of two key workloads: Enterprise Application Integration (EAI) and B2B. Although the focus of this article is on the EAI workload, I'll describe the capabilities and concepts of each before going deeper into how to enable rich messaging endpoints with WABS EAI.
WABS B2B
First, a little bit of history.
Before this whole Internet thing came along, there weren't a lot of options for exchanging automated messages between organizations. Web Services didn't exist yet, much less SOA or the cloud (the ultimate SOA of SOAs) so companies had to pretty much set up direct links via WAN/VANs to communicate.
If a company wanted to communicate with another company in an automated manner and exchange business documents electronically (and Excel spreadsheets don't count!), EDI (Electronic Data Interchange) was really the only game in town for doing business-to-business transactions before the 21st century. EDI fueled a whole new level of communication and efficiency both between and within companies and many industries invested significantly in EDI standardization that continues to drive healthcare, transportation, retail, and manufacturing today.
Looking to the shininess of new, human-readable messaging standards like XML and the proliferation of Web services that would (rightly) fuel a whole new generation of innovation, it seemed that EDI would slowly go the way of the .ini file, due not only to the arduous nature of writing and reading EDI files, but the associated expense of standing up a VAN (Value Added Network) and managing an AS2 (Application Statement Two) infrastructure.
I'm not going to go into too much more detail on EDI here, but suffice to say that in 2014, EDI remains alive as well. From healthcare and manufacturing to transportation and retail, EDI remains a proven, reliable mechanism for transmitting data between organizations. Its pedantic rules for how and where strings of data are stored fits in very well with heavily regulated industries like healthcare, but is also very popular in retail and manufacturing.
As one example, Walmart requires the exchange of EDI messages with suppliers who do more than 3,500 invoices per year. They have no mandates as to what providers, vendor, etc. to use, but whether you are a big manufacturing facility or a small mom-and-pop start up that just hit it big after pitching your homemade fudge, if you want to be in business with the big boys, you must be able to exchange EDI transactions for purchase orders and invoicing (i.e., EDI 850s and 810s).
This is where WABS comes in. Unless you already have an EDI infrastructure, have stood up a VAN, and are an EDI pro, chances are that a cloud-hosted model can get you up and running much more quickly and for a fraction of the cost of traditional B2B providers. This is the sweet spot of WABS B2B that provides all of the strength of traditional B2B/EDI infrastructure in a cloud service model without compromise. Customers that already have investments in traditional B2B infrastructure may find a PaaS hosted model much more cost effective.
WABS B2B provides a 100% portal-based experience and does all of the heavy lifting for you. You can set up Partners, Agreements, hosted, and guest partners right from the Azure portal, as shown in Figure 1.
WABS EAI
With the proliferation of devices and clouds, businesses and developers are challenged more than ever to both enable employee productivity and take advantage of the cost benefits of cloud computing. The reality however, is that the vast majority of organizations are going to continue to invest in assets that reside both within their own data center and public clouds like Windows Azure and Amazon Web Services.
This creates two very real challenges and corresponding opportunities.
First, as companies chip away at workloads that make sense to move to the cloud, it's critical that these externally hosted workloads continue to function as an extension of the organization, as opposed to silos. To facilitate this, it's imperative that new applications, products, and solutions are able to seamlessly integrate with on-premise investments in a secure and cost-effective manner.
It's critical that externally hosted workloads continue to function as an extension of the organization, rather than new silos.
Second, I would posit that thus far, mobility has had a marginal impact on productivity. The majority of interaction between the user and a device imposes a consumer relationship between the user and the data, information, and events with which she superficially interacts.
This proliferation of devices and clouds introduces an opportunity to revolutionize mobility in the enterprise and beyond by reversing this relationship, making the user a producer, in complete command of her personal and professional world whether in the office or on the go.
However, in order to meet this challenge, you need the ability to do a number of things like:
- Enable Web and mobile clients to interact with assets both on-premise and in the cloud in a secure and simple manner regardless of where they reside.
- Externally communicate securely with assets that exist behind the firewall, like databases, ERPs, and other LOB apps.
- Support the ability to validate messages and translate from and to various message representations while supporting POX, REST, and SOAP.
- Provide the ability to broker, mediate, enrich, compose, and route messages and requests across multiple endpoints in a consistent and simple manner.
WABS focuses precisely on this challenge by introducing the concept of a Bridge. A Bridge is an entity that is hosted on Windows Azure and is exposed securely over HTTP. It supports the ability to receive messages in various formats, which can then be manipulated and routed to any combination of protocols by a set of middleware components that are available to you as a developer to configure, deploy, and manage on Windows Azure.
Figure 2 provides a logical view for how Bridges work. You have sources and destinations to work with along with a series of pipeline components that allow you to accomplish the goals listed above and much more.
Taking a Closer Look at the Bridge
Let's take a closer look at each of the components that make up an EAI Bridge.
Addressing
When you create a WABS environment, you are provided with a unique account name that corresponds to your own Azure account as follows: https://[Your Namespace].biztalk.windows.net.
A WABS environment can support multiple Bridges, which you can think of similarly to the way you address a Service Bus Queue, Topic, or Relay Endpoint. For example, in this article, you'll create a Bridge that is capable of looking up customer information in a database that lives behind the firewall and serving it up to a mobile client. The runtime address for the Bridge is the combination of your WABS service URI along with a relative path to your Bridge. For example, the following runtime address corresponds to a Bridge called CustomerLookupBridge: https://codemagwabs.biztalk.windows.net/default/CodeMag/CustomerLookupBridge.
Clearly, a Bridge provides location virtualization that decouples your client applications from unnecessary details (you'll see some examples of this in the scenario walkthrough shortly) and once you deploy your Bridge to your WABS environment, it becomes addressable from any client or application that has an Internet connection.
Sources, Destinations, and Messaging Patterns
You can send messages to a Bridge in a variety of ways, such as FTP or SFTP. The most common way is to issue an HTTP POST request with a message payload, as you'll see shortly in the sample scenario. In addition, a Bridge can accept a message from an FTP or SFTP source.
WABS EAI Bridges also support a number of destinations including SOAP and REST Web services, FTP, SFTP destinations, Azure Service Bus Queues, Topics, and Relay endpoints along with Azure Blob Storage. In addition, an optional component called BizTalk Adapter Service (BAS) enables you to expose BizTalk WCF LOB Adapters that ship with the BizTalk Adapter Pack and enable read and write operations against SQL Server, Oracle Database, and SAP ERP, to name a few.
Bridges support both one-way and request-reply message patterns via the XML One-Way and XML-Request-Reply Bridge respectively. You can also use the Pass Through Bridge when you want to support other unstructured payload types, or have no need to perform additional manipulation on the message.
You can think of these as pipelines that support the ability to work with the message in a number of stages.
Bridge Processing Stages
When using an EAI Bridge, you have a number of opportunities to manipulate the message while in flight. Most middleware and messaging vendors describe these message mediation steps using messaging and integration patterns such as validation, enrichment, extraction, transformation, and routing, often abbreviated as VETER.
An EAI Bridge supports a number of processing stages. Once a message is received, it can be optionally validated to ensure that the payload conforms to the expected schema. This is typically a good idea to ensure that the message you are expecting complies with the assumptions further down the processing pipeline stopping bad data at the gate.
After optionally validating a message, you can modify the message by changing and/or adding fields (enrichment) by doing additional lookups against other services and data sources. Additionally, you can peek into and extract values from the message while in-flight. This is helpful in supporting compositions for enrichment as well as in aiding with routing decisions.
For example, an order message from a client application may contain customer and order information. The order is submitted to the Bridge, and the Bridge looks up the fulfillment vendor for the order and adds the fulfillment ID to the message, enriching the message for submission to the vendor's fulfillment API.
Taking this example one step further, fulfillment vendors may be selected dynamically based on the customer's account level. In this stage of the Bridge processing pipeline, the account level can be looked up and then used to make a dynamic routing decision for expediting the order for premium customers.
Beyond the domain of writing Web forms and populating lists on a Web page, it becomes necessary to contend with various model representations. For example, the model for a customer or an order on a Web or mobile device will likely be as simple as possible, containing only the necessary information to render the customer and/or order. Even if the fulfillment service's model is equally simple, it's very likely to be different, and thus a mapping from the client's domain model to the service's domain model is necessary. This requires either performing cumbersome mapping at the client/service or decoupling the client from the service by using middleware.
Often, when integrating with LOB systems, databases, and ERPs, the model can be quite complex, requiring significant mapping. A typical pattern for addressing this is to build a canonical schema that models a uniform entity, such as a customer, and then building maps that transform from N client models to the common model. From there, you can reuse transforms on the source and destination that lead to a cleaner and more manageable solution.
If you've worked with BizTalk Server over the last several years, you are intimately familiar with the venerable BizTalk Mapper. The mapper provides a graphical UI that takes the pain out of writing cumbersome XSLT by providing the ability to drag and drop elements onto a canvas and execute small scripts between nodes as necessary.
In WABS, a new mapper is provided for mapping/transformation chores. Although its heritage runs deep with the on-premise version, the new WABS Mapper is optimized for PaaS and actually uses no XSLT at all (though XSLT is supported). Figure 3 shows a straightforward mapping from one version of a purchase order to another, taken from the WABS SDK samples.
Once the message leaves the final processing stage in the pipeline, it's ready to be dispatched to its destination. This can be a one-way or request-response (blocking) exchange. You can also apply simple rules to determine what WABS destination the message should be routed to.
For example, you might have a purchase order that needs to go to the credit card and fulfillment service. Or, based on some work done in the pipeline stage, you may have a rule that looks up the customer's account level and routes premium customer orders to the expedited fulfillment service.
In the rest of this article, I'll focus on pulling many of these concepts together to build a simple but realistic mobile application that can communicate with LOB systems behind the firewall in a secure and efficient manner.
Getting Started with WABS EAI
To get started, you'll need to a Windows Azure account. There are many options for creating an Azure account, including a truly free, no-strings-attached trial account. To get started, go to https://azure.microsoft.com/en-us/ and choose the option that's right for you.
You'll also want to download and install the WABSSDK (see the sidebar for details). The WABS SDK provides the BizTalk Services project template for Visual Studio 2012 and includes a number of samples and walkthroughs.
Creating your WABS Environment
Once you have an Azure account, you'll want to create a WABS environment. There are a few steps involved, including reserving your namespace and configuring your storage, Windows Azure Database, and Windows Azure Active Directory accounts. The sidebar “Provisioning a WABS Account” points you to an excellent step by step tutorial.
Overview of the BizTalk Adapter Service
Once your WABS environment has been created, you are ready to start developing modern, rich messaging solutions. At this point, you can experiment with sending messages to the Bridge and routing them to various destinations, including Azure Service Bus, Blog Storage, FTP, etc. For the sample scenario in this article, I want to take things one step further and enable support for connecting to an on-premises SQL Server database using the BizTalk Adapter Service (BAS).
Once your WABS environment has been created, you're ready to start developing modern, rich messaging solutions.
The BAS is a new capability that ships with the WABS SDK that is installed on-premise to enable new Bridge destinations that support connectivity to LOB Targets, including popular database platforms like Microsoft SQL Server and Oracle Database. BAS also provides connectivity to ERP systems such as Oracle E-Business Suite, SAP, and Siebel eBusiness Applications.
The BAS includes a management and runtime component for configuring and enabling integration with your LOB systems. The capabilities are partitioned into the design-time experience and the runtime experience. At design time, you configure your LOB Target for connecting to your LOB application via an LOB Relay. Built on Windows Azure Service Bus Relay Messaging, the LOB Relay allows you to establish a secure, outbound connection to the WABS Bridge, which safely enables bi-directional communication between WABS and your LOB target through the firewall.
For the purposes of the upcoming scenario, you can think of BAS as a destination that allows you to integrate with SQL Server on-premise.
A Modern Messaging Scenario
As I've discussed, one of the major features of WABS EAI is the ability to expose assets that reside on premise, behind the firewall to applications outside of the data center. To demonstrate just one example of what's possible with WABS EAI, I'll walk you through a simple but real-world scenario.
Consider a mobile Web application that allows a sales person to look up customer, account, and order information on their device, as shown in Figure 4.
Figure 5 shows the architecture for realizing this scenario.
All of the code supporting this scenario is available with the download as part of this article and I encourage you to crack the solution open and follow along.
Step 1: The Mobile Web Client
The Client project in the solution uses ASP.NET MVC 4 to provide a simple mobile Web client that issues a POST to the Bridge via HTTP. The controller passes the Customer ID to a service agent that uses WebClient to acquire an OAuth WRAP token from Windows Azure Active Directory and passes it on with a simple POST request as shown in the snippet below. The full code is provided in the CustomerAgent class located in the Client project that is a part of the SqlBridge solution and is shown in Listing 1.
string response;
WebClient webClient = new WebClient();
webClient.Headers[HttpRequestHeader.Authorization] = "WRAP access_token=\"" +
HttpUtility.UrlDecode(runtimeToken) + "\"";
webClient.Headers["Content-Type"] = contentType;
byte[] uploadData = webClient.UploadData(runtimeAddress, "POST", payload);
Listing 1: CustomerAgent Client Code
using System;
using System.IO;
using System.Text;
using System.Net;
using System.Linq;
using System.Collections.Specialized;
using System.Web;
using Client.Models;
using System.Xml.Linq;
using System.Xml;
namespace Client.Managers
{
public class CustomerAgent
{
public CustomerModel GetCustomer(string customerId)
{
var runtimeAddress = "https://[wabsenvironment].biztalk.windows.net/
default/bridges/CustomerLookUpBridge";
var contentType = "application/xml";
var acsNamespace = "wabs01";
var issuerName = "owner";
var issuerKey = "KEY";
string xml = "<ns0:Request xmlns:ns0='http://SqlBridge.Schemas.Request'>
<CustomerId>'" + customerId + "'</CustomerId></ns0:Request>";
var payload = Encoding.UTF8.GetBytes(xml);
string runtimeToken;
UriBuilder endpointToGetAcsTokenForBuilder = new UriBuilder(runtimeAddress);
endpointToGetAcsTokenForBuilder.Scheme = Uri.UriSchemeHttp;
endpointToGetAcsTokenForBuilder.Port = -1;
runtimeToken = GetAccessControlToken(
endpointToGetAcsTokenForBuilder.ToString(), issuerName,
issuerKey, acsNamespace);
string response;
WebClient webClient = new WebClient();
webClient.Headers[HttpRequestHeader.Authorization] = "WRAP access_token=\"" +
HttpUtility.UrlDecode(runtimeToken) + "\"";
webClient.Headers["Content-Type"] = contentType;
byte[] uploadData = webClient.UploadData(runtimeAddress, "POST", payload);
response = Encoding.UTF8.GetString(uploadData);
XmlDocument doc = new XmlDocument();
doc.LoadXml(response);
XmlNodeReader reader = new XmlNodeReader(doc);
reader.MoveToContent();
XElement customer = XElement.Load(reader);
CustomerModel customerModel = new CustomerModel();
customerModel.CustomerId = customer.Element("CustomerID").Value;
customerModel.ContactName = customer.Element("ContactName").Value;
customerModel.CompanyName = customer.Element("CompanyName").Value;
customerModel.Phone = customer.Element("Phone").Value;
return customerModel;
}
/// <summary>
/// Get the Access Control token for the Service Bus URI
/// </summary>
/// <param name="endpointUri">Represents the End Point URI</param>
/// <param name="issuerName">Issuer name for the Service Bus URI</param>
/// <param name="issuerKey">Issuer key for the Service Bus URI</param>
/// <returns>Access Control token</returns>
private static string GetAccessControlToken(string endpointUri,
string issuerName, string issuerKey, string acsNamespace)
{
string acsAddress = GetAcsAddress(acsNamespace);
return GetAcsToken(acsAddress, issuerName, issuerKey, endpointUri);
}
/// <summary>
/// Get the ACS address from the ACS namespace
/// </summary>
/// <param name="acsNamespace">Represents ACS Namespace</param>
/// <returns>ACS Address</returns>
private static string GetAcsAddress(string acsNamespace)
{
UriBuilder acsUri = new UriBuilder(Uri.UriSchemeHttps + "://" +
acsNamespace + "." + "accesscontrol.windows.net");
return acsUri.ToString();
}
/// <summary>
/// Gets the ACS token for the specified Service Bus URI
/// </summary>
/// <param name="acsAddress">Represents ACS address</param>
/// <param name="issuerName">Issuer name for the specified Service Bus namespace</param>
/// <param name="issuerKey">Issuer key for the specified Service Bus namespace</param>
/// <param name="appliesToAddress">Represents Service Bus URI</param>
/// <returns>ACS Token</returns>
private static string GetAcsToken(string acsAddress, string issuerName,
string issuerKey, string appliesToAddress)
{
using (WebClient client = new WebClient())
{
client.BaseAddress = acsAddress;
NameValueCollection values = new NameValueCollection();
values.Add("wrap_name", issuerName);
values.Add("wrap_password", issuerKey);
values.Add("wrap_scope", appliesToAddress);
byte[] responseBytes = client.UploadValues("WRAPv0.9/", "POST", values);
string response = Encoding.UTF8.GetString(responseBytes);
// Extract the SWT token and return it.
return response
.Split('&')
.Single(value => value.StartsWith("wrap_access_token=",
StringComparison.OrdinalIgnoreCase))
.Split('=')[1];
}
}
}
}
Steps 2 and 3: Mapping and Routing
The EAI Bridge receives the message from the client, mediates it, and performs some mapping and routing. The implementation can be found in the SqlBridge project, which is part of the solution available for download. The SqlBridge is a WABS project template and includes a number of key artifacts, as shown in Figure 6:
- MessageFlowItenarary.bcs: This is an XAML file that provides a designer surface for designing your Bridges. For this scenario, it includes the model for a request-response Bridge, a model for the SQL Server LOB Target and a connector.
- Schemas: This folder includes a request and response schema that I created to be used within the two maps inside the Request-Response Bridge, which I'll describe shortly.
- LOBSchemas: This folder includes schemas generated by the BAS for integrating with the SQL Server Adapter. They are used by the maps.
- Maps: This folder includes two transforms that I created using the mapper. The
RequestToCustomers.trfmfile includes the instructions for mapping the XML request message to the SQL Server Adapter SELECT schema. TheSelectResponseToResponse.trfmfile takes the SQL Server Adapter response schema and maps it to a canonical response.
Double-click on the Request-Response Bridge (called CustomerLookUpBridge). A new designer opens that provides a visual representation of the processing steps required to support a request-response message exchange. As shown in Figure 7, the left side represents the request steps and the right side represents the response pipeline. In this scenario, the shapes depicted in Figure 7 are used.
- Message Types: Allow you to bind the expected request and response schemas located in the Schemas folder of the same project. The Request.xsd schema is bound to the Request Message Type and the Response.xsd is bound to the Response Message Type, as shown in the first pipeline shape shown in Figure 8.
- Transform: Maps the
Request.xsdto theTableOperation.dbo.Customers.xsdschema on the response and theTableOperation.dbo.Customers.xsdschema to theResponse.xsdschema.
The first transform shape (shown on left request side of the pipeline in Figure 7) is very simple and uses the String Concatenate operation to map the CustomerID element from the Request schema to the Query element on the Customers schema. The same operation type is used again to map the “*” string to the Query element.
Once the message is processed by the request map in the Request-Response Bridge, it's transformed to the following message, which conforms to the SQL Server Adapter schema for issuing a SELECT statement:
<ns1:Select
xmlns:ns0=http://SqlBridge.Schemas.Request
xmlns:ns1="trimmed for brevity"
xmlns:ns3=" trimmed for brevity ">
<ns1:Columns>*</ns1:Columns>
<ns1:Query>where CustomerId='ALFKI'</ns1:Query>
</ns1:Select>
The message is now ready to be routed to the LOB destination. Click on the connector and click properties. In the properties pane, click the ellipses next to Route Actions. As shown in Figure 9, you can see that the message will be mapped to the ‘TableOp/Select/dbo/Customers’ SOAP action. This step is important because the Bridge prepares the message as a SOAP message using WCF and sends it over the wire using the LOB Relay that is configured using the BAS on-premise. The routing action would be different for a different WCF service or could be targeting an HTTP service given a request header, for example.
Step 4: Adding a SQL Adapter Destination and Deploying the Bridge
The SQL Adapter consists of two main components that are supported by the BAS as briefly discussed before: The LOB Relay enables connectivity to the on-premise network and the LOB Target accepts connections from the LOB Relay and passes them on to the SQL Server Adapter.
In order to enable the SQL Adapter as a destination, be sure you've enabled the Runtime options from the WABS SDK installer. The Developer SDK only includes the necessary components for developing WABS solutions but doesn't install the BAS (if you haven't done so, you can re-run the installer and add it now). Enabling this option allows you to connect your bridges running in Windows Azure to on-premise LOB systems supported by the BizTalk Adapter Pack (which is also installed automatically for you along with the WCF LOB Adapter Framework).
Be sure to install the following pre-requisites as well:
- IIS 7+ and WAS
- SQL Server 2008 or 2012
- AppFabric 1.1 for Windows Server (optional): While optional, there are many benefits to installing AppFabric anytime you are working with WCF or WF services on-premises.
- The installation process prompts you for key information, including the account to run the application pool that hosts the management and runtime services and a password for encrypting key settings stored by the management service in SQL Server.
If you have issues with the installation, take a look at the following tips which will hopefully save you some time:
- To install the MSI, open a command prompt as Administrator. Navigate to the location of the unpacked MSI and issue the following command:
msiexec /i BizTalkAdapterService.msi /l*vx install_log.txt in
This will ensure that the MSI runs under both Administrator and log details should something go wrong. If it takes you a couple of tries, try these tips:
- If you're installing on an un-joined domain computer, enter the name of your computer (i.e., the hostname) under Domain on the Management Service Application Pool step if you're using the Windows option (recommended).
Don't worry about enabling SSL on the Management Service Site Binding step. As shown in Figure 10, this is optional and will work just fine over port 8080.
As shown in Figure 11, if you right-click on the SQL target (or any available target) you get the option to add a target. This kicks off the configuration wizard for getting your LOB Target and LOB Relay set up:
- On the Connection Parameters screen, specify connection information for the database you want to connect to and select the security option you'd like to use, as shown in Figure 12. These credentials are used at design time for browsing the adapter metadata.
The credentials in this step will be used to create the WCF/Relay endpoints in IIS/AppFabric as well as to connect to the database you specify to enumerate the operations available in the next step.
- The next screen allows you to select the operations that you want to enable. At this point, you're interacting with the SQL Adapter API, which does a great job of enumerating all of the operations supported, consistent with the experience in BizTalk Server, as shown in Figure 13.
- On the Runtime security screen, specify the credentials that will be used to connect to the SQL Server database you specified in Step 1.
- Finally, in the Deployment screen, you're prompted to provide Azure Service Bus Relay connection details as well as the relative URI for the LOB Relay.
This step is critical as it provides the integration glue between the WABS Bridge in Windows Azure and the SQL Server Adapter on-premise.
Note that you need to either have an existing Azure Service Bus Relay namespace set up or hop onto the Azure portal to create one.
- When everything is ready, you'll see a summary of the options you selected, as shown in Figure 14. Click Create to complete the configuration.
If everything goes well, you'll see a Relay endpoint representing the LOB Relay under the SQL LOB Types, as shown in Figure 15. If you have problems with this step, be certain that you're running Visual Studio in Administrator mode.
- Right-click the endpoint and select “Add schemas to SqlBridge”.
This results in the SQL Adapter destination being placed on the designer surface, as shown in Figure 16, along with the required schemas being generated and added to the LOBSchemas folder, as discussed in Steps 2 and 3: The EAI Bridge.
With everything configured, you can right-click anywhere in the designer surface of the MessageFlowItinerary.bcs file and select Validate All" or build the project (Ctrl + Shift + B) to ensure that there are no issues with the model.
If everything checks out, deploy the Bridge to Windows Azure by right-clicking on the SqlClient project and selecting Deploy, as shown in Figure 17.
Step 5: The LOB Relay
As I mentioned before, the LOB Relay is a very critical piece of the solution because it enables WABS to communicate with SQL Server on-premise.
If you open IIS/AppFabric, at first glance, the endpoint looks like any other WCF endpoint. But if you poke around in the Web.config file, you'll find that aside from the connection string to the database, there's not a lot of other configuration.
This is because the WABS team took a repository approach to managing the configuration of the LOB Relays, LOB Targets, and the corresponding operations. It's enabled by storing the relay, target and operation information in a database called BAService that was created when you installed the BAS.
This provides a number of benefits, including being scale- and farm-friendly. Speaking of which, because the LOB Relay is implemented as a WCF service, you benefit significantly by installing AppFabric for Windows Server, (which provides a number of useful features for WCF and WF services. These include hosting, monitoring and enhanced management of the LOB Relay endpoints.
Figure 18 shows the AppFabric Dashboard that provides health information about the service and the ability to drill into any errors or issues without having to write a bunch of configuration.
Step 6: The Northwind Database
Once the request is passed off to the LOB Target from the LOB Relay, the SQL Adapter issues a simple SELECT statement based on the transformation in the Mapping and Routing step of the solution.
The result is returned over a Tabular Data Stream (TDS) to the SQL Adapter, and the SQL Adapter then converts the record to an XML message that conforms to the Customer response schema stored in the LOBSchema folder as generated by the Add LOB Target step. The LOB Target then passes the response message to the LOB Relay for delivery back to the Bridge.
Keep in mind that while there is a lot going on here, the resulting message is still in the context of the original request. All of this obviously happens very quickly, but you always want to conduct load testing to ensure that you're able to meet the expectations of your users. While the cloud can help with scaling out the Bridge, there's no replacement for conducting load and performance testing.
Step 7: Response Processing
Once the message comes back to the WABS Bridge from the LOB Relay, it goes back through the Request-Response Bridge where the second map is executed.
The map transforms the SQL Adapter response to the canonical Customer schema found in the Schemas folder that is now ready to be delivered to the client, as shown in Figure 19.
Step 8: Delivering the Results to the Client
The controller for the client is very lean and simple. It merely listens for HTTP requests and passes the heavy lifting on to the CustomerAgent (you can see the full code listing in Listing 2):
public ActionResult Index()
{
CustomerAgent agent = new CustomerAgent();
var customer = agent.GetCustomer(customerId);
return View(customer);
}
Listing 2: Controller Code
namespace Client.Controllers
{
public class HomeController : Controller
{
public ActionResult Index()
{
string customerId = null;
(Request.HttpMethod == "POST")
{
customerId = Request["CustomerID"];
}
else
{
customerId = "ALFKI";
}
CustomerAgent manager = new CustomerAgent();
var customer = manager.GetCustomer(customerId);
return View(customer);
}
}
}
As discussed in the first step of the messaging architecture, the CustomerAgent implements the Service Agent/Gateway pattern and is responsible for authentication/authorization, preparing the messages for transmission over the wire, managing encoding/decoding chores, and taking care of any mapping that needs to happen when dealing with different message representations.
As you can see in the following code snippet (and shown in the full code listing in Listing 1), the CustomerAgent takes a byte array representing the response and encodes it as UTF8 so that it can be loaded into an XmlDocument.
response = Encoding.UTF8.GetString(uploadData);
XmlDocument doc = new XmlDocument();
doc.LoadXml(response);
From there, you take the XmlDocument instance and load it into an XmlNodeReader so you can use LINQ to manipulate it using XElement, which is very powerful:
XmlNodeReader reader = new XmlNodeReader(doc);
reader.MoveToContent();
XElement customer = XElement.Load(reader);
Finally, you instantiate the CustomerModel and map the fields using the XElement as shown in the following snippet:
CustomerModel customerModel = new CustomerModel();
customerModel.CustomerId = customer.Element("CustomerID").Value;
customerModel.ContactName = customer.Element("ContactName").Value;
customerModel.CompanyName = customer.Element("CompanyName").Value;
customerModel.Phone = customer.Element("Phone").Value;
Management and Monitoring
In Step 5, you looked at how to deploy the EAI Bridge. I want to wrap things up by walking through some helpful features provided by the Windows Azure portal.
The WABS Dashboard is your single cockpit view into the health of your WABS environment.
The WABS Dashboard is your single cockpit view into the health of your WABS environment. It includes a summary of configuration information regarding your namespace and a number of configurable metrics such as messages sent, messages received, latency, CPU time, and memory, as shown in Figure 20.
From the dashboard, you can also scale up your WABS compute units. Be sure to check out the pricing information provided on the Windows Azure website.
The next level of detail is provided via the BizTalk Services portal. In the portal, you can see the entities that have been deployed for both EAI and EDI workloads. You can also get more in-depth tracking information, which is very useful when something goes wrong. Figure 21 shows the Tracking view that provides traces of all of the processes that have executed thus far and allows you to view more details in the event of an error.
Conclusion
Today's messaging scenarios require new ways of thinking and corresponding tools for addressing new challenges brought forth by hybrid-cloud computing.
The growing proliferation of devices and cloud computing introduces new challenges and opportunities for all organizations and developers making integration an imperative for any company that wants to remain both relevant and competitive.
In this article, I provided you with an overview of the need for iPaaS (integration platform as a service) capabilities and provided a walk-through of the first release of Windows Azure BizTalk Services EAI Bridges. Taking a simple, but common challenge of exposing organizational data that lives in the data center to mobile devices and applications that live outside of the firewall, you exposed a CRM database over a rich messaging endpoint enabling a responsive Web application to query the database over a secure internet connection.
In addition, I provided a glimpse into the power of EAI Bridges, which provide true integration and messaging capabilities for addressing validation, enrichment, transformation and routing, allowing you to keep your application and back-end domains clean and separated while focusing on supporting simple message exchanges with minimal friction.
Although this is a first release of the product, it's built on a solid foundation of over two years of development working closely with customers, partners, influencers, and MVPs and is a great first step into the iPaaS space for Microsoft.
I believe that this product provides a glimpse into the future of Microsoft's integration strategy and I look forward to seeing this product continue to grow with the development of additional sources and destinations by both Microsoft and the vendor/developer community at large.
