Have you noticed what a cornucopia of buzz-words the technology world is? It seems that we're in a never-ending quest to put as many words before “-driven development” and “-oriented architecture” as we can. One such term that's been thrown around for the last few years is “microservices.” The problem is that there are a lot of opinions about what they are and how to do them right. A further problem is that many of these opinions and definitions have the correct intent while differing a bit from one another. So, let me further stir the pot by now offering you my opinion. I've always tried to be a realist and not fall into the hype of the new “shiny coin.” I try to concentrate on what can truly bring value to my customers. I've found true value in approaching my designs using microservices to different extents in different scenarios, but I've approached it a little differently from what you may have read elsewhere. I've approached things in a holistic fashion.

Microservices are not really a particular type of service; they're a part of an architecture that makes the services, the act of calling the services, and the clients that call the services, behave a certain way and provide certain characteristics to an application. The more accurate definitions and explanations of microservices I have seen are those that approach it from an architectural perspective. I'm not implying that microservices require a tight coupling with the application that consumes them. But I am saying that microservices are as much about the architecture of a system as they are about the implementation technology and the tools used to create them. The goal of this article is to give you clarity in understanding microservices by way of an overview of their characteristics and that of an architecture that implements them. How the concepts I'm going to discuss are implemented remains a large and complex topic, and one of possibly several potential future articles.

Microservices are part of an architecture that makes the services and clients behave a certain way and provide certain characteristics to an application.

If you're asking yourself, “do we really need another description of microservices,” I'll tell you why I think the answer is yes. This is large topic with a number of different actors and ways to implement those actors. Learning about various points of view has always helped me take an eclectic stance and adopt ideas from more than one scenario. I hope, if you do the same, I'm contributing to that.

Looking at the Big Picture

If you've made the transition from developer to architect, whether in a full- or part-time capacity, the concept of the “big picture” should not be strange to you. As an architect, one of the tasks that falls under my responsibility is to talk to the developers across the entire stack and be mindful of what their needs are, both individually and collectively. How are service boundaries defined? Are services too coarse or too fine? How are services accessed? What parts of a client are dependent on another? How much can fail on the service-side before the entire client-side is rendered useless? These are just a few questions that architects ask when looking at building a new system. In fact, these are important questions when looking at an existing system that may require modification or even overhaul.

So, can microservices benefit the system you're designing? There are many who say absolutely. But you can't really know without having a good understanding of what they are, how they're accessed, and if your application can be decomposed to accommodate them. And then there is the matter of the technology implementation choices you face. This is looking at the big picture.

The “Micro” in Microservice

The biggest misunderstanding about microservices is the prefix the term uses: “micro.” Linguistically, this implies small services. Really small services. That's not exactly the case. Microservices are about orchestrating and decomposing a system into services that are loosely coupled and that encapsulate areas of volatility. When done correctly, yes, this results in smaller services that expose operations in as fine-grained logical groupings as possible. This is because in microservices, the service boundaries are closer to the center, resulting in a more specific responsibility. This is probably where the term “micro” comes from.

A microservice is designed to do a very specific thing and only that thing, and to operate as autonomously as possible.

The decomposition that leads to how many services to have and what they do differs greatly from system to system, as does the size of the service. If this sounds a lot like the good old Service-Oriented Architecture (SOA) that you've read about plenty in the past, that's because microservices have been described as SOA done right. SOA is about breaking up a monolithic system into a set of coarse-grained, somewhat autonomous components and then distributing the hosting of these components, or services, for shared client access. Systems are now separated into areas of responsibility so they can evolve and be deployed independently and not compromise the entire application. This also has the advantage of eliminating quite a bit of responsibility from the client regarding things like database access and details of workflow steps that can now be housed, orchestrated, and exposed by a service layer. On the Microsoft stack, you'd typically use either WCF or Web API to write and expose services. A client can now be a lot “dumber” and need only to know where a service is and how to access it. With WCF, this is done with a client channel (usually crated by a proxy class); in Web API, it uses the HttpClient class. Of course, in the case of the client being pure JavaScript, services are limited to a REST implementation of some kind.

The biggest misunderstanding about microservices is the prefix the term uses, “micro.”

More than likely, I haven't told you anything you didn't already know, so how are microservices different? Well, the immediate answer is that in microservices, the granularity of services gets finer, making microservices an evolution of traditional SOA, but it's about much more. Let's go back to my original statement about microservices being as much about architecture as they are about the implementing technology, and how I approach things in a more holistic fashion. This means I try to think about the entire application and how it will be laid out.

In traditional SOA, you think about services on the server side, obviously. You spec out your service decomposition there, but traditionally the client doesn't take any of that into consideration. It tends to stay as far away from that design as it possibly can, with the expectation that it will be able to use the services later. When architecting a microservice-based system, this can change a little. I'm not saying that the services are designed to cater to a specific client. By no means should that be the case. The decomposition on the server side remains decoupled from whatever clients it services. What I mean is to give thought to how a client is broken out because that becomes important when you start introducing the technology that accesses services; as you'll see, that's a big part of a microservice architecture.

Client Composition

Did you think you'd be reading about client design when discussing services? Let me be clear. I'm not saying that microservices should be designed around their potential clients. I'm saying you should be thinking about them when architecting a complete system that will use microservices. Now, before some of you start yelling loud enough to reach me here in New Jersey, let me elaborate on that statement. If you're talking about the services exclusively, the clients are, for the most part, irrelevant. But I'm talking about a larger picture here, remember? I'm talking about organizing a composite system with regard to both the services and the clients. The relationship here is loose, but it's a relationship nonetheless.

Most definitions of microservice-based solutions discuss fault tolerance to some degree, as I'll also do later. But what about complete failure of a service? On the service side, this may very well affect other services, either crippling them partially or rendering them completely useless. This would obviously affect the client as well, but it certainly shouldn't render an entire client useless. Although there's service decomposition that takes place when the service layer is designed, there's also decomposition of a client so that failure of a service or groups of services manifests on the client in a controlled fashion, making only a part or parts of the client inoperable - hopefully temporarily of course. Note that the decomposition of a client is not particularly a new concept, nor one that is tied to the service-oriented world. This is one of the things that leads to what is sometimes referred to as a composite application.

A Practical Example

Let me put this into perspective using a real-world example in a business sector with which we are all familiar: eCommerce.

An eCommerce system, such as Amazon.com, can have an entire service layer powering it. The decomposition of all the needed functionality resulted in groups of services that handle areas such as User Account, Shopping, and Checkout. In fact, these areas can be further broken into sub-areas, each somewhat autonomous. The User Account area can consist of Account Creation, Login, Profile Management, and management of a user's definition, all specific to this system. In the case of Amazon, this could include the one-click setting, Kindle devices, etc. The Shopping area can be broken out into Product Browsing and Building a Shopping Cart. And even Product Browsing can be split into browsing various product categories. The Checkout area, which, on the surface, may seem like a self-contained area in its finest grained form already, can allow the as-is checkout versus the changing of checkout options, such as credit card or shipping address.

All of these areas, sub-areas, or sub-sub-areas I'm mentioning have their own service or small groups of services that handle the fulfillment of tasks that cover that area's responsibilities. The finer grain I can decompose service-side areas into, the more “micro” my services appear, right? Yes, but that fine-grain decomposition alone doesn't give this eCommerce system a microservice architecture. Some of these services may depend on the availability of others. Some of this dependency may even span areas of composition. An infrastructure is necessary to make these determinations. As I'll talk about a little later, this is one of the key characteristics of a microservice architecture and it's known as discoverability. Now let's shift to the client a little bit.

All these areas, as I've called them, that are driven by services have some kind of manifestation on the client. More than likely it's not a one-to-one relationship. For example, Product Browsing may be a part of the site that is serviced by several microservices. But the temporary failure of a service or a group of services that are needed for Product Browsing to operate shouldn't affect the user's ability to purchase what's currently in the user's shopping cart. And if the service that handles user login is down, the site should be able to accommodate shopping under an anonymous identity. Of course, the Checkout section of the site would be offline until the Log-in service is back up. And then, it would also rely on its own services being available.

Each part of the client must accommodate a failure in accessing the services it calls upon. The discovery facility to which I alluded earlier forms a large part in facilitating this. Discoverability is one of many characteristics of a microservice architecture. This and all the other characteristics I'm going to talk about have more than one type of implementation possibility. There are products on the market, free and otherwise, that assist in providing your architecture with said characteristics. I'm not going to point you to any specific solution in this article. In fact, in many cases I've written custom implementations to accommodate my particular needs. My goal here is to make you aware of characteristics that your architecture and the actors within need to have if you want to design a clean, scalable, and robust microservice-based solution.

Microservice Architecture Characteristics

As you've probably figured out by now, fully embracing microservices is about more than just accessing a smaller service. A system properly architected to be microservice-based needs to look at having certain characteristics, some of which apply only to the services, but others to the system as a whole. Keep in mind that not only does the implementation of each of the characteristics I'll discuss vary, but also a microservice architecture isn't necessarily limited to only these characteristics. In fact, even the depth to which each of these carries can vary among implementations. This list represents items that you find in every well thought-out microservice architecture.

  • Redundant hosting
  • Isolation
  • Dependency checking
  • Service discovery
  • Easy client access
  • Failover and exception management

I'll limit each item to a conceptual explanation as anything beyond it both outside the scope of this article and beyond the room we have. You may find that these characteristics overlap a bit.

Redundant Hosting

Good services shouldn't be limited to one-location hosting. For systems to scale successfully, you don't want the same service in the same process in the same computer being the only point of availability to clients. But of course, hosting the same service from multiple addresses may introduce the question of which one to use. This is going to depend on how your services are hosted. If you're using virtual containers, such as Docker, your addressing may only differ by a host name. Whereas if you're simply hosting services on the same computer but different processes, a port name may differentiate the address. A load balancer may or may not exist in front of a group of the hosted services, or a discovery service may be the differentiator.

There's no exact right answer to this and there's no one solution for how to handle it. Note that with regard to how many services a host should handle, the ideal answer is one. But when looking at the business space to which the service provides value, one service may take a dependency on another. Whether this multitude of services is grouped into a single host or each is provided with a host of their own is going to depend on your environment, discovery facility, hardware, and several other factors.

Isolation

A service or group of services hosted together should be exposed in a fashion that doesn't put any other service or group of services at risk. This can only be guaranteed by isolating their hosting from one another, ranging from process isolation to computer isolation. If the desire is to have computer isolation in conjunction with deployment facilitation, this is where virtual containers can assist. But make no mistake, containers do introduce their own element of complexity.

Containers introduce their own element of complexity.

Dependency Checking

You can't always guarantee that a service can or even should handle all its responsibility internally. This eventually leads to breaking that golden rule of reusability that you've been taught and have been practicing for years. A common task can easily be required by more than one service. Duplicating twice may be fine, three times may be acceptable, but twenty times is unfeasible and blatantly unmanageable. If one service is going to depend on another, the availability of the other needs to be checked. This can and should be handled by whatever discovery solution you choose but also, it shouldn't be limited to the service in need. Ideally, a client shouldn't be able to call a service that has unavailable dependencies. The extent of the dependency can vary. It can be assigned to an entire service or only a particular operation. Such information needs to be provided in some way. Again, you have choices. You can prevent the call all the way to the client level, or allow the call and adjust what gets returned to the client by the called service. How and where service dependencies are defined varies depending on the solution you choose.

Service Discovery

This key characteristic can be implemented in various ways but they all come down to a client not knowing specifically where a service is, and simply having a way to ask for it. This can be as broad as having a way to determine a hosting address, with the client still suppling the details of the call, including the resource (URL) and the payload. But it can also be more fine-grained where a client can ask for a service by a designated label, for example “CheckoutService”. In either scenario, the discovery process handles looking for the appropriate service and returning it to the client. Checks for unavailable services and redundantly provided services should be built into this process and remain transparent to the client. As I mentioned in “Dependency Checking,” a part of service discovery can also be to confirm that any of the dependencies are available as well.

Although many can disagree about what's important in a microservice architecture, it's pretty safe to say that discoverability is at the top of the list. A centralized discovery server is at the heart of all solutions and there are several off-the-shelf solutions for this. All have similarities, including the ability to keep a registry of what services you have and where they are. All other details, like constant heartbeat checking or automatic registering, may differ from solution to solution. How the act of discovery takes place should be exposed to the client in a clean and organized means. That task typically falls to the API Gateway.

Easy Client Access

In a microservice architecture, a client uses an API Gateway to perform a service call. How far from discoverability to the actual operation call an API Gateway can handle varies depending on your solution. An API Gateway is a key component in a microservice architecture because it's the first line of access for the client. In its simplest form, it can receive a request for a service and merely check a repository in order to return the address for that service.

How much it leaves for the client to do on its own or how much it handles for the client differs greatly. A robust and feature-filled API Gateway knows how to identify a service down to details about the operation desired and even other services on which it might depend. A common and key component of a well-written gateway is the ability to handle potential failover scenarios. It should be up to the API Gateway to handle a failure and try an alternative endpoint for a service that either the client or another service requested. It isn't until all avenues have been exhausted that the client should receive a failure notice from the API Gateway.

An API Gateway should also be the first line of security. If a call handled by an API Gateway successfully authenticates, that authentication should be held and passed through to the actual service call. The API Gateway is the part of a microservice solution that's very commonly developed in-house, as its intelligence, more often than not, needs to cater to and interact with just about every other characteristic in a microservice architecture.

Failover and Exception Management

Handling what happens when a service or group of services is down is vital to the continuous operation of an application. This is an area that again makes you look at the application as-a-whole. The down-state of a service should affect only the area of the application that needs that service and ideally not any other. Realistically, this depends on the application. A down-state of an area of a website may impede a particular workflow and prevent usage past a certain point. Knowing this information upon service request should be part of the responsibilities of a good API Gateway, and knowing how to manifest that failure gracefully to a user should be part of the application's overall architecture. A good microservice design includes redundant hosting for every service, ideally on both the same computer and across separate computers. An API Gateway should be able to take a request and simply find the service applicable to it. Selecting it from a series of locations based on availability and/or load balancing can be the responsibility of a really smart gateway or can be a shared responsibility of a gateway and a load balancer.

The API Gateway is the part of a microservice solution that's very commonly developed in house, as its intelligence will more often than not need to cater to and interact with just about every other characteristic in a microservice architecture.

Design for Failure

A lot of these characteristics are centered around a concept known as “design for failure.” I give credit to Martin Fowler and James Lewis on possibly coining this as a software term. Put simply, this is the software architecture's version of Murphy's Law. Applications using a microservice architecture should have the layers above the service layer designed as autonomously as possible, following the similar concept already recommended in services. Again, this in no way means that that the client is coupled with the services, but it's certainly logically dependent on them; after all, services drive the clients. So, as I explained earlier with Amazon as an example, this means that sections of the client should be partitioned in such a way that they depend on each other as little as possible. Failures of any kind from the service layer and down should be gracefully handled by the client. The only real way to ensure this while you're developing an application is to always count on things failing.

When designing for failure, a service needs to know exactly what to do if any part of it fails or if a dependent call to another service fails. Similarly, a client needs to know what to do if a service call fails and restrict as little of itself as it can afford. A failure of a service can easily be because an error in a down-level object call was left unhandled. It can also mean that the database being accessed is offline. We've all seen all these scenarios in applications with all types of architectures, but the isolation you put into a microservice architecture should prohibit failures like this from toppling over an entire service layer or worse, an entire application. This doesn't mean to simply catch errors and tell a user that “the application is temporary offline.” The point here is about failures only affecting a part of an application without prohibiting use of another.

Handling failure in an application properly should have nothing to do with microservices. This is just good architecture and design that every application should consider. A microservice architecture makes it so that failures on the service side don't prohibit usage of the entire application.

Depth of Encapsulation

This is perhaps one of the more controversial topics when researching definitions and descriptions of microservices. It goes to exactly how autonomous a microservice should be regarding its specific tasks and the resources on which it depends. The reason for the controversy is valid, because in a way, it can violate the reusability principles you've been following for a long time. To most people, the encapsulation of a microservice with its resources is one of its defining characteristics. This means that when you talk about a microservice, you're also talking about the objects it calls, and even the database it uses. All of it is considered part of the microservice - or is it?

This isn't an easy one to define, figure out, or even justify. Like many things in the wonderful world of software, it's very easy to define something and say, “this is what it is and this is how you do it,” but you know that in the real world, absolute mindsets don't always work out as planned. Yes, ideally a microservice carries all its dependent resources along-side and as a group, all of it defining its autonomousity (is that even a word? It should be!).

When you talk about a microservice, you're also talking about the objects it calls, and even the database it uses.

Remember those reusability principles I mentioned? Think about it. You have object libraries to reuse. You have databases that span an entire application. You reference utility objects all the time. Well, object references are easy enough to reuse. Each microservice gets its own reference and when deployed, doesn't clash with that of another microservice. The hard one is databases. And there's no easy way around this challenge because there's no easy solution. A dedicated database that services a single microservice (or set of jointly hosted microservices) is easy enough to implement, but what about the data in it that will then be used by another microservice? Again, there's no easy solution here. A common approach is a hybrid solution where an application doesn't have a single database, but it doesn't have one-per-microservice either.

Another alternative is to use database triggers that keep data in sync across databases. This can depend on how your particular workplace feels about database actors such as triggers and stored procedures. Although it may raise an eyebrow when I say it, there are indeed shops that refuse to use these database actors and instead rely on code-based DALs to handle all that. I've also seen dedicated “sync services” that watch and monitor multiple data updates and then syncs them across databases. Having a database professional on your team helps a lot here because a properly designed database is often crucial in choosing your particular answer to this problem. This is also a scenario where services calling other services often happens. Remember “dependencies?”

Microservices Product Choices

So, do all these things that I've discussed fall into the responsibility of the developers to write? If you need service discoverability, do you need to write a discovery hub yourself? What about hosting isolation and deployment? Well, they can be your responsibility. I've certainly written them and with very good reason and specific requirements in mind when doing so. But no, you don't necessarily have to develop every part of this infrastructure yourself. I'm a control-freak and I love to write infrastructure products and plumbing code, but that's just me (#gluttonforpunishment). There are products and services out there that provide you with a lot of this. A number of them are free, and some of them are not. I'm not here to push any particular product, as most of my microservice work has been home grown (I told you I was a control freak).

Discovery

There are discovery products out there, like Microphone and Consul, that allow you to create a registry of your services. I can't speak to their capabilities of self-registering of services that come up and auto-deregistering of services that go down. This is a feature I put into a home-grown discovery hub. In any case, these discovery solutions are designed to become the first point of contact for clients, through the API Gateway. The scope of information that you provide in a discovery service depends on the product and how it was written.

Hosting

There are many solutions for hosting services and for even running service code in various ways. Microsoft Azure for example, has the ability host services in a scalable fashion, allowing you to spin instances up and down as your needs require. Azure also offers Azure Functions, which lets you run code written in a variety of languages that responds to different events or is exposed RESTfully. How you turn a feature like this into part of your microservice solution depends on your architecture. Amazon's competition to Azure Functions is AWS Lambda. These are also event-driven code constructs.

API Gateway

I told you that the API Gateway is a crucial piece in a microservice architecture. Well, if you Google the term, you'll see that everyone has their own implementation idea about this. There are many people who've put their own versions of API Gateways on GitHub. I've even seen API Gateways that integrate the discovery piece within themselves. I don't consider this a good idea because it puts a lot of weight on the client by forcing it to keep the registry of all the services. I've always been fan of writing my own API Gateways; I've never really found any commercial products that cater to the specific tasks of a well-designed gateway, perhaps because it does have custom demands, especially in the area of interacting with a discovery mechanism. If the API Gateway is going to interact with a discovery mechanism, as it should, which discovery mechanism you use is going to steer how your API Gateway works. Some providers give you some help but you have to be using their product stack. If you're hosting services on Amazon, they have a developer's guide for an “Amazon API Gateway.”

What's Next?

Make no mistake, microservices and a microservice architecture may not exactly simplify an application. All these things I've mentioned need to be thought out and discussed. Decomposing an application and distributing its responsibilities isn't a trivial task, but it's a crucial one; when looking at the whole architecture, you can't dismiss microservices in a client either. Designing for failure affects all parts of an application and in every layer, including the client. Seeing an application in a holistic way means understanding not only how the individual components work, but how they may or may not continue to work in the absence of some other component.

It's very common for systems to be built monolithically first, then refactored into a microservice architecture later. There are many who'll balk at this thought, and this isn't advice. I'm merely saying that not every team has the resources, budget, or time to get a first-version product out the door using a microservices approach. And speaking of teams, the breakdown is completely different. Whereas a monolithic architecture breaks out the teams by the layer of the application they control and know best, a microservice architecture's teams have a more vertical breakdown. Each team controls and owns a slice of the application that spans from the client to the database and everything in between.

This doesn't mean that the teams are kept sheltered within their own slice. Any dependencies, however loose, among services or among parts of the client need to be known and communicated on an ongoing basis. And there may be a team or two responsible for some horizontal slices of reusable object libraries that may be shared by the other teams. One particular advantage that's always discussed when you read about microservices is that teams may be developing their microservices using a different technology stack from another team. It's going to be up to you to decide if this is feasible or not. Yeah, on the surface it sounds like a great amount of freedom, but in reality, that approach may not be a value proposition for your environment. It's very easy for some teams to run off and jump on the “new shiny toy” without thinking about whether or not it actually brings value.

One advantage in using microservices is that one team might develop their microservices using a different technology stack from another team.

When discussing the decision to go toward microservices, you need to ask yourself to what level are you going to take it. There's no microservice bible that says, “do it this way only.” In fact, I suspect you're reading this because you've found that there isn't even a single definition of microservices or microservice architecture, and that got you even more confused. Yeah well, it's confused me as well, believe me.

Are all the characteristics I discussed necessary for every microservice architecture? Although it would be nice to have them all, you may decide that “failover handling” is just not important enough to you. I don't mean exception handling, I mean failover. Remember, this is one service going down and another hosted instance picking up the call in its place. Whether your reasons are time, budget, or resources, it's very possible to build an app in a microservice fashion and only have one host for each service. Of course, you certainly shouldn't ignore exception handling in this case, or any case. If a service goes down, you better be handling the problem gracefully all the way up to the client.

What about discovery? You may not have the time to write your own discovery server or the time to even look into and set up a commercial service that gives you one. Maybe you're refactoring parts of a monolithic app into microservices. This, in conjunction with not having failover support may mean that each service has only one address. Your API Gateway wouldn't be asking a discovery hub for a particular service but instead just addressing a service directly. The good thing about a microservice architecture is that some of these characteristics can be added later, so you can prioritize your needs. If you encapsulate all client access into at least a simple API Gateway, you give your clients one point of contact with the services and can modify and enhance the gateway's internals later.

The biggest priority hasn't changed since the days of conventional SOA, so you should start by concentrating on the decomposition of your system and how your services are going to be broken out. Lining up your microservice boundaries by business needs is crucial in your application's success. It affects everything else going forward.

I've given you my views on microservices and microservice architecture throughout this article, and I hope you weigh my information in conjunction with a lot of other good information out there on this topic. Earlier, I wondered if you were asking yourself, “do we really need another description of microservices,” and I rhetorically answered that I thought we did. I sincerely hope that I've added to your arsenal of knowledge on this topic, and if you pulled some food for thought on developing your own ideas, then I guess your answer to my question is also, “yes.”