Utilize unit testing in your Visual Studio .NET development.

Automated unit testing enables a team to exercise its entire code base against a battery of tests. This facilitates a quick, reactive environment by providing instant feedback during development. Changes to the code will be tested for validity and any errors will become apparent. Your code will become simpler and you will have great example documentation for using your code.

Since unit tests are used to test individual units of a system, you can ensure that the unit implements the correct functionality and encapsulates the appropriate error handling.

It is rare to find a software development team that does not practice some form of testing. This typically takes the form of a QA department performing system tests at the end of a development cycle, right before a release. Any bugs found during this testing must be fixed and retested before the final release; potentially causing delays and release date slips.

With the increased interest in agile software development, unit testing has taken a more prominent role in the development cycle. Using automated, repeatable unit tests is fast becoming commonplace for many development teams to verify and test their code. Development teams do this as an active part of development, rather than at the end of development. Using an automated testing framework, teams can create a set of repeatable tests for anyone on the team to run against the current build of the system. Through these tests, development teams can verify that new code additions or a change does not break existing code.

In this article, I will introduce automated unit testing in a .NET environment. Many of the concepts described within are not new to Java and Smalltalk developers, however, many Visual C++ and Visual Basic developers, as well as .NET developers, may not be familiar with them.

Before I dive into an example component and its tests, let me address some of the core concepts and definitions for unit testing.

Unit Testing Overview

Unit tests exercise a small piece of functionality within a system. Typically, a test runs individual functions against known inputs, with the results verified against expected results. For example:

Debug.Assert( 2 == someFunction( 1 ) );

The above statement verifies that the output of someFunction() is 2 when passed a value of 1. This is an example of an inline unit test. You usually put inline unit tests inside your code and execute them at run time. The other type of unit test is the external unit test. You run external unit tests during development, usually as part of some build process. They are external in that there is no trace of external unit tests within the actual shipping code. Either a human or a machine can run these external tests.

Unit tests don't just test for successful outputs; they can also test for appropriate error and boundary conditions. The core concept of the unit test is that they appropriately exercise how the code will be used, and what the results should be.

Why Unit Test?

Since unit tests test individual units of a system, you can ensure that the unit implements the correct functionality and encapsulates the appropriate error handling. Units, in this case, can be methods, classes, packages, or even complete inheritance hierarchies.

You can use unit tests to document the correct and incorrect usage of a unit. A suite of tests organized into fine-grained individual tests provides an excellent example of how to use (and not use) the unit. Instead of a somewhat incomplete set of examples that a QA group must maintain, a suite of exhaustive unit tests provides a complete view of a system and its functionality.

Some teams use unit tests as a safety net for the team. When taken as a whole, they comprise a very powerful battery of tests to use whenever you wish. This means that activities such as fixing bugs and refactoring will go more smoothly because any changes that break existing code will automatically cause the tests to fail. Developers receive feedback as soon as the next time they run the tests.

Approaches to Unit Testing

You can test code manually or automatically. There are several benefits and drawbacks to each form of testing, which I will discuss below:

Manual unit testing usually involves a debugging session within an IDE, utilizing breakpoints and step-through debugging.

• Benefits
• Highly visible to current developer.
• Since the code is fresh in the developers' minds, any bugs can usually be fixed rather quickly.
• Drawbacks
• The developer must manually setup any initial state to perform the testing. This can be time consuming and difficult, especially if a complex state is required.
• This type of testing can be error prone and time consuming because the developer cannot test all possible inputs / outputs.
• There is no way to track the results of this type of test.
• There is no way to track the code coverage of this type of testing. This could lead to untested code.
• There is no way to integrate your testing with the testing that is done by the rest of your team. This could lead to changes in your code negatively affecting another team member's code.
• Because developers must run this type of test manually, there is no way to build up a suite of tests and make them repeatable.
• Any external resources used or modified during the test must be manually cleaned up. This could include database records, mainframe records, or temp files on the file system.

Automated unit testing involves programmatically writing tests for your code. Typically you collect these tests into a specific collection of tests called a suite. You can then run the suite of tests against your code using several different means, manual or automated.

• Benefits
• Allows you to build a battery of tests to run your code against at any time.
• Keeps code syntax fresh in your mind by providing comprehensive documentation for your code in the form of examples.
• Provides an easy way to make sure new code changes did not break existing code.
• Changes to the code are easier to make because of automatic feedback and coverage that automated unit testing provides. Any code breaking changes will be known because the tests will fail.
• The entire battery of tests can be run against the code without user intervention. This enables the tests to be run during an external event, such as during the build process.
• Drawbacks
• There is a higher startup cost associated with automated unit testing.
• Even though the benefits of automated unit testing can be seen when implemented by a single developer, to reach full usefulness, the entire code base should have automated tests, not just part of it. This requires that the entire team adopt the practices of automated unit testing. This way, there can be a guarantee of complete code coverage.
• Tools must be chosen to aid in the creation and execution of the automated unit tests.

Despite the drawbacks, automated unit testing provides a clear advantage over manual unit testing. The rest of this article will focus on utilizing automated unit tests within the .NET environment.

Automated Unit Testing with .NET

.NET provides a flexible, dynamic development environment that makes automated unit testing easy to implement. In this section, I'll present a simple example of a unit test. I'll show you how to develop a class to remove duplicate elements from an ArrayList. Keep in mind that this is meant to be an educational and informative example; therefore such things as performance will not be taken into account.

.NET provides a flexible, dynamic development environment that makes automated unit testing easy to implement.

The first step is to choose a testing framework.

Choosing a Testing Framework

Before you can begin to write unit tests, you must choose a framework that you will write your tests in. There are several free, open source options for the .NET programmer to choose from:

• NUnit. A .NET implementation of the popular xUnit framework, originally written by Erich Gamma and Kent Beck, has become a popular choice among developers because of its simplicity, robustness, and availability. It is currently available for most languages, including Java, Smalltalk and C++. The current version, at the time of this writing, is 2.1, although 2.2 is in beta.
• csUnit. Another implementation of the xUnit framework, csUnit is very similar to NUnit. However, at the time of writing, the last release is dated November 09, 2003. I am unsure how active the development on this project is.
• MbUnit. A recent addition to the .NET testing scene. Among other features, MbUnit provides the ability to simplify your unit tests by relying on graphs and model-based testing. An in-depth overview of MbUnit and its features is a topic for a more advanced article, and will not be discussed here.
• Custom. The last framework option for a team is the possibility of building their own testing framework. I advise you and your team to thoroughly evaluate one or more of the above frameworks before you decide to build your own framework. Building your own framework is a complex process and unless there are special circumstances, one of the above frameworks will provide all of the functionality you will need. Another reason to choose an established framework is that it has third-party support. Many of the frameworks above come with infrastructure support, including VS.NET add-ins, NAnt tasks, and GUI test runners.

Given its popularity and simplicity, I'll use the NUnit framework throughout this article.

Getting to Know NUnit

Now I will take you though a simple example using NUnit and Visual Studio .NET.

First, download the latest copy of NUnit. Go to http://nunit.sourceforge.net and download the latest working release, which at the time of writing is version 2.2. Download the Microsoft Windows Installer .msi version of the release. Once you've downloaded the file, double-click the .msi file to install the NUnit framework. The installer offers you a few directory options, so be sure you remember which directory you install it into. Later you will need to set a reference to the main NUnit assembly.

With NUnit installed, start Visual Studio .NET. From the start page, choose New Project and decide which language you would like to create the sample in. I will provide both C# and VB .NET examples. You are welcome to attempt to use another .NET language, such as J# or Managed C++, as an advanced exercise, however, I recommend following along using C# or VB .NET. In whichever language you choose, select the Class Library option, as shown in Figure 1.

Figure 1: The testing framework is available as a new project.

Next, set a reference to the main NUnit assembly, nunit.framework.dll. This assembly contains the core NUnit attributes you will use to markup and create your test classes.

Right-click on References in the project you created and select Add Reference. With the .NET tab selected, click the Browse button. Navigate to the Bin directory where you installed NUnit to. Select the nunit.framework.dll to add a reference to it. Once you've completed the Add Reference dialog box, nunit.framework.dll should be listed in the Selected Components section. Click OK. You now have a reference to the NUnit assembly and you can use it in the project.

Creating Your First Test Class

Now that you've correctly installed and referenced NUnit within the project, you can create the sample class and its unit tests.

Delete Class1, the default class that Visual Studio inserted into the project. Add a new class called ArrayListUtility. The ArrayListUtility class is where the method to test will live.

Inside the new class, import the System.Collections namespace.

In C#:

using System;
using System.Collections;

In VB .NET:

Imports System.Collections

Next, create a new static method that will take in an ArrayList, and return a new ArrayList that is purged of duplicates.

In C#:

public static ArrayList RemoveDuplicateTokens( 
ArrayList 
inputArrayList )
{
return null;
}

In VB .NET:

Public Shared Function RemoveDuplicateTokens(ByVal 
inputArrayList 
As ArrayList) As ArrayList
   Return Nothing
End Function

Notice that the methods currently do not do anything except return null. Why? The unit tests will build up the functionality within this class.

Build your project to make sure that it compiles.

Next, add a new class named ArrayListUtilityTests. This class will holds the unit tests for the ArrayListUtility class. Import the NUnit framework and System.Collections namespace into ArrayListUtilityTests.

In C#:

using System;
using System.Collections;
using NUnit.Framework;

In VB .NET:

Imports System.Collections
Imports NUnit.Framework

Now it's time to add the first test. Open ArrayListUtilityTests and add the following code snippet:

In C#:

[Test]
public void RemoveDuplicatesNoElementsInInput()
{
   ArrayList outputArrayList = 
   ArrayListUtility.RemoveDuplicateTokens( new 
   ArrayList() );
   Assert.AreEqual( 0, outputArrayList.Count, 
   "ArrayList Count" );
}

In VB .NET:

<Test()> Public Sub 
  RemoveDuplicatesNoElementsInInput()
  Dim outputArrayList As ArrayList
  Dim inputArrayList As ArrayList
  inputArrayList = New ArrayList

  outputArrayList = 
  ArrayListUtility.RemoveDuplicateTokens_ 
  (inputArrayList)

  Assert.AreEqual(0, outputArrayList.Count, 
  "ArrayList count")
End Sub

Take a moment to notice a few things about this test method. First, it's decorated with the Test attribute. Using attributes is the means by which you inform NUnit that the method is a test. Second, it is a regular method that returns nothing, as indicated by the void keyword in the method signature. Within the NUnit framework, test methods are regular methods with void return types. The only action performed by this test is to verify that if passed an empty ArrayList, an empty ArrayList comes out. It is a simple test, but it does just enough to get started.

Finally we'll mark the test class as an actual NUnit test class. In the ArrayListUtilityTests class, apply the TestFixture attribute as shown below.

In C#:

[TestFixture]
public class ArrayListUtilityTests

In VB .NET:

<TestFixture()> Public Class ArrayListUtilityTests

Now the test class is ready for its first run. Compile the project and make sure that there are no errors or warnings. At this time, you need to load NUnit in order to run the test class.

Running the Tests

With the test assembly compiled, it is time to run the tests. NUnit offers two ways to run test assemblies: a windows GUI test runner and a console test runner.

NUnit GUI Test Runner

Most developers prefer to use the graphical GUI test runner to run their unit tests during day-to-day development. It provides a graphical representation of your tests, as well as verbose error information in the case of failing tests.

After you installed NUnit, you should have an NUnit GUI 2.2 icon on your desktop. Double-click the icon to open up the NUnit GUI. When the GUI loads up, from the File menu choose Open (or just press Ctrl-O). This opens a dialog box. Navigate to where your test assembly was compiled. Double-click on the SimpleUnitTest.dll to load the assembly into NUnit.

Figure 2: Looking at SimpleUnitTest from within the NUnit GUI.

Your screen should look something like Figure 2.

On the left side of the NUnit GUI, you should see a tree hierarchy of your test assembly. In the top-level element you should see the path to the physical test assembly. For each sub-element, NUnit will display each class that contains unit tests, which NUnit denotes by the TestFixture attribute in the source code. Underneath each class, every test method, denoted by the Test attribute in the source code, will be displayed. Next to each test method, a small circle represents the result of the test method after that method has run. A red circle means a failed test, a green circle means a successful test, and a yellow circle represents an ignored test.

On the right side of the NUnit GUI, you should note the following three sections. The top contains two buttons that allow you to start and stop tests. You'll also see a status bar that displays the status of all the tests in the assembly. If any test fails, the status bar will turn red.

The middle section is where any output from running the unit tests will appear. Each of the four tabs display a different piece of information. The tab of the most interest is the one visible by default: Errors and Failures. This is where any error messages that are returned from the test methods are displayed.

You'll use the last section in conjunction with the Errors and Failures tab. It is where a detailed stack trace for any error selected will be displayed.

Finally, the status bar at the bottom of the screen displays the number of test fixtures as well as the number of tests run. Also located in the status bar after a test run is the number of failures as well as the amount of time it took to run the tests.

The NUnit GUI contains several more advanced aspects including multiple configurations, projects, and categories. However, those topics are beyond the scope of this article.

NUnit Console Test Runner

You can also use the NUnit console test runner to run tests. To use the console test runner, make sure the Bin directory of the NUnit installation is in your path. Then open a command prompt and change to the directory that contains your compiled test assembly. Run your tests by typing the following command:

nunit-console SimpleUnitTest.dll

Listing 1 shows what your output should look like.

As you can see, NUnit generates a large amount of output, even for just one test. In the output you can see the same statistics that the GUI runner reported, including the number of failures and the total amount of time it took to run the tests.

The console runner includes several features that make it optimal for running automated tests from build scripts, such as generating detailed test reports and silent mode. However, because of the highly visible nature of the NUnit GUI runner, I will use it for the remainder of the article.

Running the Tests Using the GUI

For now, simply click the Run button. Your one test should fail, leaving your screen looking like Figure 3. Notice that both the status bar on the right side, as well as the circle corresponding to the individual test method on the left, turn red. This indicates that the test failed, which is not surprising because nothing has been implemented yet. Let's implement the method, as well as some more tests to further exercise the functionality.

Figure 3: A failed test.

Rounding Out the Class

Add the code in Listing 2 (C#) or Listing 3 (VB) to the ArrayListUtilityTests class, below the original test method, RemoveDuplicatesNoElementsInInput().

Now, recompile the project. The NUnit GUI should automatically refresh and load the assembly that contains the new tests. Verify that you can see five tests in the tree view on the left. Click Run and verify that all five tests fail.

Now let's implement the function that will make the tests pass. The implementation is fairly simple and straightforward.

In C#:

public static ArrayList RemoveDuplicateTokens( 
ArrayList inputArrayList ) 
{
ArrayList purgedArrayList = new ArrayList();
   foreach ( object item in inputArrayList ) 
   {
      if ( ! purgedArrayList.Contains( item ) ) 
      {
         purgedArrayList.Add( item );
      }
   }
   return purgedArrayList;
}

In VB .NET:

Public Shared Function RemoveDuplicateTokens(ByVal 
InputArrayList As ArrayList) As ArrayList
Dim purgedArrayList As ArrayList
       Dim item As Object

       purgedArrayList = New ArrayList

For Each item In inputArrayList
       If Not purgedArrayList.Contains(item) Then
           purgedArrayList.Add(item)
       End If
    Next
    Return purgedArrayList
  End Function

If you run the NUnit GUI, all the tests should now pass and the status bar should be green. Congratulations, you have created your first unit tests and corresponding class.

The power of having an automated suite of tests to run against code at any time should now be evident. Although this example is fairly simple, you can now apply these simple concepts to start utilizing unit testing on your own code.

Shortcomings of Unit Testing

No product or process is a silver bullet, and unit tests are no different. Using the unit testing process and frameworks described in this article, you should also consider several shortcomings when deciding how to test your code.

• Can you effectively test code that uses expensive external resources (such as a database, a mainframe, or a fax server)?
• Can you test units that depend on other units?
• How will you depth-test objects?
• Do you know how to verify the internal state of an object without violating encapsulation?
• How can you test against code that has not been written yet?
• What if the tests being written are not enough to completely test the unit? Or what if the tests themselves are not correct?

Attributes of Effective Unit Tests

Writing effective unit tests requires practice. The tests should exercise enough of the possible inputs of a function without testing everything. Here are several attributes of effective unit testing that you should keep in mind when you're writing unit tests. These attributes are similar to the ACID attributes of transactions.

• Atomic. All assertions within each unit test should pass, or the entire test should fail. Within one logical test you may have several different assertions. For example, you might be trying to test an entire process, where changes to several different components occur. You want to verify all changes before declaring the test successful.
• Consistent. Results of a test run should remain the same as long as the inputs remain the same. Sometimes objects can keep track of state internally, when this happens, you should make sure the state remains the same between calls. If any assertions fail when running the same method call over and over again, then that means there are internal properties that are not being cleared.
• Isolated. Tests should be isolated from each other and be able to be run in any order. That means you should be able to test any part of the system without having to set up a separate part of the system. As a concrete example, you should need to run test method testA() in order for a separate test method, testB(), to pass.
• Durable. Tests should be resilient enough so that all other tests run, regardless of previous tests failures. If you have a suite of 10,000 tests and one fails, then the other tests should continue to run, without knowledge of any previous failures. The result of one test should not affect the outcome of any other test. For example, all tests should cleanup after themselves, including removing any files created or any database records added. This can be tested by immediately running your tests twice, just to make sure that there are no residual effects of the tests being left behind.

Summary

Automated unit testing is an essential tool for any programmer. Your code will become simpler and you will find yourself spending less time debugging and more time playing foosball! Additionally, the ability to change and refactor your code while knowing that you have a built-in safety net in the form of automated unit tests provides a huge advantage.

NUnit version 2.1.91
Copyright (C) 2002-2003 James W. Newkirk, Michael C. Two, 
Alexei A. Vorontsov, Charlie Poole.
Copyright (C) 2000-2003 Philip Craig.
All Rights Reserved.

OS Version: Microsoft Windows NT 5.1.2600.0    .NET Version: 
1.1.4322.985

.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.015625 seconds

Failures:
1) SimpleUnitTestCSharp.ArrayListUtilityTests.
RemoveDuplicatesNoElementsInInput: System.NullReferenceException :
Object reference not set to an instance of an object.
   at SimpleUnitTestCSharp.ArrayListUtilityTests.
RemoveDuplicatesNoElementsInInput() in 
arraylistutilitytests.cs:line 14

[Test]
public void RemoveDuplicatesOneElement() 
{
   ArrayList inputArrayList = new ArrayList();
   inputArrayList.Add( "value1" );
   ArrayList outputArrayList = 
   ArrayListUtility.RemoveDuplicateTokens( inputArrayList );
   Assert.AreEqual( 1, outputArrayList.Count, "ArrayList Count" );
   Assert.AreEqual( "value1", outputArrayList[0] );
}
[Test]
public void RemoveDuplicatesTwoElements()
{
   ArrayList inputArrayList = new ArrayList();
   inputArrayList.Add( "value1" );
   inputArrayList.Add( "value2" );
   ArrayList outputArrayList = 
   ArrayListUtility.RemoveDuplicateTokens( inputArrayList );
   Assert.AreEqual( 2, outputArrayList.Count, "ArrayList Count" );
   Assert.AreEqual( "value1", outputArrayList[0] );
   Assert.AreEqual( "value2", outputArrayList[1] );
}
[Test]
public void RemoveDuplicatesOneElementLeft()
{
   ArrayList inputArrayList = new ArrayList();
   inputArrayList.Add( "value1" );
   inputArrayList.Add( "value1" );
   ArrayList outputArrayList = 
   ArrayListUtility.RemoveDuplicateTokens( inputArrayList );
   Assert.AreEqual( 1, outputArrayList.Count, "ArrayList Count" );
   Assert.AreEqual( "value1", outputArrayList[0] );
}
[Test]
public void RemoveDuplicatesTwoElementLeft()
{
ArrayList inputArrayList = new ArrayList();
   inputArrayList.Add( "value1" );
   inputArrayList.Add( "value2" );
   inputArrayList.Add( "value1" );
   ArrayList outputArrayList = 
   ArrayListUtility.RemoveDuplicateTokens( inputArrayList );
   Assert.AreEqual( 2, outputArrayList.Count, "ArrayList Count" );
   Assert.AreEqual( "value1", outputArrayList[0] );
   Assert.AreEqual( "value2", outputArrayList[1] );
}

<Test()> Public Sub RemoveDuplicatesOneElement()
       Dim outputArrayList As ArrayList
       Dim inputArrayList As ArrayList
       inputArrayList = New ArrayList

       inputArrayList.Add("value1")
       outputArrayList = 
       ArrayListUtility.RemoveDuplicateTokens(inputArrayList)
       Assert.AreEqual(1, outputArrayList.Count, "ArrayList Count")
       Assert.AreEqual("value1", outputArrayList(0))
End Sub
<Test()> Public Sub RemoveDuplicatesTwoElements()
       Dim outputArrayList As ArrayList
       Dim inputArrayList As ArrayList
       inputArrayList = New ArrayList

       inputArrayList.Add("value1")
       inputArrayList.Add("value2")
       outputArrayList = 
       ArrayListUtility.RemoveDuplicateTokens(inputArrayList)
       Assert.AreEqual(2, outputArrayList.Count, "ArrayList Count")
       Assert.AreEqual("value1", outputArrayList(0))
       Assert.AreEqual("value2", outputArrayList(1))
End Sub
<Test()> Public Sub RemoveDuplicatesOneElementLeft()
       Dim outputArrayList As ArrayList
       Dim inputArrayList As ArrayList
       inputArrayList = New ArrayList

       inputArrayList.Add("value1")
       inputArrayList.Add("value1")
       outputArrayList = 
       ArrayListUtility.RemoveDuplicateTokens(inputArrayList)
       Assert.AreEqual(1, outputArrayList.Count, "ArrayList Count")
       Assert.AreEqual("value1", outputArrayList(0))
End Sub
<Test()> Public Sub RemoveDuplicatesTwoElementLeft()
       Dim outputArrayList As ArrayList
       Dim inputArrayList As ArrayList
       inputArrayList = New ArrayList

       inputArrayList.Add("value1")
       inputArrayList.Add("value2")
       inputArrayList.Add("value1")
       outputArrayList = 
       ArrayListUtility.RemoveDuplicateTokens(inputArrayList)
       Assert.AreEqual(2, outputArrayList.Count, "ArrayList Count")
       Assert.AreEqual("value1", outputArrayList(0))
       Assert.AreEqual("value2", outputArrayList(1))
End Sub