Microsoft included its T4 generation language in the box in Visual Studio 2008 and added important new features in Visual Studio 2010. Visual Studio 2010 makes generation easier to find and supplies a powerful new feature called preprocessed templates. Code generation lets you automatically create significant portions of your application. It has the potential to

decrease bugs and increase your ability to alter code across your application as needs change. Microsoft’s generation language is T4 and it is included in the box starting with Visual Studio 2008. Visual Studio 2010 makes T4 easier to find and supplies a powerful new feature called preprocessed templates. I’ll show you how to use T4 in Studio 2005 and beyond.

Microsoft’s code-generation tool emerged from the DSL Toolkit. DSL stands for Domain Specific Language and code generation transforms information written in a DSL into a .NET language, which creates the run-time artifact for representing the DSL. While I won’t cover DSL in this article, aspects of T4 are easier to understand in connection with this perspective. When you have time to explore it, the DSL Toolkit provides a nice visual DSL, particularly in Visual Studio 2010. Oslo will provide a textual DSL. This article discusses using the T4 generation language outside DSL to generate portions of your applications.

I’ll show the generation language itself, and then return to cover a few approaches to executing templates and which portions of your applications are most likely to benefit from generation.

Starting Out with T4

T4 stands for Text Templating Transformation Toolkit. If you’re running Visual Studio 2005, you’ll need to download and install the DSL Toolkit to have T4 available. If you’re running Visual Studio 2008, T4 is already on your box, although it’s well hidden. Visual Studio 2010 includes T4 in the Add Item dialog. The T4 language is very similar in all of these versions of Visual Studio. T4 is part of Visual Studio, not the .NET Framework because it’s a design-time tool.

Generation is a transformation process.

Generation is a transformation process. You take some sort of template and some sort of input data and create a new file as output. The input data is called metadata. The output file can be anything-including documentation, configuration, and code files such as C#, Visual Basic .NET, or T-SQL. You can create one or many files depending on the capabilities of your tool. With the default Visual Studio behavior, T4 has limited metadata capabilities and outputs only one file per template.

To create a T4 template in Visual Studio 2005 (after you’ve installed the DSL Toolkit) or 2008, create a text file and save it with a .tt extension in a Visual Basic .NET or a C# project. If you’re working in Visual Studio 2010, select Text Template in the Add Item dialog. Visual Studio provides a default custom tool based on the file extension.

To create your first template, type the following into the editor window:

<#@ template debug="true" #> 
//Hello World

When you save, you’ll find a dependent file created with a .cs extension. The dependent file appears as a child node to your template file. If you can’t see the dependent file after saving, select “Show All Files” at the top of Solution Explorer. You can also right-click and explicitly run the custom tool. The dependent file created from the template above contains the single Hello World comment.

If you’re using Visual Basic .NET, you’ll need to specify the output extension, change the comment character, and then select “Show All Files” in Solution Explorer to find the dependent file:

<#@ template hostspecific="true" #>
<#@ output extension=".vb" #> 
'Hello World

All current versions of Visual Studio offer only a “Notepad” editing experience for T4-no code coloration and no IntelliSense. In Visual Studio 2010, you can download the Tangible T4 Editor from Tangible Engineering. In earlier versions, you can download the Visual T4 Editor (professional or community edition) from Clarius Consulting. Thanks to both companies for offering free versions to get you started.

All code-generation templates interleave code that represents the logic of the template with the output code.

All code-generation templates interleave code that represents the logic of the template with the output code. Good editor support is essential to distinguish these two types of code, especially in the common case where both languages are the same. I’ll use these tools to provide code coloration in the rest of this article.

Building Templates

The Hello World template uses two of T4’s four types of blocks. The syntax <#@ initiates a directive that supplies instructions to the T4 engine.

Text which is not contained in any type of angle bracket syntax is called a text block and is output exactly as written. The Hello World comment in the earlier samples is a text block.

You can add variable data to your output file with the expression block. The offsetting syntax is <#=. The following code snippet adds the current time to the previous C# template:

<#@ template hostspecific="true" #>
// Hello World. It's <#= System.DateTime.Now #>

In addition to adding variable data, templates often need to execute conditional, looping, or other logic. The statement block uses the <# syntax and allows you to include logic. The resulting template begins to look a bit like ASP.NET, which is why T4, CodeSmith, and several other generation languages are sometimes called “ASP.NET style generation languages”. You can use statement blocks for looping and conditional logic:

<#@ template hostspecific="true" #>
<# System.DateTime now = System.DateTime.Now;  #>
<# if (now.Hour < 11) { #> 
// Good Morning World. It's <#= now #> 
<# } else {#> 
// Hello World. It's <#= now #> 
<# } #>

The T4 engine parses your template and creates a TransformText method. It uses specialized methods and StringBuilder’s Append methods to express the intent of your template. Statement and expression blocks become actual code in the TransformText method. One version of the code generated by Visual Studio for this template fragment is shown in Listing 1.

A template whose logic is in Visual Basic .NET can output C# code.

The parser understands both C# and Visual Basic .NET. The default language of the template is C#, so you need to specify Visual Basic .NET when working in that language. Note that there are two different languages involved-the language of the output and the language of the template. These can be different languages, for example a template whose logic is in Visual Basic .NET can output C# code. You specify the language of the template with the language parameter:

<#@ template hostspecific="true" language="VB"#>
<# Dim now As System.DateTime = System.DateTime.Now  #>
<# If now.Hour < 11 Then #> 
// Good Morning World. It's <#= now #> 
<# Else #> 
// Hello World. It's <#= now #> 
<# End If #>

Class Feature Block

Templates are often easier to understand and provide better reuse if they include other subroutines, classes, properties, and members. T4 provides this capability by the fourth block type-the class feature block. You indicate class feature blocks by enclosing them in <#+ #>., T4 outputs the code within the class feature block verbatim outside the TransformText method. Class feature blocks can include additional members and nested classes.

A supporting method can contain logic for specific tasks, such as determining whether it’s morning. An updated version of the HelloWorld template shows how you can include a supporting method in a class feature block. Class feature blocks are important in providing reuse across templates:

<#+ 
bool IsMorning(DateTime now)
{ return (now.Hour < 11); }
#>

I’ve summarized the available block types in Table 1.

Directives

Directives instruct T4 how to build the file containing the TransformText method and how to output your application code. Directives do not become part of the interim code-in other words, they don’t become part of the TransformText method.

Like any other .NET file, you can include .NET namespaces you’d like considered in resolving symbols. In C# this would be a using statement; in Visual Basic .NET it would be an Imports statement.

The TransformText method sits in a file created by the T4 engine. Like any other .NET file, you can include .NET namespaces you’d like considered in resolving symbols. In C# this would be a using statement; in Visual Basic .NET it would be an Imports statement. A few namespaces such as System are always included. You can include any other namespaces with the import directive:

Templates often call members of classes contained in other assemblies, either in your own supporting code or .NET assemblies. You can make assemblies available to your template code using the assembly attribute.

The include directive incorporates the contents of additional template files at the location of the include statement. This allows you to build templates as reusable blocks. This ability to compose templates is a very important capability of T4:

Together class feature blocks and include directives let you create complex generation systems entirely in T4, although you should consider maintainability. You can use text, statement, and expression blocks in include files. T4 places the file contents at the point of the include directive. Include files can also contain include directives, allowing you to reuse common directive patterns or have include files incorporate additional include files to any depth.

I’ve summarized available directives in Table 2.

If you’re using the Clarius editor in Visual Studio 2008, you’ll find another directive listed called “property”. If you use this directive with the techniques discussed so far, you’ll receive this cryptic error:

No processor was specified for a directive named 'property'. The transformation will not be run.

The Clarius editor includes the property directive because it’s an important feature of a T4 engine/host that was released as part of GAX from the Patterns and Practices team at Microsoft. However, the default Visual Studio T4 engine/host does not recognize the property directive, so you get an error.

The ability to add custom directives through custom hosts is one of several extensibility points of T4.

The ability to add custom directives through custom hosts is one of several extensibility points of T4. I’m skipping the mucky details of how to create a custom host. The primary reason you would write a custom directive would be to provide data to the template.

The DSL team added a parameter directive at least for some scenarios (preprocessed templates) in Visual Studio 2010 Beta 2. The parameter directive will be very similar to the property directive of GAX.

Executing Templates

How exactly does a template run to generate code? The answer is-it depends. If you’re running in a separate graphical or command-line build tool, it depends on that tool. If you’re running inside Visual Studio, it depends on the custom tool. Visual Studio supports a concept called “Custom Tool.” When a custom tool is present, it runs on save and when explicitly called.

Visual Studio 2008 Behavior

Visual Studio 2008 supplies the TextTemplatingFileGenerator custom tool by default for all files with a .tt extension. The custom tool transforms your template into executable code, which includes the TransformText method. When this method runs, it creates a string containing your output, runs that code, and then sends the result to a single file. This final output file is the dependent file you see in Solution Explorer.

Note that T4 creates and compiles two things during this process. The first is temporary code, which is your template transformed into C# or Visual Basic .NET including the TranformText method. Because your output file is contained within the Visual Studio project, it is also the second thing compiled. Thus you can get a compiler error from either of these two sources, as illustrated in Figure 1. You can tell which file produces a particular error by checking the file name in the Error window.


Unless you create your own host or use a third-party host, you can’t explore the interim code in Visual Studio 2005 or 2008.

Because the template runs in the same AppDomain, if you make changes to an assembly referenced by a template, you must reload to reflect your changes. Also, for Visual Basic .NET projects, at least in Visual Studio 2005 and 2008, you must set the Option values at the project level because you cannot set them within the template. It does not appear that Visual Studio 2010 will address this issue.

If you name your file .t4 instead of .tt, the editors will recognize the file as a template file, but the default custom tool will not be applied. In Visual Studio 2008, this is helpful if you want to run your template from a generation harness.

Visual Studio 2010

Visual Studio 2010 incorporates two new T4 template types in the New Items dialog: Text Template and Preprocessed Text Template. Selecting Text Template gives you similar behavior to Visual Studio 2008. The dependent file is the final output of the template.

The new Preprocessed Text Template also provides a dependent file, but instead of the final output, it contains the interim code that creates the final output-meaning it includes the TransformText method. This class can act as a partial class, so you can add any additional features to your template. This immediately means that you can move features from class feature blocks into your partial class where you have full design time and debugging support of normal Visual Studio code.

T4 generates code required by the template into each preprocessed template file.

Microsoft is working out details of exactly what the interim file will look like in Visual Studio 2010 during the beta cycle, but a few things are clear. This class will have the capacity to work within a larger infrastructure, and allow you to define your own base class. To provide this flexibility, T4 generates code required by the template into each preprocessed template file. Avoid having any partial class code rely on non-public details of the preprocessed class as its details and the concept of preprocessed templates in general may evolve in future versions of Visual Studio. If you define a base class, T4 expects it to fulfill these tasks and outputs only the TransformText method.

There are several major benefits of preprocessed templates. At design time you get immediate feedback for any syntax errors within statement, expression, or class feature blocks. You get to see the actual code behind these errors, and these issues are completely isolated from any compile errors in your output code. You can open the preprocessed template and have your normal editing environment to solve a particular problem or use code snippets, although you must remember to immediately copy your fixes back into the .tt file. Finally, when you run your template to create output, you debug in a familiar environment, although you then need to find the corresponding location in your template to fix. I love to debug in preprocessed templates because I’m confident of exactly what’s going on.

Another major benefit of preprocessed templates is that your template is embedded in the broader context of your code. You have a partial class available and you can easily define an unrestricted base class and implemented interfaces. (You can provide an alternate base class for non-preprocessed templates, but it must derive from a specific base class and it’s more difficult to use).

Executing Outside Visual Studio

Preprocessed templates must be executed by some external harness because Visual Studio does not execute the interim code to produce final output.

Before considering this a problem, let’s consider the aspects of Visual Studio custom-tool generation you might want to do differently:

• Outputs to a single file
• Puts all output in the template assembly
• Hides application code as template dependent
• Offers no capacity to control overwriting
• Allows no data to be passed to template
• Provides no logging or tracing system

You can overcome most of these issues with non-preprocessed templates if you create a generation system in base classes and class feature blocks, including alternate outputting mechanisms and reading data from configuration files or other mechanisms. This is the approach taken by the T4 Toolbox available on CodePlex.

I find it preferable to have an external harness running from a command prompt that can be incorporated into build scripts.

T4 was designed as a transformation tool to output code. I think combining this task with the myriad of supporting tasks required for code generation, including mechanisms for grabbing metadata, outputting files, scripting what types of files are created, logging, project file management, loading databases, and other tasks is a separation of concerns issue. I find it preferable to have an external harness running from a command prompt that can be incorporated into build scripts. I’ve got one free and available at www.AppVenture.com in the blogs section. This harness is based on MEF to independently collect input metadata, templates, and output mechanisms. I discuss this in an article in the May 2009 edition of Visual Studio Magazine (http://visualstudiomagazine.com/articles/2009/05/01/visual-studios-t4-code-generation.aspx).

You can also run non-preprocessed templates through an external tool.

Ecosystems, Extensibility, and Generation

The harness tool I use embraces the fact that problems around code generation are open, meaning that I cannot predict what you will want to do with code gen. Any closed tool, including the non-preprocessed template approach and any harness which provides non-replaceable features, will become a hindrance at some future point in your evolution with code generation. For example, if you are offered no naming support, you’ll waste a lot of time managing plurality and other naming variations. However, I can almost guarantee that at some point in your project development the rules anyone else writes will not fit your scenario.

If we move to open composable tools, the harness simply kicks off the process and manages an environment where it can find parts requested by individual templates. In .NET today, you can achieve this kind of composability using an MEF ecosystem. The harness manages the ecosystem, finds and executes all items with a particular interface, and then manages looping because that’s a bit tricky.

Returning to the naming problems-I supply a set of interfaces and simple implementation for my perception of U.S. English to get you started. When you need something more sophisticated, you simply provide an alternative implementation of the interfaces by deriving from my class or starting over.

I don’t care what interfaces you use, as long as your templates anticipate them.

But an open system takes this concept one step further. I don’t care what interfaces you use, as long as your templates anticipate them. You are free to solve problems in entirely new ways, and you are free to solve entirely new problems. That’s an ecosystem approach that illustrates the power of MEF.

The harness is responsible for managing the ecosystem, including managing prioritization. If I give you an empty slate, you have too much work to do to get started. However, you may need to override anything in the system. To allow this, everything provided as a starting point with the tool has a priority of negative one. If you create parts with a higher explicit priority or you don’t provide a priority, your part wins and is used. Prioritization is a custom extension of MEF.

It often makes sense to run template processing in groups, and it’s helpful to define a few things like project directory structures in a configuration file. For this reason, there is a small set of configuration anticipated by the harness. The default loads a configuration file, but the open nature of the harness allows you to replace the configuration mechanism as well.

Other than the entry point interface, the harness has no demands, so you can replace the entire system. It’s significant that the core entry-level interface is called IProcessWrapper. This naming acknowledges that you might have processes that run database scripts or modify project and solution files. But it’s also a reminder that you can do anything where an ecosystem approach makes sense.

Parameters

You must pass data into your templates. Code generation is rather useless without metadata. But it’s a bit problematic to supply metadata to a template sitting inside Visual Studio. So, at this point generation diverges into one of four directions:

If you’re working with DSL, you have metadata available from your model.. If DSL fits your needs, I’d suggest the Addison-Wesley book Domain-Specific Development with Visual Studio DSL Tools by Steve Cook, Gareth Jones, Stuart Kent, and Alan Cameron Wills.

For Visual Studio 2008 and below and with non-preprocessed templates in Visual Studio 2010, one approach is to inherit from a custom base class. This requires installing the Visual Studio SDK to access the Microsoft.VisualStudio.TextTemplating DLL (in Visual Studio 2010 Beta 2, this requires you download the DSL SDK). Your base class can access any metadata your base class makes available.

Another approach is to trick Visual Studio into doing what you want it to do. Since the template becomes code, it can include any other code. This approach and the base class approach are documented on Oleg Sych’s blog http://www.olegsych.com/ and supported by the T4 Toolbox on CodePlex.

With the MEF ecosystem approach, templates are parts that can simply request metadata via MEF for parameters defined in the template.

Where is T4 Effective?

Generation is most effective where there are a large number of files that follow similar patterns. A great deal of CRUD behavior, including stored procedures, meets this criteria and it’s a common initial point to explore code generation. Other areas include scripts to create databases, business objects, validation, and authorization. User interfaces are more difficult to generate because of variations caused by different platforms, but you can often generate common forms over data screens, such as utility screens.

Both DSL and code generation push us into new thinking about how we define our applications. You currently define your application in language- and technology-specific code. Keeping business logic isolated is nearly impossible. DSL, code generation, and run-time metadata driven systems, such as NHibernate, push us toward a new level of abstraction. But this jump won’t be as easy as the jump into third-generation languages, because there are about a dozen core metadata sources each probably a separate DSL. These include workflow, UI, authorization, validation, data mapping, database design, rules engines, services, and configuration.

The good news is that you can get started today. Play with T4 on your own, download my tool from AppVenture.com, buy a commercial tool, or explore the T4 Toolbox on CodePlex.

At the detail level, templates are easy to construct. They use a handful of language elements, rely on a familiar language, and have access to the power of the .NET Framework. The challenge is expressing full architectures at the detail level reflecting the sophistication and variability we have come to expect in handcrafted architectures.

public virtual string TransformText()
{ 
  System.DateTime now = System.DateTime.Now;  
  if (now.Hour < 11) { 
    this.Write(" \r\n// Good Morning World. It\'s ");
    this.Write(this.ToStringHelper.ToStringWithCulture(now));
    this.Write(" \r\n");
 } else {
    this.Write(" \r\n// Hello World. It\'s ");
    this.Write(this.ToStringHelper.ToStringWithCulture(now));
    this.Write(" \r\n");
  } 
  return this.GenerationEnvironment.ToString();
}