If you're serious about testing before long you'll bump into the need to do some mocking. You can of course chose to write your own mocks but that is hardly time effective or trivial.  More than likely you'll want to use a framework, but how do you choose between the most popular options?

There's been some discussion on Twitter between myself and Daniel Cazzulino (@kzu), the creator of Moq (pronounced "Mock-you" or just "Mock"), a newer and popular .NET mocking, about mocking frameworks and what goes into choosing a mocking framework.  Daniel and I share a passion in mocking and by extension testing.  As the creator of Moq, I hold him in very high regard.

The discussion centered around some work that Andrew Kazyrevich has been doing recently.  Andrew is passionate about mocking.  As such he's created an open source project which compares the "big dawgs" of .NET mocking frameworks.  In his introductory post he explains the intent, which is message I support and want to help spread:

I've started a small open source project. It provides a unified set of tests written against Moq, NMock2, Rhino Mocks and Typemock Isolator, so that you can easily compare the frameworks and make an informed decision when picking one up.

In Andrew's most recent post he compares the execution time of the various frameworks.  My response on the RhinoMocks mailing list was terse but sincere:

Curious about speed in your tests, is the a first level concern when choosing a testing framework?

Daniel answered the question in a tweet:

IMO, API design, usability and expressiveness is a much more valuable comparison. #moq is not optimized for runtime perf.

To which I responded:

I would also throw in documentation, discoverability, and community into your equation.

So, fine blog reader, I have a question for you.  If you are using a mocking framework, what are your first-level concerns when you chose your framework?  For those who have yet to utilize a framework, what will you base your decision on when you do choose?

• Execution Speed
• API Design
• Usability
• Expressiveness
• Documentation
• Discoverability (possibly coincides with API Design)
• Community
• Other?

1 Thessalonians 5:21

• "Test everything. Hold on to the good." (NIV)
• "Prove all things; hold fast that which is good." (King James Version)

Shhh...don't tell anyone but we don't unit test all of our code.  We're striving to get all developers and managers on board with unit testing but we're not there yet.  Despite our delinquency in writing tests one thing we do try our very best to do is keep mistakes from happening again.  If we find a bug in the system, before fixing it, we go and write a unit test for that problem.  We verify the test fails, the go and write the code that fixes it.  This recognizes that bugs will appear in the system, but once fixed we'll insulate ourselves from that bug happening again.

We had one such case last week.  We were getting a yellow-screen error after a bunch of commits by developers.  The error stemmed from a component in Windsor that did not have all of it's dependencies met.  In other words, we registered a component which relied on another component which was not registered.  We already had written a test to compile and test our Windsor configuration, however that test was not mature enough and needed some tweaking. 

In our application we leverage the new ASP.Net MVC.  We also take advantage of the ability to leverage an IoC container, Windsor, to create controller instances when a web request comes in.  When the application starts, we register all of the controllers in the application into Windsor using reflection.  Therefore our Windsor configuration at runtime is made up of two things, the XML configuration and the controllers which are registered on startup.  The "bug" here was that one of our controllers had a dependency on it that was not registered in Windsor.  When you ran the test by itself, the test would pass, since it was only loading up the components from the Windsor configuration.

Wanting to protect us in the future, we acknowledged there was something missing from our test and set out to find it.  To create the failing test was relatively simple.  All that we needed to do replicate in our test what we were doing at runtime.  Therefore we needed to register all implementers of IController, just like we do on application startup.  Below I've

Before:

   1: IWindsorContainer container = new WindsorContainer("windsor.config.xml");

   2:  

   3: foreach (var handler in container.Kernel.GetAssignableHandlers(typeof(object)))

   4: {

   5:     Console.WriteLine("{0}\n\t- {1}", handler.ComponentModel.Service, handler.ComponentModel.Implementation);

   6:     container.Resolve(handler.ComponentModel.Service);

   7: }

After:

   1: IWindsorContainer container = new WindsorContainer("windsor.config.xml");

   2:  

   3: var assm = new HomeController(null).GetType().Assembly;

   4: //add all controllers

   5: foreach (Type type in assm.GetTypes())

   6: {

   7:     if (typeof(IController).IsAssignableFrom(type) && !type.IsAbstract)

   8:     {

   9:         container.AddComponent(type.Name.ToLower(), type);

  10:     }

  11: }

  12:  

  13: foreach (var handler in container.Kernel.GetAssignableHandlers(typeof(object)))

  14: {

  15:     Console.WriteLine("{0}\n\t- {1}", handler.ComponentModel.Service, handler.ComponentModel.Implementation);

  16:     container.Resolve(handler.ComponentModel.Service);

  17: }

Now that this test is in place, I am much more confident that we won't see the error we were seeing in product again, we've got a test which warns us if something isn't quite right.  Our initial test didn't quite get it all right.  Mistakes/bugs in code will happen, hopefully few and far between, but when a bug does show up, go write a test that verifies the bug before going off an fixing it.  Then make the test pass, which should fix the bug.

This past weekend I attended the Continuous Improvement in Software Development Conference in Austin, Texas.  This conference brought together many brilliant minds in the .NET community for the expressed purpose of continuous improvement.  The conference was very good, but some of the best chances for learning happened outside the conference sessions at dinner or sitting around a table at the hotel bar.  One such discussion seemed to point out one problem we have in our community, minutiae.

I will preface the remainder of this post by saying I tend to be a purist at times and must always remind myself what the ultimate goal is.  Acknowledging my tunnel vision, sometimes purists, can miss the point if we're not careful.  One evening a few of us got on the discussion of test-driven development.  Someone at the table used the phrase test-first development in the conversation.  That person was quickly halted by others at the table because the terms, in their opinion are quite different and should not be used interchangeably.  If I were to ask you define both, could you adequately define the difference?  Should you be able to?  Does it really matter?

I would say that for the development community out there that the differences, if in fact they do exist, don't matter.  A quick and informal poll revealed that 8 of 10 developers I know aren't unit testing at all.  I readily admit that my sampling is probably an inaccurate representation of the total developer population.  I don't think though that my sample poll is off by an enormous amount.  Think of the developers you know; are they testing at all?  I'm not talking TDD, but plain old unit tests?  Chances are good that you know a few that aren't testing at all.  Are they served by the bantering back and forth about differences between TDD and TFD?  Absolutely not.

The important thing to remember is that fundamentally the TDD and the TFD practitioner believe the same thing, that is, you need to be testing your code. They may disagree about small things, but those small things should take a back seat to the fundamentals.  We don't have to look far to see another example of this type of behavior.  Think of two Christian denominations, both of which are trying to convert nonbelievers to Christianity, bickering among themselves about whether children should be baptized or not.  The debate becomes so ardent, that neither camp accomplishes their original goal or converting nonbelievers because they're too focused on each other.  I fear the same outcome will befall those of us preaching testing.  We get so religious about the non-essential aspects of testing that we miss the people we intended to target, the people who aren't testing their code at all.

I think we could all take a page from St. Augustine who had it right over 1600 years ago: "In essentials, unity; in non-essentials, liberty; in all things, charity."

I want to invite any of you who are within a few hours of Des Moines, Iowa to attend the second Iowa Code Camp on November 8th.  The first Iowa Code Camp back in May was a huge success. From Derik's write-up:

Yesterdays event was awesome.  They had about 125-150 people show up for the first ever Code Camp in Iowa.  The venue could not have been any nicer and setup any better.

I will be giving one presentation on using RhinoMocks and one workshop on TDD.  Below are the abstracts.  If you have the chance, take part, and register today.

TDD: A Workshop in Driving Your Design with Tests

If you've heard about Test Driven Development (TDD) and wondered what it was or how to do it, then this workshop is for you.  We'll take a practical, introductory approach to getting started with TDD.  We'll introduce fundamental object-oriented design principles including separation of concerns, dependency injection/inversion, and more.  This will be a hands on lab, so bring your laptops and a copy of Visual Studio 2008 and expect to learn.

Easing your Testing With RhinoMocks

When learning about testing you'll see trivial examples illustrating how to write tests. However most production code is non-trivial, making calls to configuration files or to a database which makes testing in isolation hard.  The use of mock objects allows you to isolate code you want to test by providing fake objects to your methods, allowing you to set up complex scenarios to test specific conditions.  In this session we'll first dig into some code that is not very testable and refactor it to make it more testable.  After the refactoring we'll use and explore RhinoMocks to see how we can test different scenarios in our code and verify our code is working as it should.

Often times I get the sense that others think I'm a nut for testing.  There are plenty of others out there who are far more dogmatic about testing than I am.  In fact right now I'd be ashamed to share with you the test coverage of our current project.  I will say that the old codebase from which we are migrating had 0% coverage and I'm proud to say, confidently, that we're beating that number handily.

Testing, at it's very core, for me, is about ensuring correctness.  Whether you're testing using an automated testing framework such as NUnit, MbUnit, or xUnit or you test by hitting F5 and running your project, you're seeking correctness.  For the moment forget words such as "Unit Test", "Integration Test", or "System Test".  Those terms only indicate different scopes on which you can test for correctness.  Often they confuse more than they clarify.  They have their place, just not here right now.

I came across the following code the other day:

   1: public Address GetBillToAddress()

   2: {

   3:     return shippingAddress;

   4: }

See the problem?  Yeah, it was an easy one.  This code, as it sat, was not tested.  The problem with the code above is that it's so simple that it's easy to think it doesn't need to be tested.  Most users use the same billing address as their shipping address, thus masking the error for the vast majority of users.  The danger lies in a simple bug like this masquerading as something more insidious.  For example, this method is part of the "User" class in an eCommerce application.  The application verifies the users credit card by sending credit card information along with billing address to a credit card validation component.  We've already said that most users would have the same billing address as their shipping address.  However, imagine when a user comes by who has different shipping and billing addresses and now the user is being told that their credit card cannot be validated due to the billing address not matching the address on the file with the card.  I see a couple of scenarios happening but two that I want to address specifically:

• In the worst scenario, the user will just go to a different site, knowing their credit card is good and assuming our site is wrong.  We lose the sale and are not alerted to the problem on the site.
• The second, less likely scenario is that the user calls our customer service department to tell us the problem.  The customer service department verifies the problem and gets in touch with the manager of the eCommerce department.  The eCommerce manager verifies the problem as well and gets with one of the developers to find/fix the problem.  The developer then begins to research the problem and may find it immediately or may start looking in the wrong place, the credit card authorization component.

While I'm being a bit overly dramatic here, it's for good reason.  How hard,even if you know very little about testing or are new to automated testing, do you think it is to test the method above?  5 minutes? 10 minutes? An hour?  Compare any of those estimates for putting an automated test in place versus the money lost in the first scenario or the time spent in second scenario.

Too often people look at testing for only the non-trivial functions, for example their custom implementation of the Great-circle distance algorithm.  It's easy to discount testing the simple stuff.  However, a simple bug like this making it's way to production validates the point Steve McConnell makes in Code Complete; the later a bug is found the more it costs to fix it.

If you aren't testing your code, start.  Don't worry about whether or not you're doing it right or not if you're new to testing.  Any test is better than none, despite what many in the business will tell you.  I'd take 10 badly written "unit tests" (that zealots are quick to point out are really integration tests) rather than one well-written one.  Testing isn't an all or nothing proposition.  It's about ensuring correctness.  It's about continuous improvement, being better than you were yesterday.  Testing a bit more than you were yesterday.  Go on, take that first step, write a test or two.

In church last week we were talking about evolution when the term "Irreducible Complexity" came up.  For those who aren't familiar with the concept irreducible complexity is a term coined by Michael Behe to illustrate that complex system could not have evolved and therefore must have been created intelligently.  Behe defines irreducible complexity as:

A single system which is composed of several interacting parts that contribute to the basic function, and where the removal of any one of the parts causes the system to effectively cease functioning. (Darwin's Black Box p39 in the 2006 edition)

In his book Behe uses a mousetrap to illustrate an irreducible complex system. 

"If any one of the components of the mousetrap (the base, hammer, spring, catch, or holding bar) is removed, then the trap does not function. In other words, the simple little mousetrap has no ability to trap a mouse until several separate parts are all assembled. Because the mousetrap is necessarily composed of several parts, it is irreducibly complex." (Behe, 1996).

(This post does not seek to discuss the controversy of evolution vs. creationism vs. intelligent design)

How does this translate to software?  An irreducibly complex software is software that cannot be easily tested.  The "removal of any one of the parts causes the system to effectively cease functioning" really points to dependencies in software.  Do your tests rely on a database being there?  Do you get emails whenever some piece of code runs because as part of the method an email is sent?

Creationist Design

How often have you stepped into some code and have seen a method calling database, sending an email, doing some validation, and then returning some value.  Here's an example I pulled from Jimmy Bogard's post on separation of concerns:

   1: [DataObjectMethod(DataObjectMethodType.Select, false)]

   2: public static List<Customer> GetCustomers(int startRowIndex, int maximumRows)

   3: {

   4:     List<Customer> customers = null;

   5:     string key = "Customers_Customers_" + startRowIndex + "_" + maximumRows;

   6:  

   7:     if (HttpContext.Current.Cache[key] != null)

   8:     {

   9:         customers = (List<Customer>) HttpContext.Current.Cache[key];

  10:     }

  11:     else

  12:     {

  13:         customers =

  14:             (

  15:                 from

  16:                     c in DataGateway.Context.Customers

  17:                 orderby

  18:                     c.CustomerID descending

  19:                 select

  20:                     c

  21:             ).Skip(startRowIndex).Take(maximumRows).ToList();

  22:  

  23:         if ((customers != null) && (customers.Count > 0))

  24:             HttpContext.Current.Cache.Insert(key, customers, null, DateTime.Now.AddDays(1), TimeSpan.Zero);

  25:     }

  26:  

  27:     return customers;

  28: }

The code above will work however it's a bear to test, the code absolutely requires other items (database, HttpContext, ect) to be there in order to work.  I would venture to say that the person who wrote it did not build up the code through small iterations, but rather wrote it all in one fell swoop.

Evolutionary Design

In software, reducible complexity should be a goal.  A reducible complex system allows for pieces to be substituted for others so you can focus on one particular area.  Don't have a database in place, no problem, simply put a placeholder object in for all calls made to the database.  (Yes I'm alluding to mocking).  You can approach creating a reducibly complex system in two ways:

• Create the system as a whole, then reduce it's complexity
• Create the system incrementally

I prefer the former to the latter as I'm working on my TDD skillz.

Conclusion

While the analogies used here are not quite perfect, I do feel they illustrate the point adequately.  As a developer your goal shouldn't be to simply write code.  You should concern yourself with the lifetime of your application/code base.  Irreducibly complex systems, while working, can also have short life spans.  Think about the last time you've ever seen someone fix a broken mousetrap.  All the parts are too interconnected, the mousetrap is simply replaced.  If you're not careful your code may be viewed too difficult and time consuming to fix, and simply replaced.