Norbert Eder

It’s so important to test your software automatically. Unit Tests are really great for testing small, atomic parts. But you have to ensure that your complete systems works as expected. This is where web API tests comes into play.

First, some words about the motivation behind this article: A lot of people don’t want to write test code and one of the reasons most state first is that they have too little time for it. In reality, tests reduce maintenance costs enormously – not only for the development team , but also for support or DevOps team. The most important thing to do is, to integrate your tests into your CI tool to execute them automatically and periodically.

There are several starting points into this topic. Let’s have a look.

Available API specification

Some of you might specify your web API using API Blueprint or Swagger. Both provides a high-level API description language. Those enable you to generate your web API as well as your tests from the API specification.

There are – of course – reasons to create code, but it’s not advisable to generate critical parts of a web API automatically. To create the tests automatically can be a good thing though – certtain limitations apply, which will be discussed further on.

Using tools to document API

Very often you have to work with an existing web API, but got no documentation about it. To get “help” some install tools like Swagger to generate documentation at runtime. This is pretty cool to get an overview of all existing endpoints.

For Swagger you can use Swashbuckle.AspNetCore. To install the package run

dotnet add package Swashbuckle.AspNetCore

To generate tests another packages needs to be added by running:

dotnet add package Swashbuckle.AspNetCore.Examples

This gives you the possibility to add examples for each endpoint, which is necessary to generate the tests. To achieve this, you have to implement several interfaces have to be implemented, and that . Ayields s a result you have a lot of source code to be able to generate tests from your API documentation.

So there are two possible solutions:

• Take the effort and create a specification based on your existing API. As a benefit, your will be able to automatically generate tests from that specification.
• Implement your tests manually.

Existing API without any documentation

Not everyone knows about API specifications, Open API and so on. A lot of developers just write their APIs. You can do that, but you won’t have any specifications, which also makes it harder for third parties (like buisness partners etc.) to your your API.

Of course, you can create a specification later, but doing this upfront saves a lot of effort. By using tools like Swagger, you are able to create at least a helpful documentation of your web API. To generate your tests automatically from this documentation needs a lot of additional source code. That’s not to be preferred.

You have two possibilities:

• Create your own coded testing project
• Script something using tools like HTTPie and bash etc.

Creating a testing infrastructure by code

There are important reasons to also implement testing code manually:

• It’s not enough to just test your API against its specification, you also need to know if your implementation does really work
• Testing APIs also requires to test (business) workflows. This can’t be done by tests that were created automatically for single endpoints.

For my tests I use XUnit. This is – originally – a unit testing framework, but it also provides interfaces for ordering your tests. Ordered tests are necessary to test workflows.

First, you need to implement an attribute to be able to order your tests:

public class TestOrderAttribute : Attribute
{
  public int Priority { get; set; }

  public TestOrderAttribute(int priority)
  {
    Priority = priority;
  }
}

Then you need a mechanism to order your tests within a single test case:

public class TestPriorityOrderer : ITestCaseOrderer
{
  public IEnumerable<ttestcase> OrderTestCases<ttestcase>(IEnumerable<ttestcase> testCases) where TTestCase : ITestCase
  {
    SortedList<int ttestcase="ttestcase"> sortedTestCases = new SortedList<int ttestcase="ttestcase">();
    foreach (var testCase in testCases)
    {
      var methodInfo = testCase.TestMethod.Method;
      var attribute = methodInfo.GetCustomAttributes((typeof(TestOrderAttribute).AssemblyQualifiedName)).FirstOrDefault();
      var priority = attribute.GetNamedArgument<int>("Priority");
      sortedTestCases.Add(priority, testCase);
    }
   return sortedTestCases.Values.ToList();
  }
}

Last but not least, you need to order all existing test cases:

public class TestCollectionOrderer : ITestCollectionOrderer
{
  public IEnumerable<itestcollection> OrderTestCollections(IEnumerable<itestcollection> testCollections)
  {
    return testCollections.OrderBy(GetOrder);
  }

  private static int GetOrder(ITestCollection testCollection)
  {
    var i = testCollection.DisplayName.LastIndexOf(' ');
    if (i <= -1)
    return 0;

    var className = testCollection.DisplayName.Substring(i + 1);
    var type = Type.GetType(className);
    if (type == null)
      return 0;

    var attr = type.GetCustomAttribute<testorderattribute>();
    return attr?.Priority ?? 0;
  }
}

At a single point within your testing assembly set necessary configurations:

[assembly: collectionbehavior(disabletestparallelization = true)][assembly: testcollectionorderer("integrationtests.infrastructure.testcollectionorderer", "integrationtests")]
[assembly: CollectionBehavior(DisableTestParallelization = true)]
[assembly: TestCollectionOrderer("TestCollectionOrderer", "IntegrationTests")]

To configure your test class, set the test case orderer as well as the test order:

[TestCaseOrderer("TestPriorityOrderer", "IntegrationTests")]
[TestOrder(10)]
public class AuthenticationTest { ... }

Your test methods (facts) have to be configured as well:

[Fact, TestOrder(20)]
public async void LoginWithCorrectCredentials_ShouldBeOk()

Alright, this is all you need to run your web API tests.

You will gain the bests results when you combine the requirement that all tests need to run successfully BEFORE new code is commited into your source repository, with also setting up automatic test runs within your CI infrastructure.

Happy coding!