Quite often when you are developing software you need to take a dependency on other software solutions, and more and more often this dependency takes the form of a web service. This creates problems when you want to start testing your software at greater than a unit test level granularity, as you often have to provide a piece of software to masquerade as the service that you are coupled to if you want consistent and reliable tests.
In this post I want to dig into this topic a little, as it seems that there is no real clear approach to this, no obvious set of practices, and although there are a lot of tools they do vary significantly in approach and capability. Think of this as my structured brain dump on service stubbing or “service virtualisation” (yeah, subtly different and we can argue, but run with it ok?).
First of all, lets look at the primary ways you may want to implement service stubbing or service virtualisation for a web service (although lets not completely limit ourselves here, every time I write “web service” pretend I wrote “some external dependency behind a communication protocol that is defined and abstract enough to be impersonated by some other implementation”).
At the code level
When we implement stubbing at the code level, what we are looking to do is implement specific hooks as part of the code that we are writing. We may choose to put an abstraction in place, but we are really talking hooks at the communication level within our actual implementation that can choose to return from the real service or another, “stubbed” source (this may just be a local xml/json file). This is a really specific implementation, and does not have much in the way of generic reuse other than as a pattern. It also has a strong developer bias, as this is often as simple as using an IoC container or dependency injection.
At the pipeline level
With the pipeline approach, we are injecting into the runtime mechanism without the code specifically being aware of it, although it is probably easier if it is. This may take the form of HTTP headers at request and response points in a HTTP pipeline to insert and inject responses (in IIS you may implement this as HTTP Handlers/HTTP Modules). This approach may have some reuse, but will still be runtime and possibly application/payload specific.
At the proxy level
This approach requires that your client application supports HTTP proxies, and it essentially allows you to provide a third party piece of software that has knowledge of what should and should not be stubbed, and to pass through or stub based on configuration or convention. The benefit here is that you can use the proxy to record responses in pass-through mode, and replay them when required. It also provides a very targeted approach where even small subsets of an API can be stubbed depending on config. Example implementations of this would be FauxPI, which is something I was knocking up to provide some of this functionality and is currently quite incomplete (in all honesty I started writing it to learn Go!).
At the endpoint level
This approach sees us spin up a service that presents as the underlying dependency, which is then addressed directly as if it were our target service. This is the first case where we are actually not relying on anything other than the default client behaviour (pointing at an endpoint) for our stubbing to work. You can also configure this type of server to passthrough and record/replay, but you generally need to know up front what requests you are going to be making on what port so that you can configure a service to respond. There are plenty of examples of this type of implementation, but some of my favourite so far are puffing-billy, Canned and Mountebank.
At the network level
A combination approach to this uses IPTables and a transparent proxy/dynamic-direct services to provide all the benefits of 3. and 4. without the requirement for proxy-aware client software. This also provides for a single point that can be used for introspection, MITM, logging etc that can sit beside your application instances without actually requiring any redefinition of configuration. An example of this would be a Vagrant definition of a three tier application, “web”, “app” and “db”. With the addition of a “routing/stubbing” instance to the Vagrant definition, with all network access routing through it, we can suddenly apply many of the stubbing techniques above without modifying the actual application as we can do this at a network level (albeit there might be some certificate fiddling if we require SSL).
Once we have an approach to the actual service stubbing itself, we need to address the problem of discovery and routing. With a proxy based stubber, this can be quite simple as we only require a proxy-aware client to support stubbing any endpoint/host/url. Where we are using direct service stubbing, we have to be a little more targeted around configuration as we will need to explicitly align configuration with the service supporting the stubbing. There are a few approaches we can use for this as well:
Direct configuration manipulation
Ok, its json/xml/yaml config in your app that is pointing to the endpoint, right? You didn’t hardcode it, right? So…we can always just modify the config to point wherever we instantiate the stub service. This does mean we have to provide configuration transforms for our stubbing, but it is a known problem with a lot of good solutions.
Network environment manipulation
So we typically want the type of testing that requires stubs to occur in an orchestrated virtualisation provider of some sort (et Vagrant/Docker). If we are in that type of environment, we can start manipulating the network to skip over the requirement to modify config by directly injecting things like host file entries to manipulate DNS, or even implement something like SkyDNS/Consul to provide service discovery at the DNS level. Docker makes this particularly easy with its “link” capability, which automates the DNS modifications for us. So you leave your config as it is, and modify what that config resolves to (generally the box that is doing the stubbing).
Via a Proxy
Again, if our client is proxy aware, passing all requests via a proxy allows us to leave the config as is, and stub based on the request URI. This provides for the easiest configuration story, although there are some caveats around connectivity, for instance if you use production configuration values how do you ensure that you cannot impact production (again, solvable via network configuration under virtualisation, but a constraint none the less).
Via Network Gateway
Don’t change the config, don’t set a proxy, but have a network level stubbing gateway as described in 5. above.
Once you get past “how” to stub, then you get into the “what”. REST services are the easiest, and implementations such as FauxPI and Canned provide a good example of how you can easily stub these based on URI. But things can get more complicated. So, what functionalities are we looking for? Well:
Smart or Dumb
A “smart” stub provides the ability to introspect the request and modify the response based on details contained within. Generally this requires a tool to identify the portions of interest within the request (for example XPath/Regex for SOAP) and a ruleset for the transform of a template response based on the request data. The sort of use case for this is where you want to stub a service that takes an int and multiplies it by 4. The response is subject to the request, and for tests to work consistently for anything other than a very simple static set you will need to have some smarts in the stub. There is obviously a line to draw here regards complexity (cough ITKO Lisa cough). “Dumb” stubbing generally just returns a single static response for a particular request or endpoint.
If it is just REST and json, there are a lot of tool options. When you start getting a little more bespoke or interesting (SOAP, RPC, random binary) then you get a little more specific on the tooling. Probably the most general purpose tool that I am aware of is Mountebank.
Stub storage and CD/DevOps Friendliness
Some tools are just designed for large enterprises, central teams and fence-hoisted software (yep, looking at you ITKO Lisa…again…). You probably want to take into account how the stubs are stored, whether they are on disk as readable, manipulable files stored with the code of your application or the application to stub, or in some centrally controlled, managed and secured database instance separate in time and space from any application you are actually working on. The latter just doesn’t fit with my experience of Continuous Delivery or DevOps, and you should probably try and avoid if you can. Keep the stubs close, verify they match the service you are stubbing (and yes you can write test for this) and you will have a much better experience managing you ability to test your software.
Service stubbing used to be considered the cheap and cheerful developer-led approach to abstracting away service dependencies, whilst service virtualisation used to be the one that ITKO or IBM or some similarly large corporation would sell your corporation for a staggering yearly fee. This is blurring based on the capability that open source software is providing, with tools like Fiddler.Core and GoProxy making the entry level to sophisticated stubbing pretty low (super low, I build FauxPI with a few lines on top of GoProxy!). This availability is fantastic for software quality and developer productivity.
Finally, these are just my thoughts condensed on the whole service stubbing/virtualisation topic. There is probably a lot that I have missed. If I have, please drop me a line or add a comment, as I am interested to capture as much on this topic as I can.Tweet