About the author
Darko Špoljarić
VP of AI
If you’ve ever had to integrate with an external HTTP-based system from another vendor, but they provided only API docs and a production instance, not a sandbox or test environment, this piece is for you.
Wiremock is a fantastic piece of kit. Besides all the imaginable stub mappings you can create with it, it also has an admin HTTP REST interface. It enables setting up all the stubs without any downtime or restarts. That means you can set up a simulated external system simply by invoking some REST endpoints on a blank instance. And if you’re a QA, you can set up Wiremock for one test scenario, and just for REST HTTP call. No restarts necessary.
And while that’s all nice and fine, as soon as you restart an instance where you configured stubs by using the admin REST API, the stub mappings are gone. Also, if you’re using the Wiremock Docker container, the files that you may have uploaded are also gone. So what do we do?
We need some kind of persistence for the mappings and files.
And this is where the Wiremock extension facilities come in. There are about a dozen interfaces you can implement and plug into Wiremock. So what we’re gonna do in this blog is look at the extension options and create custom ones. It will persist the stub mappings into the Mongo database, and load them up on WireMock start. In that way, we’re going to make your new external system stub fully cloud-friendly.
Which Wiremock extensions?
We’ll build two extensions in total, but we’re gonna use three interfaces for that:
StubLifecycleListener
It’s an interface you implement if you want to hook into operations on stubs done via the admin REST interface. We’re gonna use these callbacks to insert, modify, and delete stub mappings into/from MongoDB.
Here’s the code for hooking into the creation of a stub mapping.
import static com.coinme.wiremock.extension.Constants.STUBS;
import com.github.tomakehurst.wiremock.common.Json;
import com.github.tomakehurst.wiremock.extension.StubLifecycleListener;
import com.github.tomakehurst.wiremock.stubbing.StubMapping;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
public class StubPersister implements StubLifecycleListener {
MongoDatabase database;
public StubPersister() {
String connectionString = System.getProperty("mongodb.uri");
String dbName = System.getProperty("mongodb.name");
MongoClient mongoClient = MongoClients.create(connectionString);
database = mongoClient.getDatabase(dbName);
}
@Override
public void afterStubCreated(StubMapping stubMapping) {
String json = Json.write(stubMapping);
database.getCollection("wiremock_stubs").insertOne(Document.parse(json));
}
}Doesn’t get much simpler than that does it? The code for updating and deleting a stub is just as simple (see the complete code in the my GitHub Wiremock Persisted repo).
MappingsLoaderExtension
Now that we have the mappings in Mongo, it would be nice to load them from there once the system starts. Here’s how.
public class StubPersister implements MappingsLoaderExtension {
MongoDatabase database;
public StubPersister() {
...
}
@Override
public void loadMappingsInto(StubMappings stubMappings) {
MongoCollection<Document> stubs = database.getCollection(STUBS);
stubs.find().forEach(document -> {
document.remove("_id");
StubMapping stubMapping = Json.read(document.toJson(), StubMapping.class);
stubMappings.addMapping(stubMapping);
});
}
}Other than the fact we had to remove _id attribute from the Mongo document which is added automatically on insertion – and which would result with an exception on deserialization – the code is as straightforward as it gets.
A Wiremock caveat ⚠️
Now, at this point, we need to account for a particular Wiremock behaviour that doesn’t really suit us. When both MappingsLoaderExtension and StubLifecycleListener are plugged in, the stubs that MappingsLoaderExtension loads will be processed by the StubLifecycleListener.
This is going to result in a stub being loaded, which is what we want, but also with another mapping being inserted into the DB via the stub creation callback, which is what we don’t want.
How to prevent this?
We could check if the database already contains the mapping by their UUID before creating a new one, but a global variable would serve the same purpose. So let’s create the variable and populate it on startup, then check its contents when stub manipulation callbacks are invoked to avoid creating duplicates.
public class StubPersister implements StubLifecycleListener, MappingsLoaderExtension {
MongoDatabase database;
private final List<UUID> persistedStubsAtStartup = new ArrayList<>();
public StubPersister() {
...
}
@Override
public void loadMappingsInto(StubMappings stubMappings) {
MongoCollection<Document> stubs = database.getCollection(STUBS);
stubs.find().forEach(document -> {
document.remove("_id");
StubMapping stubMapping = Json.read(document.toJson(), StubMapping.class);
persistedStubsAtStartup.add(stubMapping.getId());
stubMappings.addMapping(stubMapping);
});
}
@Override
public void afterStubCreated(StubMapping stubMapping) {
if (!persistedStubsAtStartup.contains(stubMapping.getUuid())) {
String json = Json.write(stubMapping);
database.getCollection(STUBS).insertOne(Document.parse(json));
System.out.println("[afterStubCreated] inserted stub maping");
}
}
}Working with files in stubs
The code above works nice and fine, but there’s one particular use case that isn’t covered. Remember how you can reference files when setting up a Wiremock response via the bodyFileName attribute? Like this?
POST http://localhost:8080/__admin/mappings
Content-Type: application/json
{
"request": {
"method": "GET",
"url": "/some-endpoint"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "file.json"
}
}The problem is that to make such a stub mapping, you have to have the referenced file.json uploaded already and residing in the file system (__files folder). On top of that, the admin endpoints for working with files (uploads, deletions) won’t invoke any of the extension endpoints we’ve mentioned so far.
Sounds like a bummer, but let’s take a look at what else is available in the extensions department.
AdminRequestFilterV2
AdminRequestFilterV2 looks full of potential. It’s an interceptor-like (or servlet filter if you’d like) component that invokes every request fired at any admin endpoints. So what we can do is check if the endpoint that’s being intercepted is for file operations, and look at the HTTP verb that’s used, and only react to PUT and DELETE.
import static com.github.tomakehurst.wiremock.extension.requestfilter.RequestFilterAction.continueWith;
import org.bson.Document;
import org.bson.types.Binary;
// other imports ommited
public class FilePersister implements AdminRequestFilterV2 {
public static final String FILES_PREFIX = "/files";
public static final String FILENAME = "filename";
public static final String BYTES = "bytes";
private final MongoDatabase database;
public FilePersister() {
...
}
@Override
public RequestFilterAction filter(Request request, ServeEvent serveEvent) {
if (!request.getUrl().startsWith(FILES_PREFIX)) {
return continueWith(request);
}
if (request.getMethod().isOneOf(RequestMethod.PUT)) {
persistFile(request);
}
if (request.getMethod().isOneOf(RequestMethod.DELETE)) {
deleteFile(request);
}
return continueWith(request);
}
private void persistFile(Request request) {
String filename = getFilename(request);
byte[] bytes = request.getBodyAsString().getBytes(StandardCharsets.UTF_8);
Document fileDoc = new Document().append(FILENAME, filename).append(BYTES, new Binary(bytes));
database.getCollection(FILES).insertOne(fileDoc);
}
private void deleteFile(Request request) {
String filename = getFilename(request);
Document filter = new Document().append(FILENAME, filename);
database.getCollection(FILES).deleteOne(filter);
}
private static String getFilename(Request request) {
return request.getUrl().substring((FILES_PREFIX + "/").length());
}
}Also, we have to load the files onto the filesystem when Wiremock is started because it might be a container reschedule, and the old files are long gone.
import static com.coinme.wiremock.extension.Constants.FILES;
// other imports ommited
public class FilePersister implements AdminRequestFilterV2, MappingsLoaderExtension {
public static final String FILENAME = "filename";
public static final String BYTES = "bytes";
private final MongoDatabase database;
public FilePersister() {
...
}
@Override
public void loadMappingsInto(StubMappings stubMappings) {
MongoCollection<Document> files = database.getCollection(FILES);
files.find().forEach(file -> {
String filename = file.getString(FILENAME);
Binary bytesBinary = file.get(BYTES, Binary.class);
byte[] bytes = bytesBinary.getData();
try {
Path path = Files.createFile(Path.of(WireMockApp.FILES_ROOT, filename));
Files.write(path, bytes);
} catch (FileAlreadyExistsException ex) {
// process restarted, do nothing
} catch (IOException e) {
throw new RuntimeException(e);
}
});
}
}Putting it all together
The way we’re gonna package this is also quite simple. All of the code above will sit in a simple Maven Java project, we’re gonna pull in the wiremock-standalone dependency and a Mongo client. Then we will use the maven shade plugin to make a fat jar, use Wiremock.run as the main class, and use the final artefact in a Dockerfile when creating an image.
Dependencies:
<dependencies>
<dependency>
<groupId>org.wiremock</groupId>
<artifactId>wiremock-standalone</artifactId>
<version>3.9.2</version>
</dependency>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-driver-sync</artifactId>
<version>4.11.1</version>
</dependency>
</dependencies>Shade plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.6.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>wiremock.Run</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>Dockerfile
FROM eclipse-temurin:17-jre-alpine
COPY target/wiremock-db-backed-*.jar /opt/wiremock/wiremock-db-backed.jar
ENTRYPOINT java -Dmongodb.uri=${MONGO_DB_URI} -Dmongodb.name=${MONGO_DB_NAME} -jar /opt/wiremock/wiremock-db-backed.jar --extensions com.coinme.wiremock.extension.FilePersister,com.coinme.wiremock.extension.StubPersister
The nice thing about making a fat jar is that you don’t have to deal with classpath in the Dockerfile, and yes – you can make a fat jar with another fat jar as a base :)
There’s also a Docker standard way of passing the mongo DB information, so you can start the container with -e flags.
> docker build -t my-org/db-backed-wiremock .
> docker run -e MONGO_DB_URI="mongodb://mongo-host:27017" -e MONGO_DB_NAME=wiremock -p 8080:8080 my-org/db-backed-wiremock
Alternatives to Wiremock extensions
When talking about alternatives, the Wiremock Cloud option – is certainly a much better one. You get a very nice GUI for editing stubs and browsing recorded requests, but that comes with rate restrictions for non-paying customers of 1000 requests per month.
And that‘s probably only good enough for local dev testing, not for a staging environment with much more activity. But sure, if you don’t mind paying, Wiremock Cloud is probably the way to go.
Wiremock, when you don’t have a testing sandbox
As I’ve said at the very beginning, this is for teams that have to connect to an external system in a staging environment, but they don’t have a testing sandbox. Usually, SaaS vendors have them, but not all of them, which means testing with a production system in staging, unless you’ve done some other trickery and avoided calls to the external system altogether. Which, on a side note, is only sometimes a good testing strategy.
But the whole beauty of testing with this container is that the same image can be used for multiple external systems, you can even tie them to different databases, and the QA has the complete freedom to set the system up exactly how they want, without depending on a developer.
Sure, you have to learn a bit about WireMock and its admin API, but it’s a very mild learning curve for the most part.
