How to test FTP file exchanges with MockFtpServer

An application running in complete isolation is quite rare these days. Often, they integrate with other services to increase the value they deliver. When dealing with legacy applications, I often see the pattern of file transmission via FTP. Because the correct and stable integration of services can be an important part of business it should be tested properly.
In this blog post, I will show you how you can test the file transmission via FTP with MockFtpServer.

What is MockFtpServer?

MockFtpServer is a project that provides two FTP server implementations, that can be used to test your FTP client code.
The first implementation is the FakeFtpServer. It is a higher-level abstraction of an FTP server providing features like users, virtual filesystems, and files and folders. It is useful for most testing scenarios and this blog post will focus on it.
The second implementation is the StubFtpServer. Other than the FakeFtpServer, it provides a low-level-API that allows the configuration of FTP commands.

The Example

Let’s imagine our company has built a custom shopping system a decade ago. While everything is still functioning as intended, the user-facing parts just don’t feel up-to-date. Our Sales Team wants it to be updated. Because we want to reduce the blast radius of our changes, we decided to split the shopping system into two parts. While remaining the order processing as is, we extract the shopping part into a separate deployable artifact with a new tech stack. Because of some infrastructure constraints, we decide to transfer the placed orders as an XML file via FTP to a server, where they get asynchronously processed.

A Deployment diagram of the new architecture. The shop (cloud) and order processing (datacenter) are seperated. The shop writes data to the ftp server (datacenter), the order processor reads it.

Testing the sending of Data

The first thing we want to test is the successful transmission of order files from the shop to the FTP server. Because the transmission itself is not the focus of this blog post, I will just summarize it. We’ll make use of the Spring Integration project. We start by adding the FtpHandler - a MessagingGateway as described in the documentation. Next, we configure the outbound adapter in our FtpConfiguration , as described here. Now we can use the FtpHandler in our OrderService to transfer the generated order XML file to the FTP server.

In our test we will use the MockFtpServer library, to test the successful transmission of the generated file.

OrderServiceIT.java 
@SpringBootTest(properties = {"ftp.port=12021"})
class OrderServiceIT {

    private FakeFtpServer fakeFtpServer;

    @Autowired
    private OrderService orderService;

    @BeforeEach
    void setup() {
        fakeFtpServer = new FakeFtpServer();
        fakeFtpServer.setServerControlPort(12021);
        fakeFtpServer.addUserAccount(new UserAccount("admin", "admin", "/"));

        FileSystem fileSystem = new UnixFakeFileSystem();
        fileSystem.add(new DirectoryEntry("/"));
        fakeFtpServer.setFileSystem(fileSystem);

        fakeFtpServer.start();
    }

    @AfterEach
    void cleanup() {
        fakeFtpServer.stop();
    }

    @Test
    void orderFile_isTransmittedSuccessfully() throws IOException {

        final var customerId = UUID.fromString("add46359-60b0-44c5-b00b-f22367c0533d");
        final var itemId = UUID.fromString("80cea71c-b024-4e84-8c1c-af9580728132");
        final var orderId = orderService.place(customerId, itemId);

        final var orderXml = fakeFtpServer.getFileSystem().getEntry("/" + orderId + ".xml");
        assertNotNull(orderXml);
    }
}
  • We start the complete application context for our test and set the property ftp.port to 12021. This is the port the mocked FTP server will listen to.
  • In the test setup, we instantiate a FakeFtpServer provided by MockFtpServer. We also set its control port, create a user, and a file system and start it.
  • To minimize the chance of side effects we stop the FakeFtpServer after each test. This way each test will start with a clean filesystem.
  • We call the method that creates and transfers the order XML file
  • The FakeFtpServer provides an API to access the content of its file system. We use it to check if a file with our expected name exists.

Testing the receiving of Data

Now that we tested the sending of data to an FTP server, we want to test that the application can successfully receive and process the data. Again, we use Spring Integration for communication with the FTP server. We extend our FtpConfiguration by adding an ApplicationEventPublishingMessageHandler and an IntegrationFlow. The latter polls files from the FTP server, wraps them into custom OrderReceived -Events and publishes them. The event is handled by the OrderListener which parses the order XML file and converts it into an Order . This object is then passed into the OrderProcessingService where it is finally persisted in the database. If the order XML file could be processed successfully, it will be deleted from the FTP server.

In our test, we want to make sure that our order gets persisted in the database and that the order XML file is removed from the FTP server.

OrderProcessingIT.java 
@SpringBootTest(properties = {"ftp.port=12021"})
class OrderProcessingIT {

    private FakeFtpServer fakeFtpServer;

    @Autowired
    private OrderRepository orderRepository;

    // setup and cleanup of FakeFtpServer ommited for brevity

    @Test
    void orderFile_isProcessedSuccessfully() {
        final var orderId = UUID.fromString("221f5cb1-65f0-4688-8d8e-4176bf37423e");
        final var customerId = UUID.fromString("c98d9bfa-fd98-4166-8596-9ccc0f7f06b8");
        final var itemId = UUID.fromString("caec7bae-b6e8-482c-bc39-c1bb2b9f7b8a");

        final var orderFileName = "/" + orderId + ".xml";
        final var orderFileContent = """
                                     <order>
                                         <id>%s</id>
                                         <customer-id>%s</customer-id>
                                         <item-id>%s</item-id>
                                     </order>
                                     """.formatted(orderId, customerId, itemId);

        fakeFtpServer.getFileSystem().add(new FileEntry(orderFileName, orderFileContent));

        await().atMost(Duration.ofSeconds(5))
               .until(() -> orderRepository.count() == 1L);

        final var processedOrder = orderRepository.findById(orderId);
        assertTrue(processedOrder.isPresent());
        assertEquals(customerId, processedOrder.get().getCustomerId());
        assertEquals(itemId, processedOrder.get().getItemId());

        await().atMost(Duration.ofSeconds(5))
               .until(() -> fakeFtpServer.getFileSystem().getEntry(orderFileName) == null);

    }
}
  • We add an order XML file to the filesystem of the FakeFtpServer.
  • The added file gets processed asynchronously. With Awaitility we wait until an order is persisted in the database.
  • We use the OrderRepository to receive our order from the database and check its content afterward.
  • Again, we use Awaitility to wait until the order XML file is deleted from the FakeFtpServer.

Conclusion

If you need to test your application’s integration with an FTP server, the MockFtpServer project got you covered. The higher-level API of the FakeFtpServer is intuitive and lets you set up your test scenarios quite easily. Because everything runs in-memory, you won’t need to configure extra infrastructure and must not fear unavailabilities that would cause flaky tests. On the other hand, in memory solutions also have their downsides. They are not the real thing you will integrate within the production environment. Same as you may want to use a real PostgreSQL database in your integration tests, you may want to use a real FTP server as well. As always it’s a tradeoff you have to make.

Resources

Spring Data  MockFtpServer  Spring Integration