Integration Testing

Integration tests are a type of software testing where individual components or modules of a system are combined and tested as a group to ensure they work together seamlessly. In our case, these tests ensure the correct behavior of the server.

Apart from testing all the parts of our server together, They serve as a valuable substitute for unit tests in parts of the program that would otherwise be too difficult to isolate with techniques such as mocks - like controllers and services - due to complex dependencies. This is especially true when certain components, such as repositories, are used by these elements.

Much like various other testing methodologies, creating new integration tests becomes crucial upon the introduction of new features, changes, or bug fixes that are implemented and made accessible through the API, thereby becoming visible to users.

Additionally, it's essential to highlight that the API Documentation is formulated while creating integration tests. Further details regarding this can be found on the specialized wiki page.

Integration vs Unit Tests

Just like it's explained at the start of this page, integration testing has a different purpose compared to unit testing. However, before getting into integration tests (ITs), it's important to know how to create unit tests since they use the same testing framework and structure. This section brings together the main similarities and differences between these two testing methods and how we apply them in our project.

Similarities

Their combined effort ensures the accurate behavior of the system.
They rely on JUnit as the testing framework.
All JUnit's features can be used, such as @DisplayName and @BeforeEach.
They reside in the test/ package.

Differences

Instead of invoking a simple method, integration tests (ITs) employ an actual instance of the application with a test database.
ITs work like Spring components and can make use of dependency injection (read Understanding Spring).
Instead of simple operations, ITs test the state of the system before and after making a request, as well as its response.
Other than JUnit's features, ITs use Spring's repositories and MockMvc to interact with the application.
In our case, all ITs are located within the test/controller package and refer to their respective controllers.

How to Test Requests

As previously highlighted, Integration Tests (ITs) test application behavior by sending requests, verifying the expected response, and evaluating the system's post-request state. In the Spring framework, the majority of ITs can be composed utilizing the methods demonstrated in the example below.

The test creation process involves three key stages: setup (performed directly in the test method or facilitated by JUnit's @BeforeEach/@BeforeAll), executing the request, and validating assertions. If needed, a cleanup phase can be included (performed directly in the test method or facilitated by JUnit's @AfterEach/@AfterAll).

@SpringBootTest
@AutoConfigureMockMvc
@AutoConfigureTestDatabase
class AccountControllerTest @Autowired constructor(
    val mockMvc: MockMvc,
    val repository: AccountRepository
) {

    val testAccount = Account(
        "Test Account",
        "test_account@test.com",
        "test_password",
        "This is a test account",
        TestUtils.createDate(2001, Calendar.JULY, 28),
        null,
        "https://linkedin.com",
        "https://github.com"
    )

    @Nested
    inner class DeleteAccount {
        @BeforeEach
        fun addAccount() {
            repository.save(testAccount)
        }

        @Test
        fun `should delete the account`() {
            mockMvc.perform(delete("/accounts/{id}", testAccount.id)).andExpectAll(
                status().isOk,
                content().contentType(MediaType.APPLICATION_JSON),
                jsonPath("$").isEmpty
            )

            assert(repository.findById(testAccount.id!!).isEmpty)
        }
    }
}

Here's a closer look at the example:

The main test class is established with annotations configuring and booting the Spring application and components, MockMvc for request handling, and the test database. @Autowired is used to inject Spring components, as the test class itself is not a component.
@Nested identifies inner classes within the main test class, aiding in organizational clarity.
The method annotated with @BeforeEach generates a test account before each test case within the class, effectively setting up the tests.
Requests are executed through mockMvc.perform, using the appropriate HTTP method and URL. This returns a ResultActions instance, offering extensive options for assessing the response.
jsonPath can be utilized to validate aspects of the response's body, such as format, content, and values.
Lastly, assert statements, combined with the repository, are employed to verify the application's final state.

We'll now delve into slightly more advanced techniques that are commonly used in ITs.

Handling Transactions

Similar to the behavior observed in the Services section, operations responsible for managing relationships (or any actions involving multiple entities) will not be persisted correctly unless they are marked with the @Transactional annotation. If such operations are executed within a non-transactional test context, the relationships will not be stored, potentially leading to unexpected outcomes or runtime errors. This phenomenon is tied to Hibernate's concept of sessions, which is elaborated further in this article.

Applying the @Transactional annotation to a test or a test class ensures that the code within the test runs within the same session and all operations triggered by requests will behave as expected. However, it's crucial to note that these modifications will only be persisted upon the test's completion. As a result, prior to executing any assertions involving repository operations, it's necessary to manually end the current transaction and start a new one. To streamline this process, there exists a function within TestUtils designed to handle this:

fun startNewTransaction(rollback: Boolean = false) {
    if (rollback) {
        TestTransaction.flagForRollback()
    } else {
        TestTransaction.flagForCommit()
    }
    TestTransaction.end()
    TestTransaction.start()
}

It's worth highlighting that committing during tests is not problematic due to the fact that the database is automatically reset before the start of each test method. More insights regarding this can be found in the Database Cleanup section.

Mocking

Even though integration testing eliminates the need for many of the mocks that would be required in unit testing scenarios, there are instances where mocking remains highly advantageous. In the project, we employ the widely-used Mockito framework.

An excellent illustration of the indispensability of mocks is demonstrated through the ability to predict system-generated UUIDs, a critical aspect when testing file storage functionality:

// ...

private val uuid: UUID = UUID.randomUUID()
private val mockedSettings = Mockito.mockStatic(UUID::class.java)

@BeforeAll
fun setupMocks() {
    Mockito.`when`(UUID.randomUUID()).thenReturn(uuid)
}

@AfterAll
fun cleanup() {
    mockedSettings.close()
}

@Test
fun `should create the account with valid image`() {
    val expectedPhotoPath = "${uploadConfigProperties.staticServe}/profile/${testAccount.email}-$uuid.jpeg"

    mockMvc.multipartBuilder("/accounts/new")
        .addPart("account", testAccount.toJson())
        .addFile()
        .perform()
        .andExpectAll(
            status().isOk,
            content().contentType(MediaType.APPLICATION_JSON),
            jsonPath("$.photo").value(expectedPhotoPath)
        )

// ...

Note that Spring also offers mocking features, such as @MockBean and @SpyBean, that might be useful in some cases. The mock will replace or wrap any existing bean of the same type in the application context. For example:

It's important to note that Spring also offers its own mocking features, such as @MockBean and @SpyBean, which can prove beneficial in certain scenarios. These mocks are designed, respectively, to replace and encapsulate any existing beans of the same type within the application context. For instance:

@SpringBootTest
@AutoConfigureTestDatabase
class MockBeanAnnotationIntegrationTest @Autowired constructor(
    @MockBean val mockRepository: RoleRepository,
    val context: ApplicationContext
) {
    @Test
    fun `should returned mocked value`() {
        Mockito.`when`(mockRepository.count()).thenReturn(123L)

        // Simulate repository used in the application
        val contextRepository = context.getBean(RoleRepository::class.java)
        val roleCount = contextRepository.count()

        Assertions.assertEquals(123L, roleCount)
        Mockito.verify(mockRepository).count()
    }
}

Sources: Baeldung and Reflectoring.

Custom Test Utilities

In addition to the Spring-provided utilities, there are custom utilities available. The primary ones are presented below for your reference. Feel free to create your own utilities when required and include them in this list!

Custom Annotations

Custom annotations can be designed to group together a set of other annotations. This enhances the readability of tests and guarantees consistent configurations across them.

`@ControllerTest`

The @ControllerTest annotation is responsible for configuring the main test class of a controller.

@Target(AnnotationTarget.CLASS)
@Retention(AnnotationRetention.RUNTIME)
@SpringBootTest
@AutoConfigureMockMvc
@AutoConfigureRestDocs
@AutoConfigureTestDatabase
@TestExecutionListeners(
    listeners = [DbCleanupListener::class],
    mergeMode = TestExecutionListeners.MergeMode.MERGE_WITH_DEFAULTS
)
internal annotation class ControllerTest

It is composed of the following annotations:

@SpringBootTest: Initializes SpringBoot and sets up all related configurations.
@AutoConfigureMockMvc: Automatically configures the MockMvc instance, used for sending requests to the application.
@AutoConfigureRestDocs: Automatically configures the generation of Spring REST documentation.
@AutoConfigureTestDatabase: Automatically configures and starts an in-memory test database.
@TestExecutionListeners: Registers all listeners used by the test class. Currently, a listener is employed to reset the database before each test method. The merge mode is set to MERGE_WITH_DEFAULTS to ensure that defaults are not excluded.

`@NestedTest`

The @NestedTest annotation is employed to configure any test classes within a controller test.

@Target(AnnotationTarget.CLASS)
@Retention(AnnotationRetention.RUNTIME)
@Nested
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal annotation class NestedTest

It is composed of the following annotations:

@Nested: Signals that the annotated class is a nested, non-static test class that shares setup and state with the enclosing class.
@TestInstance: Setting the lifecycle to PER_CLASS enables the utilization of @BeforeAll and @AfterAll features within JUnit. This setting is essential as test methods would otherwise need to be static, which bloats the code and doesn't work with Kotlin's inner classes. In practice, one instance of the test class is reused for invoking all test methods within the class.

TestUtils

TestUtils is a class containing compact, static utility methods that are recurrently utilized across diverse test classes. It's important to emphasize that employing TestUtils should be considered as a final option. This approach is advisable when annotations, extensions, or standard methods are not viable or would lead to redundant code.

class TestUtils {
    companion object {
        fun createDate(year: Int, month: Int, day: Int): Date {
            return Calendar.getInstance(TimeZone.getTimeZone("UTC"))
                .apply { set(year, month, day, 0, 0, 0) }
                .time
        }

        // ...
    }
}

Multipart Builder

To simplify the creation of requests using multipart form data, we utilize an extension function for the MockMvc class:

fun MockMvc.multipartBuilder(uri: String): MockMvcMultipartBuilder {
    return MockMvcMultipartBuilder(this, uri)
}

This MockMvcMultipartBuilder class is also custom and offers techniques to construct and execute the request, much like regular requests. Illustrative methods of this class include:

addPart(key: String, data: String)
addFile(name: String, filename: String, content: String, contentType: String)
asPutMethod()
perform()

For instance, this extended function could be used in the following manner:

mockMvc.multipartBuilder("/events/new")
    .addPart("event", objectMapper.writeValueAsString(testEvent))
    .addFile(name = "image", filename = "image.pdf")
    .perform()
    .andExpectAll(
        status().isBadRequest,
        content().contentType(MediaType.APPLICATION_JSON),
        jsonPath("$.errors.length()").value(1),
        jsonPath("$.errors[0].message").value("invalid image type (png, jpg,  jpeg or webp)"),
        jsonPath("$.errors[0].param").value("createEvent.image")
    )

Validation Tester

When it comes to validating user input, it's essential to test it in nearly every controller. To address this requirement, we've developed a class that simplifies the process and offers a set of functions for testing commonly used validations.

ValidationTester operates by taking the request for an endpoint in its constructor along with any necessary fields, which would otherwise lead to unexpected missing field errors. You can then configure the param field to be set differently in each request, and optionally specify the expected parameterName in the response (which defaults to param). The last step is to call one of the validation functions, such as isRequired.

An example implementation is provided below:

class ValidationTester(
    private val req: (Map<String, Any?>) -> ResultActions,
    private val requiredFields: Map<String, Any?> = mapOf()
) {
    lateinit var param: String
    var parameterName: String? = null

    private fun getParamName(): String {
        return parameterName ?: param
    }

    fun isRequired() {
        val params = requiredFields.toMutableMap()
        params.remove(param)
        req(params)
            .expectValidationError()
            .andExpectAll(
                jsonPath("$.errors[0].message").value("required"),
                jsonPath("$.errors[0].param").value(getParamName())
            )
    }

    fun hasSizeBetween(min: Int, max: Int) {
        val params = requiredFields.toMutableMap()
        val smallValue = "a".repeat(min - 1)
        params[param] = smallValue
        req(params)
            .expectValidationError()
            .andExpectAll(
                jsonPath("$.errors[0].message").value("size must be between $min and $max"),
                jsonPath("$.errors[0].param").value(getParamName()),
                jsonPath("$.errors[0].value").value(smallValue)
            )

        val bigValue = "a".repeat(max + 1)
        params[param] = bigValue
        req(params)
            .expectValidationError()
            .andExpectAll(
                jsonPath("$.errors[0].message").value("size must be between $min and $max"),
                jsonPath("$.errors[0].param").value(getParamName()),
                jsonPath("$.errors[0].value").value(bigValue)
            )
    }

    private fun ResultActions.expectValidationError(): ResultActions {
        andExpectAll(
            status().isBadRequest,
            content().contentType(MediaType.APPLICATION_JSON),
            jsonPath("$.errors.length()").value(1)
        )
        return this
    }
}

And an example of its usage:

@NestedTest
@DisplayName("Input Validation")
inner class InputValidation {
    private val validationTester = ValidationTester(
        req = { params: Map<String, Any?> ->
            mockMvc.multipartBuilder("/events/new")
                .addPart("event", objectMapper.writeValueAsString(params))
                .addFile(name = "image")
                .perform()
                .andDocumentErrorResponse(documentation, hasRequestPayload = true)
        },
        requiredFields = mapOf(
            "title" to testEvent.title,
            "description" to testEvent.description,
            "dateInterval" to testEvent.dateInterval,
            "image" to testEvent.image,
            "slug" to testEvent.slug
        )
    )

    @NestedTest
    @DisplayName("title")
    inner class TitleValidation {
        @BeforeAll
        fun setParam() {
            validationTester.param = "title"
        }

        @Test
        fun `should be required`() = validationTester.isRequired()

        @Test
        @DisplayName(
            "size should be between ${ActivityConstants.Title.minSize}" +
                " and ${ActivityConstants.Title.maxSize}()"
        )
        fun size() =
            validationTester.hasSizeBetween(ActivityConstants.Title.minSize, ActivityConstants.Title.maxSize)
    }

Implementation - Controller Tests

Following the previous sections, you should be able to understand the implementation of our controller tests. The structure of their hierarchy is as follows (from outer to inner classes):

A single class annotated with @ControllerTest, referring to a specific controller in the application. This class will hold dependencies used throughout all tests, such as MockMvc.
Inner classes annotated with @NestedTest and @DisplayName, referring to an endpoint inside the controller.
Optional inner classes annotated with @NestedTest, used for organizational purposes. Feel free to use and abuse this feature.
Test functions annotated with @Test.

Note that all classes can hold special functions such as @BeforeEach or @AfterAll.

There are many examples of this in the code but the implementation should look something like this:

@ControllerTest
internal class EventControllerTest @Autowired constructor(
    val mockMvc: MockMvc,
    val objectMapper: ObjectMapper,
    val repository: EventRepository,
    val accountRepository: AccountRepository
) {
    ...

    @NestedTest
    @DisplayName("POST /events/new")
    inner class CreateEvent {
        ...

        @BeforeEach
        fun addAccount() {
            accountRepository.save(testAccount)
        }

        ...

        @NestedTest
        @DisplayName("Input Validation")
        inner class InputValidation {
            ...

            @NestedTest
            @DisplayName("title")
            inner class TitleValidation {
                ...

                @Test
                fun `should be required`() = validationTester.isRequired()

                ...
            }

            ...
        }

        ...
    }
    ...
}

Cleaning the Database

A significant issue with Spring's default implementation of integration tests (ITs) is that it reuses the database across all tests within a class. Consequently, modifications made in one test can impact the execution of subsequent tests. This behavior is undesirable and can lead to numerous mistakes when creating new tests or modifying existing ones.

To address this issue, we've developed a TestExecutionListener that resets the database before each test method is executed. It's worth noting that we previously attempted a method using @DirtiesContext (#46 and #95), but it significantly impacted test execution performance because it required creating a new Spring context every time the database was cleaned, which was overkill.

The logic of the listener goes as follows:

Implement the `TestExecutionListener` interface

We start by implementing the TestExecutionListener interface and overriding the beforeTestMethod method to execute custom code before each test case is executed. We then add this listener to our @ControllerTest annotation:

// DbCleanupListener.kt
class DbCleanupListener : TestExecutionListener, Logging {
    ...

    override fun beforeTestMethod(testContext: TestContext) {
        super.beforeTestMethod(testContext)
        ...
    }

    ...
}

// ControllerTest.kt
...
@TestExecutionListeners(
    listeners = [DbCleanupListener::class],
    mergeMode = TestExecutionListeners.MergeMode.MERGE_WITH_DEFAULTS
)
internal annotation class ControllerTest

Clean the Data Source

To successfully reset the database, specific actions are required:

Creation of a statement object to write and execute SQL statements.
Deactivation of constraints, thereby disabling referential integrity.
Truncation of all tables and the restart of their identity to prevent ID number issues.
Resetting of sequences to ensure ID generation begins at 1.
Re-enabling constraints.

For a more detailed implementation, please refer to src/test/utils/listeners/DbCleanupListener.kt. The inspiration for this method can be traced back to this article.

Clean the Entity Manager

Spring uses an entity manager equipped with a sequence generator. In addition to the database mechanism, this generator utilizes an optimizer that won't reset ID generation unless we set it to null, thereby triggering a re-initialization at a later point.

For the actual code pertaining to this aspect, it's a bit complicated and may not be critical for our case. Nevertheless, you can examine it in detail in src/test/utils/listeners/DbCleanupListener.kt. If you need further insights, you can refer to this thread.

Excluding any auxiliary methods, the final function appears as follows:

override fun beforeTestMethod(testContext: TestContext) {
    super.beforeTestMethod(testContext)
    logger.info("Cleaning up database...")

    val dataSource = testContext.applicationContext.getBean(DataSource::class.java)
    cleanDataSource(dataSource)

    val entityManager = testContext.applicationContext.getBean(EntityManager::class.java)
    cleanEntityManager(entityManager)

    logger.info("Done cleaning up database...")
}

Sources: Youtube playlist that introduces integration testing, Spring Docs, Baeldung.

Getting Started

Architecture Details

Implementation Details

Testing

Documentation

API documentation

Deployment

How to Deploy

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration Testing

Contents

Integration vs Unit Tests

Similarities

Differences

How to Test Requests

Handling Transactions

Mocking

Custom Test Utilities

Custom Annotations

`@ControllerTest`

`@NestedTest`

TestUtils

Multipart Builder

Validation Tester

Implementation - Controller Tests

Cleaning the Database

Implement the `TestExecutionListener` interface

Clean the Data Source

Clean the Entity Manager

Clone this wiki locally

Integration Testing

Contents

Integration vs Unit Tests

Similarities

Differences

How to Test Requests

Handling Transactions

Mocking

Custom Test Utilities

Custom Annotations

@ControllerTest

@NestedTest

TestUtils

Multipart Builder

Validation Tester

Implementation - Controller Tests

Cleaning the Database

Implement the TestExecutionListener interface

Clean the Data Source

Clean the Entity Manager

Clone this wiki locally

`@ControllerTest`

`@NestedTest`

Implement the `TestExecutionListener` interface