Integration Testing - Database state management
Posted by tech-man-ua@reddit | ExperiencedDevs | View on Reddit | 22 comments
I am currently setting up integration test suite for one the RESTful CRUD apis and the frameworks I use put some limitations.
Stack: Java 21, Testcontainers, Liquibase, R2DBC with Spring
I want my integration tests to be independent, fast and clean, so no Spin up a new container per each test.
Some of the options I could find online on how I can handle:
- Do not cleanup DB tables between test methods but use randomised data
- Make each test method Transactional (can't use it out of the box with R2DBC)
- Spin up a single container and create new database per each test method
- Create dump before test method and restore it after
- ....
Right now I am spinning up a single container per test class, my init/cleanup methods look like following:
@BeforeEach
void initEntities() {
databaseClient.sql("""
INSERT INTO .........
""")
.then()
.subscribe();
}
@AfterEach
void cleanupEntities() {
databaseClient.sql("TRUNCATE <tables> RESTART IDENTITY CASCADE")
.then()
.subscribe();
}
which theoretically works fine. Couple of things I am concerned about are:
- I insert test data in the test class itself. Would it be better to extract such pieces into .sql scripts and refer these files instead? Where do you declare test data? It will grow for sure and is going to be hard to maintain.
- As we are using PostgreSQL, I believe TRUNCATE RESTART IDENTITY CASCADE is Postgre-specific and may not be supported by other database systems. Is there a way to make cleanup agnostic of the DB system?
Any better ways to implement integration test suite? Code examples are welcomed. Thanks
Cell-i-Zenit@reddit
You can run all your tests in a transaction. When the test is done everything is getting rolled back. You can do that by just having the @Transactional annotation at the class level.
Be aware that this changes the dynamic since everything runs then in the same transaction, which is not always the case.
tech-man-ua@reddit (OP)
It's not provided out-of-the-box with R2DBC, as I mentioned, unfortunately.
Otherwise that should work perfectly, thanks for the suggestion anyway.
BestUsernameLeft@reddit
Either random data or
@TestTransaction
are good.With respect to managing test data, use scenarios and user personas. So you have a
ScenarioBuilder
class (or classes) andPersonaBuilder
and you can do something like this for your new airline reservations app:Create new scenarios and user personas when you have "some" tests duplicating the same setup. Avoid putting too much setup into a single scenario or persona.
RedditIsBadButActive@reddit
Was going to suggest just this - builder pattern is great for building easily repeatable and readable scenarios.
mmcnl@reddit
I'm not familiar with your stack, but I don't like managing the database using code in tests. Too low-level, too many moving parts, too much maintenance, which reduces the confidence the test results give me.
For Postgres databases, I spin up a Docker container for every single test and mount a .sql database dump in /docker-entrypoint-initdb.d (for MariaDB the directory is different but the idea is the same), which the official image automatically runs before starting the database. After the tests are done I destroy the containers.
rapture_survivor@reddit
How long does your entire test suite take to run, what is the average time per test? This approach seems like it would take quite a while to execute.
originalchronoguy@reddit
Modern infrastructure is ephemeral.
So it is common to programmatically update, manage database via code. This is part of the 12-factor tenants of "administrative processes."
Table alters, views, we do those as code that is in git so it can be tracked and versioned. This process also makes it easier for other developers to get latest updated test data or schema changes.
And if you follow chaos monkey theory, you should test before and after schema changes. System should still do proper error handling if a DB was altered, the error handling on the app should detect and fail on those assertions. Because in the real world, database changes may not sync and that level of chaos should be anticipated.
In short, this is considered a good modern practice.
mmcnl@reddit
Ofcourse you manage your database and migrations using code. However if you need to set a certain state for tests then I prefer to not set the state myself and using simple import/export functionality.
tech-man-ua@reddit (OP)
That makes sense and I've seen that approach, I could mount init scripts to be picked up on the container.
However, as init happens once, that would work only if you spin up a new container per each test method, wouldn't it?
Or in case you are restoring the dump after each test method.
hibbelig@reddit
Does Postgres have snapshots? Oracle calls them flash back points i think. You could set up basic/common test data, then save a snapshot, then take the snapshot prior to each test.
Expensive_Garden2993@reddit
Yes, that's called savepoint, I'm using exactly this to clean up tests.
The whole test suite is in transaction, every individual test is starting a savepoint, rolls back to it after running. Works like a charm and I love this approach, being using it for years.
tech-man-ua@reddit (OP)
Is that's what you are referring to?
https://github.com/testcontainers/testcontainers-java/discussions/4845
nikita2206@reddit
No this isn’t it. The following is the docs for savepoints https://www.postgresql.org/docs/current/sql-savepoint.html
tech-man-ua@reddit (OP)
Got it.
That's sounds like good way to solve the problem, but to achieve Transactional I would need to have some custom code. R2DBC does not support it out-of-the-box as I mentioned.
Expensive_Garden2993@reddit
No, not using test containers, I mean this:
https://www.postgresql.org/docs/current/sql-savepoint.html
tech-man-ua@reddit (OP)
Good idea, I believe that's what that guy did, I noted down that page.
IamWildlamb@reddit
You could spin up separate container and run tests in paralel if your concern is speed.
That being said, I am not sure why you can not use Transactional? I am pretty sure that SpringBoot has support for it just fine even in case of reactive dbs?
As for scripts.. I would keep them separate somewhere in resources and use Sql annotation with execution phase settings to run them.
tech-man-ua@reddit (OP)
JUnit does not support Transactional with R2DBC. Reactive puts some limitations on useful features / frameworks.
About the scripts, same thing, R2DBC does not support Sql annotation 😁
dublinvillain@reddit
I had a pipeline that basically did 1. Load seed data in base schema, 2. Perform migrations 3. Test migrations 4. Run integration tests. During ITs we never modified the DB directly, only via api. We never cleaned the DB. This of course meant tests weren't idempotent which is bad (worked around with naming in the data). But it was surprisingly robust, reasonably quick to run, and easy to onboard. We used something like wiremock / microcks to mock external apis. Fwiw I think your approach is good. I think it's the most valuable type of testing but I don't think you can get around maintaining schema data in your test code.
Rocketninja16@reddit
The only part of this stack I use is Testcontainers, so this info might be garbage:
We use fixtures to spin up and seed the test database for the test suites and then test against that.
If some tests require specific state in the DB, it gets its own fixture that overrides the default ones.
It still takes a moment to spin a container up and such but it’s fairly quick.
This is do dotnet, using Xunit as the test framework so overall ymmv with what you’re able to do with your own stack.
tech-man-ua@reddit (OP)
Interesting, I've seen some .Net examples, including Respawn
Need to check if there are some kind of fixtures in Java world
elprophet@reddit
> I want my integration tests to be independent, fast and clean, so no Spin up a new container per each test.
You might want to reevaluate this, it's much cheaper than you expect. You might consider some amount of sharding or checkpointing, but generally, this is what I've seen teams land on.