Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make JUnit Tests reliable again #669

Open
hohwille opened this issue Sep 30, 2024 · 2 comments · Fixed by #671
Open

Make JUnit Tests reliable again #669

hohwille opened this issue Sep 30, 2024 · 2 comments · Fixed by #671
Assignees
Labels
enhancement New feature or request refined Issue in new state was discussed with the team and is clear

Comments

@hohwille
Copy link
Member

As a IDEasy developer, I want to have deterministic JUnit tests so that I can rely on their results and CI status.

It all started with #648 where we changed our generated start-script that delegates to the actual tool to run in background.

Since that is now processing asynchronous in the background the process may terminate for Java before the output is printed and received.
As a workaround we started writing the output to a file instead of echoing it and assuming that it will be captured and logged.
Therefore also the assertions all had to be refactored to assert the message not from the logs anymore but from the file where the script is writing the output to.
Now in #667 we had the effect that even that failed because the file was created in background while the JUnit in the foreground ran the assertion for the file before it was actually created in the background.

Thoughts:

  • For KISS we could question if the & is really that important for the end-user or if we simply want to prevent all this background and async processing. Still we start the real process in BACKGROUND mode - but however in JUnits we mock the process mode in ProcessContextTestImpl so we never actually run processes directly with ProcessMode.BACKGROUND.
  • For maintainability we should also consider more design for writing JUnits and stop the copy&paste mentality also in tests: We need to understand that we have a common pattern in all these tests and instead of copy&pasting this to many many test methods, we need to write reusable code for such checks. Then we can adjust in a central place to stabilise the tests if they are flickering instead of finding all the copy&pasted spots spread around multiple JUnit classes and methods.
@hohwille hohwille added the enhancement New feature or request label Sep 30, 2024
@github-project-automation github-project-automation bot moved this to 🆕 New in IDEasy board Sep 30, 2024
@jan-vcapgemini jan-vcapgemini self-assigned this Oct 1, 2024
@jan-vcapgemini jan-vcapgemini moved this from 🆕 New to 🏗 In progress in IDEasy board Oct 1, 2024
jan-vcapgemini added a commit to jan-vcapgemini/IDEasy that referenced this issue Oct 1, 2024
moved checkInstallation before run test validation
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in IDEasy board Oct 1, 2024
@hohwille hohwille added this to the release:2024.09.002 milestone Oct 1, 2024
@hohwille
Copy link
Member Author

hohwille commented Oct 1, 2024

I assume the error will come back. We can simply reopen and remove milestone if that happens.

@jan-vcapgemini
Copy link
Contributor

Issue still occurs as seen in: https://github.com/devonfw/IDEasy/actions/runs/11340850609/job/31537989280?pr=674
So we need to re-open this issue.

@jan-vcapgemini jan-vcapgemini added the refined Issue in new state was discussed with the team and is clear label Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request refined Issue in new state was discussed with the team and is clear
Projects
Status: ✅ Done
Development

Successfully merging a pull request may close this issue.

2 participants