Make integration tests using test suites #11634
Conversation
xiangfu0
force-pushed
the
integration-tests-suite
branch
3 times, most recently
from
c0b4919
to
477bff9
Compare
Codecov Report
@@ Coverage Diff @@
## master #11634 +/- ##
============================================
- Coverage 63.12% 62.96% -0.17%
Complexity 1140 1140
============================================
Files 2343 2343
Lines 126306 126330 +24
Branches 19419 19421 +2
============================================
- Hits 79733 79545 -188
- Misses 40905 41108 +203
- Partials 5668 5677 +9
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 25 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
xiangfu0
force-pushed
the
integration-tests-suite
branch
2 times, most recently
from
1b29f55
to
82382e8
Compare
xiangfu0
force-pushed
the
integration-tests-suite
branch
3 times, most recently
from
371d0e6
to
8d4161c
Compare
Integration 1 and 2 are for balancing. |
xiangfu0
force-pushed
the
integration-tests-suite
branch
5 times, most recently
from
58c0042
to
4c77a6d
Compare
| @@ -39,7 +39,7 @@ | |||
| /** | |||
| * Integration test for floating point data type (float & double) filter queries. | |||
| */ | |||
| @Test(suiteName = "CustomClusterIntegrationTest") | |||
| @Test(suiteName = "CustomClusterIntegrationTest", groups = {"custom-integration-suite"}) | |||
There was a problem hiding this comment.
so before this change the @BeforeSuite and @BeforeClass is basically the same b/c they are not group into suite is that correct? (we should see huge test time reduction if so)
There was a problem hiding this comment.
Only this package has the suite implementation, which is already tested using mvn test -P github-actions,custom-cluster-integration-test-suite || exit 1. But in this case, we are missing the code coverage report for this.
So just wanna merge these two suite to run together.
|
We have 3 integration tests profile right now:
1 and 2 are just for rebalancing Right now we split the integration tests in two parts: For 2, just one command: The issue with 1 is that we are missing the code coverage report for the first command. |
xiangfu0
force-pushed
the
integration-tests-suite
branch
from
4c77a6d
to
321d410
Compare
| </run> | ||
| </groups> | ||
| <packages> | ||
| <package name="org.apache.pinot.integration.tests.*"/> |
There was a problem hiding this comment.
this looks like in includes everything (*) and the other *-2 also have the same package name, do we plan to still do the first letter split to make it eaiser (e.g. no need to annotate each test class).
b/c i can image adding a new file without remembering to include the suite name - one will be able to manually trigger that test in IDE but will be ignored in CI and mvn test
There was a problem hiding this comment.
nvm i actually see you do exclusion from the *-2 xml... (maybe to call it out in comment)
| </run> | ||
| </groups> | ||
| <packages> | ||
| <package name="org.apache.pinot.integration.tests.*"/> |
There was a problem hiding this comment.
nvm i actually see you do exclusion from the *-2 xml... (maybe to call it out in comment)
| <exclude name="custom-integration-suite"/> | ||
| <exclude name="integration-suite-1"/> |
There was a problem hiding this comment.
good to call out that integration-suite-2 is a catch all and theoretically it doesn't need name annotation in @Test
|
lgtm can you fix the test failures |
xiangfu0
force-pushed
the
integration-tests-suite
branch
6 times, most recently
from
2d82e5b
to
53cf473
Compare
xiangfu0
force-pushed
the
integration-tests-suite
branch
21 times, most recently
from
afd762b
to
6272c77
Compare
xiangfu0
force-pushed
the
integration-tests-suite
branch
4 times, most recently
from
5bfefc7
to
a325a48
Compare
xiangfu0
force-pushed
the
integration-tests-suite
branch
from
a325a48
to
ffc08cd
Compare
Split integration tests into two suites:
integration-suite-1andintegration-suite-2