Is sharding already possible?

[cambiph]

Hey,

I read through the docs, PR's and discussions but it's a bit unclear to me what the current state is.
Is there already a way to use sharding on Firebase?
We are hitting the 60m limit so I am looking for a solution :)

Thanks in advance.
Cheers

[bartekpacia]

Sharding is possible and should work out of the box. We tested it and it works on Firebase, emulator.wtf, BrowserStack.

Check out our example app - you can try running it on FTL as a demo.

[cambiph]

So to have sharding enabled, I do not have to do anything? Because now I am running the trests as following, and sharding does not seem to be done:

gcloud firebase test android run 
--device model=MediumPhone.arm,version=33 
--type instrumentation 
--use-orchestrator 
--environment-variables clearPackageData=true 
--app build/app/outputs/apk/debug/app-debug.apk 
--test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk 
--results-bucket #{results_bucket} 
--results-dir #{results_dir} 
--format=json 1>#{output_file} 
--timeout 60m

[bartekpacia]

By default gcloud firebase test android doesn't use sharding. Consult docs of this command here: https://cloud.google.com/sdk/gcloud/reference/beta/firebase/test/android/run#--num-uniform-shards

[cambiph]

I saw that but we need the beta SDK for that. Guess there's no other way at the moment?

[bartekpacia]

Maybe you could try curling Google Cloud's REST endpoints. But I think using beta is the simplest solution.

[cambiph]

I checked Flank and I am experimenting with Fladle (https://runningcode.github.io/fladle). The only issue left is that the way MainActivityTest is setup, fladle sees it as one test and complains about it:
WARNING: num-uniform-shards (5) is higher than number of test cases (1) from gs://test-lab-h3pxbib2yh4f0-y4jh297cm3nvx/2023-10-12_13-01-35.953242_QrKj/app-debug-androidTest.apk

I think if we can find a way to split the tests in MainActivityTest, it will run fine.
Can this be easily achieved, you think?

[bartekpacia]

I think it might be a problem on Flank's/Fladle's side. Our MainActivityTest is a valid Android instrumented test.

I think if we can find a way to split the tests in MainActivityTest, it will run fine. Can this be easily achieved, you think?

Achieved - maybe, easily - not so sure. MainActivityTest is set up the way it is to allow for fixing one of the most annoying problems with the integration_test plugin – flutter/flutter#115751.

[cambiph]

Thanks for the feedback. I think I will just split up our tests in logical groups and run them in parallel on our CI. This will have the least impact and the result should be the same.

Thank you for Patrol and your amazing efforts!

[cambiph]

Mmmm, I cannot seem to find a way to split up the tests to run in parallel on Firebase as the gcloud command does not support the -t integrations_tests/... flag. Is there a workaround for this?

[bartekpacia]

Hi, how is that information relevant to this Discussion?

[cambiph]

The cloud beta is not installed on our build agents and I cannot seem to find the docs telling me how to install the beta version. I also cannot find it in the gcloud-sdk-installer bucket they provide. Op vr 13 okt. 2023 11:53 schreef Bartek Pacia ***@***.***>:

…

I think you'd have to split tests at build time – that is, when you call patrol build. Build a few APKs and then upload them with separate gcloud cli invocations. Why not just use gcloud beta, btw? — Reply to this email directly, view it on GitHub <#1789 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMNQU2NPWUEVYJF2EVHA63X7EFRZAVCNFSM6AAAAAA52HIPPWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TENZSGI3TS> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bartekpacia]

I think you can just do gcloud beta:

$ gcloud beta
You do not currently have this command group installed.  Using it
requires the installation of components: [beta]


Your current Google Cloud CLI version is: 439.0.0
Installing components from version: 439.0.0

┌─────────────────────────────────────────────┐
│     These components will be installed.     │
├──────────────────────┬────────────┬─────────┤
│         Name         │  Version   │   Size  │
├──────────────────────┼────────────┼─────────┤
│ gcloud Beta Commands │ 2023.07.14 │ < 1 MiB │
└──────────────────────┴────────────┴─────────┘

For the latest full release notes, please visit:
  https://cloud.google.com/sdk/release_notes

Do you want to continue (Y/n)?  y

... omitted

[bartekpacia]

or maybe run gcloud beta firebase test android run, it should "just work"

[cambiph]

Already tried just running it and then it complains that it does not recognize the beta flags. Let me try if I can install it in some sort of way.

[cambiph]

Switched to Browserstack, works like a charm :)

[bartekpacia]

Happy to hear that!

Were these docs useful to you in this process?

[cambiph]

Yes, I used them. Only thing to note is that you need to manually install jq before you can use ns_android. Other than that, it works like a charm, kiddos! Op vr 13 okt. 2023 18:32 schreef Bartek Pacia ***@***.***>:

…

Happy to hear that! Did these docs <https://patrol.leancode.co/integrations/browserstack> come in useful? — Reply to this email directly, view it on GitHub <#1789 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMNQUYTFCH5QGMGLXRJKFLX7FUKDAVCNFSM6AAAAAA52HIPPWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TENZVGY3DO> . You are receiving this because you authored the thread.Message ID: ***@***.***>