Add end-to-end emulator test running on CI #553
Conversation
stevesoltys
force-pushed
the
feature/e2e-test
branch
16 times, most recently
from
5480103
to
259ec52
Compare
app/src/androidTest/java/com/stevesoltys/seedvault/e2e/BackupRestoreTest.kt
Outdated
Show resolved
Hide resolved
stevesoltys
force-pushed
the
feature/e2e-test
branch
9 times, most recently
from
0b7b55b
to
5048c09
Compare
stevesoltys
force-pushed
the
feature/local-development-improvements
branch
from
eaf4e6d
to
a922b69
Compare
stevesoltys
force-pushed
the
feature/e2e-test
branch
from
5048c09
to
59cef31
Compare
stevesoltys
force-pushed
the
feature/e2e-test
branch
2 times, most recently
from
7353753
to
3fb5506
Compare
stevesoltys
force-pushed
the
feature/e2e-test
branch
5 times, most recently
from
12a9dd9
to
daa7602
Compare
stevesoltys
force-pushed
the
feature/e2e-test
branch
from
daa7602
to
29bd71b
Compare
| load(null) | ||
| }.apply { | ||
| deleteEntry(KEY_ALIAS_MAIN) | ||
| deleteEntry(KEY_ALIAS_BACKUP) |
There was a problem hiding this comment.
do we need to do all this manually or is there another way like resetting/clearing all app-data?
There was a problem hiding this comment.
The reason I did it this way is because clearing the app data (with adb) seems to close the app, failing the test.
Since we need to clear the state before each test and after restoring the baseline backup, I couldn't think of a better solution. Agree that it's not the best... open to any ideas 😄
| DEVELOPMENT_DIR=$SCRIPT_DIR/.. | ||
| ROOT_PROJECT_DIR=$SCRIPT_DIR/../../.. | ||
|
|
||
| EMULATOR_DEVICE_NAME=$($ANDROID_SDK_HOME/platform-tools/adb devices | grep emulator | cut -f1) |
There was a problem hiding this comment.
this limits execution to emulators. is this intentional? why not allow physical test devices?
There was a problem hiding this comment.
The reason I did this was to avoid the person running the tests accidentially targeting their personal phone (like me 😆) when running these tests, since it will do some destructive things like uninstalling packages, clearing backups, wiping seedvault, etc.
I can't test on a physical device unfortunately, but if we could think of away around the above could avoid this I'm sure?
There was a problem hiding this comment.
We already have some instrumentation tests like this. Do these tests even work on personal phones? Doesn't it require the platform signing key?
There was a problem hiding this comment.
It does, but for instance this script in particular could be run against a personal device and clear the Seedvault data. Maybe just need to enforce only one device is connected?
There was a problem hiding this comment.
@chirayudesai is there a way to check for a userdebug build device? Maybe that would be more useful than single device?
There was a problem hiding this comment.
getprop ro.build.type - user, userdebug, or eng
Also, ro.build.tags. test-keys for dev, release-keys for prod (typically)
|
|
||
| val createNewFolderButton = findObject { text("CREATE NEW FOLDER") } | ||
|
|
||
| val useThisFolderButton = findObject { text("USE THIS FOLDER") } |
There was a problem hiding this comment.
ok these are specific to manage documents permission missing, so it won't work on my test phones which isn't a problem right now, but prevent me from running the tests outside of CI (because I can't run emulators).
There was a problem hiding this comment.
Ahh yeah. Sorry about that. One option is if we can somehow get the generic system images to boot in an emulator, we can test with the permission. I couldn't get this working though 😞. If you know how let me know and we can switch over, it would be better without all the Google apps anyways.
There was a problem hiding this comment.
What we could do instead is not request the MANAGE_DOCUMENTS permission for instrumentation test (via a modified manifest). So every device would behave the same.
app/src/androidTest/java/com/stevesoltys/seedvault/e2e/BackupRestoreTest.kt
Outdated
Show resolved
Hide resolved
No description provided.