-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add end-to-end emulator test running on CI #553
Conversation
5480103
to
259ec52
Compare
app/src/androidTest/java/com/stevesoltys/seedvault/e2e/BackupRestoreTest.kt
Outdated
Show resolved
Hide resolved
0b7b55b
to
5048c09
Compare
eaf4e6d
to
a922b69
Compare
5048c09
to
59cef31
Compare
7353753
to
3fb5506
Compare
12a9dd9
to
daa7602
Compare
daa7602
to
29bd71b
Compare
load(null) | ||
}.apply { | ||
deleteEntry(KEY_ALIAS_MAIN) | ||
deleteEntry(KEY_ALIAS_BACKUP) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to do all this manually or is there another way like resetting/clearing all app-data?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason I did it this way is because clearing the app data (with adb) seems to close the app, failing the test.
Since we need to clear the state before each test and after restoring the baseline backup, I couldn't think of a better solution. Agree that it's not the best... open to any ideas 😄
DEVELOPMENT_DIR=$SCRIPT_DIR/.. | ||
ROOT_PROJECT_DIR=$SCRIPT_DIR/../../.. | ||
|
||
EMULATOR_DEVICE_NAME=$($ANDROID_SDK_HOME/platform-tools/adb devices | grep emulator | cut -f1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this limits execution to emulators. is this intentional? why not allow physical test devices?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason I did this was to avoid the person running the tests accidentially targeting their personal phone (like me 😆) when running these tests, since it will do some destructive things like uninstalling packages, clearing backups, wiping seedvault, etc.
I can't test on a physical device unfortunately, but if we could think of away around the above could avoid this I'm sure?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We already have some instrumentation tests like this. Do these tests even work on personal phones? Doesn't it require the platform signing key?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does, but for instance this script in particular could be run against a personal device and clear the Seedvault data. Maybe just need to enforce only one device is connected?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@chirayudesai is there a way to check for a userdebug build device? Maybe that would be more useful than single device?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
getprop ro.build.type
- user, userdebug, or eng
Also, ro.build.tags
. test-keys
for dev, release-keys
for prod (typically)
|
||
val createNewFolderButton = findObject { text("CREATE NEW FOLDER") } | ||
|
||
val useThisFolderButton = findObject { text("USE THIS FOLDER") } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok these are specific to manage documents permission missing, so it won't work on my test phones which isn't a problem right now, but prevent me from running the tests outside of CI (because I can't run emulators).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahh yeah. Sorry about that. One option is if we can somehow get the generic system images to boot in an emulator, we can test with the permission. I couldn't get this working though 😞. If you know how let me know and we can switch over, it would be better without all the Google apps anyways.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What we could do instead is not request the MANAGE_DOCUMENTS permission for instrumentation test (via a modified manifest). So every device would behave the same.
No description provided.