- Clone the repository.
- Install Conda environment for the project using this environment file for Mac or environment file for Windows. you can use
conda env create -f environment.yml
to create the environment and activate the environment byconda activate usage
. - install Appium Desktop
- add app's configuration in App_Config.py -- you can find relevant info in news_app_info.csv and shopping_app_info.csv
- start Android Emulator -- make sure the AUT (App Under Test) is installed on the Emulator, e.g., abcnews.
- start Appium Server on Appium desktop (you do not need to change the default port).
- Change file paths in global_config.py. Final-Artifact can be downloaded here.
- run main method in dynamic_evaluation.py
At this point you should be able to see an output similar to the output below on your command line interface:
The screen classifier top1 guess for the screen: category The screen classifier top5 guesses for the screen: ['category', 'home', 'items', 'menu', 'popup'] Choose the closest screen tag from the top5 guesses: [
You should type the closest screen tag on your cli
]home
id:0 floating_search_view - matched with: to_search id:1 rl_search_box - matched with: to_search id:2 navigation_home - matched with: home id:3 navigation_feed - matched with: menu Choose the id of the widget you want to interact with:[
You should type the id on your cli
]:0
Please enter the ground truth IR for the widget you chose:to_search executing event: resource-id floating_search_view click
Enabling REMAUI feature
- download REMAUI Java program
- install Eclipse for Java if you don't already have it
- use Eclipse to import the REMAUI Java program
- configure all the build paths (the REMAUI repo should already contain all the necessary jars except for openCV) -- see screenshots below regarding build path. you need to edit the paths of jar files to the file paths on your machine
- follow this tutorial to enable openCV (see notes below first if using Mac)
- double check the main method in
edu.wm.cs.semeru.redraw.REMAUI
, it should only haverun_REMAUI_single_image(args[0]);
. - run the main method in
edu.wm.cs.semeru.redraw.REMAUI
through Eclipse's "run configurations" and give one argument which is the path of a screen image (see below). (Make sure to change file path inrun_REMAUI_single_image
function, includingprojectRootDirectory
, andscreenshots
String which should be the folder name that contains the image). REMAUI output should be in the parent folder of the image path. - After running sccessfully, copy the command and replace
REMAUI_cmd
in state.py
If you're using Mac, you need to build openCV yourself in order to have that "build" folder. follow steps below:
- after downloading openCV's source, make a
build
directory in it. - then you build opencv using cmake command. If you are in the build folder you just created,
cmake ..
will do. Then runmake -j8
- from the tutorial above, you should put
build/lib
as the native library location
usage_root_dir
should point to V2S's results of a usage, e.g., "Combined/SignIn/".- run
1_step_extraction/step_extraction.py
to generateclicked_frames
folder - go to
binaryClassifier
and follow the instructions on the next section to generate finaltyping_result.csv
. - run
1_step_extraction/step_cleaning.py
to generatesteps_clean
folder. - run
1_step_extraction/special_step_recognition.py
to append special actions insteps_clean
folder. - run
2_ir_classification/screen_widget_extraction.py
to generateir_data_auto
folder with screens only. - get UIED results (run UIED v2.3's
run_mybatch.py
) -- check terminal for running time as this process is time-consuming. - run
2_ir_classification/screen_widget_extraction.py
to add widget crops toir_data_auto
folder.