This file contains the instructions on how to install DeepMetis-UnityEyes on a real machine equipped with Windows 10.
NOTE: We cannot ensure that DeepMetis-UnityEyes works on other Windows versions, e.g. Windows 7.
Download and install Java SE 11. Official instructions here.
Install Python 3.7.9
Check that you are using the correct version of python:
py.exe -V
This command should produce as output the following string: Python 3.7.9
To easily install the dependencies with pip, we suggest to create a dedicated virtual environment. For example, you can use venv and create a virtual environment called .venv inside the current folder (DeepMetis-UE):
python -m venv .venv
At this point, you need to activate the virtual environment:
.\.venv\Scripts\activate
At this point, upgrade pip:
py.exe -m pip install --upgrade pip
Update setuptools:
pip install setuptools --upgrade
Finally, install the other dependencies:
pip install -r requirements.txt
- Download a free Windows version of UnityEyes from the official website.
- Edit the UNITYEYES_PATH in properties.py by inserting the path to your UnityEyes folder where the UnityEyes application is located.
- Pin the UnityEyes application to the taskbar: instructions here
- Copy the sikulix_scripts folder to Local Disk (C:)
The folder Sikuli-jars contains a version of SikuliX downloaded from the official website. We use it to allow the interaction of DeepMetis with UnityEyes via GUI. SikuliX works based on scanning the screen for particular elements to interact with, for example, an app icon to click on. Therefore, for each system the user should provide screenshots of GUI widgets to interact with. As screen resolutions and colours might differ from one computer to another, the screenshots we provide with our SikuliX scripts might not work on other computers. In the following, we will provide instructions on how to re-capture these images. To this aim, we provide the whole window along with the specific widget to crop, i.e. the one highlighted with a pink frame.
- In the taskbar, take a screenshot of the highlighted component (i.e., UnityEyes icon) and save it as eye.png
- Start UnityEyes and from the starting window take a screenshot of the highlighted component (i.e., play button) and save it as play.png
- Press the play button and from the main window take a screenshot of the highlighted component (i.e., first edit text widget) and save it as first.png
- From the UnityEyes' main window take a screenshot of the highlighted component (i.e., second edit text widget) and save it as second.png
- From the UnityEyes' main window take a screenshot of the highlighted component (i.e., start button) and save it as start.png
- From the UnityEyes' main window take a screenshot of the highlighted component (i.e., close window button) and save it as x.png
- Save all the captured images in the sikulix_scripts/unityeyes.sikuli folder
NOTE: We already provided examples of these images in the sikulix_scripts/unityeyes.sikuli folder but you most probably have to replace them to match your own screen resolution.
- Run the Sikulix IDE in Sikulix_jars/sikulixide-2.0.4.jar (you can simply double click on it). It will automatically install the Jython standalone version.
- Open sikulix_scripts/unityeyes.sikuli/unityeyes.py inside the Sikulix IDE.
- Press the Run button to verify that the Sikulix script is able to find and interact with all the GUI widgets
NOTE: If Sikulix cannot find a widget, please capture it again (try to focus on the element and capture pixels that will always be present around the element).
NOTE: Please note that a computer should have a monitor connected for SikuliX to work. Please, also note that pop-up windows (such as the notification of a low battery) can disrupt the work of SikuliX.
At this point, you can go to Step 2 of the instructions.