diff --git a/README.md b/README.md index 696fa83..8e463ec 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,8 @@

- - + +

@@ -37,10 +37,10 @@ This is the official repository for the **Hackability@Sherlock** project. The goal is to develop a **3D printed design object** which easily lets **visually impaired people** (but also normally sighted people) get a quick and informative **audio description of a small indoor space** - such as an hotel room, to get an understanding of where and which objects are present in the room. Sherlock will be composed of the following components: -* an audio speaker. -* an electronic circuit to "read" the user's input (through the pressing of buttons). -* a Raspberry Pi to control everything. -* a 3D printed casing, with buttons and Braille text. +* an **audio speaker**. +* an **electronic circuit** to "read" the user's input (through the pressing of **buttons**). +* a **Raspberry Pi** to control everything. +* a **3D printed casing**, with buttons and Braille text. We will try to make Sherlock as simple and yet configurable as possible, where users can just drag-and-drop their audio tracks to be reproduced and Sherlock will be able to reproduce them. @@ -48,48 +48,60 @@ Our vision is to create an object which integrates well into any environment and ## Installation -Make sure your electronic circuit and PCB is built following the schema in `SherlockSketch.fzz`. +0. Make sure your electronic circuit and PCB is built following the schema in `SherlockSketch.fzz`. -Open a terminal window in your RaspberryPi (you can either connect through `SSH` or directly to the device). +1. Open a terminal window in your RaspberryPi (you can either connect through `SSH` or directly to the device). -Clone the repository into your preferred location: +2. Clone the repository into your preferred location: ``` cd /path/to/your/folder git clone https://github.com/Quellichenonsannofareuncazzo/Sherlock cd Sherlock ``` -**[Optional]** Create a virtual environment for the project: +3. **[Optional]** Create a virtual environment for the project: ``` python3 -m venv source /bin/activate ``` -Install project dependencies: +4. Install project dependencies: ``` -pip install -r requirements.txt +pip3 install -r requirements.txt ``` ## Usage +1. Check and update the [`config/sherlock_parameters.yaml`](config/sherlock_parameters.yaml) configuration file with the actual pins used and with your preferred settings. More details on individual settings can be found in the `Sherlock` class [docstring](src/sherlock.py#L8). -From the command line, run: +2. From the command line, run: ``` -python3 main.py +python3 src/main.py ``` and enjoy the experience! +**N.B.**: at the moment, only `.mp3` audio files are supported. Please, do convert your audio files to the supported file formats. + ## To-do List Below, a non-comprehensive list of stuff we should do in the future: -* Update `README.md` with the exact RaspberryPi model used for prototyping and testing (including Ubuntu distro, Python version, etc.) for reproducibility purposes. -* Create a file with detailed technical specifications of the electronic components (resistors, LEDs, etc.). -* Clean and update `requirements.txt` and check all dependencies (eventually try to see if we can work with the latest releases to get better long-term support). -* Use conda venvs instead of python venv, so that we can also control the Python version -* Add the outer case 3D CAD model file to the repository. -* Add images of Sherlock's final prototype, and possibly of videos of it working. Also, add other images to use in the `README.md` file (e.g., ~~Sherlock and Hackability logos~~, electronic circuit, 3D CAD model, etc.). -* Add shields for release, license, etc. +* **[VERY HIGH]** `set_pos` only sets position with respect to `get_pos` (which does not update when calling `set_pos`). Therefore, need to define an attribute in the Sherlock class with the time from playback start (else, fast-forward/fast-backward won't work). + +* **[HIGH]** Implement fast-backward. +* **[HIGH]** Test fast-for/backward functions when end/start of track is reached. + +* **[MEDIUM]** Update `README.md` with the exact RaspberryPi model used for prototyping and testing (including Ubuntu distro, Python version, etc.) for reproducibility purposes. +* **[MEDIUM]** Clean and update `requirements.txt` and check all dependencies (eventually try to see if we can work with the latest releases to get better long-term support). +* **[MEDIUM]** Add the outer case 3D CAD model file to the repository. + +* **[LOW]** Create a file with detailed technical specifications of the electronic components (resistors, LEDs, etc.). +* **[LOW]** Use conda venvs instead of python venv, so that we can also control the Python version. +* **[LOW]** Add images of Sherlock's final prototype, and possibly of videos of it working. Also, add other images to use in the `README.md` file (e.g., ~~Sherlock and Hackability logos~~, electronic circuit, 3D CAD model, etc.). + +* **[VERY LOW]** Add shields for release, license, etc. Use GH Issues to open MRs/PRs for new improvements and adding functionalities. -## Acknowledgements & Contacts +Task priorities are in brackets. + +## Acknowledgements The Sherlock project was realized by [Hackability@Milano](http://www.hackability.it/hackabilitymilano/), in partnership with [Fondazione G. Brodolini](https://www.fondazionebrodolini.it/) and [Associazione Nazionale Subvedenti](https://www.subvedenti.it/), whose contributions were essential for the brainstorming and development of Sherlock. @@ -100,6 +112,7 @@ The project was initially ideated in the context of the [EU's ECOS4IN project](h Sherlock is a byproduct of the **ECOS4IN Workshop** organized by Fondazione G. Brodolini with makers from Hackability@Milano, and inclusion stakeholders from Associazione Nazionale Subvedenti. +## Contacts If you have any questions, want to contribute, or want more information, feel free to reach out to us. * **Hackability@Milano**, [milano@hackability.it](mailto:milano@hackability.it) * Teo Bistoni, [@TeoBistoni](https://github.com/TeoBistoni) @@ -116,4 +129,4 @@ The Sherlock project is licensed under the [Creative Commons Attribution-NonComm [![CC BY-NC-SA 4.0][cc-by-nc-sa-image]][cc-by-nc-sa] [cc-by-nc-sa]: http://creativecommons.org/licenses/by-nc-sa/4.0/ -[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png \ No newline at end of file +[cc-by-nc-sa-image]: https://licensebuttons.net/l/by-nc-sa/4.0/88x31.png diff --git a/config/sherlock_parameters.yaml b/config/sherlock_parameters.yaml new file mode 100644 index 0000000..b0bc4ab --- /dev/null +++ b/config/sherlock_parameters.yaml @@ -0,0 +1,12 @@ +TRACKS_DIR: './tracks' +FW_PIN: 3 +BW_PIN: 5 +PLAY_PIN: 11 +OUT_PIN: 13 +BOUNCE: 200 # [milliseconds] +SKIP_TIME: 10 # [seconds] +LONG_PRESS_TIME: 2 # [seconds] +CURRENT_IDX: 0 +PLAY_STATE: False +RESTART_TIME: 2 # [seconds] +SUPPORTED_FORMATS: ['mp3'] \ No newline at end of file diff --git a/main.py b/main.py deleted file mode 100644 index e664e06..0000000 --- a/main.py +++ /dev/null @@ -1,51 +0,0 @@ -import pygame -import json -import glob -from time import sleep -import os -import sys -import RPi.GPIO as GPIO - -def play_music(index): - music.stop() - music.load("sound/"+soundtracks[index]) - music.play() - while music.get_busy() == True: - continue - -print("Ciao") -GPIO.setmode(GPIO.BOARD) -GPIO.setup(3,GPIO.IN,pull_up_down=GPIO.PUD_UP) -GPIO.setup(5,GPIO.IN,pull_up_down=GPIO.PUD_UP) -GPIO.setup(11,GPIO.IN,pull_up_down=GPIO.PUD_UP) -mixer = pygame.mixer -mixer.init() -music = mixer.music - -index_audio = 0 -soundtracks = os.listdir("./sound") -print(soundtracks) -while True: - if(GPIO.input(3) == GPIO.LOW): - if(index_audio == len(soundtracks)-1): - index_audio = 0 - else: - index_audio += 1 - print("avanti: " , index_audio) - play_music(index_audio) - - if(GPIO.input(5) == GPIO.LOW): - if(index_audio == 0): - index_audio = len(soundtracks)-1 - else: - index_audio -= 1 - print("indietro: " , index_audio) - play_music(index_audio) - - if(GPIO.input(11) == GPIO.LOW): - music.stop() - print("stop") - - - - diff --git a/requirements.txt b/requirements.txt index 5ce6a1e..5eacd18 100644 --- a/requirements.txt +++ b/requirements.txt @@ -56,6 +56,7 @@ pyinotify==0.9.6 PyJWT==1.7.0 pyOpenSSL==19.0.0 pyserial==3.4 +pyyaml pyxdg==0.25 rainbowhat==0.1.0 requests==2.21.0 diff --git a/src/main.py b/src/main.py new file mode 100644 index 0000000..831919e --- /dev/null +++ b/src/main.py @@ -0,0 +1,75 @@ +import os +import yaml + +from sherlock import Sherlock + +def welcome_sherlock(): + '''Prints welcome and instructions for usage.''' + # Print instructions + print('#'*10) + print('WELCOME TO SHERLOCK! :)') + print('#'*10) + print('\n') + print('#'*5) + print('INSTRUCTIONS') + print('#'*5) + print('1. Press the right button (NEXT) to skip to the next track.') + print('2. Press the central button (PLAY/PAUSE) to play/pause the track.') + print('3. Press the left button (BACK) to go back to the previous track.') + print('4. Long press the right button (NEXT) to fast-forward the current track.') + print('5. Long press the left button (BACK) to fast-backward the current track.') + print('\n') + print('Have fun!') + +def goodbye_sherlock(): + '''Prints closing message for KeyboardInterrupt exception.''' + # Print goodbye + print('#'*10) + print('GOODBYE! COME BACK SOON! :)') + print('#'*10) + +def read_yaml(config_path='./config/sherlock_parameters.yaml'): + ''' + Reads a .yaml file from the specified path. Returns a dict of parameters. + + Args: + config_path (str) : path to .yaml config file + + Returns: + params_dict (dict) : dict containing the parameters in config file + ''' + # Load parameters file + with open(config_path, 'r') as param_file: + params_dict = yaml.load(param_file, Loader=yaml.FullLoader) + + return params_dict + +def main(): + '''Starts main loop by initializing the Sherlock device.''' + # Load configuration parameters + sherlock_params_dict = read_yaml() + + # Initialize the device + sherlock = Sherlock(**sherlock_params_dict) + + # Continue listening to events + while True: + pass + +if __name__=='__main__': + ''' + Wraps main loop in a try-except exception handling to catch + KeyboardInterrupt as quit event. + ''' + try: + # Welcome message and instructions + welcome_sherlock() + # Start main loop + main() + # Catch CTRL+C command for quitting + except KeyboardInterrupt: + # Say goodbye + goodbye_sherlock() + # Deal elegantly with other errors and quit + except Exception as ex: + print('Encountered the following error: ', ex) \ No newline at end of file diff --git a/src/sherlock.py b/src/sherlock.py new file mode 100644 index 0000000..809bac4 --- /dev/null +++ b/src/sherlock.py @@ -0,0 +1,247 @@ +import os +import time + +import pygame +import RPi.GPIO as GPIO +from gpiozero import Button + +class Sherlock: + ''' + A Sherlock object contains all methods and attributes to setup and control + a Sherlock device. It builds upon the GPIO libray and the pygame.mixer object. + + Methods: + init_board : initializes RaspberryPi board + init_player : initializes pygame.mixer.music + _play : stops current track, loads track indicated by self.current_idx, then plays it + _forward : skips to next track or fast-forwards the current track. Used as callback in GPIO.add_event_detect + _fastforward : fast-forwards the current track by incrementing track position + _backward : restart current track or goes back to previous track. Used as callback in GPIO.add_event_detect + _fastbackward : fast-backwards the current track by decrementing track position + _play_pause : pause/unpause the current track. Used as callback in GPIO.add_event_detect + _long_press : detects long-pressing of any button and trigger specific action + + Examples: + >>> device1 = Sherlock(1, 2, 3, 4, 5) # dummy pins + >>> + >>> device2 = Sherlock(6, 7, 8, 9, 10, CURRENT_IDX=1, SUPPORTED_FORMATS=['wav', 'oog']) + ''' + def __init__( + self, + FW_PIN, + BW_PIN, + PLAY_PIN, + OUT_PIN, + TRACKS_DIR='./tracks', + BOUNCE=200, # [milliseconds] + SKIP_TIME=10, # [seconds] + LONG_PRESS_TIME=2, # [seconds] + CURRENT_IDX=0, + PLAY_STATE=False, + RESTART_TIME=2, # [seconds] + SUPPORTED_FORMATS=['mp3'] + ): + ''' + Store all parameters, then (1) initialize the board, (2) initialize + te pygame.mixer.music object. + + Args: + FW_PIN (int) : RaspberryPi board pin for NEXT button. + BW_PIN (int) : RaspberryPi board pin for BACK button. + PLAY_PIN (int) : RaspberryPi board pin for PLAY/PAUSE button. + OUT_PIN (int) : RaspberryPi board pin for OUTPUT. + TRACKS_DIR (str) : path to the folder where the tracks are stored. Defaults to: '.tracks' + BOUNCE (int) : time delay (in [ms]) to compensate bounce effect. Defaults to: 200 + SKIP_TIME (int) : time (in [s]) to skip when long-pressing the NEXT button. Defaults to: 10 + LONG_PRESS_TIME (int) : time (in [s]) to long-press the NEXT button before triggering the fast-forward. Defaults to: 2 + CURRENT_IDX (int) : index of track to start the playback (assuming tracks are ordered). Defaults to: 0 + PLAY_STATE (bool) : flag for play/pause status. Defaults to: False + RESTART_TIME (int) : time (in [s]) AFTER which the BACK button restarts the current track instead of going back to the previous one. Defaults to: 2 + SUPPORTED_FORMATS (list): list of supported adio formats as strings. Defaults to: ['mp3'] + ''' + # RaspberryPi board setup + self.fw_pin = FW_PIN # NEXT + self.bw_pin = BW_PIN # BACK + self.play_pin = PLAY_PIN # PLAY/PAUSE + self.out_pin = OUT_PIN # OUTPUT + self.bounce = BOUNCE # Compensate bounce effect + + # Other parameters + self.tracks = [os.path.join(TRACKS_DIR, track) for track in os.listdir(TRACKS_DIR) for fmt in SUPPORTED_FORMATS if track.endswith(fmt)] # store tracks + self.skip_time = SKIP_TIME # How many seconds to skip forward when long-pressing NEXT button + self.long_press_time = LONG_PRESS_TIME # How long to keep pressing the NEXT button to trigger the fast-forward + self.current_idx = CURRENT_IDX # From which track to start the playback (assuming the tracks in the folder are ordered) + self.is_playing = PLAY_STATE # Flag for play/pause status + self.restart_track_time = RESTART_TIME # after how many seconds the BACK button pressing restarts the current track instead of going back to the previous track + + ### INITIALIZATIONS ### + # 1. Initialize GPIO + print('Initializing board...') + self.init_board() + # 2. Initialize audioplayer + print('Initializing audioplayer...') + self.init_player() + + print('Ready! Press the PLAY/PAUSE button to start!') + + def init_board(self): + '''Initialize GPIO input/output pins and event detection.''' + ### GPIO SETUP ### + GPIO.setmode(GPIO.BOARD) + + # Set pins modes + GPIO.setup(self.fw_pin, GPIO.IN, pull_up_down=GPIO.PUD_UP) + GPIO.setup(self.bw_pin, GPIO.IN, pull_up_down=GPIO.PUD_UP) + GPIO.setup(self.play_pin, GPIO.IN, pull_up_down=GPIO.PUD_UP) + GPIO.setup(self.out_pin, GPIO.OUT) + GPIO.output(self.out_pin, GPIO.HIGH) + + # Detect button pressing events + GPIO.add_event_detect(self.fw_pin, GPIO.FALLING, callback=self._forward, bouncetime=self.bounce) + GPIO.add_event_detect(self.bw_pin, GPIO.FALLING, callback=self._backward, bouncetime=self.bounce) + GPIO.add_event_detect(self.play_pin, GPIO.FALLING, callback=self._play_pause, bouncetime=self.bounce) + + def init_player(self): + '''Initialize pygame.mixer object.''' + ### PYGAME SETUP ### + self.mixer = pygame.mixer + self.mixer.init() + self.player = self.mixer.music + + def _play(self): + '''Play the current track (self.current_idx).''' + # Stop the playback if it's playing + self.player.stop() + # Load the current track + self.player.load(self.tracks[self.current_idx]) + # Start playing + self.player.play() + + # Set is_playing flag + self.is_playing = True + + def _forward(self, channel): + ''' + Pressing the NEXT button lets the user skip to the next track. + Long-pressing the NEXT button for `N` seconds lets the user fast-forward the track by `skip` seconds. + + Args: + channel: parameter passed by GPiO.add_event_detect callback. It is the pin number + ''' + # Detect long-press + long_press_flag = self._long_press(self.fw_pin, self._fastforward) + + # If single, short press, skip to next track + if (not long_press_flag): + if(self.current_idx == len(self.tracks)-1): + self.current_idx = 0 + else: + self.current_idx += 1 + # Play next track + self._play() + + print(f"Avanti. Traccia corrente #{self.current_idx+1} - ({self.tracks[self.current_idx]})") + elif long_press_flag: + print(f'Fast-forward. Traccia corrente #{self.current_idx+1} - ({self.tracks[self.current_idx]})') + + def _fastforward(self): + ''' + Long-pressing the NEXT button for `N` seconds lets the user + fast-forward the current track by `self.skip_time` seconds. + + Note: depending on the audio format, pygame.mixer.music.set_pos + behaves differently. For .mp3 audio files, set_pos sets the new + position relatively to the current position - i.e., if you do + set_pos(5), it will skip to 5 seconds after the current position. + + More info on the official documentation: + https://www.pygame.org/docs/ref/music.html#pygame.mixer.music.set_pos + ''' + # Skip by 'fforward_skip' seconds + self.player.set_pos(self.skip_time) + + def _backward(self, channel): + ''' + Pressing the BACK button lets te user either restart the current track + or go back to the previous one if the time from the start of the current + track is less than a pre-defined time interval. + + Note: get_pos() returns time from start of playback in [milliseconds]. + + Args: + channel: parameter passed by GPiO.add_event_detect callback. It is the pin number + ''' + # Detect long-press + long_press_flag = self._long_press(self.bw_pin, self._fastbackward) + + if (not long_press_flag): + # If self.restart_track_time has passed, then restart current track + if(self.player.get_pos()//1000 > self.restart_track_time): + self._play() + else: + # Go to previous track + if(self.current_idx == 0): + self.current_idx = len(self.tracks)-1 + else: + self.current_idx -= 1 + self._play() + print(f"Indietro. Traccia corrente #{self.current_idx+1} - ({self.tracks[self.current_idx]})") + elif long_press_flag: + print(f'Fast-backward. Traccia corrente #{self.current_idx+1} - ({self.tracks[self.current_idx]})') + + def _fastbackward(self): + ''' + Long-pressing the BACK button for 'self.long_press_time' seconds + lets the user go back of the current track by 'self.skip_time' seconds. + + Note: see self._fastforward() note. + ''' + # Go back by 'self.skip_time' seconds + raise NotImplementedError + + def _play_pause(self, channel): + ''' + Pressing the center PLAY/PAUSE button, plays or pauses the current track. + + Args: + channel: parameter passed by GPiO.add_event_detect callback. It is the pin number + ''' + if(self.is_playing): + self.player.pause() + print(f"Pausa. Traccia corrente #{self.current_idx+1} - ({self.tracks[self.current_idx]})") + self.is_playing = False + else: + self.player.unpause() + print(f"Play. Traccia corrente #{self.current_idx+1} - ({self.tracks[self.current_idx]})") + self.is_playing = True + + def _long_press(self, pin, action, **action_kwargs): + ''' + Detect long-pressing of any button. + + Note: The action triggered by the long-pressing is repeated every half-second. + + Args: + pin (int) : board pin to detect the long-pressing from + action (function) : which action to trigger when long-pressing + action_kwargs (kwargs) : any kwargs needed by the action function (do not specify any if the function does not need any) + + Returns: + long_press_flag (bool) : whether the long-press was detected or not + ''' + # Compute time from when button press is detected + start_press = time.time() + + # Detect if the NEXT button has been long-pressed + long_press_flag = False + + while GPIO.input(pin) == GPIO.LOW: + time.sleep(.5) + if(time.time()-start_press > self.long_press_time): + # Long-pressing triggers a specific action + action(**action_kwargs) + + # Do not go to the next track + long_press_flag = True + + return long_press_flag