Skip to content

yushakareem/Teleop_MIRO

 
 

Repository files navigation

Teleop_MIRO

Software Architecture: Teleoperation with MIRO using gesture and speech Using Motion Capture for localization and gesture recognition

Objective: ROS-based software architecture that drives the MIRO robot to successfully perform a given task, specifically to go to a certain position with respect to a object in the workspace, using pointing gestures and issuing commands via speech which could be interpreted by the system. We use as feedback the information provided by a visual tracking system, in this case the Motion Capture by OptiTrack.

Usage Instructions:

  • Clone the master branch of Teleop_MIRO repository inside the src folder of your catkin_ws
  • The following 2 OpenCV steps are optional. You can either follow them, or comment out relevant code in miro_teleop/src/command_logic.cpp
  • Run catkin_make from inside catkin_ws
  • Download MIRO MDK and MIRO app from http://labs.consequentialrobotics.com/miro/mdk/
  • Turn on MIRO. Run the app and connect with MIRO, to find MIRO's IP. Ensure your machine is on the same network as MIRO.
  • Edit the ~/.profile file in root@<MIRO's IP>, set ROS_MASTER_IP with machine IP running the roscore. Source it.
  • Run/Enable bridge on the app. You might need to do this at frequent intervals if MIRO stops responding.
  • In the Motive software for Mocap Optitrack, enable Data Streaming and set broadcast IP to the machine running the roscore.
  • Setup up markers in the Motion Capture area. Create Rigid Bodies in Motive (Robot, Obstacle, Gesture in order) after aligning required local axes with global axes.
  • Launch the application using roslaunch miro_teleop miro_teleop.launch
  • Spatial Reasoner will run immediately displaying the first direction of mapping. Press any key to continue after each such image is displayed. Spatial Reasoner will display 4 images, whereas Pertinence Mapping will generate an image plot each time look command is entered in the interpreter.
  • User points towards required target. Then, enter look command in Interpreter.
  • Wait till path is generated by RRT* (see display on command logic terminal). Then issue go command on Interpreter.
  • You can also issue stop command while MIRO is moving to pause. Entering go will make it resume its originak trajectory.
  • MIRO halts once goal is reached.
  • Pointing to a different direction and issuing look command can be used asynchronously in between actions to reset MIRO's trajectory with a new path.
  • Keep an eye on the Motive screen. If rigid bodies are not being tracked properly, you might need to create rigid bodies again with proper alignment.

Accomplishments:

MiRo and obstacle in the motion capture area with markers on them.

Pointing gesture on the ground is used to calculate a goal-point where MiRo has to reach.
Voice command - look (for now typed in the workstation) is used to orient MiRo towards the goal.
Montecarlo simulation is used to generate an approximate point near the goal-point.
RRT* planner is used to generate a path from MiRo to the montecarlo given goal-point.
Voice command - go (for now typed in the workstation) is used to move MiRo towards the goal, as it avoids the obstacle.
MiRo reaches the goal position.
MiRo can also be stopped by a voice command - stop (for now typed in the workstation).

Modules (files) in the system:

File 'miro_teleop/src/command_logic.cpp' - Main Command Logic node.
File 'miro_teleop/src/interpreter.cpp' - Interpreter node.
File 'miro_teleop/src/gesture_processing.cpp' - Gesture Processing server.
File 'miro_teleop/src/monte_carlo.cpp' - Monte Carlo Simulation server.
File 'miro_teleop/src/pertinence_mapping.cpp' - Pertinence Mapping server.
File 'miro_teleop/src/spatial_reasoner.cpp' - Spatial Reasoner server.
File 'miro_teleop/src/robot_controller.cpp' - Robot Controller node.
File 'miro_teleop/srv/GestureProcessing.srv' - Gestutre Processing service
File 'miro_teleop/srv/MonteCarlo.srv' - Monte Carlo Simulation service
File 'miro_teleop/srv/PertinenceMapping.srv' - Pertinence Mapping service
File 'miro_teleop/srv/SpatialReasoner.srv' - Spatial Reasoner service
File 'miro_teleop/srv/rrtStarSRV.srv' - RRT* service
File 'miro_teleop/msg/Path.msg' - msg used to transfer trajectory from Command Logic to Robot Controller
File 'rrtStar/src/rrts_main.cpp' - RRT* server
File 'mocap_optitrack-master/launch/mocap.launch' - Launches mocap node
File 'miro_teleop/launch/miro_teleop.launch' - Launches the whole application including mocap nodes
File 'run_miro_teleop' - Shell script to run all nodes in separate terminals
File 'doc/html/index.html' - Doxygen generated documentation

Limitations and proposed extensions of the system:

  • Dependence on Motion Capture data: Code in robot_controller.cpp to be modified for implementing other sensors like odometry.
  • Workspace is 4mx4m square with origin at center: HSIZE and VSIZE in all cpp files to be modified for changing dimensions. Modify RES in all files for required discretization resolution.
  • One static obstacle in workspace: Modify spatial_reasoner.cpp to include more.
  • Basic string matching to interpret commands: Call grammar parsing ROSJAVA service from interpreter.cpp.

About

Software Architecture: Teleoperation with MIRO

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • HTML 64.3%
  • C++ 24.4%
  • CSS 5.5%
  • C 4.2%
  • CMake 1.6%