# Troubleshooting

If you have a problem, check below if it is already answered. If the steps don't help, please ask in MS Teams.

### Something is going strangely

Try these things in order:

1. **Never forget to source after building**: `cd ~catkin_ws && source devel/setup.bash` (or just `s` in the container)

2. Clean and rebuild your workspace: `cd ~catkin_ws && catkin clean -y && catkin build`

3. Run `o2ac-build-catkin-workspace` (this builds both workspaces).

4. Run `o2ac-reset-catkin-workspace` (this cleans and rebuilds both workspaces).

5. On the host computer, run `cd ~/o2ac-ur && git submodule update --init` and retry from step 3.

### catkin build does not work / Error during build

Are you using the newest changes in the develop branch? Did you rebuild the docker image? Have you tried `cd ~catkin_ws && catkin clean -y && catkin build` (this removes old build files and starts fresh)? In the host PC, have you initialized your submodules on the host PC (not inside the docker container) by executing `cd ~/o2ac-ur && git submodule update --init --recursive`?

Also, try `o2ac-reset-catkin-workspace` once again.

### A camera or sensor I want to use does not work

- Newly installed drivers and software need to be added to the installation commands in the dockerfile for our shared environment. Add the installation commands near the bottom of the Dockerfile so that the previous instructions can be reused.
- You need to restart the docker container when you plug in a new USB device: `docker restart [container_name]` (press tab to auto-complete)

### moveit does not load, xacro / param errors

If you just cloned the repository on a new machine, you may need to run `rosrun o2ac_parts_description generate_urdf_from_meshes.py` and `... generate_assembled_piece.py`?

### Cannot access my files (no rights, access errors, could not save, permission denied etc.)

Inside docker, run `o2ac-fix-permission-issues.sh` (press tab to auto-complete)

Explanation: When files are created inside docker, they belong to the user "root". The host computer does not have the rights to those files. The script sets the host PC's account as the owner of the files.

### "segmentation fault" / Rviz and Gazebo do not start up

Your graphics driver is not configured correctly.

* Do you have an Nvidia GPU? If not, follow [these instructions](http://wiki.ros.org/docker/Tutorials/Hardware%20Acceleration#Intel)
* Have you checked out the develop branch and followed all the installation instructions, including the new install scripts?
* Are you putting "sudo" in front of any commands where it does not belong? You should enter the container without sudo.
* If you enter "echo $DOCKER_RUNTIME" on the host, what is the result? It needs to be "nvidia" if you are using a GPU and "runc" if you do not have an nvidia GPU. You may need to update it in your ~/.bashrc file.

### Couldn't find an AF_INET address for [...]

Did you enter the docker container in all terminals? Are all the hostnames in /etc/hosts set correctly? What is ROS_IP set to?

If you are connecting a new machine to the system, this is related to the network configurations. Are all the hostnames entered in the `/etc/hosts` of each machine? Is `/etc/hostname` correct? Does `echo $ROS_MASTER_URI` point to the hostname of the PC running roscore (there can only be one)? Also check ROS_HOSTNAME, ROS_IP.

### The UR is "overshooting" the target; does not stop at the target pose; the movement keeps going

Confirm that the "Speed" setting on the pendant is set to 100%. If that does not fix the problem, power off the robot and restart it. If that does not help, reset the robot to its factory settings.

### Cannot connect to docker

`ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?`

Your user account is probably not in the "docker" group. Follow these instructions: https://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo

### Something is still not going right

To do a clean rebuild of both workspaces (takes 7+ minutes), run `o2ac-reset-catkin-workspace`. 

If that does not help, consider asking for help, or rebuilding the docker image from scratch (pass the `--no-cache` argument and/or run `docker system prune` after removing the container). This takes a long time.

### Problem with the realsense camera inside the docker

* Unplug/Replug the USB cable.
* Verify that the camera is plugged on a USB3.0 port. If the Realsense camera doesn't show up with `dmesg | grep Real` on the host then the cable might have a problem. You can also use `realsense-viewer` on the host.
* If none of the above works and realsense is not installed on the host. Install the realsense camera on the host following the instruction and test that the installation went well with realsense-viewer.
* After reboot, the camera might not work. In this case type the command `sudo modprobe uvcvideo`
* Test `realsense-viewer` on the host and it should work again. 
* If it's working, restart the docker so that the change in the kernel are effective inside the docker too.
* Try `realsense-viewer` inside the docker and it should work.

## Problems with the Panda

- If you have connection problems, make sure the WiFi is turned off on both machines (the Panda desktop and the master PC (`o2ac-ur`))
- If the joints cannot be unlocked or something is wrong, restart the end effector.
- If an error is displayed, either restart the controller with `roslaunch franka_control osx_franka_control.launch` or call the error recovery function as shown in `o2ac_routines/src/panda_control.py`

## Problems with the UR

- Can you ping the robot at its IP? Does its network connection look active? The cable may need to be unplugged and plugged back in after starting up the robot.
- Did you start the `external_control` program on the teaching pendant? The line "Robot ready to receive control commands" needs to appear in the terminal.
- If the joints are purple in Rviz, this means that the joint limits are exceeded. Check the teaching pendant if the joints are overextended or have an additional rotation.

## The gripper does not open/close

- The robot needs to be put into `Remote Control` mode, otherwise it will not execute scripts sent to it. It seems that `connect_real_robots.launch` also needs to be restarted after activating remote control mode on the robots.

### Ethernet connection does not work when WiFi is on

The Wireless connection is taking higher priority than the ethernet for the robots' IP address range. We had this problem when the robots were on 192.168.0.x and 192.168.1.x address ranges, and the subnet mask was set to 255.255.0.0. Apparently the problem was that the wifi had the 192.168.0.0/24 network and the ethernet the 192.168.0.0/16 network. The former is more specific, so it always took priority.

We solved this by setting all the robots and desktop PCs to the 192.168.1.X range and setting the subnet mask to 255.255.255.0 on the wired connection.

### Arms do not connect (connect_real_robots dies), fastening tools do not connect

Is the USB hub connecting the tools turned on? Press the ON button on the hub.

### Docker image needs to be rebuilt from scratch

Add " --no-cache" to the last line of `BUILD-DOCKER-IMAGE.sh` and execute it once. Don't forget to remove it afterwards.


### Rviz does not start up / libGL error: failed to load driver: swrast

Most likely, your Nvidia graphics driver is not working correctly. Try following these, making sure that the `nouveau` driver is not selected:

https://linuxconfig.org/how-to-install-the-nvidia-drivers-on-ubuntu-20-04-focal-fossa-linux/

https://askubuntu.com/questions/363200/e-unable-to-correct-problems-you-have-held-broken-packages

If you are running the realtime kernel, make sure that `echo $DOCKER_RUNTIME` returns `runc` (by running `export DOCKER_RUNTIME=runc` ) on the hostß