Web visualization

[diegoferigo]

This issue is a reminder to try the web visualization. It could be an interesting alternative to the current GUI. Trying it should not require any modification in code. Adding the following plugin to the loaded world should suffice:

  <plugin name='ignition::launch::WebsocketServer'
          filename='ignition-launch-websocket-server'>
    <port>9002</port>
  </plugin>

Alternatively, the plugin could be loaded programmatically from the APIs.

If this works, it could be implemented similarly to the existing GazeboSimulator::gui. Note that also this method would not scale automatically to multiple worlds simulated in parallel in the same machine, unless each world uses a different port. This is similar to the current GUI limitation in a multi-simulator setup.

A dynamic allocation of the port, if possible, would be great. In a setup where multiple simulators are used in parallel to train a RL policy, it could also allow to visualize the progress by attaching to one of the instances (also from a remote server!). However, we should investigate if the plugin streams continuously even if there's nobody listening. If this is the case, with many parallel simulators, the network traffic could be relevant.

Even though we currently support only GNU/Linux, this visualization method should work flawlessly on all OSs.

Upstream documentation: https://github.com/ignitionrobotics/docs/blob/master/dome/web_visualization.md.

[vfdev-5]

@diegoferigo I was trying to check that while running cartpole example for 1000 epochs.
However launching:

ign launch -v 4 websocket.ign
[Wrn] [WebsocketServer.cc:546] Partial SSL configuration specified. Please specify:        <ssl>
          <cert_file>PATH_TO_CERT_FILE</cert_file>
          <private_key_file>PATH_TO_KEY_FILE</private_key_file>
        </ssl>.
Continuing without SSL.
[Err] [WebsocketServer.cc:836] Failed to get the scene information for default world.
[Err] [WebsocketServer.cc:866] Failed to get the particle emitter information for default world.

and connecting via to the webapp nothing is shown and the error is displayed in the terminal.

Maybe I'm missing something ?

[diegoferigo]

Hi @vfdev-5, I'm not really familiar with the websocket-based visualization and I'm not sure how much it's updated upstream, I didn't see anything recent enough using it (but maybe it just didn't show up in my watch list). Right now this visualization doesn't have a high priority on our side, we're currently mainly interested to get the GUI running in the same process of the server, but there are still some bits missing from upstream.

If you're interested in exploring the web-visualization support with gym-ignition, please keep us posted with your results. First, did you try to get it working with a simple world (like rolling shapes) using plain Ignition Gazebo? Then, I would suggest to start with a simple example that just uses the low-level scenario APIs (you can also load the websocket plugin from Python as an alternative of adding it to the sdf file), and only then moving to a more complex gym-ignition example like the cartpole.

[vfdev-5]

@diegoferigo thanks for your reply !
Yes, I'm very interested in web visualization as I'm working with a server machine and no native gui visualization is possible.
First I was also checking if things were working as in upstream docs (Upstream documentation: https://github.com/ignitionrobotics/docs/blob/master/dome/web_visualization.md)
So, yes, I could visualize fuel.sdf or empty.sfd as they suggested:

ign gazebo -v 4 -s -r fuel.sdf &
ign launch -v 4 websocket.ign

I'll try next to use scenario api as you are suggesting and report back if could find anything working or not.

[vfdev-5]

TL;DR: there is something missing in the configuration in the default world configuration to be visible with web sockets. Not sure how it is possible though

Scenario API results:

1. Not working config

• Based on the code snippet from : https://robotology.github.io/gym-ignition/master/getting_started/scenario.html
  Visualization does not show anything and the command ign launch -v 4 websocket.ign gives only;

[Err] [WebsocketServer.cc:836] Failed to get the scene information for default world.
[Err] [WebsocketServer.cc:866] Failed to get the particle emitter information for default world.

2. Working configuration:

• Modified code based on the code snippet from : https://robotology.github.io/gym-ignition/master/getting_started/scenario.html

main.py

# from https://robotology.github.io/gym-ignition/master/getting_started/scenario.html

import time
import gym_ignition_models
from scenario import gazebo as scenario_gazebo

scenario_gazebo.set_verbosity(scenario_gazebo.Verbosity_debug)

# Create the simulator
gazebo = scenario_gazebo.GazeboSimulator(step_size=0.001,
                                         rtf=1.0,
                                         steps_per_run=1)


assert gazebo.insert_world_from_sdf("local_empty.sdf", "empty")

# Initialize the simulator
gazebo.initialize()

# Get the default world and insert the ground plane
world = gazebo.get_world()
world.insert_model(gym_ignition_models.get_model_file("ground_plane"))

# Select the physics engine
world.set_physics_engine(scenario_gazebo.PhysicsEngine_dart)

time.sleep(1)
gazebo.run(paused=True)

# Insert a pendulum
world.insert_model(gym_ignition_models.get_model_file("pendulum"))
gazebo.run(paused=True)
time.sleep(1)

# Get the pendulum model
pendulum = world.get_model("pendulum")

# Reset the pole position
pendulum.get_joint("pivot").to_gazebo().reset_position(0.01)
gazebo.run(paused=True)
time.sleep(1)

print("Simulate 30 seconds")
for _ in range(int(30.0 / gazebo.step_size())):
    gazebo.run()
    print("", end=" .")
    time.sleep(0.5)

print("")

print("Close the simulator")
time.sleep(5)
gazebo.close()

In the code I'm inserting local empty world (gazebo.insert_world_from_sdf("local_empty.sdf", "empty")) which is just a copy of ignition gazebo empty.sdf + renamed "ground_plane" to avoid name collision.

local_empty.sdf

<?xml version="1.0" ?>
<!--
  Try inserting a model:

ign service -s /world/empty/create \
--reqtype ignition.msgs.EntityFactory \
--reptype ignition.msgs.Boolean \
--timeout 300 \
--req 'sdf: '\
'"<?xml version=\"1.0\" ?>'\
'<sdf version=\"1.6\">'\
'<model name=\"spawned_model\">'\
'<link name=\"link\">'\
'<visual name=\"visual\">'\
'<geometry><sphere><radius>1.0</radius></sphere></geometry>'\
'</visual>'\
'<collision name=\"visual\">'\
'<geometry><sphere><radius>1.0</radius></sphere></geometry>'\
'</collision>'\
'</link>'\
'</model>'\
'</sdf>" '\
'pose: {position: {z: 10}} '\
'name: "new_name" '\
'allow_renaming: true'

  Then try deleting it:

ign service -s /world/empty/remove \
--reqtype ignition.msgs.Entity \
--reptype ignition.msgs.Boolean \
--timeout 300 \
--req 'name: "new_name" type: MODEL'

  Try inserting a light:

ign service -s /world/empty/create --reqtype ignition.msgs.EntityFactory --reptype ignition.msgs.Boolean --timeout 300 --req 'sdf: '\
'"<?xml version=\"1.0\" ?>'\
'<sdf version=\"1.6\">'\
'<light name=\"spawned_light\" type=\"directional\">'\
'<pose>0 0 10 0.1 1.0 0</pose>'\
'</light>'\
'</sdf>"'

  Then try deleting it:

ign service -s /world/empty/remove \
--reqtype ignition.msgs.Entity \
--reptype ignition.msgs.Boolean \
--timeout 300 \
--req 'name: "spawned_light" type: LIGHT'

  Insert a light using a message and allow_renaming:

ign service -s /world/empty/create \
--reqtype ignition.msgs.EntityFactory \
--reptype ignition.msgs.Boolean \
--timeout 300 \
--req 'allow_renaming: true, light: {name: "spawned_light", type: 2, diffuse: {r: 1}}'

-->
<sdf version="1.6">
  <world name="empty">
    <physics name="1ms" type="ignored">
      <max_step_size>0.001</max_step_size>
      <real_time_factor>1.0</real_time_factor>
    </physics>
    <plugin
      filename="ignition-gazebo-physics-system"
      name="ignition::gazebo::systems::Physics">
    </plugin>
    <plugin
      filename="ignition-gazebo-user-commands-system"
      name="ignition::gazebo::systems::UserCommands">
    </plugin>
    <plugin
      filename="ignition-gazebo-scene-broadcaster-system"
      name="ignition::gazebo::systems::SceneBroadcaster">
    </plugin>
    <plugin
      filename="ignition-gazebo-contact-system"
      name="ignition::gazebo::systems::Contact">
    </plugin>

    <light type="directional" name="sun">
      <cast_shadows>true</cast_shadows>
      <pose>0 0 10 0 0 0</pose>
      <diffuse>0.8 0.8 0.8 1</diffuse>
      <specular>0.2 0.2 0.2 1</specular>
      <attenuation>
        <range>1000</range>
        <constant>0.9</constant>
        <linear>0.01</linear>
        <quadratic>0.001</quadratic>
      </attenuation>
      <direction>-0.5 0.1 -0.9</direction>
    </light>

    <model name="local_ground_plane">
      <static>true</static>
      <link name="link">
        <collision name="collision">
          <geometry>
            <plane>
              <normal>0 0 1</normal>
              <size>100 100</size>
            </plane>
          </geometry>
        </collision>
        <visual name="visual">
          <geometry>
            <plane>
              <normal>0 0 1</normal>
              <size>100 100</size>
            </plane>
          </geometry>
          <material>
            <ambient>0.8 0.8 0.8 1</ambient>
            <diffuse>0.8 0.8 0.8 1</diffuse>
            <specular>0.8 0.8 0.8 1</specular>
          </material>
        </visual>
      </link>
    </model>

  </world>
</sdf>

Thus, running the following:

ign launch -v 4 websocket.ign &
SCENARIO_VERBOSE=1 python -u main.py
> 
...
[Msg] [Model.cpp:147] Model: [13] pendulum                         
[Msg] [Model.cpp:160] Links:                                       
[Msg] [Link.cpp:147]   [14] support                                 
[Msg] [Link.cpp:147]   [17] pendulum                                
[Msg] [Model.cpp:169] Joints:                                      
[Msg] [Joint.cpp:115]   [21] pivot                                 
[Wrn] [Physics.cc:993] Model's parent entity [1] not found on world / model map.
[Wrn] [Physics.cc:1028] Link's parent entity [13] not found on model map.
[Wrn] [Physics.cc:1028] Link's parent entity [13] not found on model map.
[Wrn] [Physics.cc:1086] Collision's parent entity [14] not found on link map.
[Wrn] [Physics.cc:1086] Collision's parent entity [17] not found on link map.
[Wrn] [Physics.cc:1281] Joint's parent entity [13] not found on model map.
[Err] [Physics.cc:2191] Internal error: link [5] not in entity map  
[Err] [Physics.cc:2191] Internal error: link [10] not in entity map
[Err] [Physics.cc:2191] Internal error: link [14] not in entity map 
[Err] [Physics.cc:2191] Internal error: link [17] not in entity map 
[Err] [Physics.cc:2191] Internal error: link [5] not in entity map  
[Err] [Physics.cc:2191] Internal error: link [10] not in entity map
[Err] [Physics.cc:2191] Internal error: link [14] not in entity map 
[Err] [Physics.cc:2191] Internal error: link [17] not in entity map 
Simulate 30 seconds  

gives on https://app.ignitionrobotics.org/visualization once connected:

@diegoferigo any ideas on what happens with original example and why there are errors like [Err] [Physics.cc:2191] Internal error: link [17] not in entity map ? Thanks

[diegoferigo]

Can you try to remove from the sdf file all the plugins? I mean these lines:

    <plugin
      filename="ignition-gazebo-physics-system"
      name="ignition::gazebo::systems::Physics">
    </plugin>
    <plugin
      filename="ignition-gazebo-user-commands-system"
      name="ignition::gazebo::systems::UserCommands">
    </plugin>
    <plugin
      filename="ignition-gazebo-scene-broadcaster-system"
      name="ignition::gazebo::systems::SceneBroadcaster">
    </plugin>
    <plugin
      filename="ignition-gazebo-contact-system"
      name="ignition::gazebo::systems::Contact">
    </plugin>

The upstream Physics system is not compatible with ScenarIO. The errors [Err] [Physics.cc:2191] Internal error: link [17] not in entity map occur typically when both the upstream and our own Physics systems are loaded in the same simulation.

[vfdev-5]

@diegoferigo looks like

    <plugin
      filename="ignition-gazebo-user-commands-system"
      name="ignition::gazebo::systems::UserCommands">
    </plugin>
    <plugin
      filename="ignition-gazebo-scene-broadcaster-system"
      name="ignition::gazebo::systems::SceneBroadcaster">
    </plugin>
    <plugin
      filename="ignition-gazebo-contact-system"
      name="ignition::gazebo::systems::Contact">
    </plugin>

are important to include...

However, now it seems to me that using this web visualization we can not start simulation ... Looks like we can see a static image.

I'm wondering how can we get pendulum joint position (if it really moves?):

# ...
# Get the pendulum model
pendulum = world.get_model("pendulum")

# Reset the pole position
pendulum.get_joint("pivot").to_gazebo().reset_position(0.01)
gazebo.run(paused=True)
time.sleep(1)

print("Simulate 30 seconds")
for _ in range(int(30.0 / gazebo.step_size())):
    gazebo.run(paused=True)
    time.sleep(0.5)
    print("-", pendulum.get_joint("pivot").to_gazebo().position(0))  # Should this work and print different values as pendulum moves ?

[diegoferigo]

Can you then try to just disable the Contacts and the Physics systems? I'm pretty sure they are not necessary also there. Maybe the SceneBroadcaster is required, and we load it only when the GUI is opened from the APi, which is not your case.

[vfdev-5]

@diegoferigo yes, you are right, it works without the Contacts and Physics systems.
However, nothing moves and I'm wondering if this is a limitation of https://app.ignitionrobotics.org/visualization or still there is something not configured.
I would tend to say that https://app.ignitionrobotics.org/visualization may be limited as even ign gazebo -v 4 -s -r fuel.sdf if visualized with desktop gazebo, it shows more agents on the scene and some of them are moving...

Do you know how to get pendulum joint position and to print it in the console if it moves ?
I tried print("-", pendulum.get_joint("pivot").to_gazebo().position(0)) but it shows 0.01 the value it is set before.