Gazebo crash on startup #161

erdnaxe · 2022-01-07T21:43:35Z

I installed nix-ros-overlay.rosPackages.noetic.gazebo. When I run gazebo --verbose, I get:

Gazebo multi-robot simulator, version 11.9.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org

[Msg] Waiting for master.
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 192.168.56.1
Gazebo multi-robot simulator, version 11.9.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org

[Msg] Waiting for master.
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 192.168.56.1
[Msg] Loading world file [/nix/store/qgrfkh8bqddgzkawb8irnwrxxrkdq8q5-gazebo-11.9.0/share/gazebo-11/worlds/empty.world]
[ALSOFT] (EE) Failed to set real-time priority for thread: Operation not permitted (1)
[ALSOFT] (EE) Failed to set real-time priority for thread: Operation not permitted (1)
[Err] [Scene.cc:227] Service call[/shadow_caster_material_name] timed out
[Err] [Scene.cc:249] Service call[/shadow_caster_render_back_faces] timed out
[Wrn] [Scene.cc:462] Ignition transport [/scene_info] service call failed, falling back to gazebo transport [scene_info] request.
[Wrn] [GuiIface.cc:120] Qt has caught an exception thrown from an event handler. Throwing
exceptions from an event handler is not supported in Qt.
You must not let any exception whatsoever propagate through Qt code.
If that is not possible, in Qt 5 you must at least reimplement
QCoreApplication::notify() and catch all exceptions there.

[Err] [main.cc:37] Ogre Error:RuntimeAssertionException: Ogre/ShadowExtrudePointLight not found. Verify that you referenced the 'ShadowVolume' folder in your resources.cfg in initialise at /build/source/OgreMain/src/OgreShadowVolumeExtrudeProgram.cpp (line 70)

I am seeing a window with a black 3D viewport for some seconds before Gazebo crash. I am using Intel integrated graphics on NixOS 21.11.
Should Gazebo work out of the box? Did I miss something somewhere?

The text was updated successfully, but these errors were encountered:

lopsided98 · 2022-01-09T05:19:54Z

Yes, Gazebo is broken right now. A few weeks ago I fixed some of the issues, but it still ended crashing a little later in the startup process. I'm not sure I got the same error as you, but I do remember that the window would appear for a short time before it crashed. Unfortunately, I'm not sure when I will have the time to really dig into it.

If you are able to debug it, that would be really helpful. I'm not certain when it stopped working, but it may have been when I upgraded to 11.9.0, so you may want to try reverting that update to help narrow down the cause.

erdnaxe · 2022-01-09T15:13:28Z

I get the same issue when I change version in pkgs/gazebo/default.nix to:

11.5.1
11.8.1
11.9.0 (master)
11.9.1

My test command is nix-build . -A gazebo && ./result/bin/gazebo --verbose.

I am now trying to narrow down the issue by playing with Ogre version.

edit: my setup was wrong, for some reasons I was still running Gazebo 11.9.0.

erdnaxe · 2022-01-09T15:17:32Z

This issue looks like gazebosim/gazebo-classic#2700

erdnaxe · 2022-01-10T09:08:56Z

~~Gazebo built with ogre replaced by ogre1_9 or ogre1_10 also crashes.~~

~~gazebo_9 also crashes on master.~~

~~Same error if I checkout effc6ce and build gazebo_9.~~

~~Maybe it is caused by an external library that got updated in nixpkgs?~~

erdnaxe · 2022-01-10T09:12:23Z

Ah something is wrong with my setup:

$ ls -l
[...]
lrwxrwxrwx  1 erdnaxe users   57 Jan 10 10:09 result -> /nix/store/nwl4zdpsspw5kabcwbj5a7m7xvhz79sd-gazebo-9.18.0
$ ./result/bin/gazebo --verbose
Gazebo multi-robot simulator, version 11.9.0
[...]

For some reasons running gazebo_9 bin/gazebo is running 11.9.0... So all my past tests are wrong.

traversaro · 2022-01-10T09:12:27Z

Gazebo built with ogre replaced by ogre1_9 or ogre1_10 also crashes.

Do you get the same error:

[Err] [main.cc:37] Ogre Error:RuntimeAssertionException: Ogre/ShadowExtrudePointLight not found. Verify that you referenced the 'ShadowVolume' folder in your resources.cfg in initialise at /build/source/OgreMain/src/OgreShadowVolumeExtrudeProgram.cpp (line 70)

even for Ogre 1.9 and Ogre 1.10? That would be quite strange.

beezow · 2022-01-23T08:37:28Z

I was able to successfully run gazebo by using gazebo_9 rather than gazebo. While this is not a fix for the bug, I wanted to point it out as a temporary bypass for people that need to get gazebo running.

beezow · 2022-01-23T09:53:48Z

I also tried reverting to ogre1_10 within gazebo 11.9. I was able to successfully launch and run it, albeit, with the following error.

[Err] [Scene.cc:227] Service call[/shadow_caster_material_name] timed out
[Err] [Scene.cc:249] Service call[/shadow_caster_render_back_faces] timed out

However, everything did work as expected. Based on other issues, pinning the ogre version seems to be the accepted fix. Maybe @traversaro can comment more on this as they seemed to be involved on the upstream issues, and I am not familiar with either project. My branch is here, beezow/nix-ros-overlay. One caveat to this is that @lopsided98 explicitly updated the ogre version in 6178a8c. Perhaps I am missing something?

beezow · 2022-03-14T22:43:01Z

So while gazebo was working, when trying to use gazebo_ros there were still some problems.

➤ rosrun gazebo_ros gzserver -e ode worlds/emtpy.world --verbose $final
Gazebo multi-robot simulator, version 11.9.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org

munmap_chunk(): invalid pointer
/nix/store/8fj7m50ag8si1gzgdya7j2v2b9r5z1dw-ros-noetic-gazebo-ros-2.9.2-r1/lib/gazebo_ros/gzserver: line 41: 179679 Aborted                 (core dumped) GAZEBO_MASTER_URI="$desired_master_uri" GAZEBO_MODEL_DATABASE_URI="$desired_model_database_uri" gzserver $final

I was able to track this down the libgazebo_ros_paths_plugin.so as triggering the fault. When removed from the shared library list for gzserver, the error goes away. I tested this by directly calling gzserver with the args from gazebo_ros. This can be found in /nix/store/8fj7m50ag8si1gzgdya7j2v2b9r5z1dw-ros-noetic-gazebo-ros-2.9.2-r1/lib/gazebo_ros.

gzserver -e ode worlds/empty.world --verbose -s /nix/store/8fj7m50ag8si1gzgdya7j2v2b9r5z1dw-ros-noetic-gazebo-ros-2.9.2-r1/lib/libgazebo_ros_paths_plugin.so -s /nix/store/8fj7m50ag8si1gzgdya7j2v2b9r5z1dw-ros-noetic-gazebo-ros-2.9.2-r1/lib/libgazebo_ros_api_plugin.so

I dont know if this is related or not but wanted to document my efforts to get gazebo working again

beezow · 2022-03-15T03:50:37Z

After doing some debugging,
ros::package::getPlugins("gazebo_ros","gazebo_media_path",gazebo_media_paths);
in libgazebo_ros_paths_plugin.cpp is causing problems. In fact, every call to ros::package::getPlugins will trigger the fault.
My GDB stacktrace:

#0  0x00007ffff3b98baa in raise () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#1  0x00007ffff3b83523 in abort () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#2  0x00007ffff3bd92e8 in __libc_message () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#3  0x00007ffff3be0b0a in malloc_printerr () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#4  0x00007ffff3be0f3c in munmap_chunk () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#5  0x00007ffff3be5ac3 in free () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#6  0x00007ffff3c221bd in closedir () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#7  0x00007ffff4cec375 in boost::filesystem::detail::dir_itr_close(void*&, void*&) () from /nix/store/m53nb05wzx9d15k4xp8i29mdf104a896-boost-1.73.0/lib/libboost_filesystem.so.1.73.0
#8  0x00007fffe7f1519b in boost::filesystem::detail::dir_itr_imp::~dir_itr_imp (this=0xd69e00, __in_chrg=<optimized out>) at include/boost/filesystem/directory.hpp:292
#9  boost::sp_adl_block::intrusive_ptr_release<boost::filesystem::detail::dir_itr_imp, boost::sp_adl_block::thread_safe_counter> (p=0xd69e00) at include/boost/smart_ptr/intrusive_ref_counter.hpp:173
#10 boost::intrusive_ptr<boost::filesystem::detail::dir_itr_imp>::~intrusive_ptr (this=0x7ffffffcf6b0, __in_chrg=<optimized out>) at include/boost/smart_ptr/intrusive_ptr.hpp:98
#11 boost::filesystem::directory_iterator::~directory_iterator (this=0x7ffffffcf6b0, __in_chrg=<optimized out>) at include/boost/filesystem/directory.hpp:307
#12 rospack::Rosstackage::isStackage (this=this@entry=0x7fffe7f45200 <rospack::ROSPack::run(int, char**)::rp>, path=...) at /build/rospack-release-release-noetic-rospack-2.6.2-1/src/rospack.cpp:341
#13 0x00007fffe7f19feb in rospack::Rosstackage::crawlDetail (this=0x7fffe7f45200 <rospack::ROSPack::run(int, char**)::rp>, path=..., force=false, depth=2, collect_profile_data=false, profile_data=..., 
    profile_hash=...) at /build/rospack-release-release-noetic-rospack-2.6.2-1/src/rospack.cpp:1484
#14 0x00007fffe7f1a3fc in rospack::Rosstackage::crawlDetail (this=this@entry=0x7fffe7f45200 <rospack::ROSPack::run(int, char**)::rp>, path=..., force=<optimized out>, depth=depth@entry=1, 
    collect_profile_data=collect_profile_data@entry=false, profile_data=..., profile_hash=...) at /build/rospack-release-release-noetic-rospack-2.6.2-1/src/rospack.cpp:1546
#15 0x00007fffe7f1b721 in rospack::Rosstackage::crawl (this=this@entry=0x7fffe7f45200 <rospack::ROSPack::run(int, char**)::rp>, search_path=..., force=force@entry=false)
    at /build/rospack-release-release-noetic-rospack-2.6.2-1/src/rospack.cpp:392
#16 0x00007fffe7f2d04d in rospack::rospack_run (argc=argc@entry=4, argv=argv@entry=0xc74630, rp=..., output=...) at /build/rospack-release-release-noetic-rospack-2.6.2-1/src/rospack_cmdline.cpp:218
#17 0x00007fffe7f29319 in rospack::ROSPack::run (this=this@entry=0x7fffe7f5c0c0 <ros::package::command(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::rp>, 
    argc=argc@entry=4, argv=argv@entry=0xc74630) at /build/rospack-release-release-noetic-rospack-2.6.2-1/src/rospack_backcompat.cpp:46
#18 0x00007fffe7f297aa in rospack::ROSPack::run (this=this@entry=0x7fffe7f5c0c0 <ros::package::command(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::rp>, cmd=...)
    at /build/rospack-release-release-noetic-rospack-2.6.2-1/src/rospack_backcompat.cpp:80
#19 0x00007fffe7f4f8e6 in ros::package::command (_cmd=...) at /build/ros-release-release-noetic-roslib-1.15.8-1/src/package.cpp:55
#20 0x00007fffe7f4fbfa in ros::package::command (cmd=..., output=...) at /build/ros-release-release-noetic-roslib-1.15.8-1/src/package.cpp:73
#21 0x00007fffe7f501f1 in ros::package::getPlugins (package=..., attribute=..., packages=..., plugins=..., force_recrawl=<optimized out>) at /build/ros-release-release-noetic-roslib-1.15.8-1/src/package.cpp:118
#22 0x00007fffe7f50bb4 in ros::package::getPlugins (package=..., attribute=..., plugins=..., force_recrawl=<optimized out>) at /build/ros-release-release-noetic-roslib-1.15.8-1/src/package.cpp:155
#23 0x00007fffe5a6ede1 in gazebo::GazeboRosPathsPlugin::LoadPaths (this=<optimized out>) at /home/beezow/opt/nix-ros-overlay/testing/catkin_ws/src/gazebo_ros_pkgs/gazebo_ros/src/gazebo_ros_paths_plugin.cpp:88
#24 0x00007fffe5a6f8db in gazebo::GazeboRosPathsPlugin::GazeboRosPathsPlugin (this=0xd8d860)
    at /home/beezow/opt/nix-ros-overlay/testing/catkin_ws/src/gazebo_ros_pkgs/gazebo_ros/src/gazebo_ros_paths_plugin.cpp:54
#25 gazebo::RegisterPlugin () at /home/beezow/opt/nix-ros-overlay/testing/catkin_ws/src/gazebo_ros_pkgs/gazebo_ros/src/gazebo_ros_paths_plugin.cpp:52
#26 0x00007ffff7fa869e in ?? () from /nix/store/rcj6ykfdv59ixdqdwb2cr6ia6k3rs6kg-gazebo-11.9.0/lib/libgazebo.so.11
#27 0x00007ffff7fa8b87 in ?? () from /nix/store/rcj6ykfdv59ixdqdwb2cr6ia6k3rs6kg-gazebo-11.9.0/lib/libgazebo.so.11
#28 0x00007ffff7f762a8 in gazebo::Server::ParseArgs(int, char**) () from /nix/store/rcj6ykfdv59ixdqdwb2cr6ia6k3rs6kg-gazebo-11.9.0/lib/libgazebo.so.11
#29 0x000000000040a118 in ?? ()
#30 0x00007ffff3b84780 in __libc_start_main () from /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59/lib/libc.so.6
#31 0x000000000040a1fa in ?? ()

If anyone has any insights on what may be causing the issue that would be great.
I notice the call stack goes through boost so I wonder if somehow multiple versions are being pulled in? Or maybe something with the rospack dependencies are mucked up?

beezow · 2022-03-15T05:19:46Z

It seems gazebo was pulling in boost 173, while gazebo_ros was pulling in boost 177. By upgrading gazebo to boost177, the segfaults went away. Can finally run gazebo_ros again! You should be able to use the staging branch as it incorporates all the updates needed.

Can someone else can confirm the staging branch works for gazebo_ros?

The only remaining problem is with the shadows. There are some errors related to the shadow caster. The shadows from light sources do not work, with the light shining right through any obstacles.

lopsided98 · 2022-03-16T20:30:56Z

I finally got a chance to test this and can confirm that staging works. Staging has now been merged into master.

Do you know if the shadows not working is a new issue?

beezow · 2022-03-16T20:49:29Z

I finally got a chance to test this and can confirm that staging works. Staging has now been merged into master.

Do you know if the shadows not working is a new issue?

Not sure if it is new. It seems to be present with both gazebo_9 and gazebo_11 though. Were you able to replicate it?

lopsided98 · 2022-03-16T22:18:17Z

I don't see any shadows, but I don't see any shadow related errors either. I did notice that attempting to add more than one light results in the Gazebo freezing. It looks like the same problem as gazebosim/gazebo-classic#2373, and I was able to fix it by using OGRE 1.9.

beezow · 2022-03-17T04:44:11Z

I tried reverting to OGRE 1.9,ant the shadows seem to be working, but I still get the error messages. Additionally can confirm that using OGRE 1.9 fixes the crash with multiple light sources.

Introduce version of Ogre removed from Nixpkgs. Required by lopsided98#161 Taken from NixOS/nixpkgs@40f52d9

Introduce version of Ogre removed from Nixpkgs. Required by #161 Taken from NixOS/nixpkgs@40f52d9

traversaro mentioned this issue Feb 15, 2022

Gazebo 11 does not run correctly with Ogre 1.12 from conda-forge gazebosim/gazebo-classic#2700

Open

lopsided98 mentioned this issue Apr 28, 2023

Gazebo not building? (ogre1_9) #260

Closed

jtojnar mentioned this issue May 7, 2023

ogre1_9: init at 1.9.1 #261

Merged

jtojnar added a commit to jtojnar/nix-ros-overlay that referenced this issue May 7, 2023

ogre1_9: init at 1.9.1

8ccaa46

Introduce version of Ogre removed from Nixpkgs. Required by lopsided98#161 Taken from NixOS/nixpkgs@40f52d9

lopsided98 pushed a commit that referenced this issue Jun 19, 2023

ogre1_9: init at 1.9.1

e346d59

Introduce version of Ogre removed from Nixpkgs. Required by #161 Taken from NixOS/nixpkgs@40f52d9

lopsided98 closed this as completed Apr 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gazebo crash on startup #161

Gazebo crash on startup #161

erdnaxe commented Jan 7, 2022

lopsided98 commented Jan 9, 2022

erdnaxe commented Jan 9, 2022 •

edited

Loading

erdnaxe commented Jan 9, 2022

erdnaxe commented Jan 10, 2022 •

edited

Loading

erdnaxe commented Jan 10, 2022 •

edited

Loading

traversaro commented Jan 10, 2022 •

edited

Loading

beezow commented Jan 23, 2022

beezow commented Jan 23, 2022

beezow commented Mar 14, 2022

beezow commented Mar 15, 2022

beezow commented Mar 15, 2022

lopsided98 commented Mar 16, 2022

beezow commented Mar 16, 2022

lopsided98 commented Mar 16, 2022

beezow commented Mar 17, 2022

Gazebo crash on startup #161

Gazebo crash on startup #161

Comments

erdnaxe commented Jan 7, 2022

lopsided98 commented Jan 9, 2022

erdnaxe commented Jan 9, 2022 • edited Loading

erdnaxe commented Jan 9, 2022

erdnaxe commented Jan 10, 2022 • edited Loading

erdnaxe commented Jan 10, 2022 • edited Loading

traversaro commented Jan 10, 2022 • edited Loading

beezow commented Jan 23, 2022

beezow commented Jan 23, 2022

beezow commented Mar 14, 2022

beezow commented Mar 15, 2022

beezow commented Mar 15, 2022

lopsided98 commented Mar 16, 2022

beezow commented Mar 16, 2022

lopsided98 commented Mar 16, 2022

beezow commented Mar 17, 2022

erdnaxe commented Jan 9, 2022 •

edited

Loading

erdnaxe commented Jan 10, 2022 •

edited

Loading

erdnaxe commented Jan 10, 2022 •

edited

Loading

traversaro commented Jan 10, 2022 •

edited

Loading