Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow dashboard loading #17548

Open
2 tasks done
rafaelse opened this issue Jul 18, 2024 · 11 comments
Open
2 tasks done

Slow dashboard loading #17548

rafaelse opened this issue Jul 18, 2024 · 11 comments
Assignees

Comments

@rafaelse
Copy link

rafaelse commented Jul 18, 2024

Code of Conduct

  • I agree to follow this project's Code of Conduct

Is there an existing issue for this?

  • I have searched the existing issues

Version

10.0.16

Bug description

Some graphs from the dashboard are very slow to load or often don't load at all. It mainly applies to that graph that shows the amount of tickets by status through the months, but similar symptoms are noticed on graphs on top of the ticket listing.

We are running our workloads on AWS EKS, and the database on a db.t4g.small instance in RDS.
As can be seen on the image below, the load on the database is ok for most of its usage, except for the case mentioned above.

GLPI DB load

I wonder if there is any optimization that can be done to the database that will improve this scenario. Maybe adding some new indexes, or changing some MySQL parameters.

Even on a larger RDS instance, it takes a while to load some of the graphs.

Relevant log output

No response

Page URL

No response

Steps To reproduce

No response

Your GLPI setup information

Information about system installation and configuration
GLPI 10.0.16 ( => /var/www/html)
Installation mode: TARBALL
Current language:en_GB

Server
 
Operating system: Linux production-7c77d8d784-85nrl 5.10.219-208.866.amzn2.x86_64 #​1 SMP Tue Jun 18 14:00:06 UTC 2024 x86_64
PHP 8.3.9 fpm-fcgi (Core, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, bz2, cgi-fcgi, ctype, curl, date, dom, exif,
fileinfo, filter, gd, hash, iconv, imap, intl, json, ldap, libxml, mbstring, mysqli, mysqlnd, openssl, pcre, pdo_sqlite, posix,
random, readline, redis, session, soap, sodium, sqlite3, standard, tokenizer, xml, xmlreader, xmlrpc, xmlwriter, zip, zlib)
Setup: max_execution_time="300" memory_limit="768M" post_max_size="20M" safe_mode="" session.save_handler="files"
upload_max_filesize="20M"
disable_functions="pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,exec,passthru,shell_exec,system,proc_open,popen,curl_multi_exec,parse_ini_file,show_source"

Software: nginx
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
Server Software: Source distribution
Server Version: 8.0.36
Server SQL Mode: NO_ENGINE_SUBSTITUTION

PHP version (8.3.9) is supported.
Sessions configuration is OK.
Allocated memory is sufficient.
mysqli extension is installed.
Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter.
curl extension is installed.
gd extension is installed.
intl extension is installed.
zlib extension is installed.
The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present.
Database engine version (8.0.36) is supported.
No files from previous GLPI version detected.
The log file has been created successfully.
Write access to /var/lib/glpi/_cache has been validated.
Write access to /var/lib/glpi/_cron has been validated.
Write access to /var/lib/glpi has been validated.
Write access to /var/lib/glpi/_dumps has been validated.
Write access to /var/lib/glpi/_graphs has been validated.
Write access to /var/lib/glpi/_lock has been validated.
Write access to /var/lib/glpi/_pictures has been validated.
Write access to /var/lib/glpi/_plugins has been validated.
Write access to /var/lib/glpi/_rss has been validated.
Write access to /var/lib/glpi/_sessions has been validated.
Write access to /var/lib/glpi/_tmp has been validated.
Write access to /var/lib/glpi/_uploads has been validated.

Web server root directory configuration seems safe.
Sessions configuration is secured.
OS and PHP are relying on 64 bits integers.
exif extension is installed.
ldap extension is installed.
openssl extension is installed.
Following extensions are installed: bz2, Phar, zip.
Zend OPcache extension is installed.
Following extensions are installed: ctype, iconv, mbstring, sodium.
Write access to /var/www/html/marketplace has been validated.
Timezones seems loaded in database.

GLPI constants
 
GLPI_ROOT: "/var/www/html"
GLPI_CONFIG_DIR: "/etc/glpi"
GLPI_VAR_DIR: "/var/lib/glpi"
GLPI_LOG_DIR: "/var/log/glpi"
GLPI_MARKETPLACE_DIR: "/var/www/html/marketplace"
GLPI_USE_CSRF_CHECK: "1"
GLPI_CSRF_EXPIRES: "7200"
GLPI_CSRF_MAX_TOKENS: "100"
GLPI_USE_IDOR_CHECK: "1"
GLPI_IDOR_EXPIRES: "7200"
GLPI_ALLOW_IFRAME_IN_RICH_TEXT: false
GLPI_SERVERSIDE_URL_ALLOWLIST: ["/^(https?|feed):\/\/[^@:]+(\/.*)?$/"]
GLPI_TELEMETRY_URI: "https://telemetry.glpi-project.org"
GLPI_INSTALL_MODE: "TARBALL"
GLPI_NETWORK_MAIL: "glpi@teclib.com"
GLPI_NETWORK_SERVICES: "https://services.glpi-network.com"
GLPI_MARKETPLACE_ALLOW_OVERRIDE: true
GLPI_MARKETPLACE_MANUAL_DOWNLOADS: true
GLPI_USER_AGENT_EXTRA_COMMENTS: ""
GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: "1"
GLPI_AJAX_DASHBOARD: "1"
GLPI_CALDAV_IMPORT_STATE: 0
GLPI_DEMO_MODE: "0"
GLPI_CENTRAL_WARNINGS: "1"
GLPI_TEXT_MAXSIZE: "4000"
GLPI_DOC_DIR: "/var/lib/glpi"
GLPI_CACHE_DIR: "/var/lib/glpi/_cache"
GLPI_CRON_DIR: "/var/lib/glpi/_cron"
GLPI_DUMP_DIR: "/var/lib/glpi/_dumps"
GLPI_GRAPH_DIR: "/var/lib/glpi/_graphs"
GLPI_LOCAL_I18N_DIR: "/var/lib/glpi/_locales"
GLPI_LOCK_DIR: "/var/lib/glpi/_lock"
GLPI_PICTURE_DIR: "/var/lib/glpi/_pictures"
GLPI_PLUGIN_DOC_DIR: "/var/lib/glpi/_plugins"
GLPI_RSS_DIR: "/var/lib/glpi/_rss"
GLPI_SESSION_DIR: "/var/lib/glpi/_sessions"
GLPI_TMP_DIR: "/var/lib/glpi/_tmp"
GLPI_UPLOAD_DIR: "/var/lib/glpi/_uploads"
GLPI_INVENTORY_DIR: "/var/lib/glpi/_inventories"
GLPI_NETWORK_REGISTRATION_API_URL: "https://services.glpi-network.com/api/registration/"
GLPI_MARKETPLACE_PLUGINS_API_URI: "https://services.glpi-network.com/api/marketplace/"
GLPI_I18N_DIR: "/var/www/html/locales"
GLPI_VERSION: "10.0.16"
GLPI_SCHEMA_VERSION: "10.0.16@b13256c443dd4fdb27b4a0d3b8fea8caba4dfaa9"
GLPI_MARKETPLACE_PRERELEASES: false
GLPI_MIN_PHP: "7.4.0"
GLPI_MAX_PHP: "8.4.0"
GLPI_YEAR: "2024"

Libraries
 
htmlawed/htmlawed version 1.2.14 in (/var/www/html/vendor/htmlawed/htmlawed)
phpmailer/phpmailer version 6.8.0 in (/var/www/html/vendor/phpmailer/phpmailer/src)
simplepie/simplepie version 1.5.8 in (/var/www/html/vendor/simplepie/simplepie/library)
tecnickcom/tcpdf version 6.7.5 in (/var/www/html/vendor/tecnickcom/tcpdf)
michelf/php-markdown in (/var/www/html/vendor/michelf/php-markdown/Michelf)
true/punycode in (/var/www/html/vendor/true/punycode/src)
iamcal/lib_autolink in (/var/www/html/vendor/iamcal/lib_autolink)
sabre/dav in (/var/www/html/vendor/sabre/dav/lib/DAV)
sabre/http in (/var/www/html/vendor/sabre/http/lib)
sabre/uri in (/var/www/html/vendor/sabre/uri/lib)
sabre/vobject in (/var/www/html/vendor/sabre/vobject/lib)
laminas/laminas-i18n in (/var/www/html/vendor/laminas/laminas-i18n/src)
laminas/laminas-servicemanager in (/var/www/html/vendor/laminas/laminas-servicemanager/src)
monolog/monolog in (/var/www/html/vendor/monolog/monolog/src/Monolog)
sebastian/diff in (/var/www/html/vendor/sebastian/diff/src)
donatj/phpuseragentparser in (/var/www/html/vendor/donatj/phpuseragentparser/src/UserAgent)
elvanto/litemoji in (/var/www/html/vendor/elvanto/litemoji/src)
symfony/console in (/var/www/html/vendor/symfony/console)
scssphp/scssphp in (/var/www/html/vendor/scssphp/scssphp/src)
laminas/laminas-mail in (/var/www/html/vendor/laminas/laminas-mail/src/Protocol)
laminas/laminas-mime in (/var/www/html/vendor/laminas/laminas-mime/src)
rlanvin/php-rrule in (/var/www/html/vendor/rlanvin/php-rrule/src)
ramsey/uuid in (/var/www/html/vendor/ramsey/uuid/src)
psr/log in (/var/www/html/vendor/psr/log/Psr/Log)
psr/simple-cache in (/var/www/html/vendor/psr/simple-cache/src)
psr/cache in (/var/www/html/vendor/psr/cache/src)
league/csv in (/var/www/html/vendor/league/csv/src)
mexitek/phpcolors in (/var/www/html/vendor/mexitek/phpcolors/src/Mexitek/PHPColors)
guzzlehttp/guzzle in (/var/www/html/vendor/guzzlehttp/guzzle/src)
guzzlehttp/psr7 in (/var/www/html/vendor/guzzlehttp/psr7/src)
glpi-project/inventory_format in (/var/www/html/vendor/glpi-project/inventory_format/lib/php)
wapmorgan/unified-archive in (/var/www/html/vendor/wapmorgan/unified-archive/src)
paragonie/sodium_compat in (/var/www/html/vendor/paragonie/sodium_compat/src)
symfony/cache in (/var/www/html/vendor/symfony/cache)
html2text/html2text in (/var/www/html/vendor/html2text/html2text/src)
symfony/css-selector in (/var/www/html/vendor/symfony/css-selector)
symfony/dom-crawler in (/var/www/html/vendor/symfony/dom-crawler)
twig/twig in (/var/www/html/vendor/twig/twig/src)
twig/string-extra in (/var/www/html/vendor/twig/string-extra)
symfony/polyfill-ctype not found
symfony/polyfill-iconv not found
symfony/polyfill-mbstring not found
symfony/polyfill-php80 not found
symfony/polyfill-php81 not found
symfony/polyfill-php82 in (/var/www/html/vendor/symfony/polyfill-php82)
league/oauth2-client in (/var/www/html/vendor/league/oauth2-client/src/Provider)
league/oauth2-google in (/var/www/html/vendor/league/oauth2-google/src/Provider)
thenetworg/oauth2-azure in (/var/www/html/vendor/thenetworg/oauth2-azure/src/Provider)
phpCas version 1.4.0 in (/usr/local/lib/php)

SQL replicas
 
Not active

Plugins list
 
	news                 Name: Alerts                         Version: 1.12.3     State: Enabled                                 
		Install Method: Marketplace
	formcreator          Name: Form Creator                   Version: 2.13.9     State: Enabled                                 
		Install Method: Marketplace
	glpiinventory        Name: GLPI Inventory                 Version: 1.3.5      State: Enabled                                 
		Install Method: Marketplace
	itilcategorygroups   Name: Grupos ItilCategory            Version: 2.5.1      State: Not installed                           
		Install Method: Marketplace
	manufacturersimports Name: Importação de fornecedores     Version: 3.0.8      State: Enabled                                 
		Install Method: Marketplace

Anything else?

We ended up removing the slow graphs from the dashboards as to not affect the overall user experience with the system.
Any improvement on this front would be most welcome.

@cconard96
Copy link
Contributor

cconard96 commented Jul 18, 2024

Please enable debug mode for your user (need to be on s super-user profile). Then when on the page with the dashboard, click the SQL requests widget in the debug bar at the bottom of the screen (2nd button). Click the "Time" column header to sort by the slowest.

Can you identify the slow query(s)?

@rafaelse
Copy link
Author

We didn't know about the debug mode yet, but upon following your instructions, we noticed that it seems like it isn't the database that is slowing down the loading of the dashboard, but something related to ajax requests. However, I mentioned a possible database issue because the loading of the dashboard seemed to have slowed down after reducing the size of the DB instance.

Here are the screenshots that show the time it takes to load the dashboard:

GLPI - Slow dashboard

It takes quite a while to load the "Network inventoried devices" widget, but there aren't any slow query at all:

GLPI - dashboard queries

Do you have any idea of what is causing all this delay in loading this specific widget and the dashboard as a whole?

@cconard96
Copy link
Contributor

It looks like there is a bug that kept the data for the requests made to load the dashboard cards from being saved on the server side. So, none of the SQL requests are included in the debug bar. For the request timings, this is the entire request time including the time spent stalled. In debug mode, your session file isn't closed early so each request has to wait for the previous ones to finish in order to get a lock on the session file to be able to open it. You can see more detailed timing info in your browser's developer tools (F12 key > Network tab > Select a request > Timing sub-tab in Chrome).

For the missing data from the dashboard requests, I opened #17567 that seems to resolve the issue.

@cconard96 cconard96 self-assigned this Jul 21, 2024
@rafaelse
Copy link
Author

I'll wait for the next release to include the fix from issue #17567 so that we can get more information on which queries are slowing down the loading of some dashboard cards.

If this problem occurs to more people, then maybe some patch would be necessary. Otherwise, we'll see how we can optimize our database to improve the user experience.

@trasher
Copy link
Contributor

trasher commented Aug 9, 2024

Since the patch is quite small and has a quite low impact, you maybe can consider applying it right now; so you can report problematic queries as soon as possible.

@MinusCortex
Copy link

MinusCortex commented Oct 7, 2024

Hello everyone, i'm having the same issue as @rafaelse

we are running GLPI 10.0.16 behind an AWS Application loadbalancer (ALB) , with 2 ec2 instances and an EFS to store GLPI configuration the database is handled by a db.t4g.small RDS instances with mariadb 10.6.18.

dashboards cards are slow to load and some of the cards won't load at all, we discovered this issue when we started using the helpdesk dashboard who has a lot of issue while loading.
one of the workaround was to remove some cards and keep only the strict necessary but it's not ideal.

image

some of them reach the maxium request time and throw a 504 error

image

Copy link
Contributor

github-actions bot commented Dec 6, 2024

There has been no activity on this issue for some time and therefore it is considered stale and will be closed automatically in 10 days.

If this issue is related to a bug, please try to reproduce on latest release. If the problem persist, feel free to add a comment to revive this issue.
If it is related to a new feature, please open a topic to discuss with community about this enhancement on suggestion website.

You may also consider taking a subscription to get professionnal support or contact GLPI editor team directly.

@github-actions github-actions bot added the Stale label Dec 6, 2024
@rafaelse
Copy link
Author

I would like to add that those boxes in "Personal View" and "Group View" are also very slow to load (>=10s), but I noticed that they only take that long when we have plugins activated.

I should also mention that this is happening with our staging environment where we're testing using AWS S3 mountpoint and EFS for volumes, whereas our production environment backed by EBS volumes is running fine. I've also tried EBS in this staging environment, but the slowness still persists.

In case you guys have any hint as to what the problem might be, it would be most welcome. Otherwise, I will try to replicate the production environment and apply changes one by one until the cause comes up.

As of now, we ruled out database (RDS) as the source of delay.

@github-actions github-actions bot removed the Stale label Dec 10, 2024
@rafaelse
Copy link
Author

rafaelse commented Jan 5, 2025

This problem afore mentioned was already solved, still keeping S3 and EFS as persistent volumes.

However, returning to the original issue, the referred box still takes a while to load. After applying this fix, it is now noticeable that the above average delay is due to a long SQL query:

SELECT DISTINCT `glpi_networkequipments`.`id` AS id, '789899' AS currentuser,
                        `glpi_networkequipments`.`entities_id`, `glpi_networkequipments`.`is_recursive`,  `glpi_networkequipments`.`name` AS `ITEM_NetworkEquipment_1`,
                        `glpi_networkequipments`.`id` AS `ITEM_NetworkEquipment_1_id`,
                        `glpi_entities`.`completename` AS `ITEM_NetworkEquipment_80`,  `glpi_manufacturers`.`name` AS `ITEM_NetworkEquipment_23`,  `glpi_networkequipmentmodels`.`name` AS `ITEM_NetworkEquipment_40`,  `glpi_locations`.`completename` AS `ITEM_NetworkEquipment_3`,   GROUP_CONCAT(DISTINCT CONCAT(IFNULL(`glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`.`name`, '__NULL__'),
                                               '$#$',`glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`.`id`) ORDER BY `glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`.`id` SEPARATOR '$$##$$')
                              AS `ITEM_NetworkEquipment_126`,
                  
                   GROUP_CONCAT(DISTINCT CONCAT(IFNULL(`glpi_vlans_30b720f4ff8116eb3dfe981ed77541e0`.`name`, '__NULL__'),
                                               '$#$',`glpi_vlans_30b720f4ff8116eb3dfe981ed77541e0`.`id`) ORDER BY `glpi_vlans_30b720f4ff8116eb3dfe981ed77541e0`.`id` SEPARATOR '$$##$$')
                              AS `ITEM_NetworkEquipment_88`,
                  
                  `glpi_networkequipments`.`serial` AS `ITEM_NetworkEquipment_5`,  `glpi_networkequipments`.`otherserial` AS `ITEM_NetworkEquipment_6`,  `glpi_networkequipments`.`id` AS `ITEM_NetworkEquipment_178_id`, `glpi_networkequipments`.`name` AS `ITEM_NetworkEquipment_178_name`,  COUNT(DISTINCT `glpi_problems_ad1d1102b08981d196811ea88b1a2f20`.`id`) AS `ITEM_NetworkEquipment_140`,
                     `glpi_networkequipments`.`date_creation` AS `ITEM_NetworkEquipment_121`,  `glpi_networkequipments`.`date_mod` AS `ITEM_NetworkEquipment_19`,  `glpi_autoupdatesystems`.`name` AS `ITEM_NetworkEquipment_72` 
FROM `glpi_networkequipments`LEFT JOIN `glpi_entities` 
                                          ON (`glpi_networkequipments`.`entities_id` = `glpi_entities`.`id`
                                              )LEFT JOIN `glpi_manufacturers` 
                                          ON (`glpi_networkequipments`.`manufacturers_id` = `glpi_manufacturers`.`id`
                                              )LEFT JOIN `glpi_networkequipmentmodels` 
                                          ON (`glpi_networkequipments`.`networkequipmentmodels_id` = `glpi_networkequipmentmodels`.`id`
                                              )LEFT JOIN `glpi_locations` 
                                          ON (`glpi_networkequipments`.`locations_id` = `glpi_locations`.`id`
                                              ) LEFT JOIN `glpi_ipaddresses`  AS `glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`
                                          ON (`glpi_networkequipments`.`id` = `glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`.`mainitems_id`
                                              AND `glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`.`mainitemtype` = 'NetworkEquipment'
                                               AND `glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`.`is_deleted` = '0' AND  NOT (`glpi_ipaddresses_3714572b4d6732a91ac0d68a00a3c328`.`name` = '') )  LEFT JOIN `glpi_networkports` 
                                          ON (`glpi_networkequipments`.`id` = `glpi_networkports`.`items_id`
                                              AND `glpi_networkports`.`itemtype` = 'NetworkEquipment'
                                              )  LEFT JOIN `glpi_networkports_vlans` 
                                             ON (`glpi_networkports`.`id` = `glpi_networkports_vlans`.`networkports_id`
                                                 )LEFT JOIN `glpi_vlans`  AS `glpi_vlans_30b720f4ff8116eb3dfe981ed77541e0`
                                          ON (`glpi_networkports_vlans`.`vlans_id` = `glpi_vlans_30b720f4ff8116eb3dfe981ed77541e0`.`id`
                                              ) LEFT JOIN `glpi_items_problems` 
                                          ON (`glpi_networkequipments`.`id` = `glpi_items_problems`.`items_id`
                                              AND `glpi_items_problems`.`itemtype` = 'NetworkEquipment'
                                              ) LEFT JOIN `glpi_problems`  AS `glpi_problems_ad1d1102b08981d196811ea88b1a2f20`
                                          ON (`glpi_items_problems`.`problems_id` = `glpi_problems_ad1d1102b08981d196811ea88b1a2f20`.`id`
                                               )LEFT JOIN `glpi_autoupdatesystems` 
                                          ON (`glpi_networkequipments`.`autoupdatesystems_id` = `glpi_autoupdatesystems`.`id`
                                              ) 
WHERE  `glpi_networkequipments`.`is_deleted` = 0  AND `glpi_networkequipments`.`is_template` = 0  AND (    (`glpi_autoupdatesystems`.`name`  LIKE '%GLPI Native Inventory%' ) ) GROUP BY `glpi_networkequipments`.`id` ORDER BY `ITEM_NetworkEquipment_1` ASC

It takes approximately 7s to run and is the only slow box in our dashboard.
Any improvements to this query would be most welcome. Otherwise, I will close this issue since it is not a big deal and we now know how to use the debug mode to find possible sources of delay.

@freelol95
Copy link

I upgrade from 10.0.16 to 10.0.17 and now i have the same slow loading in the dashboard ;-/

Any fix about it ?

@freelol95
Copy link

Solution, tuner le mod_evasive d'apache ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants