Long time to process BOMs and stats

[dc-re]

It may take days or week to process BOMs and update statistics.

We are running DT 3.8.0 in a docker container, Docker is hosted in a t3.large instance in AWS. No other containers have been deployed in this Docker instance. CPU load of the whole instance rarely exceeds 10%.

We have 114 projects with ~11000 components.

In the logs I don't see anything wrong, apart from several the OSS index analysis tasks taking up to 1-2 minutes to complete:

2021-02-15 10:23:20,968 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Analyzing 100 component(s)
2021-02-15 10:24:08,773 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Analyzing 100 component(s)

Is this behaviour (taking days to process BoMs and update stats) expected with our organisation size?

[dc-re]

I apologize in advance for the lack of details, but I couldn't find anything wrong with our installation. This started happening when the number of projects increased, when we had only 10-20 projects, we didn't experience any significant delay between the BOM upload and its processing.

[stevespringett]

DT 4.0 and higher have some performance improvements, especially with regard to metrics. But I've never seen an installation take days to process a BOM.

The largest DT install I've seen is 1000 projects. So I know it can be done. I would recommend looking at the containers. They are typically underpowered. T3 likely isn't an ideal choice as it's not optimized for CPU and memory. It t3.large only has 2 CPUs which is the minimum recommended by DT. I would also look at the database server and see if there's some performance tuning that can be done there. RDS can be slow if not provisioned correctly.

[dc-re]

Thank you @stevespringett, I hadn't realised we are using a micro burstable instance for the DB, which seems to be the bottleneck. We are going to upgrade it tomorrow.