Use direct call to sphinx build for faster build

[khancyr]

remove subprocess call for building (make things a bit faster)
use 2 cpu for inventory parsing and building. We could probably raise this more build it will collide with the parallel build.

On rover wiki building it save 30s.

[khancyr]

need newer sphinx version to work and updated youtube plugin

[khancyr]

hum ... look like the current CI is just failing silently :

2023-10-31T22:21:23.7942369Z Handler <function download_images at 0x000001C8FBBED3F0> for event 'env-updated' threw an exception (exception: [Errno 22] Invalid argument: 'D:\\a\\ardupilot_wiki\\ardupilot_wiki\\copter\\build\\html\\_video_thumbnail\\-Db4u8LJE5w?t=103.jpg')

https://pipelinesghubeus23.actions.githubusercontent.com/gvVmBJgcaBYml9XHduM82e3qv4adNpnuFd0H59d2aO1VWNxrWB/_apis/pipelines/1/runs/8942/signedlogcontent/2?urlExpires=2023-10-31T22%3A42%3A49.7261743Z&urlSigningMethod=HMACV1&urlSignature=3aXk2tC%2FzkYoRNVwjHh3S8mp0lqPwPsyxaMneu66Myc%3D

Will have to find a solution for this to pass CI

[khancyr]

removing ?t=103 should do the trick

[khancyr]

@Ryanf55 thx for the review, I didn't do py3 codestyle update as last time it was a blocker for Peter. I will do in a following PR

[peterbarker]

@Ryanf55 are you happy with this now?

@khancyr is the wiki build output identical before/after this change? i.e. with diff -ur

[Ryanf55]

@Ryanf55 are you happy with this now?

@khancyr is the wiki build output identical before/after this change? i.e. with diff -ur

From a code perspective yes. We still saving time doing this after the latest change? I don't have access to a computer to test till Sunday.

[khancyr]

@peterbarker that show some small diff on the doctrees for the nav bar, but that seems unrelated. An example I saw is terrain navigation for copter. It is linked in two place : First time setup / configuration / failsafes mechnism and Mission Planning / terrain following.
On master terrain following page is hold by failsafes mechanism, so when you reach this page the navbar always show you the First time setup navtree.

With the build I did with this PR, it was hold by Mission Planning

This is just the navigation tree that change.

@Ryanf55 on copter wiki, I gain about 30-40s per build using this, mostly per the multi cpu use for the parsing.

Notice that using the sphinx-build is what make is using. If you open the makefile it calls sphinx-build. So we were creating a new process to call make to call sphinx-build. This PR save some ressources on this.

I can still remove the part that use multiple cpu and stick with 1 per default as before, I just tried to max out the cpu usage with some ruling that are what they are !

It could be interesting to test on the build server if it is faster to do build sequencially with full cpu usage or continue to try parallele build with lower cpu count.

[peterbarker]

Well, I'm OK with this - Henry?

[Ryanf55]

@peterbarker that show some small diff on the doctrees for the nav bar, but that seems unrelated. An example I saw is terrain navigation for copter. It is linked in two place : First time setup / configuration / failsafes mechnism and Mission Planning / terrain following. On master terrain following page is hold by failsafes mechanism, so when you reach this page the navbar always show you the First time setup navtree.

With the build I did with this PR, it was hold by Mission Planning

This is just the navigation tree that change.

@Ryanf55 on copter wiki, I gain about 30-40s per build using this, mostly per the multi cpu use for the parsing.

Notice that using the sphinx-build is what make is using. If you open the makefile it calls sphinx-build. So we were creating a new process to call make to call sphinx-build. This PR save some ressources on this.

I can still remove the part that use multiple cpu and stick with 1 per default as before, I just tried to max out the cpu usage with some ruling that are what they are !

It could be interesting to test on the build server if it is faster to do build sequencially with full cpu usage or continue to try parallele build with lower cpu count.

Thanks for the info, great to see!