MSBuild Server - Agreeing and achieving desirable usage mode

[JanKrivanek]

Context

MSBuild server is currently opt-in. There might be scenarios where it might bring performance gains. There as well might be possibility to further leverage MSBuild Server (though there is a overlap with current VS build caching techniques). The value and trade-off might not be fully clear

Goal

• Collect feedback from stakeholders (PM/EM), suggest the path forward (if any) and secure stakeholders buy-in for those
• If needed actions are identified - start properly tracking and prioritizing those

Blockers

• NuGet.Build.Tasks caches CredentialProvider device flow timeout. NuGet/Home#12540
• Server: Flush console output periodically #10231

[baronfel]

One of the reasons it's off by default is a known issue with the NuGet credential providers used in Restore - which has been fixed as of a couple weeks ago! We should be clear to pick this effort back up now.

[baronfel]

The general approach we should use here is:

• enabling Server for the MSBuild Repo and dogfooding for a bit
• enabling Server for dotnet/runtime, dotnet/aspnetcore, and dotnet/sdk as a second 'ring' of testing
• changing the default in the .NET SDK so that starting in a specific .NET SDK 10 preview release the default is back on. This will require a breaking change doc in the SDK change docs, but that is a well-understood process.