[release 0.21] remove prototype stuff #3553

Job	Run time
linux (linux.12xlarge, cpu) / linux-job	49s
linux (linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job	1m 1s
macos (macos-m1-stable) / macos-job	1m 12s
windows (windows.4xlarge, cpu) / windows-job	1m 38s
windows (windows.g5.4xlarge.nvidia.gpu, cuda, 11.8) / windows-job	1m 30s
	6m 10s

Provide feedback