Expand torch.compile feature set, including new backends

[ScottTodd]

(Issue migrated from nod-ai/SHARK-ModelDev#94)

---

In general, the torch.compile path could use some attention and new features.

dynamo/backends/ currently only includes cpu.py, which is registered here:

iree-turbine/setup.py

Lines 101 to 105 in bd6ebc6

	entry_points={
	"torch_dynamo_backends": [
	"turbine_cpu = shark_turbine.dynamo.backends.cpu:backend",
	],
	},

Would other backends be added as separate registrations, or as a single registration with options?

Does Dynamo even offer a way to pass options to custom backends? I see mode and option parameters on torch.compile (https://pytorch.org/docs/stable/generated/torch.compile.html), but those look overly specific to Inductor, Triton, etc. The contract between torch.compile and custom backends doesn't seem to include options? See also https://pytorch.org/docs/main/torch.compiler_custom_backends.html :

A backend function has the contract (gm: torch.fx.GraphModule, example_inputs: List[torch.Tensor]) -> Callable.

[ScottTodd]

Oh, the options parameter dictionary looks freeform enough that we could pass through our own flags there. I see some sample code online of people using options similar to those at https://pytorch.org/TensorRT/py_api/ts.html, for example. The torch.compile documentation just says "Some notable ones to try out are" and focuses on the in-tree backends.