Turn on TMA by default for row-wise GEMM (#2450)

Summary:
Pull Request resolved: #2450

X-link: facebookresearch/FBGEMM#189

Enabling the TMA row-wise GEMM by default it TMA appears to give quite some speedup across-the-board, up to 40% for some shapes.

Reviewed By: choutim

Differential Revision: D62212842

fbshipit-source-id: 59220cec90e222fe91be9f53a3477f1c38e02e2a

torchbenchmark/operators/fp8_gemm_rowwise/operator.py

@@ -24,7 +24,7 @@ def parse_args(args: List[str]) -> argparse.Namespace:
     parser.add_argument(
         "--no_fp8_fast_accum", dest="fp8_fast_accum", action="store_false"
     )
-    parser.add_argument("--use_tma", action="store_true")
+    parser.add_argument("--no_use_tma", dest="use_tma", action="store_false")
     args = parser.parse_args(args)
     return args