Cisco-8000 test pfcwd function background traffic revision

[rbpittman]

Description of PR

Summary:

• Revise Cisco-8000's current method of sending background traffic from an extra "verify_tx_egress" call to use the pre-existing background traffic support. The verify tx egress operation could fail if some packets leakout due to slow pfc_gen.py frames from fanout.
• Remove the test failure catcher, as it simply rethrows an exception that is more difficult to debug. Using a simple "try-finally" pattern allows the "finally" clause to always execute even with an exception, but also continues to throw the exception without catching it, resulting in the required test failure in that case.

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Back port request

• 202012
• 202205
• 202305
• 202311
• 202405

Related PRs

#14711 is not in 202405, but is requested for the branch. Without that PR in 202405, the double commit of this PR will have a either have a merge conflict or possibly fail if it happens to not conflict. Possible to get that PR in 202405?

Approach

What is the motivation for this PR?

Fix flaky test on Cisco-8000.
Clarify exceptions that do occur for all vendors.

How did you do it?

How did you verify/test it?

Verified pass 10 times on Cisco-8122 T0 on 202405 branch. But master/202405 aren't in sync yet due to missing 14711 PR.

Any platform specific information?

The verify_tx_egress change is cisco-only.
The exception catcher removal should improve exception readability for all vendors.

Supported testbed topology if it's a new test case?

Documentation

[alpeshspatel]

Pinged Kevin/Storm to back port the pr 14711 to 202405

[kevinskwang]

LGTM

[mssonicbld]

Cherry-pick PR to 202405: #15841

[auspham]

Hi @rraghav-cisco, this could be the potential issue for pfc_function that failed in having pfcwd storm detection step and still send out packet in verify_tx_egress. Could you have a look please.