[action] [PR:13865] [Snappi] New testcases for PFC-ECN.

[mssonicbld]

Description of PR

This pull-request has changes specifically for the following commit-IDs:
a82b489
180af4d
3da40bc

This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added.

Summary:
Fixes # (issue)
#13655
#13215

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Back port request

• 202012
• 202205
• 202305
• 202311
• 202405

Approach

What is the motivation for this PR?

Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode).

How did you do it?

1. Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test.

2. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list.

3. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps.
   This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively.

4. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed.

5. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format.

6. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports.

7. Testcases:
   a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py:
   -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN.
   b. tests/snappi_tests/pfc/test_pfc_port_congestion.py:
   -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities.
   c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py:
   -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress.

How did you verify/test it?

Test case was executed on local clone.

Results of the verification:

• Test cases executed for 100Gbps interfaces.
• Two combinations - single-line-card-multi-asic and multiple-dut
  Non-congestion:

19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic
19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv
PASSED [ 16%]
snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 
19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic
19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv
PASSED [ 33%]
snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 
19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic
19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv
PASSED [ 50%]
snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 
19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic
19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv
PASSED [ 66%]
snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 
19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic
19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv
PASSED [ 83%]
snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 
19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic
19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv
PASSED [100%]
 

Over-subscription:

20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic
20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv
PASSED [ 12%]
snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 
20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic
20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv
PASSED [ 25%]
snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 
20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic
20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv
PASSED [ 37%]
snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 
20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic
20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv
PASSED [ 50%]
snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 
20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic
21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv
PASSED [ 62%]
snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 
21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic
21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv
PASSED [ 75%]
snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 
21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic
21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv
PASSED [ 87%]
snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 
21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic
21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv
PASSED [100%]

PFCWD:

01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv
PASSED [ 10%]
01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv
PASSED [ 20%]
01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv
PASSED [ 30%]
01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv
PASSED [ 40%]
01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv
PASSED [ 50%]
02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv
PASSED [ 60%]
02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv
PASSED [ 70%]
02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv
PASSED [ 80%]
02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv
PASSED [ 90%]
02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv
PASSED [100%]

Any platform specific information?

The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT.

Supported testbed topology if it's a new test case?

Documentation

[mssonicbld]

/azp run

[mssonicbld]

Original PR: #13865

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run Azure.sonic-mgmt

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).