chapter 6

code/unreadable_opt.svh

@@ -0,0 +1,19 @@
+
+// Assignment: Create a counter with a direction flag
+
+// student submission (13 logic cells)
+always_comb begin
+    for (int i = 0; i<WIDTH; i++) begin
+        // logic here is to toggle if counting up and lowerbits = max
+        // or counting down and lowerbits = 0
+        lower_mask = (1<<i) - 1;
+        count_d[i] =
+        count_q[i] ^ (~dir_i & (count_q & lower_mask) == lower_mask |
+                       dir_i & (count_q & lower_mask) == 0);
+    end
+end
+
+// teacher solution (14 logic cells)
+always_comb begin
+    count_d = count_q + ((dir_i) ? -1 : 1);
+end

figures/unreadable_opt.tex

@@ -0,0 +1,7 @@
+
+\begin{figure}[t]
+    \centering
+    \inputminted[frame=single]{systemverilog}{code/unreadable_opt.svh}
+    \caption{Student drastically reduced readability and transferability in order to save 1 logic cell over the teacher solution.}
+    \label{fig:unreadable_opt}
+\end{figure}

tex/chapters/6_autograders.tex

@@ -6,19 +6,21 @@ \chapter{Autograders}
 
 \section{Autograders offer instant, high-quality feedback.}
 
-Students are empowered to submit their code multiple times, enabling them to refine their solutions and learn from their mistakes in real time. This back-and-forth approach ensures that students can practice a Verilog concept and receive as much help as they need until they pass all the instructor-defined tests. Autograders fall under the educational approach known as ``Ungrading,'' where the emphasis shifts strongly toward providing valuable feedback over assigning traditional grades. In the autograders that I set up, it's worth noting that a significant majority of students eventually achieve a 100\% score by the assignment deadline. This phenomenon essentially transforms the grading system into a confidence-building mechanism rather than a competitive ranking system. Ungrading has been shown to help students by reducing stress, inspiring creativity, and encouraging healthy risk taking. [ref] However, arguably Ungrading's largest downside is that the instructor may not have time to provide personalized feedback to all students. [ref] Fortunately, an intrinsic attribute of software, (such as HDL implementations), is that code quality and correctness can be run with automatic, subjective computer algorithms. Therefore, by implementing autograders, Verilog educators can tap into this pedagogical insight extremely easily, offering students a more effective way to grasp digital design principles.
+Students are empowered to submit their code multiple times, enabling them to refine their solutions and learn from their mistakes in real time. This back-and-forth approach ensures that students can practice a Verilog concept and receive as much help as they need until they pass all the instructor-defined tests. Autograders fall under the educational approach known as ``Ungrading,'' where the emphasis shifts strongly toward providing valuable feedback over assigning traditional grades. In the autograders that I set up, it's worth noting that a significant majority of students eventually achieve a 100\% score by the assignment deadline. This phenomenon essentially transforms the grading system into a confidence-building mechanism rather than a competitive ranking system. Ungrading has been shown to help students by reducing stress, inspiring creativity, and encouraging healthy risk taking. \cite{kohn:book, blum:article} However, arguably Ungrading's largest downside is that the instructor may not have time to provide personalized feedback to all students. Fortunately, an intrinsic attribute of software, (such as HDL implementations), is that code quality and correctness can be run with automatic, subjective computer algorithms. Therefore, by implementing autograders, Verilog educators can tap into this pedagogical insight extremely easily, offering students a more effective way to grasp digital design principles.
 
 \section{Autograders can run remotely without complex local-setup.}
 \label{section:complex_tool_setups}
 
 When instructing students on crafting Verilog code that maintains accurate synthesizability across various platforms, it is essential to follow the industry standard of verifying a design with a wide selection of tools. Autograders streamline this process, making it accessible and efficient for students to perform comprehensive testing without the need for local installation. For example, the autograders that I created for ECE 152A, 154A, and 154B would consistently use anywhere from 6 to 10 different tools, sometimes requiring complex installation and setup procedures. Expecting students to complete these setup procedures is often tedious and counterproductive. Therefore, simply giving students access to a fully prepared autograder can remove the setup barrier completely.
 
-As mentioned, an autograder test suite that closely mirrors industry quality should follow all the verification steps demonstrated in Figure [fig]. First, it is important to run behavioral simulations with multiple tools such as Icarus, which supports propagation of unknown (\mintinline{systemverilog}{x}) values, and simulation with Verilator, which has stronger restrictions on bad syntax. Only by passing simulations with both tools should the autograder grant full points. Furthermore, code linters such as Verilator and Verible can ensure adherence to essential coding standards and practices, checking for issues like latches in \mintinline{systemverilog}{always_comb} blocks, correct use of blocking and non-blocking assignments, net-width discrepancies, and more. Considering that the frontends of tools do not always offer helpful warnings, this detailed syntax checking from Verilator and Verible is invaluable for students when fixing otherwise cryptic issues. Then, to deploy SystemVerilog synthesis with open-source tools, Yosys and Nextpnr must be paired with a frontend such as Surelog or zachjs/sv2v. The Yosys synthesis and Nextpnr layout process can verify if students are using too many logic cells, if their design is too slow, or if their design infers prohibited logic cells. As a final post-synthesis step, Icarus can be run one final time on the Yosys output to initiate a gate-level simulation (GLS) with unknown value propagation, and Yosys EQY can be run to perform logical equivalence checking (LEC). All of these features (*aside from LEC with EQY {ran out of time}) have been successfully implemented in autograders from ECE 152A, 154A, and 154B.
+As mentioned, an autograder test suite that closely mirrors industry quality should follow all the verification steps demonstrated in \autoref{fig:asic_flow}. First, it is important to run behavioral simulations with multiple tools such as Icarus, which supports propagation of unknown (\mintinline{systemverilog}{x}) values, and simulation with Verilator, which has stronger restrictions on bad syntax. Only by passing simulations with both tools should the autograder grant full points. Furthermore, code linters such as Verilator and Verible can ensure adherence to essential coding standards and practices, checking for issues like latches in \mintinline{systemverilog}{always_comb} blocks, correct use of blocking and non-blocking assignments, net-width discrepancies, and more. Considering that the frontends of tools do not always offer helpful warnings, this detailed syntax checking from Verilator and Verible is invaluable for students when fixing otherwise cryptic issues. Then, to deploy SystemVerilog synthesis with open-source tools, Yosys and Nextpnr must be paired with a frontend such as Surelog or zachjs/sv2v. The Yosys synthesis and Nextpnr layout process can verify if students are using too many logic cells, if their design is too slow, or if their design infers prohibited logic cells. As a final post-synthesis step, Icarus can be run one final time on the Yosys output to initiate a gate-level simulation (GLS) with unknown value propagation, and Yosys EQY can be run to perform logical equivalence checking (LEC). All of these features (*aside from LEC with EQY {ran out of time}) have been successfully implemented in autograders from ECE 152A, 154A, and 154B.
 
 \section{``For-fun'' leaderboards can excite and inspre students.}
 \label{section:leaderboard}
 
-Aside from a grade assigned by the autograder, another form of feedback can be provided through a class leaderboard. Students can see how their submission compares to the rest of the class on statistics such as logic cell usage, max clock frequency, or branch predictor hit percentage. Since the leaderboards in ECE 152A and 154B didn't count towards any points, students really enjoyed seeing how each of their designs compared to their peers' designs, then would try to beat their friends for bragging rights. Because of this, I added leaderboards to every assignment that I could. However, it is important to clarify to students that code readability should be prioritized over moving up in the leaderboard by saving 1-2 logic cells. However, because autograders running purely open-source tools must only rely on Yosys and ABC for synthesis, students may be incorrectly rewarded for submissions that are not well-optimized for other more prevalent synthesis tools such as Design Compiler or Vivado. This was a rare edge case that only visibly affected 1 submission ([fig]) total across the 800+ different submissions from the classes I introduced leaderboards to, but is important to monitor. (See more in \autoref{section:optimizations_in_netlist_graph_viewers}.) Overall, the leaderboard increased student enthusiasm without bringing additional stress or responsibilities.
+\input{figures/unreadable_opt}
+
+Aside from a grade assigned by the autograder, another form of feedback can be provided through a class leaderboard. Students can see how their submission compares to the rest of the class on statistics such as logic cell usage, max clock frequency, or branch predictor hit percentage. Since the leaderboards in ECE 152A and 154B didn't count towards any points, students really enjoyed seeing how each of their designs compared to their peers' designs, then would try to beat their friends for bragging rights. Because of this, I added leaderboards to every assignment that I could. However, it is important to clarify to students that code readability should be prioritized over moving up in the leaderboard by saving 1-2 logic cells. However, because autograders running purely open-source tools must only rely on Yosys and ABC for synthesis, students may be incorrectly rewarded for submissions that are not well-optimized for other more prevalent synthesis tools such as Design Compiler or Vivado. This was a rare edge case that only visibly affected 1 submission total across the 800+ different submissions from the classes I introduced leaderboards to, (see \autoref{fig:unreadable_opt}), but is important to monitor. (See more in \autoref{section:optimizations_in_netlist_graph_viewers}.) Overall, the leaderboard increased student enthusiasm without bringing additional stress or responsibilities.
 
 \section{Autograders can foster community and collaboration.}
 

tex/thesis.bib

@@ -1,22 +1,5 @@
 % https://www.bibtex.com/e/entry-types/
 
-@book{kohn:book,
-  title = {Ungrading: Why Rating Students Undermines Learning (and What to Do Instead) (Teaching and Learning in Higher Education)},
-  author = {Kohn, Alfie and Blum, Susan D.},
-  publisher = {West Virginia University Press},
-  isbn = {1949199827,9781949199826,9781949199833},
-  year = {2020}
-}
-
-@article{blum:article,
-  author = {Blum, Susan},
-  year = {2017},
-  month = {11},
-  title = {The significant learning benefits of getting rid of grades},
-  journal = {Inside Higher Ed},
-  url = {https://www.insidehighered.com/advice/2017/11/14/significant-learning-benefits-getting-rid-grades-essay}
-}
-
 @misc{sutherland,
   author = {Sutherland, Stuart and Mills, Don},
   title = {Synthesizing SystemVerilog: Busting the Myth that SystemVerilog is only for Verification},
@@ -261,3 +244,20 @@ @misc{ChipDev
 	howpublished = {\url{https://chipdev.io/}},
 	note = {[Accessed 19-09-2023]},
 }
+
+@book{kohn:book,
+  title = {Ungrading: Why Rating Students Undermines Learning (and What to Do Instead) (Teaching and Learning in Higher Education)},
+  author = {Kohn, Alfie and Blum, Susan D.},
+  publisher = {West Virginia University Press},
+  isbn = {1949199827,9781949199826,9781949199833},
+  year = {2020}
+}
+
+@article{blum:article,
+  author = {Blum, Susan},
+  year = {2017},
+  month = {11},
+  title = {The significant learning benefits of getting rid of grades},
+  journal = {Inside Higher Ed},
+  url = {https://www.insidehighered.com/advice/2017/11/14/significant-learning-benefits-getting-rid-grades-essay}
+}