Skip to content

Latest commit

 

History

History
76 lines (60 loc) · 8.3 KB

README.md

File metadata and controls

76 lines (60 loc) · 8.3 KB
Raw

Build Status

What is AuDoscore?

AuDoscore is an extension of JUnit that is used to evaluate Java programming exercises.

We use AuDoscore in our algorithm and data structure course to automate evaluation of student homework submissions.

Available Annotations

Annotation Required Position Arguments Semantics
@Exercises Yes class level Array of @Ex annotations contains a list of @Ex annotations; see @Ex
@Ex Yes element in @Exercises list exId: String, points: Double for every (sub)exercise, create an @Ex annotation with unique exID
@Points Yes test case level (method) exId: String, bonus: Double, malus: Double, comment: String, defaults to method name Student earns bonus / sum(bonus) * Ex.points points for passing this test case. Student looses malus / sum(bonus) * Ex.points points for not passing this test case.
@Forbidden No class level value: Array of Strings, type: Forbidden.Type.PREFIX (default), Forbidden.Type.FIXED, or Forbidden.Type.WILDCARD Arguments specify forbidden classes/methods/etc., depending on the type. PREFIX: Arguments are interpreted as prefix to forbidden elements. FIXED: Arguments are exactly the forbidden elements. WILDCARD: Like PREFIX but also supports wildcards. See also @NotForbidden
@NotForbidden No class level value: Array of Strings, type: Forbidden.Type.PREFIX (default), Forbidden.Type.FIXED, or Forbidden.Type.WILDCARD Arguments specify allowed classes/methods/etc., these take precedence over @Forbidden; see also @Forbidden
@Replace No test case level (method) Array of Strings Strings refer to methods in the student's code. For this test case, all methods mentioned in the @Replace annotation will be replaced with their cleanroom counterparts. Note: @Replace can only be used in secret tests!
@SecretClass No class level None Marks a test class to be secret. Results will not be shown to students before the submission deadline.
@CompareInterface No class level Array of String Checks if methods and fields of students have the same signature as their cleanroom counterparts. Possible Strings: "Classname.Methodname, "Classname.Fieldname", "Classname". If only the Classname is given all public methods/fields are checked.
@InitializeOnce No static field value: String At first execution, the method given as value is called and its result is stored in a temporary file. At subsequent executions, the annotated attribute is initialized using the precomputed result from the file. Note: @InitializeOnce can only be used in secret tests and the result of the given method must be serializable!
@SafeCallers No class level value: Array of Strings Fully qualified class names of classes to be granted additional permissions through the security manager (otherwise blocked e.g. for student code in order to prevent malicious behavior, file/net access, access to grading infrastructure, ...).

Local Usage

AuDoscore comes with a test script that can be used to locally grade a submission. The test script requires that the files belonging to an exercise are organized in a specific directory structure:

exercise_dir
  ↦ cleanroom
      ↦ Solver.java
  ↦ interfaces
      ↦ ISolver.java
  ↦ junit
      ↦ UnitTest.java
      ↦ SecretTest.java
  ↦ skeleton
      ↦ Solver.java
  ↦ student
      ↦ Solver.java

The cleanroom directory contains the cleanroom solution (i.e. the sample solution from the lecturer). The annotated unit tests reside in the junit directory. The interfaces directory is optional and contains the interfaces to be implemented by a student submission. The skeleton directory is optional and contains skeleton code provided to the students (thus it is ignored by the AuDoscore infrastructure). A student submission can optionally be placed in the student directory (if placed in another folder, this folder must be explicitly provided as argument to the test script).

To run AuDoscore on this example, cd to the exercise_dir and execute the test.sh script from the AuDoscore repository. The test.sh script takes the name of the directory containing the submission as an optional parameter, so you can also test submissions in other directories than student (e.g. the cleanroom itself). The test script produces a nicely formatted output of the passed and failed tests and also prints the scored points.

Branches

There are currently several active branches in this repository. Development usually happens in the master branch. The release branch just contains the version of AuDoscore currently in use by us.

License

The project is licensed under the Apache License, Version 2.0.

Contact

Please report bugs via the issue tracker.

Feedback: cs2-aud@fau.de