-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
258f2f5
commit 7ee4cdf
Showing
18 changed files
with
975 additions
and
87 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,4 +3,5 @@ target/ | |
Antidote.iml | ||
.gradle/ | ||
build/ | ||
gen/ | ||
gen/ | ||
out/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
#Sun May 26 12:58:48 CEST 2019 | ||
distributionBase=GRADLE_USER_HOME | ||
distributionPath=wrapper/dists | ||
distributionUrl=https\://services.gradle.org/distributions/gradle-5.0-bin.zip | ||
zipStoreBase=GRADLE_USER_HOME | ||
zipStorePath=wrapper/dists | ||
distributionUrl=https\://services.gradle.org/distributions/gradle-5.0-all.zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,343 @@ | ||
# Java SmallCheck | ||
|
||
Java SmallCheck is a Java library for property based testing. | ||
It will generate all possible inputs (up to a certain size) for testing your functions. | ||
|
||
This library is based on ideas from "Smallcheck and lazy smallcheck: automatic exhaustive testing for small values" (Colin Runciman, Matthew Naylor, and Matthew Naylor; Haskell 2008) | ||
|
||
|
||
## Basic Example | ||
|
||
The following function is supposed to compute the maximum out of 3 integers. | ||
Unfortunately it contains a bug. | ||
Can you find a counter example where it would fail? | ||
|
||
static int max3(int x, int y, int z) { | ||
if (x > y && x > z) { | ||
return x; | ||
} else if (y > x && y > z) { | ||
return y; | ||
} else { | ||
return z; | ||
} | ||
} | ||
|
||
With SmallCheck it is easy to write a test that finds a minimal counter example: | ||
|
||
|
||
import org.junit.runner.RunWith; | ||
import smallcheck.SmallCheckRunner; | ||
import smallcheck.annotations.Property; | ||
|
||
import static org.junit.Assert.assertTrue; | ||
|
||
@RunWith(SmallCheckRunner.class) | ||
public class Max3Example { | ||
|
||
@Property | ||
public void testMax3(int x, int y, int z) { | ||
int result = Max.max3(x, y, z); | ||
assertTrue(result >= x); | ||
assertTrue(result >= y); | ||
assertTrue(result >= z); | ||
} | ||
|
||
} | ||
|
||
When executed this test produces the following counter example: | ||
|
||
java.lang.AssertionError: Test failed when calling testMax3 with the following arguments: | ||
x = 1 | ||
y = 1 | ||
z = 0 | ||
at org.junit.Assert.fail(Assert.java:86) | ||
at org.junit.Assert.assertTrue(Assert.java:41) | ||
at org.junit.Assert.assertTrue(Assert.java:52) | ||
at Max3Example.testMax3(Max3Example.java:26) | ||
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | ||
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) | ||
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | ||
at java.base/java.lang.reflect.Method.invoke(Method.java:566) | ||
|
||
|
||
|
||
## Getting Started | ||
|
||
We recommend using [Gradle](https://gradle.org/) as the build tool. | ||
If you are using a different build tool, follow the instructions at [https://jitpack.io/#peterzeller/java-smallcheck](https://jitpack.io/#peterzeller/java-smallcheck). | ||
|
||
|
||
For Gradle follow these steps: | ||
|
||
|
||
|
||
1. Add Jitpack to your repositories in `build.gradle`: | ||
|
||
repositories { | ||
jcenter() | ||
maven { url 'https://jitpack.io' } | ||
} | ||
|
||
2. Add the SmallCheck dependency in `build.gradle`: | ||
|
||
dependencies { | ||
// Smallcheck Testing library | ||
testCompile group: 'com.github.peterzeller', name: 'java-smallcheck', version: '-SNAPSHOT' | ||
} | ||
|
||
Change `-SNAPSHOT` to a fixed tag to get a stable build. | ||
|
||
3. (optional) Configure Gradle to keep parameter names. This will give you better error messages. | ||
|
||
compileTestJava.options.compilerArgs.add '-parameters' | ||
|
||
|
||
## Writing Property Tests | ||
|
||
To write property based tests using SmallCheck, two ingredients are required: | ||
|
||
1. The class containing the tests must be annotated with | ||
|
||
@RunWith(SmallCheckRunner.class) | ||
|
||
2. Each property is written in a public method annotated with `@Property`, similar to the `@Test` annotation for normal Junit tests. | ||
|
||
|
||
SmallCheck will invoke each property method with all possible parameters up to a certain depth. | ||
|
||
@Property | ||
public void generate(int x, char c) { | ||
System.out.println(x + ", " + c); | ||
} | ||
|
||
This simple example will be invoked with the following inputs: | ||
|
||
// depth 1 | ||
0, a | ||
1, a | ||
-1, a | ||
0, b | ||
1, b | ||
-1, b | ||
// depth 2 | ||
0, a | ||
1, a | ||
2, a | ||
-1, a | ||
-2, a | ||
0, b | ||
1, b | ||
2, b | ||
-1, b | ||
-2, b | ||
0, c | ||
1, c | ||
2, c | ||
-1, c | ||
-2, c | ||
// depth 3 | ||
0, a | ||
1, a | ||
2, a | ||
3, a | ||
-1, a | ||
-2, a | ||
-3, a | ||
0, b | ||
1, b | ||
2, b | ||
3, b | ||
-1, b | ||
-2, b | ||
-3, b | ||
0, c | ||
1, c | ||
2, c | ||
3, c | ||
-1, c | ||
-2, c | ||
-3, c | ||
0, d | ||
1, d | ||
2, d | ||
3, d | ||
-1, d | ||
-2, d | ||
-3, d | ||
// depth 4 | ||
... | ||
|
||
By default values up to depth 5 will be tested. | ||
|
||
Many standard Java types (like `int` and `char` above) are supported with built-in generators. | ||
For other types, custom generators can be written. | ||
|
||
|
||
|
||
### Property Settings | ||
|
||
The `@Property` annotation provides the following parameters for configuring SmallCheck: | ||
|
||
- **int maxDepth** (default 5) | ||
|
||
The maximum depth for generated inputs. | ||
|
||
- **int maxInvocations** (default 100000) | ||
|
||
The maximum number of different inputs used in testing. | ||
|
||
- **long minExamples** (default 20) | ||
|
||
The minimum number of valid examples that must be produced. | ||
The test will fail if not enough generated arguments pass the preconditions giving with JUnits `Asssume` methods. | ||
|
||
- **int timeout** (default -1) | ||
|
||
A maximum execution time in seconds. | ||
The test fails if the overall execution takes longer than the given duration. | ||
|
||
|
||
The following example shows a property with custom configuration: | ||
|
||
@Property(maxDepth = 6, maxInvocations = 1000, minExamples = 100, timeout = 30) | ||
public void configuredExample(int x, char c) { | ||
... | ||
} | ||
|
||
|
||
## Custom Generators | ||
|
||
A generator for a type `T` is defined by creating a subclass of `SeriesGen<T>`: | ||
|
||
public abstract class SeriesGen<T> { | ||
|
||
public abstract Stream<T> generate(int depth); | ||
|
||
public T copy(T obj) { | ||
return obj; | ||
} | ||
} | ||
|
||
The `generate` method creates a stream of elements up to a certain depth. | ||
|
||
For example the builtin `IntegerGen` generates values the numbers `0, 1, 2, 3, -1, -2, -3` for `depth = 3`. | ||
|
||
The `copy` method must be overridden for mutable datatypes. | ||
It should create a copy of a `T` object. | ||
This is used when generating collections like a `List<T>`. | ||
The List generator must generate many lists starting with the same element and thus needs to call `copy`. | ||
|
||
The following example shows a custom number generator that only generates even numbers. | ||
|
||
|
||
public static class CustomNumberGen extends SeriesGen<Integer> { | ||
@Override | ||
public Stream<Integer> generate(int depth) { | ||
return IntStream.range(0, 1 + depth).map(i -> 2 * i).boxed(); | ||
} | ||
} | ||
|
||
public static class CustomCharGen extends SeriesGen<Character> { | ||
@Override | ||
public Stream<Character> generate(int depth) { | ||
return IntStream.range(0, 1+depth).mapToObj((int i) -> { | ||
if (i % 2 == 0) { | ||
return (char) ('a' + i / 2); | ||
} else { | ||
return (char) ('A' + i / 2); | ||
} | ||
}); | ||
} | ||
} | ||
|
||
|
||
### Registering Custom Generators | ||
|
||
To use the custom generators they can be registered for a test method using the `@RegisterGenerator` annotation as shown in the example below. | ||
|
||
|
||
@Property | ||
@RegisterGenerator(CustomNumberGen.class) | ||
public void genList(List<Integer> list) { | ||
System.out.println(list); | ||
} | ||
|
||
@Property | ||
@RegisterGenerator(CustomNumberGen.class) | ||
@RegisterGenerator(CustomCharGen.class) | ||
public void genMap(Map<Integer, Character> m) { | ||
System.out.println(m); | ||
} | ||
|
||
|
||
### Configuring Custom Generators with @From | ||
|
||
It is also possible to use the `@From` annotation on a specific parameter to use the generator only for that case: | ||
|
||
@Property | ||
public void genList2(List<@From(CustomNumberGen.class) Integer> list) { | ||
System.out.println(list); | ||
} | ||
|
||
### Custom Generators from Static Factories | ||
|
||
Another option for generating custom values is to define a static factory. | ||
A static factory is a class that provides static methods that generate instances of a certain type. | ||
|
||
SmallCheck will try all combinations of these methods to create values of the type. | ||
|
||
In the example below we use this feature to create arithmetic expressions and find the smallest expression that evaluates to a value greater than or equal to 8. | ||
The expression found is `((2 * 2) * 2)`. | ||
|
||
|
||
@Property(maxInvocations = 5000000) | ||
@StaticFactory(ExprFactory.class) | ||
public void testExpr(Expr e) { | ||
assertTrue(e.evaluate() < 8); | ||
} | ||
|
||
|
||
public static class ExprFactory { | ||
public static Number number(int i) { | ||
return new Number(i); | ||
} | ||
|
||
public static Plus plus(Expr a, Expr b) { | ||
return new Plus(a, b); | ||
} | ||
|
||
public static Mult mult(Expr a, Expr b) { | ||
return new Mult(a, b); | ||
} | ||
} | ||
|
||
|
||
|
||
## The StateGen Generator | ||
|
||
So far we have only seen SmallCheck generate inputs in the form of parameters. | ||
The stateful generator `StateGen` provides an alternative way to generate inputs for your tests. | ||
|
||
To use this, simply add a parameter of type `StateGen` to your test method and use the `StateGen.gen` methods to generate values. | ||
|
||
@Property | ||
public void testMax3S(StateGen sg) { | ||
int x = sg.gen(Integer.class); | ||
int y = sg.gen(Integer.class); | ||
int z = sg.gen(Integer.class); | ||
int result = max3(x, y, z); | ||
assertTrue(result >= x); | ||
assertTrue(result >= y); | ||
assertTrue(result >= z); | ||
} | ||
|
||
SmallCheck will invoke the test method multiple times and choose different values each execution. | ||
|
||
First all values for `z` (the last choice) are tested. | ||
When all values for `z` are ok, SmallCheck will backtrack and try the next value for `y`. | ||
|
||
Since the last generated values are explored first, the counter examples given with `StateGen` differ from the ones found with normal parameters. | ||
|
||
|
||
|
||
|
Oops, something went wrong.