Incorporate ITF1788 tests into the test suite

[dpsanders]

Can we somehow incorporate the relevant ITF1788 tests directly into our test suite?
I believe the Apache license is compatible with MIT?

Cc @oheim

[dpsanders]

This could be by making the ITF1788 suite a (test) dependency of ValidatedNumerics, for example. But that may be painful for Travis.

Can we just copy over the files that ITF1788 outputs and add them to our test collection? (And then update when necessary.)

[lbenet]

I guess this depends on the license; I really have no idea.

Some numeric tests I implemented (#56 and #63) check if the result using @bigfloatinterval @biginterval is contained in the result of @interval (@floatinterval); while this does not check the true validity of the result, is a good check.

EDIT: corrected a misprint...

[oheim]

1. The GPL is compatible with your Expat license, and so is the (more permissive) Apache license. Since your code does not depend on/link to the test code, you can easily mix the files in your package in either case. The test files would have a different license, but that's no problem. The main differences are: If you use the Apache license (from the ITF1788 package), you may chose to redistribute only the compiled unit tests (object form), whereas with the GPL (from the octave package) you must (additionally) redistribute the itl files (source form, preferred form for making changes). With the Apache license you must also include the NOTICE file.
2. The pros and cons of including the compiled tests or the itl tests: This mainly depends on how often you plan to change the ITF1788 plugin for your package, because small changes to the generator will produce big diffs for the generated files in your repo. Also you have to consider whether your users, redistributors, or travis need to install ITF1788 to be able to use the tests. Are you able to distribute compiled unit tests only as part of your release or is there no difference between the release and the development sources?

[lbenet]

@oheim Once #56 and #63 are merged, I will push an update to your repo of the arithm.yaml file for Julia's API. While there are still things to be done to be fully compliant, what is implemented passes the ITF1788 tests, which is a pretty good sign. I also found this paper by M. Nehmeier, which is clearer than the standard.

Regarding our tests, we can still check compatibility (off-line) with ITF1788, and maybe add a line to README.md (and in the docs) about the actual status, or a gist.

[lbenet]

I just saw #67; I'll do that list later today.

[lbenet]

Updated Julia API pushed; see oheim/ITF1788@964a67c

[oheim]

@lbenet sounds great! You are really making progress. Please note that you might want to give lower priority to the operations that are only “recommended” in the standard document, compared to the required operations.

[lbenet]

Thanks for the advice; too late, I realized that :-)

Incidentally, can you explain me what is the case function referred by Nehmeier in this paper? I can't really find it in the standard nor in the ITF1788 tests...

[oheim]

It has been planned to include the following function in the standard together with its interval extension:
case(c,g,h) = if (c < 0) g else h
However, the idea has been discarded in 2012. You should be careful with the paper, because it is older than the standard and might differ in some aspects.

[lbenet]

@oheim Thanks for the explanation; I have edited the list and stroke it through.

You are right, that paper is older, but the standard is really not easy to follow...

[dpsanders]

Having discussed a bit with @lbenet, I think the best thing to do is to directly include the .jl files that the ITF1788 test suite generates.

[dpsanders]

cc @lbenet

[oheim]

Looks good. Some minor remarks:

• You might want to merge the LICENSE with your top level license file such that the latter contains both licenses and an explanation which license applies to what. This makes it easier for users and re-distributors of your software to find out about their rights without digging through subdirectories.
• You could include the content of the NOTICE file in your documentation if you wish, instead of placing the file in the test directory.
• The correct reference to the standard document is “IEEE Std 1788-2015” or “IEEE Standard for Interval Arithmetic” or “IEEE Std 1788-2015, IEEE Standard for Interval Arithmetic”.
• If you also included the .itl sources, this could simplify the use of more tests in .itl format in the future. However, you can still change this later.

[dpsanders]

Thanks very much, @oheim. I'll implement your first 3 suggestions.

For the last suggestion, we can certainly include the .itl sources, but I'm not sure how useful that is without the whole machinery of the rest of the ITF1788 test suite to process those .itl files into the output files.

An alternative would just be to include the whole of your repository -- or, at least, the part that's relevant for Julia -- inside ours.

[dpsanders]

So far, just the .jl files have been included in #78.