'Constrained' variables

[lelemmen]

In several optimizations, there are often relations between several variables. For example, AP1roG is a special case of APIG, where some of the APIG variables are constrained to be 1 and others to be 0. The code would therefore benefit if some kind of 'constrain' mechanism could be used.

Here are some first thoughts.

The main class that could be used is a Variables class. Since we always formulate the variables in optimization problems as vectors, there should be a method Variables.asVector() that returns the variables in their vector form.

If a constraint needs to be added, we can implement Variables.constrain(vector_index, value) to permanently set the variable at position vector_index to the given value. (I haven't really thought about symmetry, i.e. when some variable is equal to another or the negative of another, etc.) Subclasses could implement Variables.constrain(row_index, column_index, value) if a better representation of the variables is as a matrix.

If a constraint is added, the gradient and the Hessian have reduced dimensionality. Therefore, a method Variables.freeVariables() should return the vector indices (if useful: matrix indices) of the variables that are still considered as 'free' (i.e. not constrained). In this way, only the required elements of the gradient and Hessian can be used.