#3336 - Cache default LP model

[Zinoex]

Closes #3336.

The interface of the API stays the same, although we might want to consider renaming the parameter to solver_or_model, but this is a breaking change. Additionally, I have observed inconsistent naming. Specifically, in remove_redundant_constraints!, the solver (factory) parameter is named backend, which, as far as I can tell, is reserved for Polyhedra backends.

Note that this has been implemented in a thread safe manner, heavily inspired by Random's default (global) RNG state.

[schillic]

Nice, the threading solution might also fix #3315 🎉

Did you benchmark the new behavior? I would be curious.

we might want to consider renaming the parameter to solver_or_model, but this is a breaking change.

Okay to not change it for now.

I have observed inconsistent naming.

There is #1145 but we are not aware of an inconsistency. Maybe you can report problems there.

I will discuss the PR with @mforets but looks good to me apart from the two comments below.

[Zinoex]

the threading solution might also fix #3315 🎉

I do not think so. The previous implementation was consistent with JuMP's documentation for parallelism the right way with a solver and model being instantiated for each call to remove_redundant_vertices (and others), hence being thread safe.

Did you benchmark the new behavior? I would be curious.

No yet. I will do it later today.

[Zinoex]

I have now run benchmarks by adapting support.jl to HPolytope (with smaller dimensions, because generating higher dimensions take ages). The results are honestly pretty exciting. Naturally, the higher the dimension, the more the constraints and the larger the optimization problem, which takes longer to construct. Hence, the biggest gains are in sets of low dimension.

Before feature (on Master)

julia> res["ρ"][("HPolytope", "dense", 2)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  210.933 μs …   7.890 ms  ┊ GC (min … max): 0.00% … 64.90%
 Time  (median):     214.859 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   244.165 μs ± 262.025 μs  ┊ GC (mean ± σ):  2.83% ±  2.60%

  ██▅▃▁▁▂▃▂▁                                        ▁▄▂▁        ▂
  ███████████▇▆▆▇▇▇▇▇▆▇▇▇▆▆▆▆▅▆▆▆▆▆▅▄▄▅▁▅▅▄▄▁▄▅▁▅▄▃▄█████▆▆▆▆▅▆ █
  211 μs        Histogram: log(frequency) by time        408 μs <

 Memory estimate: 64.02 KiB, allocs estimate: 2012.

julia> res["ρ"][("HPolytope", "dense", 4)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  259.042 μs …   6.320 ms  ┊ GC (min … max): 0.00% … 74.19%
 Time  (median):     262.517 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   280.945 μs ± 299.838 μs  ┊ GC (mean ± σ):  4.30% ±  3.86%

  ▃▆██▆▄▄▄▃▂                    ▁                               ▂
  ███████████▇▇▆▆▇▇▇█▇▇▇▇▅▆▆▆▇▇███████▇▇▅▅▆▄▁▁▆▅▃▁▄▃▃▅▄▅▅▃▃▄▅▁▅ █
  259 μs        Histogram: log(frequency) by time        327 μs <

 Memory estimate: 116.89 KiB, allocs estimate: 2732.

julia> res["ρ"][("HPolytope", "dense", 6)]
BenchmarkTools.Trial: 5026 samples with 1 evaluation.
 Range (min … max):  836.032 μs …   6.698 ms  ┊ GC (min … max): 0.00% … 71.35%
 Time  (median):     860.827 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   989.660 μs ± 695.905 μs  ┊ GC (mean ± σ):  8.57% ± 10.40%

  █▁     ▃                                                      ▁
  ██▇▅▅▆▅█▆▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▅█ █
  836 μs        Histogram: log(frequency) by time       5.86 ms <

 Memory estimate: 848.73 KiB, allocs estimate: 11363.

julia> res["ρ"][("HPolytope", "dense", 10)]
BenchmarkTools.Trial: 72 samples with 1 evaluation.
 Range (min … max):  64.225 ms … 81.419 ms  ┊ GC (min … max): 11.69% … 8.82%
 Time  (median):     69.819 ms              ┊ GC (median):    11.06%
 Time  (mean ± σ):   69.567 ms ±  3.590 ms  ┊ GC (mean ± σ):  14.13% ± 4.08%

                        ▄█                  ▂                  
  ▆▆▃▁▁▁██▇▃▁▃▁▁▁▁▁▃▃▇▃▅██▆▆▁▃▁▁▁▁▁▃▁▁▁▁▅▃▁▁█▃▁▁▁▁▁▁▃▁▁▁▁▁▁▁▃ ▁
  64.2 ms         Histogram: frequency by time        78.8 ms <

 Memory estimate: 62.02 MiB, allocs estimate: 511298.

After feature (on zinoex/linprog_model_performance)

julia> res["ρ"][("HPolytope", "dense", 2)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  29.716 μs …  3.652 ms  ┊ GC (min … max): 0.00% … 97.82%
 Time  (median):     30.919 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   32.977 μs ± 71.977 μs  ┊ GC (mean ± σ):  4.31% ±  1.96%

     ▃▅▇▇███▇▆▅▄▂▁        ▁▂▂▃▃▃▂▂▁▁    ▁▁▁▁▂▁▁▁▁             ▃
  ▂▅██████████████▇▇▆▄▂▄▆▇███████████████████████▇▇▇▆▆▆▆▇▆▇▆▆ █
  29.7 μs      Histogram: log(frequency) by time      37.7 μs <

 Memory estimate: 19.34 KiB, allocs estimate: 328.

julia> res["ρ"][("HPolytope", "dense", 4)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  68.427 μs …   3.119 ms  ┊ GC (min … max): 0.00% … 95.75%
 Time  (median):     70.363 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   76.218 μs ± 115.360 μs  ┊ GC (mean ± σ):  5.96% ±  3.84%

   ▃▆▇██▇▆▅▄▂▁▁    ▁▁                          ▁▁▁ ▁▁          ▂
  ▆████████████████████▇▆▆▇▆▅▅▅▅▅▄▃▁▄▃▆▆▆▆█████████████████▇▇▇ █
  68.4 μs       Histogram: log(frequency) by time      89.9 μs <

 Memory estimate: 71.23 KiB, allocs estimate: 1046.

julia> res["ρ"][("HPolytope", "dense", 6)]
BenchmarkTools.Trial: 6787 samples with 1 evaluation.
 Range (min … max):  652.606 μs …   4.526 ms  ┊ GC (min … max): 0.00% … 80.92%
 Time  (median):     668.846 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   733.903 μs ± 452.945 μs  ┊ GC (mean ± σ):  7.94% ± 10.61%

  █▂▁                                                           ▁
  ███▇▆▄▁▁▆▄▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄█ █
  653 μs        Histogram: log(frequency) by time       4.11 ms <

 Memory estimate: 782.06 KiB, allocs estimate: 9673.

julia> res["ρ"][("HPolytope", "dense", 10)]
BenchmarkTools.Trial: 76 samples with 1 evaluation.
 Range (min … max):  62.557 ms … 82.828 ms  ┊ GC (min … max):  9.99% … 9.58%
 Time  (median):     64.477 ms              ┊ GC (median):    11.04%
 Time  (mean ± σ):   66.085 ms ±  3.557 ms  ┊ GC (mean ± σ):  11.96% ± 2.71%

    ▃▃▆   █▃                    ▁                              
  ▇▆███▆▇▆██▆▆▄▄▁▆▁▁▁▁▁▁▁▁▁▄▄▆▆▄█▄▄▁▁▆▄▁▄▇▁▆▄▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▄ ▁
  62.6 ms         Histogram: frequency by time        74.7 ms <

 Memory estimate: 61.02 MiB, allocs estimate: 509596.

[Zinoex]

While we are in the process of improving the performance of linprog, I just added a few more changes to reduce the number of allocations required, to gain another 10-15% speedup. Specifically, by using multiple dispatch, we avoid allocating arrays for sense, l, and u and by using views, we avoid allocating more space for A and b. Below I added the performance benchmarks (and included master for completeness). Note that these experiments were conducted on a different machine to the last batch, so the absolute values are not comparable.

If you prefer that I open a separate PR for this, just let me know. My thoughts were just that these two are very related.

Master

julia> res["ρ"][("HPolytope", "dense", 2)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  227.631 μs …  20.310 ms  ┊ GC (min … max): 0.00% … 85.42%
 Time  (median):     265.683 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   303.856 μs ± 482.131 μs  ┊ GC (mean ± σ):  3.48% ±  2.21%

  ▅▇▇██▇▅▄▃▃▂▂▁▁▁▁▁ ▁                                           ▂
  ██████████████████████▇▇▇▇▇▇▇▆▆▆▆▆▆▅▆▆▆▅▆▅▅▅▆▄▆▅▅▅▅▅▅▅▅▅▅▃▅▄▅ █
  228 μs        Histogram: log(frequency) by time        797 μs <

 Memory estimate: 63.83 KiB, allocs estimate: 2004.

julia> res["ρ"][("HPolytope", "dense", 4)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  275.476 μs …  29.547 ms  ┊ GC (min … max): 0.00% … 75.10%
 Time  (median):     328.396 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   389.699 μs ± 642.740 μs  ┊ GC (mean ± σ):  5.07% ±  3.20%

  ▃▆▆▇█▆▅▄▃▃▃▂▂▂▁▁▁▁                                            ▂
  ███████████████████████▇▇▇▆▇▇▇▆▆▇▅▆▆▇▇▆▅▆▅▆▅▅▅▆▅▅▄▅▅▄▄▅▅▂▄▂▅▃ █
  275 μs        Histogram: log(frequency) by time       1.02 ms <

 Memory estimate: 115.48 KiB, allocs estimate: 2672.

julia> res["ρ"][("HPolytope", "dense", 6)]
BenchmarkTools.Trial: 3923 samples with 1 evaluation.
 Range (min … max):  960.826 μs … 15.714 ms  ┊ GC (min … max): 0.00% … 73.55%
 Time  (median):       1.081 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):     1.268 ms ±  1.148 ms  ┊ GC (mean ± σ):  8.97% ±  9.03%

  █▆▄▃▁                                                        ▁
  ██████▇▇▆▅▄▃▁▃▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▆ █
  961 μs        Histogram: log(frequency) by time      10.4 ms <

 Memory estimate: 831.86 KiB, allocs estimate: 10643.

julia> res["ρ"][("HPolytope", "dense", 10)]
BenchmarkTools.Trial: 54 samples with 1 evaluation.
 Range (min … max):  77.613 ms … 125.274 ms  ┊ GC (min … max):  0.00% … 10.52%
 Time  (median):     91.826 ms               ┊ GC (median):    12.41%
 Time  (mean ± σ):   93.947 ms ±  10.109 ms  ┊ GC (mean ± σ):  11.63% ±  4.46%

                 █    ▃                                         
  ▅▁▁▁▁▄▄▁▄▄▄▄▄▅▄█▇▁█▅█▇▅▅▁▁▁▅▄▁▁▁▄▄▁▁▁▁▁▄▁▄▁▁▁▁▁▁▁▁▁▁▁▄▁▁▁▁▁▅ ▁
  77.6 ms         Histogram: frequency by time          124 ms <

 Memory estimate: 61.19 MiB, allocs estimate: 474928.

Without multiple dispatch and views

julia> res["ρ"][("HPolytope", "dense", 2)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  37.785 μs …  12.480 ms  ┊ GC (min … max): 0.00% … 98.53%
 Time  (median):     43.846 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   49.048 μs ± 180.766 μs  ┊ GC (mean ± σ):  6.12% ±  1.71%

   ▅▇▇▇▇██▇██▇▆▅▄▄▄▃▃▂▁▂▂▁▁▁▁   ▁▁▁                            ▃
  ██████████████████████████████████▇█▇▆▇█▇▇▆▇▇▆▇▇▆█▇▇█▇▇█▆▇▇▇ █
  37.8 μs       Histogram: log(frequency) by time      79.1 μs <

 Memory estimate: 23.09 KiB, allocs estimate: 404.

julia> res["ρ"][("HPolytope", "dense", 4)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  78.696 μs …   6.716 ms  ┊ GC (min … max): 0.00% … 97.31%
 Time  (median):     88.709 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   98.549 μs ± 195.497 μs  ┊ GC (mean ± σ):  6.44% ±  3.21%

     ▇▅▆▄▆█▁▁ ▅▃                                                
  ▂▇█████████▇██▇▅▄▄▃▃▃▃▃▃▃▃▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃
  78.7 μs         Histogram: frequency by time          146 μs <

 Memory estimate: 73.77 KiB, allocs estimate: 1070.

julia> res["ρ"][("HPolytope", "dense", 6)]
BenchmarkTools.Trial: 5155 samples with 1 evaluation.
 Range (min … max):  725.111 μs …  15.977 ms  ┊ GC (min … max): 0.00% … 88.74%
 Time  (median):     824.366 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   963.489 μs ± 778.023 μs  ┊ GC (mean ± σ):  8.18% ±  9.30%

  █▇▅▄▃▂▁                                                       ▁
  ████████▇▇▆▅▆▅▄▄▄▄▄▃▄▄▃▁▃▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆ █
  725 μs        Histogram: log(frequency) by time       6.58 ms <

 Memory estimate: 769.12 KiB, allocs estimate: 9037.

julia> res["ρ"][("HPolytope", "dense", 10)]
BenchmarkTools.Trial: 55 samples with 1 evaluation.
 Range (min … max):  79.267 ms … 136.572 ms  ┊ GC (min … max):  7.66% … 0.00%
 Time  (median):     88.583 ms               ┊ GC (median):    10.90%
 Time  (mean ± σ):   91.440 ms ±  11.186 ms  ┊ GC (mean ± σ):  10.24% ± 3.62%

        ▂ █▂▅▂▂▅█                                               
  ██▁█▁██████████▁▅█▁▅█▁█▅▁▁▁▁▅▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▅▁▁▁▁▁▁▁▁▅▁▁▁▅ ▁
  79.3 ms         Histogram: frequency by time          127 ms <

 Memory estimate: 60.19 MiB, allocs estimate: 473310.

With multiple dispatch and views

julia> res["ρ"][("HPolytope", "dense", 2)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  35.053 μs …   9.513 ms  ┊ GC (min … max): 0.00% … 98.31%
 Time  (median):     42.840 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   45.842 μs ± 129.959 μs  ┊ GC (mean ± σ):  3.93% ±  1.39%

         ▂▂▆▇▇▄ ▁   ▅█▄                                         
  ▁▂▂▂▂▄▆████████▇▆▆███▇▆▅▄▄▄▃▃▃▂▂▂▂▂▂▂▂▁▁▁▁▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃
  35.1 μs         Histogram: frequency by time         65.5 μs <

 Memory estimate: 22.31 KiB, allocs estimate: 381.

julia> res["ρ"][("HPolytope", "dense", 4)]
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max):  71.896 μs …   9.140 ms  ┊ GC (min … max): 0.00% … 95.62%
 Time  (median):     83.061 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   92.377 μs ± 211.629 μs  ┊ GC (mean ± σ):  6.89% ±  3.00%

     ▃█▄▆▃▄▂ ▂▁▃▄                                               
  ▁▂▆█████████████▆▅▄▃▃▄▄▃▃▂▂▂▂▂▂▂▂▂▂▁▂▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃
  71.9 μs         Histogram: frequency by time          134 μs <

 Memory estimate: 69.48 KiB, allocs estimate: 911.

julia> res["ρ"][("HPolytope", "dense", 6)]
BenchmarkTools.Trial: 6197 samples with 1 evaluation.
 Range (min … max):  616.408 μs …   9.289 ms  ┊ GC (min … max): 0.00% … 89.76%
 Time  (median):     705.356 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   801.840 μs ± 644.508 μs  ┊ GC (mean ± σ):  7.88% ±  8.87%

  ██▄▃▃▂▁                                                       ▁
  ████████▆▄▃▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅ █
  616 μs        Histogram: log(frequency) by time        5.9 ms <

 Memory estimate: 711.17 KiB, allocs estimate: 7221.

julia> res["ρ"][("HPolytope", "dense", 10)]
BenchmarkTools.Trial: 62 samples with 1 evaluation.
 Range (min … max):  69.862 ms … 92.054 ms  ┊ GC (min … max): 0.00% … 10.67%
 Time  (median):     80.936 ms              ┊ GC (median):    9.01%
 Time  (mean ± σ):   80.995 ms ±  4.295 ms  ┊ GC (mean ± σ):  8.72% ±  3.18%

                            ▅▂▅    ▂  ▂▂ █    █                
  ▅▁▁▁▁▁▁▁▅▁▁▅▅▁▁▁█▅█▅▅▁▅▅█▁████▁█▅██▁██▁█▅▅▁▁██▁▁▁▁▁▅▅▁▁▅▁▁█ ▁
  69.9 ms         Histogram: frequency by time        89.8 ms <

 Memory estimate: 59.68 MiB, allocs estimate: 382350.

[schillic]

Thanks a lot! Looks good and can stay in this PR from my side.