Document interface to ranges

* docs/ra-ra.texi  (Using ra:: types etc.): As stated.
  (begin, end, range): Reference stubs.
* examples/read-me.cc: Check examples from the manual.
* ra/ply.hh (begin/end/range): Provide overloads for objects that can do better
  than STLIterator.
* test/stl-compat.cc: Check random access iterator cases.

docs/ra-ra.texi

@@ -12,7 +12,7 @@
 @c References to source [ma··] or [ma···] current last is 117.
 
 @set VERSION 27
-@set UPDATED 2023 November 28
+@set UPDATED 2023 November 30
 
 @copying
 @code{ra::} (version @value{VERSION}, updated @value{UPDATED})
@@ -1315,18 +1315,46 @@ Some types are accepted automatically as scalars. These include non-pointer type
 
 Otherwise, you can bring a scalar object into @code{ra::} on the spot, with the function @ref{x-scalar,@code{scalar}}.
 
+@node Using @code{ra::} types with the STL
 @subsection Using @code{ra::} types with the STL
 
-General @code{ra::} @ref{Containers and views,views} provide STL compatible @code{std::input_iterator}s through the members @code{begin()} and @code{end()}. These iterators traverse the elements of the array (0-cells) in row major order, regardless of the storage order of the view.
+STL compatible input/output iterators and ranges can be obtained from general @code{ra::} expressions through the functions @ref{x-begin,@code{begin}}, @ref{x-end,@code{end}}, and @ref{x-range,@code{range}}. These objects traverse the elements of the expression (0-cells) in row major order.@footnote{Unqualified @code{begin()} would find @code{std::begin} through ADL since @code{Big} is parameterized by @code{std::vector}. This still works since @code{std::begin} forwards to @code{A.begin()}. It's up to you if you want to rely on such things.}
 
-For @ref{Containers and views,containers} @code{begin()} provides @code{std::random_access_iterator}s, which is handy for certain functions such as @code{std::sort}. There's no reason why all views couldn't provide @code{std::random_access_iterator}, but these wouldn't be efficient for ranks above 1, and I haven't implemented them. The container @code{std::random_access_iterator}s that are provided are in fact raw pointers.
+@example @c [ma118]
+@verbatim
+ra::Big<int, 2> A = {{3, 0, 0}, {4, 5, 6}, {0, 5, 6}};
+std::accumulate(ra::begin(A), ra::end(A), 0); // or just sum(A)
+@end verbatim
+@result{} 29
+@end example
+
+For temporary expressions that are stated once, the ranges interface is more appropriate.
+
+@example @c [ma118]
+@verbatim
+cout << std::ranges::count(range(A>3), true) << endl; // or sum(cast<int>(A>3))
+@end verbatim
+@result{} 5
+@end example
+
+One can create ranges from higher rank @code{ra::} iterators and thus use STL algorithms over cells of any rank, but note that in the current version of @code{ra::}, @ref{x-iter,@code{iter<k>()}} only works on views, not on general expressions.
+
+@example @c [ma118]
+@verbatim
+// count rows with 0s in them
+cout << std::ranges::count_if(range(iter<1>(A)), [](auto const & x) { return any(x==0); }) << endl;
+@end verbatim
+@result{} 2
+@end example
+
+For @ref{Containers and views,containers}, @code{ra::} @code{begin}/@code{end}/@code{range} provide random access iterators and ranges, which is handy for functions such as @code{std::sort}. These could be provided for general expressions, but they wouldn't be efficient for ranks above 1, and I haven't implemented them. The container @code{std::random_access_iterator}s that are provided are in fact raw pointers.
 
 @example @c [ma106]
 @verbatim
 ra::Big<int> x {3, 2, 1}; // x is a Container
 auto y = x(); // y is a View on x
-// std::sort(y.begin(), y.end()); // error: y.begin() is not std::random_access_iterator
-std::sort(x.begin(), x.end()); // ok, we know x is stored in row-major order
+// std::sort(ra::begin(y), ra::end(y)); // error: begin(y) is not std::random_access_iterator
+std::sort(ra::begin(x), ra::end(x)); // ok, we know x is stored in row-major order
 @end verbatim
 @result{} x = @{1, 2, 3@}
 @end example
@@ -1438,6 +1466,7 @@ The most common case is that a library doesn't handle steps at all, and it only
 
 @c FIXME using is_c_order, etc.
 
+@cindex BLAS
 Other libraries do accept steps, but not arbitrary ones. For example @url{https://www.netlib.org/blas}' @code{cblas_dgemm} has this prototype:
 
 @quotation
@@ -1451,7 +1480,7 @@ cblas_dgemm(order, transA, transB, m, n, k, alpha, A, lda, B, ldb, beta, C, ldc)
 Sometimes you can work around this by fiddling with @code{transA} and @code{transB}, but in general you need to check your array parameters and you may need to make copies.
 
 @cindex OpenGL
-OpenGL is another library that requires @url{https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttribPointer.xhtml,contortions:}
+OpenGL is another library that requires @url{https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glVertexAttribPointer.xhtml,contortions:}. Quote:
 
 @quotation
 @verbatim
@@ -1807,6 +1836,7 @@ The following @code{#define}s affect the behavior of @code{ra::}.
 
 The following environment variables affect the test suite under SCons:
 
+@cindex BLAS
 @itemize
 @item @code{RA_USE_BLAS} (default 0): Use BLAS for @code{gemm} and @code{gemv} benchmarks.
 @end itemize
@@ -1888,7 +1918,15 @@ This can be used for sparse subscripting. For example:
 @end verbatim
   @result{} B = @{@{101, 120@}, @{110, 121@}@}
 @end example
+@end defun
+
+@cindex @code{begin}
+@anchor{x-begin} @defun begin expr
+Create STL iterator from @var{expr}.
 
+See @ref{Using @code{ra::} types with the STL}.
+
+See also @ref{x-end,@code{end}}, @ref{x-range,@code{range}}.
 @end defun
 
 @cindex @code{cast}
@@ -1924,6 +1962,15 @@ TODO
 See also @ref{x-collapse,@code{collapse}}.
 @end defun
 
+@cindex @code{end}
+@anchor{x-end} @defun end expr
+Create STL end iterator from @var{expr}.
+
+See @ref{Using @code{ra::} types with the STL}.
+
+See also @ref{x-begin,@code{begin}}, @ref{x-range,@code{range}}.
+@end defun
+
 @cindex @code{for_each}
 @anchor{x-for_each} @defun for_each op expr ...
 Create an array expression that applies @var{op} to @var{expr} ..., and traverse it. The return value of @var{op} is discarded.
@@ -2354,6 +2401,15 @@ You don't need to use @code{ra::ptr} on STL containers and built in arrays, whic
 
 @end deffn
 
+@cindex @code{range}
+@anchor{x-range} @defun range expr
+Create STL range iterator from @var{expr}.
+
+See @ref{Using @code{ra::} types with the STL}.
+
+See also @ref{x-begin,@code{begin}}, @ref{x-end,@code{end}}.
+@end defun
+
 @cindex @code{start}
 @anchor{x-start} @defun start foreign_object
 Create array expression from @var{foreign_object}.
@@ -2560,7 +2616,7 @@ Compute @code{(0+1j)} times @var{expr}.
 @cindex @code{none}
 @anchor{x-none}
 @deffn @w{Special object} {none}
-Pass @code{none} to container constructors to indicate that the contents shouldn't be initialized. This is appropriate when the initialization you have in mind wouldn't fit in a constructor argument. For example:
+Pass @code{none} to container constructors to indicate that the contents shouldn't be initialized. This is appropriate when the initialization you have in mind doesn't fit in a constructor argument. For example:
 
 @example
 @verbatim

examples/read-me.cc

@@ -9,9 +9,9 @@
 
 // TODO Generate README.md and/or these examples.
 
-#include "ra/ra.hh"
-#include "ra/test.hh"
 #include <iostream>
+#include <numeric>
+#include "ra/test.hh" // ra/ra.hh without TestRecorder
 
 using std::cout, std::endl, ra::TestRecorder;
 using ra::mp::int_list;
@@ -151,10 +151,17 @@ int main()
         ra::Big<char, 1> A = {'x', 'z', 'y'};
         std::sort(A.begin(), A.end());
         cout << "A: " << A << "\n\n";
-
+        tr.test_eq(ra::start({'x', 'y', 'z'}), A);
+        A = {'x', 'z', 'y'};
+        std::sort(begin(A), end(A));
+        cout << "A: " << A << "\n\n";
+        tr.test_eq(ra::start({'x', 'y', 'z'}), A);
+    }
+    {
         ra::Big<float, 2> B {{1, 2}, {3, 4}};
         B += std::vector<float> {10, 20};
         cout << "B: " << B << "\n\n";
+        tr.test_eq(ra::Big<float, 2> {{11, 12}, {23, 24}}, B);
     }
     tr.section("Example from the manual [ma100]");
     {
@@ -248,5 +255,13 @@ int main()
         cout << (x(ra::all, ra::insert<2>) * a(ra::insert<1>)) << endl;
         cout << (x * a(ra::insert<1>)) << endl; // same thing
     }
-    return 0;
+    tr.section("Examples from manual [ma118]");
+    {
+        ra::Big<int, 2> A = {{3, 0, 0}, {4, 5, 6}, {0, 5, 6}};
+        tr.test_eq(sum(A), std::accumulate(ra::begin(A), ra::end(A), 0));
+        tr.test_eq(sum(cast<int>(A>3)), std::ranges::count(range(A>3), true));
+// count rows with 0s in them
+        tr.test_eq(2, std::ranges::count_if(range(iter<1>(A)), [](auto const & x) { return any(x==0); }));
+    }
+    return tr.summary();
 }

ra/bootstrap.hh

@@ -269,7 +269,7 @@ constexpr shape_manip_t
 operator<<(std::ostream & o, print_shape_t shape) { return shape_manip_t { o, shape }; }
 
 // include is_fov bc shape() may be std::vector or std::array.
-// exclude std::string_view to let it be is_fov and still print as a string [ra13].
+// exclude std::string_view so it is is_fov but still prints as a string [ra13].
 template <class A> requires (is_ra<A> || (is_fov<A> && !std::is_convertible_v<A, std::string_view>))
 constexpr std::ostream &
 operator<<(std::ostream & o, A && a) { return o << format_array(a); }

ra/ply.hh

@@ -342,7 +342,7 @@ struct STLIterator
     using value_type = value_t<A>;
 
     A a;
-    std::decay_t<decltype(ra::shape(a))> ind; // is a concrete type
+    std::decay_t<decltype(ra::shape(a))> ind; // concrete type
     bool over;
 
     STLIterator(A a_): a(a_), ind(ra::shape(a_)), over(0==ra::size(a)) {}
@@ -385,21 +385,25 @@ struct STLIterator
     }
     constexpr STLIterator & operator++() requires (ANY==rank_s<A>()) { next(rank(a)-1); return *this; }
     constexpr STLIterator & operator++() requires (ANY!=rank_s<A>()) { next<rank_s<A>()-1>(); return *this; }
-// see p0541 and p2550. Or just avoid.
-    constexpr void operator++(int) { ++(*this); }
+    constexpr void operator++(int) { ++(*this); } // see p0541 and p2550. Or just avoid.
 };
 
 template <class A> STLIterator(A &&) -> STLIterator<A>;
+
 constexpr auto begin(is_ra auto && a) { return STLIterator(ra::start(RA_FWD(a))); }
 constexpr auto end(is_ra auto && a) { return std::default_sentinel; }
 constexpr auto range(is_ra auto && a) { return std::ranges::subrange(ra::begin(RA_FWD(a)), std::default_sentinel); }
 
+// unqualified might find .begin() anyway through std::begin etc (!) Yet another footgun
+constexpr auto begin(is_ra auto && a) requires (requires { a.begin(); }) { static_assert(std::is_lvalue_reference_v<decltype(a)>); return a.begin(); }
+constexpr auto end(is_ra auto && a) requires (requires { a.end(); }) { static_assert(std::is_lvalue_reference_v<decltype(a)>); return a.end(); }
+constexpr auto range(is_ra auto && a) requires (requires { a.begin(); }) { static_assert(std::is_lvalue_reference_v<decltype(a)>); return std::ranges::subrange(a.begin(), a.end()); }
+
 
 // ---------------------------
-// i/o
+// i/o FIXME reuse general plyers once they allow specifying order
 // ---------------------------
 
-// TODO once ply_ravel lets one specify row-major, reuse that.
 template <class A>
 inline std::ostream &
 operator<<(std::ostream & o, FormatArray<A> const & fa)

ra/ra.hh

@@ -24,6 +24,7 @@
 // Enable ADL with explicit template args. See http://stackoverflow.com/questions/9838862.
 template <class A> constexpr void transpose(ra::noarg);
 template <int A> constexpr void iter(ra::noarg);
+template <class T> constexpr void cast(ra::noarg);
 
 
 // ---------------------------
@@ -145,60 +146,47 @@ template <class X> concept iota_op = ra::is_zero_or_scalar<X> && std::is_arithme
 template <is_iota I, iota_op J>
 constexpr auto
 optimize(Expr<std::plus<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(ITEM(0).n, ITEM(0).i+ITEM(1), ITEM(0).s);
-}
+{ return ra::iota(ITEM(0).n, ITEM(0).i+ITEM(1), ITEM(0).s); }
+
 template <iota_op I, is_iota J>
 constexpr auto
 optimize(Expr<std::plus<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(ITEM(1).n, ITEM(0)+ITEM(1).i, ITEM(1).s);
-}
+{ return ra::iota(ITEM(1).n, ITEM(0)+ITEM(1).i, ITEM(1).s); }
+
 template <is_iota I, is_iota J>
 constexpr auto
 optimize(Expr<std::plus<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(maybe_len(e), ITEM(0).i+ITEM(1).i, ITEM(0).gets()+ITEM(1).gets());
-}
+{ return ra::iota(maybe_len(e), ITEM(0).i+ITEM(1).i, ITEM(0).gets()+ITEM(1).gets()); }
 
 template <is_iota I, iota_op J>
 constexpr auto
 optimize(Expr<std::minus<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(ITEM(0).n, ITEM(0).i-ITEM(1), ITEM(0).s);
-}
+{ return ra::iota(ITEM(0).n, ITEM(0).i-ITEM(1), ITEM(0).s); }
+
 template <iota_op I, is_iota J>
 constexpr auto
 optimize(Expr<std::minus<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(ITEM(1).n, ITEM(0)-ITEM(1).i, -ITEM(1).s);
-}
+{ return ra::iota(ITEM(1).n, ITEM(0)-ITEM(1).i, -ITEM(1).s); }
+
 template <is_iota I, is_iota J>
 constexpr auto
 optimize(Expr<std::minus<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(maybe_len(e), ITEM(0).i-ITEM(1).i, ITEM(0).gets()-ITEM(1).gets());
-}
+{ return ra::iota(maybe_len(e), ITEM(0).i-ITEM(1).i, ITEM(0).gets()-ITEM(1).gets()); }
 
 template <is_iota I, iota_op J>
 constexpr auto
 optimize(Expr<std::multiplies<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(ITEM(0).n, ITEM(0).i*ITEM(1), ITEM(0).gets()*ITEM(1));
-}
+{ return ra::iota(ITEM(0).n, ITEM(0).i*ITEM(1), ITEM(0).gets()*ITEM(1)); }
+
 template <iota_op I, is_iota J>
 constexpr auto
 optimize(Expr<std::multiplies<>, std::tuple<I, J>> && e)
-{
-    return ra::iota(ITEM(1).n, ITEM(0)*ITEM(1).i, ITEM(0)*ITEM(1).gets());
-}
+{ return ra::iota(ITEM(1).n, ITEM(0)*ITEM(1).i, ITEM(0)*ITEM(1).gets()); }
 
 template <is_iota I>
 constexpr auto
 optimize(Expr<std::negate<>, std::tuple<I>> && e)
-{
-    return ra::iota(ITEM(0).n, -ITEM(0).i, -ITEM(0).gets());
-}
+{ return ra::iota(ITEM(0).n, -ITEM(0).i, -ITEM(0).gets()); }
 
 #endif // RA_DO_OPT_IOTA
 

test/stl-compat.cc

@@ -105,6 +105,10 @@ int main()
         // tr.test(std::weak_output_iterator<decltype(a.begin()), int>); // p2550 when ready c++
         tr.test(std::input_iterator<decltype(begin(a+1))>);
         tr.test(std::sentinel_for<decltype(end(a+1)), decltype(begin(a+1))>);
+        ra::Big<int, 2> b;
+        tr.test(std::random_access_iterator<decltype(ra::begin(b))>);
+        ra::Small<int, 2, 3> c;
+        tr.test(std::random_access_iterator<decltype(ra::begin(c))>);
     }
     tr.section("STLIterator works with arbitrary expr not just views");
     {
@@ -132,6 +136,17 @@ int main()
         std::ranges::copy(range(b*1.) | std::views::transform([](auto x) { return -x; }), begin(a));
         tr.test_eq((ra::_0 - ra::_1 + ra::_2)*(-1.), a);
     }
+    tr.section("ra::begin / ra::end use members if they exist");
+    {
+        ra::Big<char, 1> A = {'x', 'z', 'y'};
+        std::sort(ra::begin(A), ra::end(A));
+        tr.test_eq(ra::start({'x', 'y', 'z'}), A);
+    }
+    {
+        ra::Big<char, 1> A = {'x', 'z', 'y'};
+        std::ranges::sort(ra::range(A));
+        tr.test_eq(ra::start({'x', 'y', 'z'}), A);
+    }
 #if __cpp_lib_span >= 202002L
     tr.section("std::span");
     {