Skip to content

Latest commit

 

History

History
1337 lines (1026 loc) · 70 KB

index.md

File metadata and controls

1337 lines (1026 loc) · 70 KB

CppML Tutorial

Table of Contents

Introduction


CppML is a metalanguage for C++. It was designed to simplify the process of creating intricate classes, by letting the programmer design them through expressions that feel like algorithms in a functional language. It strives to be easy to write and easy to read, while being efficient. It does so by providing compositional pipelines through which parameter packs can flow without instantiating new types. Our goal is to give library developers programmatic control over the creation of class hierarchies with metaprograms that shape their structure and behaviour through metafunctional logic. This way constructions of complex designs are easily encoded in concise and readable functional expressions.

An illustrative example is generating a tagged hierarchy of classes, which is used by some implementations of a tuple. We want a metafunction MakeBase, where e.g. MakeBase<T0, T1, T2, T3> is equivalent to:

Elem<Tag<ml::Int<0>, T0>,
     Elem<Tag<ml::Int<1>, T1>,
          Elem<Tag<ml::Int<2>, T2>, Elem<Tag<ml::Int<3>, T3>>>>>;

Using CppML we can express MakeBase as a simple metaprogram:

template <typename... Ts>
using MakeBase = ml::f<
    ml::ZipWith<Tag, ml::Map<ml::Curry<ml::F<Elem>>, ml::F<ml::Compose>>>::f<
        ml::Range<>::f<0, sizeof...(Ts)>, ml::ListT<Ts...>>,
    EmptyBase>;

In this tutorial, we will go over the design of the CppML language and explore its prominent features in depth. You will learn about compositional pipelines and the flow of parameter packs through them. You will learn about the structure of metafunctions, how to understand their metafunction type, and how they integrate with pipelines. You will learn how to manipulate metafunctions using concepts like Currying, Product Maps and Branch Pipes, and how to compose them into algorithms that will build your class designs.

Interspersed throughout the tutorial are use-cases, where we will formulate a problem and break down its solution into steps, and than translate them into CppML. Through them you will learn how to encode construction of (increasingly) complex designs into elegant and concise functional expressions.

Links to CppML Reference

Throughout this tutorial, all links with the ml:: prefix (e.g. ml::Map) lead to the CppML reference for that specific metafunction. Each of them provides a specification of its structure, a definition of its metafunction type, and an example of use. You are encouraged to follow them, as you will quickly be able to learn from the CppML reference itself.

Key Concepts

Parameter pack

A parameter pack is a template argument with the syntax

template <typename ...Pack>

You can never hold a parameter pack, i.e.

template <typename ...Pack>
using T = Pack;

is a nonsensical statement, that will not compile. You can only operate on them, and pass (expand) them into another template.

template <typename ...Pack>
using T = AnotherTemplate::f<Transform::f<Pack>...>;

This mechanic is at the core of CppML. Metafunctions take in (through their f alias) a parameter pack, transform it, and pass it on to another template through a Pipe.

NOTE

In the context of CppML, we can view

template<typename T, typename U>

as parameter packs of non-variadic length, and

template<typename T, typename ...Ts>

as variadic parameter packs with minimal length. This is because CppML correctly handles mixed alias templates of pack and non-pack arguments.

Pipe

One of the pillars of CppML is composability. As such, the ability to chain transformations on parameter packs is a core feature of the language. Concretely, every metafunction template has, as their last template parameter, a Pipe into which it passes the transformed parameter pack. Please see the metafunction section to see how pipelines integrate with metafunctions and the Using Pipes section for a detailed demonstration of building pipelines.

The reader might be familiar with the concept of pipes from bash (operator |), or from R (operator %>%), or from Haskel (operator >->).

Metafunction

In CppML, a metafunction is an instantiation of a template struct with a template alias f. All metafunctions define, as their last template parameter, a metafunction Pipe into which f will pass the transformed parameter pack. In pseudo-c++ this roughly translates into

template <typename PossibleArgs...,                                // zero of more arguments
          typename Pipe = (default: ml::Identity or ml::ToList)>   // see `Default pipes` below
struct MetaFunction {
  template <typename... Args>
  using f = Pipe::f<                                               // result passed to Pipe
                    ...mfImplementation::f<
                                           PossibleArgs...,
                                           ...Args...>...>;
};

Metafunction type

Throughout the CppML reference, we use the following notation

f:: Args... -> Us... >-> Pipe

to mean:

The template alias ::f of Metafunction takes a parameter pack Args... and transforms it to a parameter pack Us... which is than passed on into Pipe (i.e. the ::f alias of Pipe is invoked on Us...).

Default pipes

Metafunctions that pass on a single parameter T have Pipe defaulted to ml::Identity, which is a metafunction with an alias f,

f:: T -> T

that simply returns the argument it was given.

Metafunctions that pass on parameter packs have Pipe defaulted to ml::ToList, which is a metafunction with an alias f,

f:: Ts... -> ml::ListT<Ts...>

that wraps the parameter pack in an ml::ListT.

Invoking metafunctions

Due to the syntax of c++, unaided invocations of metafunctions look like

using T0 = typename MetaFunction::template f<int, string, char, vector<int>>;

As this is cumbersome, CppML defines ml::f,

namespace ml {
template <typename Metafunction, typename ...Args>
using f = typename Metafunction::template f<Args...>;
}

an alias which invokes the metafunctions' f alias on the supplied arguments. Using it, T0 (above) is equivalent to a more concise

using T1 = ml::f<MetaFunction, int, string, char, vector<int>>;

Taking a concrete example, we ml::Filter a parameter pack given a predicate ml::IsClass

using T2 = ml::f<ml::Filter<ml::IsClass<>>, int, string, char, vector<int>>;
static_assert(
          std::is_same_v<
                         T2,
                         ml::ListT<
                                   string, vector<int>>>);

Using Pipes

Suppose we want a metafunction that takes the elements of the parameter pack in the range [2, 6], and from that range remove all (ml::RemoveIf) which are a class type (ml::IsClass).

using F = ml::Pivot<2,                                    // make third element the first
          /* Pipe*/ ml::Head<4,                           // take first 4 elements
          /* Pipe*/          ml::RemoveIf<ml::IsClass<>>; // filter them

To illustrate the flow of parameter packs through the pipelines, lets take the concrete example of int, char, vector<int>, int, string, double, float, and send it through F.

int, char, vector<int>, int, string, double, float 
->                                                  // Pivot<2>
vector<int>, int, string, double, float, int, char 
>->                                                 // Pipe to
vector<int>, int, string, double, float, int, char
->                                                  // Head<4>
vector<int>, int, string, double 
>->                                                 // Pipe to
vector<int>, int, string, double 
->                                                  // RemoveIf<IsClass<>>
int, double                                        
>->                                                 // Default Pipe to
int, double
->                                                  // ToList
ml::ListT<int, double>

See also ml::Pivot, and ml::Head (defined in the Pack header).

Lifting templates to metafunctions

Any template, either a struct / class

template<typename ...Ts>
struct T { /* ... */};

or an alias

template<typename ...Ts>
using AT = /* .... */;

can be lifted to a metafunction, through the use of ml::F<Template, Pipe> (see its reference). For example, we can make std::tuple a metafunction

using TupleF = ml::F<std::tuple>;

which can than be evaluated

using T = ml::f<TupleF, int, char>;
static_assert(
          ml::is_same_v<
                  T,
                  std::tuple<int, char>>);

We can now alter the previous example, and use TupleF, as the Pipe of ml::Filter

using T2 = ml::f<
                 ml::Filter<                     // Filter
                            ml::IsClass<>,       // Predicate
                            TupleF>,             // Pipe of Filter
                int, string, char, vector<int>>;
static_assert(
          std::is_same_v<
                         T2,
                         std::tuple<
                                    string, vector<int>>>);

NOTE

Although both T and AT above have variadic parameter packs, the parameter packs can be of any form (as specified in the note here); i.e. variadic, non-variadic and variadic with minimal length.

Manipulating metafunctions

CppML aims to be used and read as a (meta) functional language. As such, the key skill to master, is how to combine and compose simple metafunctions into complex metafunctions.

Composition

The most basic of manipulations of metafunctions is the composition. The idiomatic way to compose metafunctions when writing metaprograms is using pipes. The flow of composition looks like this.

f:: Ts ... -> Us... >-> Pipe

So, given that we have ml::IsClass, and ml::Not, we can define

using IsNotClass = ml::IsClass<ml::Not<>>;

IsNotClass by using pipes. But using pipes is only possible, when you have access to setting the template parameters. To illustrate the point, suppose we have concrete metafunctions

using IsClassF = ml::IsClass<>;
using NotF = ml::Not<>;

for which we do not have access to Pipe. In these cases, we can use the ml::Compose, and construct

using IsNotClassF = ml::Compose<NotF, IsClassF>;

which is equivalent to IsNotClass above. We can go further and make it an alias with a Pipe template parameter,

template <typename Pipe = ml::Identity>
using IsNotClassF = ml::Compose<Pipe, NotF, IsClassF>;

making it a proper metafunction. We can see that

static_assert(std::is_same_v<
                  ml::f<IsNotClassF<ml::Not<>>, int>
                  ml::f<ml::IsClass<>, int>>);

negating is IsNotClassF is equivalent to IsClass.

Product

A different way to combine metafunctions is to take their ml::Product<Fs..., Pipe>. An ml::Product<Fs..., Pipe>, which takes n metafunctions Fs... (see ml::Product for detailed reference), each mapping

f:: Ts... -> U

and a Pipe,

f:: Us... -> Vs...

is itself a metafunction

f:: Ts... -> Fs(Ts...)... >-> Pipe
             ^^^^^^^^^^^^
             U0...Uk...Un

which can be topologically envisioned as

              -> U0 -
            /    ...  \
f:: Ts ... -  ->  Uk -- >-> Pipe
            \    ...  /
              -> Un - 

where each arrow is one of the metafunctions Fs....

To demonstrate, we implement a metafunction partition, which takes a Predicate, and partitions a parameter pack into two parts, one containing the elements that satisfy the Predicate, and one containing those two don't. This can be achieved by taking a ml::Product of ml::Filter and ml::RemoveIf (from the Algorithm header).

template <typename Predicate, typename Pipe = ml::ToList>
using Partition = ml::Product<
                        ml::Filter<Predicate>,
                        ml::RemoveIf<Predicate>,
                        Pipe>;

To see how it works, we invoke it on int, string, bool, vector<int>.

using T = ml::f<
                Partition<ml::IsClass<>>,
                int, string, bool, vector<int>>;
static_assert(
      std::is_same_v<
          T,
          ml::ListT<
              ml::ListT<string, vector<int>>,
              ml::ListT<int, bool>>>);

NOTE

CppML defines ml::Partition in the Algorithm header. The implementation is almost identical to the one presented here.

Product map

As another way to make a product of metafunctions is to take their ml::ProductMap<Fs..., Pipe>. An ml::ProductMap<Fs..., Pipe>, which takes n metafunctions Fs... (see ml::ProductMap for detailed reference), each mapping

f:: T -> U

and a Pipe,

f:: Us... -> Vs...

is itself a metafunction

f:: Ts... -> Fs(Ts)...  >-> Pipe
            ^^^^^^^^^^^^
            U0...Uk...Un

which can be topologically envisioned as

     T0 -> U0 -
    ... -> ...  \
f::  Tk -> Uk --  >-> Pipe
    ... -> ...  /
     Tn -> Uk -
      

where each arrow is one of the metafunctions Fs....

To demonstrate we implement a metafunction that counts the number of elements of a parameter pack that satisfy a Predicate. We will do so by utilizing the ml::Reduce (from the Algorithm header). Looking at its reference, we see that it is instantiated with a metafunction

f:: Init, U -> Init'

that is used to accumulate the generalized sum. In our case, we want a metafunction that will add 1 to the current value if the predicate is true and 0 otherwise. This means

f:: Init, U -> Init, Predicate(U) >-> Add

which is equivalent to

f:: Init, U -> Identity(Init), Predicate(U) >-> Add

We now see that this is the ml::ProductMap of ml:Identity and the Predicate, which Pipes to ml::Add. Hence

template<typename Predicate, typename Pipe = ml::Identity>
using CountIf_ = ml::Reduce<
                           ml::ProductMap<
                                          ml::Identity,
                                          Predicate,
                                          ml::Add<>>,
                          Pipe>;

To see how it works, we invoke it on int, string, bool, vector<int>.

using T = ml::f<
                CountIf_<ml::IsClass<>>,
                ml::Int<0>,                      // Init as first parameter for reduce
                int, string, bool, vector<int>>;
static_assert(
              std::is_same_v<
                            T,
                            ml::Int<2>>);

For CountIf_ to fully be a CountIf metafunction, we need to get rid of the need to pass the ml::Int<0> as the initial value for ml::Reduce, because the count always starts at 0. We can do this by partially evaluating CountIf_, which is the subject of the next section.

NOTE

CppML defines ml::CountIf in the Algorithm header.

Partial evaluation

After studying how you can combine metafunctions, we take a look at how we can transform a metafunction into a different metafunction, that has some of its parameters fixed.

Partial

To partially evaluate a metafunction from the left, we employ the ml::Partial<F, Args...> from the Functional header. Partial<F, Args...> is a metafunction, that is a partial evaluation of F. When invoked on Ts..., it is as if F was invoked on Args..., Ts....

f:: Ts... -> F(Args..., Ts...)

Hence, we can use ml::Partial to partially evaluate CountIf_ from the previous section on ml::Int<0>. Full implementation looks like this

template<typename Predicate, typename Pipe = ml::Identity>
using CountIf = ml::Partial<
                        ml::Reduce<
                           ml::ProductMap<
                                  ml::Identity,
                                  Predicate,
                                  ml::Add<>>,
                           Pipe>,
                        ml::Int<0>>;

To test it, we invoke it on the same example.

using T = ml::f<
                CountIf<ml::IsClass<>>,
                int, string, bool, vector<int>>;
static_assert(
              std::is_same_v<
                            T,
                            ml::Int<2>>);

PartialR

To partially evaluate a metafunction from the right, we employ the ml::PartialR<F, Args...> from the Functional header. PartialR<F, Args...> is a metafunction, that is a partial evaluation of F. When invoked on Ts..., it is as if F was invoked on Ts..., Args....

f:: Ts... -> F(Ts..., Args...)

Bind

To give user complete control over partial evaluations, CppML provides ml::Bind<F, ml::Par<Is, Args>...> for the Functional header. ml::Bind<F, ml::Par<Is, Args>...> is a partial evaluation, where each element on the parameter pack Args... is bound to its corresponding position int ...Is.

The ml::Par<I, T> is the parameter holder for ml::Bind (see its reference page). Aliases of the form

template <typename T>
using _0 = ml::Par<0, T>

are defined for N in the range [0, 20].

To illustrate ml::Binds functionality, we state that binding to first few positions

using boundF = ml::Bind<F, ml::_0<int>, ml::_1<bool>>;

is equivalent to partially evaluating from the left.

using boundF = ml::Partial<F, int, bool>;

Please consult its reference page for details and an example demonstration of use.

Use case: A generator of linear CRTP class hierarchies

As a use case, we will implement a metaprogram that generates a linear CRTP class hiearchy from a parameter pack of templates.

Suppose we define policy classes

template <class MostDerived, class BasePolicy>
struct Policy_i : BasePolicy {
/*
 * Implementations
 */
};

which employ the base-chaining mechanism. Each of these policies defines some public methods. In order for the user to use them, he has to derive from them, by chaining them. For example, to use Policy0, Policy1 and Policy3, he has to derive from

struct Class0 : Policy0<Class0, Policy1<Class0, Policy3<Class0, EmptyBase>>> {};

which is limiting in a few ways:

  • user has to manually nest the Policies, but more importantly
  • we cannot define a general template class for the user to use, where he would simply specify the Policies he wants. Currently he must manually write the class given the policies he wants, as above with Class0.

We need a MakeBase metaprogram, that will allow us to implement a general class template

template <template <class, class> class ...Policies>
struct Class : MakeBase<Class<Policies...>, Policies...> {};

where

using Class0_ = Class<Policy0, Policy1, Policy3>;

Class0_ is equivalent to Class0 above.

Using the mechanics described so far, this can be achieved by:

This sequence is easily translated into CppML.

template <typename Derived, template <class, class> class ...Policies>
using MakeBase = ml::f<ml::Compose<ml::Partial<ml::F<Policies>, Derived>...>, ml::None>;

which concludes our implementation of Class (above), in an elegant one-liner. This is the strength of the functional approach to meta design.

Currying

With a firm grasp on partial evaluations, we step a level higher, and take a look at currying. Please take note that currying operators in CppML operate on variadic parameter packs (i.e. they can curry more than one argument at a time).

Curry

Let F be a metafunction Args... -> Us.... Than ml::Curry<F> is a metafunction that takes a parameter pack T0s..., and returns a metafunction F': T1s... -> Us..., in such a way, that F(T0s..., T1s...) == F'(T1s...).

f:: T0s... -> T1s... -> Us...

This means that you can imagine the ml::Curry<F> as partially evaluated metafunction made by lifting the template) of ml::Partial to a metafunction using ml::F, and partially evaluating it on F (the template argument of Curry). So lets write it:

template <typename F>
using Curry = ml::Partial<ml::F<ml::Partial>, F>;

Note that in CppML, ml::Curry<F> is implemented by inheritance instead of an alias, to make it a distinct type. But the logic is the same.

CurryR

Please consult the CppML reference for ml::CurryR, which is like ml::Curry, except that it partially evaluates from the right (see ml::PartialR).

Use case: A generator of tagged class hierarchies

Suppose have an object holder, which (as in our previous use case) uses base-chaining

template <typename TaggedObject, typename Base>
struct Holder : Base;

but each object being held, needs to be tagged using an ml::Int, to both give it a distinct identity, and a static index. As such, assuming

template <typename Tag, typename Object> struct Param {};

each TaggedObject is constrained to be Param<ml::Int<N>, Object>.

template <int N, typename Object, typename Base>
struct Holder<Param<ml::Int<N>, Object>, Base> : Base {
  Object object;
  /* Implementation of logic */
};

For example, this is used in some implementations of std::tuple. One creates a nested hierarchy of such Holders and derives from it. In order to be able to make use of such techniques, we need a metaprogram MakeBase

template <typename ...Ts>
struct Class : MakeBase<Ts...> {
  /* Implementation */
};

where MakeBase<T0, T1, T2, T3> is equivalent to

Holder<Param<ml::Int<0>, T0>,
                Holder<Param<ml::Int<1>, T1>,
                                Holder<Param<ml::Int<2>, T2>>,
                                                  Holder<Param<ml::Int<3>, T3>>>>;

Using the mechanics described so far, this can be achieved by:

This leaves us with a metafunction that is to be evaluated on its most bottom Base class. The sequence is easily translated into CppML.

template <typename ...Ts>
using MakeBase_f = ml::f<
    ml::ZipWith<Param, ml::Map<ml::Curry<ml::F<Holder>>, ml::F<ml::Compose>>>,
    ml::Range<>::f<0, sizeof...(Ts)>, ml::ListT<Ts...>>;

Which, after using ml::None as our bottom base class,

template <typename ...Ts>
using MakeBase = ml::f<MakeBase_f<Ts...>, ml::None>;

concludes our implementation of Class (above), which remains simple and easy to read.

Functional branching

There are two main actors behind functional branching in CppML. The first is the ml::IfElse construct, where ml::IfElse<ml::Bool<t_val>> is a metafunction

f:: T, U -> V >-> Pipe

that passes to pipe T if t_val is true, and U if t_val is false. In its bare form, it behaves like so:

using T = ml::f<ml::IfElse<ml::Bool<true>>, int, char>;
static_assert(
    std::is_same_v<
              T, int>);

The second is ml::BranchPipe, where ml::BranchPipe<Predicate, IfPipe, ElsePipe> is a metafunction which passes the parameter pack Ts..., to one of two Pipes, given the result of invoking the Predicateon Ts....

f:: Ts... -> Ts... >-> (Predicate(Ts...) ? IfPipe : ElsePipe)

BranchPipe brings the powerful notion of having the execution of metaprograms depend on introspective predicates about types into the language. In other words, it allows us to write meta algorithms that design class hierarchies, that not only stitch them together, but also change and alter their design, based on the types involved in it.

Use case: A generator of tagged class hierarchies with conditional branches

Remembering our previous use case in which we created A generator of tagged class hierarchies, where our objects were held by

template <int N, typename Object, typename Base>
struct Holder<Param<ml::Int<N>, Object>, Base> : Base {
  Object object;
  /* Implementation of logic */
};

we note that we did not take advantage of the empty-base-class optimization. Because of this, each Object occupies at least 1 byte in the hierarchy, even if they are empty (this is due to the fact that each object needs a unique address). If we instead wrote our Holder as BaseHolder,

template <int N, typename Object, typename Base>
struct BaseHolder<Param<ml::Int<N>, Object>, Base> : Object, Base {
  /* Implementation of logic */
};

all empty Objects (of different type) could share the address, and hence gain memory efficiency by the empty base class optimization. But the problem with this approach is that you can only derive from class types. Hence, if we simply substituted Holder for BaseHolder in our metaprogram from A generator of tagged class hierarchies, it would fail to compile when any type in the invoking parameter pack Ts... was a non-class type. The solution is to let the metaprogram conditionally choose which class will hold the Object, depending on whether it is a class or not.

Hence, we want to alter the metaprogram MakeBase from A generator of tagged class hierarchies, such that MakeBase<int, string, char, vector<int>> is equivalent to

Holder<Param<ml::Int<0>, int>,
               BaseHolder<Param<ml::Int<1>, string>,
                           Holder<Param<ml::Int<2>, char>>,
                                          BaseHolder<Param<ml::Int<3>, vector<int>>>>>;

We will accomplish this by using BranchPipe, which will choose between two metafunctions as pipes, namely the ml::Curry<ml::F<Holder>> and ml::Curry<ml::F<BaseHolder>>, given the result of the ml::IsClass predicate. This will be used in the place where we used ml::Curry<ml::F<Holder>> in the A generator of tagged class hierarchies. With this addition, the procedure reads:

This leaves us with a metafunction that is to be evaluated on its most bottom Base class. The sequence is easily translated into CppML.

template <typename... Ts>
using MakeBase_f = ml::f<
    ml::ZipWith<Param,
                ml::Map<ml::BranchPipe<ml::Unwrap<ml::Get<0, ml::IsClass<>>>,
                                       ml::Curry<ml::F<BaseHolder>>,
                                       ml::Curry<ml::F<Holder>>>,
                        ml::F<ml::Compose>>>,
    ml::Range<>::f<0, sizeof...(Ts)>, ml::ListT<Ts...>>;

Which, after using ml::None as our bottom base class,

template <typename ...Ts>
using MakeBase = ml::f<MakeBase_f<Ts...>, ml::None>;

concludes our implementation. Looking at the metaprogram MakeBase, it is a direct translation of our procedure in the bullet list, into the compositional pipelines of CppML. We hope it sheds light on the purpose of CppML: to give you control over class design through expressions that feel like algorithms in a functional language.

Unwrapping template arguments into metafunctions

Templates are constructs which wrap parameter packs. For example

using Tpl = std::tuple<int, char, string, bool>;

Tpl is the parameter pack int, char, string, bool that is wrapped in a std::tuple. A different example you might encounter using CppML, is ml::ListT, when you store a result of a metacomputation, where the final Pipe was ml::ToList.

Suppose you now want to invoke a metafunction on the parameter pack being wrapped. CppML defines ml::Unwrap with the metafunction type

f:: Template<Ts...> -> Ts... >-> Pipe

which is exactly what we need. For example, this can be used to compute the number of template parameters, by unwrapping them into ml::Length

f:: Template<Ts...> -> Ts... >-> ml::Length

Use case: Subsetting a std::tuple to its non-class types

Suppose we wish to create a sub-tuple of Tpl, which will contain only the elements which are non-class types. Using what we have seen so far, this is accomplished by:

This sequence is easily translated into CppML,

using NonClassSubtuple = ml::Unwrap<ml::RemoveIf<ml::IsClass<>, ml::F<std::tuple>>>;

and the NonClassSubtuple metafunction is ready to use:

using NonClassSubtuple = ml::Unwrap<ml::RemoveIf<ml::IsClass<>, ml::F<std::tuple>>>;
using SubTpl = ml::f<NonClassSubtuple, Tpl>;
static_assert(
        std::is_same_v<
              T,
              std::tuple<
                int, char, bool>>);

Aliases as type-lambdas

As we have already touched on, it is possible to lift an alias template into a metafunction. This also means that you are able to write metafunctions on the spot by lifting aliases.

One application of this concept is to use it for a different way of partially evaluating a metafunction. For example, writing an IsInt metafunction can be written using ml::IsSame, like so

template <typename T>
using IsInt_f = ml::f<ml::IsSame<>, T, int>;
using IsInt = ml::F<IsInt_f>;

Note that ml::F also accepts a Pipe, allowing us to turn the alias template IsInt_f into a proper metafunction

template <typename Pipe = ml::Identity>
using IsInt = ml::F<IsInt_f, Pipe>;

which has standard pipe-ing syntax.

using T = ml::f<
                IsInt<ml::Not<>>,
                double>;

static_assert(
        std::is_same_v<
            ml::Bool<true>,
            T>);

Use case: Introspective metafunctions

Introspection can be implemented by checking whether an invocation of a metafunction on some parameter pack Ts... is ill-formed or not. These checks can be performed by ml::IsValid, which maps a metafunction F and arguments Ts..., to an ml::Bool. We can create metafunctions, whose validity answers our introspective questions.

Suppose we are interested in whether a type defines a .size() method. Than, we create a type lambda which attempts to use it, like so:

template <typename _T>
using HasSize_f = decltype(std::declval<_T>().size());

which can be used with ml::IsValid by lifting it to a metafunctions.

using T = ml::f<
                ml::IsValid<>,
                ml::F<HasSize_f>,
                std::vector<int>>;

static_assert( std::is_same_v<
                          T, ml::Bool<true>>);

We can now define a proper metafunction HasSize, by partially evaluating ml::IsValid on ml::F<HasSize_f>. We will also give it a Pipe, to make it a proper metafunction.

template <typename Pipe = ml::Identity>
using HasSize = ml::Partial<ml::IsValid<Pipe>, ml::F<HasSize_f>>;

We now have a HasSize metafunction, which can be used in compositional pipelines of CppML as any other metafunction. Lets use it to ml::Filter a parameter pack for types which have a .size() method.

using T = ml::f<
                ml::Filter<HasSize<>>,
                std::vector<int>, int, std::map<int, int>>;
static_assert(
        std::is_same_v<
                T,
                ml::ListT<
                    std::vector<int>, std::map<int, int>>>);

The other thing one might want to do is get the result of the invocation of a metafunction on Ts... if it is not ill-formed, and get a Default type, if it is ill-formed. The construct that enables this is the ml::IfValidOr.

To demonstrate, suppose we need a metafunction that extracts the ValueType alias of a type if it defines one, or return ml::None, if it does not.

struct A { using ValueType = int;};
struct B {};

template <typename _T>
using GetType_f = typename _T::ValueType;

We will define a metafunction by partially evaluating ml::IfValidOr on ml::None (as Default) and ml::F<GetType_f>. We will also give it a Pipe, to make it a proper metafunction.

template <typename Pipe = ml::Identity>
using GetValueType = ml::Partial<ml::IfValidOr<Pipe>, ml::None, ml::F<GetType_f>>;

We now have a GetValueType metafunction, which can be used in compositional pipelines of CppML as any other metafunction:

using T = ml::f<
            ml::Map<GetValueType>,
            A, B>;


static_assert(
        std::is_same_v<
                T,
                ml::ListT<
                  int, ml::None>>);

This demonstrates how you can include introspective logic in your class design using type lambdas.

Reference: Functional

The constructs presented in this section live in the Functional header. The header also contains other constructs, which were not addressed in the section. Given what you have learned so far, you are able to find the remaining constructs in the table below, and learn about each from its reference page, where you can find a specification of its structure, a definition of its metafunction type, and an example of use.

Construct Description Type of f in ::f >-> Pipe
Bind Metafunction with args bound to specific positions. Ts... -> Us...
BranchPipe Branches to one of two Pipes, given a Predicate. Ts... -> Ts...
Compose Composition of metafunctions Fs.... Ts... -> F0(F1(...(Fn(Us...)
Constant Metafunction that always forwards U. Ts... -> U
Curry The Curry (from the left) operator T0s... -> T1s... -> Us...
CurryR The Curry (from the right) operator T1s... -> T0s... -> Us...
DelayedEval Delays an evaluation until compiler knows the arity. Ts... -> Us...
f Invokes the f alias of the metafunction F Ts... -> F::f<Ts...>
F Lifts a template to a metafunction Ts... -> Template<Ts...>
fx Invokes the metafunction on Ts... Ts... -> Us... -> Us...
Identity Identity metafunction. T -> T
IfElse Chooses between T and U. T, U -> V
Map Maps Ts... by F. Ts... -> F(Ts)...
Partial Partial evaluation of F on T0s... from the left T1... -> F(T0..., T1...)
PartialR Partial Evaluation of F on T1s... from the right T0... -> F(T0..., T1...)
Product Product of metafunctions Fs... Ts... -> Fs(Ts...)...
ProductMap Product map of metafunctions Fs Ts... -> Fs(Ts)...
ToList Wraps Ts... in an ListT Ts... -> ListT<Ts...>
ToValue Invokes ::value on Ts... Value<Ts, ts>... -> ts...
Unwrap Unwraps the template around Ts... Template<Ts...> -> Ts...

Manipulating parameter packs

Parameter packs are the streams of types that flow through our pipelines. As such, it is necessary to have a good grasp of the mechanics that control them, as you are sending them through pipes.

Choosing and Sub-packing

We can extract the first element of a parameter pack using ml::First

using T = ml::f<ml::Front<>, int, char, bool>;
static_assert(
        std::is_same_v<
              T, int>);

or extract an element at arbitrary position using ml::Get.

using T = ml::f<ml::Get<1>, int char bool>;
static_assert(
        std::is_same_v<
              T, char>);

We can sub-pack the parameter pack to the first N elements using ml::Head

using T = ml::f<ml::Head<2>, int, char, bool>;
static_assert(
        std::is_same_v<
              T,
              ml::ListT<
                int, char>>);

or sub-pack it to the last N elements using ml::Tail

using T = ml::f<ml::Tail<2>, int, char, bool>;
static_assert(
        std::is_same_v<
              T,
              ml::ListT<
                char, bool>>);

Note that all constructs that manipulate parameter packs are just metafunctions, and can be used with Pipes.

using T = ml::f<ml::Tail<2, ml::Map<ml::IsClass<>>>, int, string, bool>;
static_assert(
        std::is_same_v<
              T,
              ml::ListT<
                ml::Bool<true>, ml::Bool<false>>>);

Appending, Prepending and Inserting

You can append an element to a parameter pack by ml::Append

using T = ml::f<ml::Append<string, ml::Map<ml::IsClass<>>>, int, bool>;
static_assert(
        std::is_same_v<
              T,
              ml::ListT<
                ml::Bool<false>, ml::Bool<false>, ml::Bool<true>>>);

or prepend it using ml::Prepend.

using T = ml::f<ml::Prepend<string, ml::Map<ml::IsClass<>>>, int, bool>;
static_assert(
        std::is_same_v<
              T,
              ml::ListT<
                ml::Bool<true>, ml::Bool<false>, ml::Bool<false>>>);

To insert an element at N-th position, use ml::Insert

using T = ml::f<ml::Insert<, 1, string, ml::Map<ml::IsClass<>>>, int, bool>;
static_assert(
        std::is_same_v<
              T,
              ml::ListT<
                ml::Bool<false>, ml::Bool<true>, ml::Bool<false>>>);

NOTE

As an exercise, think about how you could implement ml::Insert, using ml::Pivot (see Algorithms on parameter packs) and ml::Prepend.

  • Pivot N-th element to the front
  • Prepend the element
  • Pivot original (now at position sizeof...(Ts) - N + 1) to the front

Because this is how ml::Insert is implemented, you can look at its implementation.

Appending multiple elements using partial evaluations

Because, as explained in parameter packs, you cannot hold a parameter pack, but only pass it forward (to a Pipe), appending to a parameter pack always happens as parameter passing. Hence, appending Us... to Ts... as we pass to F, is equivalent to an invocation of a partially evaluated F.

using T = ml::f<ml::Partial<ml::Map<ml::IsClass<>>>, string>, int, bool>;
static_assert(
        std::is_same_v<
              T,
              ml::ListT<
                ml::Bool<true>, ml::Bool<false>, ml::Bool<false>>>);

NOTE

The same holds for prepending, but we partially evaluate from the right using ml::PartialR. A similar procedure can also be performed for inserting, if we include some ml::Pivot-ing.

Reference: Pack

There are other constructs for working with parameter packs, which we did not mention in this section. Those include ml::Range, which generates a parameter pack of type-values ml::Int, in the range [From, To), with a Step that default to 1.

using Sequence = ml::Range<ml::ToList>::f<0, 6, 2>;
static_assert(
        std::is_same<
            Sequence,
            ml::List<
                ml::Int<0>, ml::Int<2>, ml::Int<4>>>);

and ml::Length which gives you the number of elements of a parameter pack. To compute the number of template arguments of type, combine it with ml::Unwrap, like we did in Unwrapping template arguments into metafunctions.

Constructs for manipulating parameter packs have an associated header CppML/Pack.hpp and a directory of the same name CppML/Pack. Every construct has a dedicated .hpp header inside the CppML/Pack/ directory. For example, the construct, ml::Head, can be found in the CppML/Algorithm/Head.hpp. Hence, you can include all algorithms at once using #include <CppML/Pack.hpp>, or include only the ones you want (e.g. #include <CppML/Pack/Head.hpp>

Find the construct of your interest in the table below. Each of them has it own reference page, where you can find a specification of its structure, a definition of its metafunction type, and an example of use.

Construct Description Type of f in ::f >-> Pipe
Drop Drops first N of Ts... T1, ... Tn, Ts... -> Ts...
Front Gets first element of Ts... T, Ts... -> T
Get Gets N-th element of Ts... T1, ... Tn, Ts... -> Tn
Head First N of Ts... T1, ... Tn, Ts... -> T1, ... Tn
Insert Inserts U as N-th element of Ts... T1, ... Tn, Ts... -> T1, ... U, Tn, Ts...
Length Length of Ts... Ts... -> Int<sizeof...(Ts)
Concat Concatenates N templates T T<A0s...>... -> T<A0s..., A1s..., Ans...>
PackExtractor Extracts N-th element. Int<N> -> T
Prepend Prepends T to Ts... Ts... -> T, Ts...
Range Pack of Int<I> in range From, To -> Int<From>..., Int<To - 1>
Tail Last N of Ts... Ts..., T_{-N}, ... T_{-1} -> T_{-N}, ... T_{-1}

NOTE

Constructs such as ml::Pivot and ml::Rotate are part of the Algorithm header.

Algorithms on parameter packs

We have already meet a few of the algorithms (like ml::ZipWith, ml::Filter, etc.) on our way to this point of the tutorial. As was apparent from their usage, they are no different from any other metafunction operating on parameter packs, so there is not much extra to be said about them. But what is important, is to know how to find them, when you need them.

Algorithms have an associated header CppML/Algorithm.hpp and a directory of the same name CppML/Algorithm. Every algorithm has a dedicated .hpp header inside the CppML/Algorithm/ directory. For example, the algorithm, ml::Sort, can be found in the CppML/Algorithm/Sort.hpp. Hence, you can include all algorithms at once using #include <CppML/Algorithm.hpp>, or include only the ones you want (e.g. #include <CppML/Algorithm/Sort.hpp>

Reference: Algorithm

We provide a detailed CppML reference, which also contains an Algorithm section. Please find the algorithm of your interest in the table below. Each of them has it own reference page, where you can find a specification of its structure, a definition of its metafunction type, and an example of use.

Construct Description Type of f in ::f >-> Pipe
AllOf Checks if a Predicate holds for all of Ts.... Ts... -> Bool<t>
AnyOf Checks if a Predicate holds for any of Ts.... Ts... -> Bool<t>
Contains Checks is Ts... contains T. Ts... -> Bool<t>
CountIf Counts Ts... for which the Predicate holds. Ts... -> Bool<t>
Filter Filters Ts..., for which the Predicate holds. Ts... -> Us...
FilterIds Filters indexes of Ts..., for which the Predicate holds. Ts... -> Int<Is>...
FindIdIf Index of Ts... for which the Predicate holds. Ts... -> Int<I>
FindIdIfNot Index of Ts... for which the Predicate does not hold. Ts... -> Int<I>
FindIf Element of Ts... for which the Predicate holds. Ts... -> T
FindIfNot Element of Ts... for which the Predicate does not hold. Ts... -> T
GroupBy Groups Ts..., given their image under By. Ts... -> ListT<Us...>...
InclusiveScan Inclusive scan under the binary F. Ts... -> T0, F(T0, T1), ...
MaxElement Get maximal element, given a Comparator. Ts... -> U
NoneOf Checks if a Predicate holds for none of Ts.... Ts... -> Bool<t>
Partition Partitions Ts... given a Predicate. Ts... -> ListT<Us...>...
PartitionIds Partitions indexes of Ts... given a Predicate. Ts... -> ListT<Int<Is>...>...
Pivot Pivots Ts... around the N-th element, making it the first. Ts... -> Us...
Reduce Reduce Ts..., given an accumulator F. Init, Ts... -> U
RemoveIdsIf Removes indexes of Ts... for which the Predicate holds. Ts... -> Us...
RemoveIf Removes elements of Ts... for which the Predicate holds. Ts... -> Int<Is>...
ReplaceIf Replace Ts..., for which the Predicate holds, by U. Ts... -> Us...
Rotate Pivots Ts... in the range [First, Middle, Last). Ts... -> Us...
Sort Sorts Ts..., given a Comparator. Ts... -> Us...
UniqueCompare Unique elements of Ts..., given a Comparator. Ts... -> Us...
Unique Unique elements of Ts.... Ts... -> Us...
ZipWith Zips two lists with a With template. Ts... -> With<Us...>...
ZipWithVariadic Zips two lists with a variadic With template. Ts... -> With<Us...>...

Examples

Optimizing the memory layout of std::tuple

The following is a condensed version of the post Optimizing the memory layout of std::tuple. The optimization comes from considering the way in which objects are laid out in memory (remember that they must be naturally aligned and thus padding may be required). We implement a class Tuple, which is an interface wrapper around std::tuple. It works by approximating the optimal permutation by the Permutation that sorts the types by their alignment. It than lays the objects out in memory in that order. It holds the Permutation as its template argument, and uses it to internally redirect the users indexing (hence the user can be oblivious to the permutation).

Please see the original post Optimizing the memory layout of std::tuple, where the entire solution is constructed step by step.

Before we begin, take a look at the result.

  Tuple<char, int, char, int, char, double, char> tup{'a', 1,   'c', 3,
                                                      'd', 5.0, 'e'};
  std::cout << "Size of out Tuple: " << sizeof(tup) << " Bytes" << std::endl;

  std::tuple<char, int, char, int, char, double, char> std_tup{'a', 1,   'c', 3,
                                                               'd', 5.0, 'e'};
  std::cout << "Size of out std::tuple: " << sizeof(std_tup) << " Bytes"
            << std::endl;

  std::cout << "Actual size of data: "
            << 4 * sizeof(char) + 2 * sizeof(int) + sizeof(double) << " Bytes"
            << std::endl;

  std::cout << get<2>(tup) << " == " << std::get<2>(std_tup) << std::endl;
  assert(tup == std_tup);

Size of Tuple: 24 Bytes
Size of std::tuple: 40 Bytes
Actual size of data: 20 Bytes
c == c


We notice that the std::tuple has 20 Bytes of wasted space (making it twice as big as the actual data), while Tuple only has 4 Bytes of wasted space.

class size [B] efficiency
Data 20 1
Tuple 24 0.84
std::tuple 40 0.5

Class Building Metaprogram

We want a TupleBase wrapper of a std::tuple, which will

template <typename Permutation, typename StdTuple> struct TupleBase;
template<int ...Is, typename ...Ts>
struct TupleBase<
            ml::ListT<ml::Int<Is>...>,
            std::tuple<Ts...>> {
/* Implementation */
};

have the Permutation that sorts the types by their alignment as its first template parameter, and the already permuted std::tuple as its second. Hence, we need a MakeBase metafunction, which will allow us to implement Tuple class like

template <typename... Ts> struct Tuple : MakeBase<Ts...> {
  using MakeBase<Ts...>::MakeBase;
};

On a concrete example, we want MakeBase

using TB0 = MakeBase<char, int, char, int, char, double, char>;
using TB1 = 
    TupleBase<ml::ListT<ml::Int<5>, ml::Int<3>, ml::Int<1>, ml::Int<6>, ml::Int<4>,
                        ml::Int<2>, ml::Int<0>>,
              std::tuple<double, int, int, char, char, char, char>>;
static_assert(
      std::is_same_v<TB0, TB1>);

This is achieved by the following sequence:

This sequence is easily translated to CppML:

template <typename ...Ts>
using MakeBase = ml::f<
    ml::ZipWith<
        Param,
        ml::Sort<ml::Map<ml::Unwrap<ml::Get<1, ml::AlignOf<>>>, ml::Greater<>>,
                 ml::Product<ml::Map<ml::Unwrap<ml::Get<0>>>,
                             ml::Map<ml::Unwrap<ml::Get<1>>, ml::F<std::tuple>>,
                             ml::F<TupleBase>>>>,
    ml::Range<>::f<0, sizeof...(Ts)>, ml::ListT<Ts...>>;

We will also need a metafunction that will compute the inverse permutation for an index I, which will allow us to internally redirect users indexing. This is done by locating the index of I in the permutation (using ml::FindIdIf). Assuming access to Permutation indexes Is... (inside the TupleBase), this is done by

template <typename I>
using Index = ml::f<ml::FindIdIf<ml::Partial<ml::IsSame<>, I>>, Is...>;

TupleBase interface

The TupleBase implementation is minimal. It will have:

  • the permuted std::tuple member _tuple
    • from its second template argument
  • f alias which will compute the inverse permutation for an index I
  • a delegate constructor:
    • It will forward the arguments Us... as a tuple to the work construcotr
  • a work constructor:
    • it will initialize the _tuple member by:
      • permuting the arguments of the forwarding tuple into its initializer
        • std::get<Is>(fwd)...
  • the get<I>() friend function, which:
    • will use the f alias to invert I in the Permutation
    • and forward the inverted index to std::get

In code, this looks like this:

template <int... Is, typename... Ts>
struct TupleBase<ml::ListT<ml::Int<Is>...>, std::tuple<Ts...>> {
private:
  std::tuple<Ts...> _tuple;
  template <typename... Us>
  TupleBase(ml::_, std::tuple<Us...> &&fwd) // work constructor
      : _tuple{
            static_cast<ml::f<ml::Get<Is>, Us...> &&>(std::get<Is>(fwd))...} {}

public:
  template <typename... Us>
  TupleBase(Us &&... us) // delegate constructor
      : TupleBase{ml::_{}, std::forward_as_tuple(static_cast<Us &&>(us)...)} {}
  template <typename I> // Compute the inverse index
  using f = ml::f<ml::FindIdIf<ml::Partial<ml::IsSame<>, I>>, ml::Int<Is>...>;
  template <int I, typename... Us>
  friend decltype(auto) get(TupleBase<Us...> &tup);
};

template <int I, typename... Us> decltype(auto) get(TupleBase<Us...> &tup) {
  return std::get<ml::f<TupleBase<Us...>, ml::Int<I>>::value>(tup._tuple);
}

Which concludes the implementation

template <typename... Ts> struct Tuple : MakeBase<Ts...> {
  using MakeBase<Ts...>::MakeBase;
};