feat(arrow-select): concat kernel will merge dictionary values for list of dictionaries

[rluvaton]

Which issue does this PR close?

Closes #6888

What changes are included in this PR?

1. Updated concat kernel to merge dictionary values for list of dictionary
2. Created merge function in BufferOffset for merging multiple BuffersOffset into one (which needed for the concat merge)

Are there any user-facing changes?

Yes, the merge function in the BufferOffset struct

[tustvold]

It would significantly reduce the codegen to instead split concatenating the dictionary values, from re-encoding the offsets. As an added bonus this could also be done recursively.

[rluvaton]

updated, please let me know it that was what you meant, I think it will be slower this way, no?

[tustvold]

I've taken a quick look and left some suggestions on how to simplify this, in particular separating the concatenation of the list values and the list offsets will result in less code, require less codegen, and likely perform better.

i.e. something like (not tested)

DataType::List(f) => {
    let values: Vec<_> = arrays.iter().map(|x| x.as_list::<i32>().values()).collect();
    let new_values = concat(&values)?;
    let offsets = OffsetBuffer::from_lengths(
        arrays.iter().flat_map(|x| x.as_list::<i32>().offsets()
             .windows(2).map(|w| w[1].as_usize() - w[0].as_usize()))
    );
    let nulls = ...
    ListArray::new(f, offsets, new_values, nulls)
}

[tustvold]

This is looking very promising, just a small comment r.e. avoiding ArrayData

I'll try to find some time for a more thorough review in the next few days if nobody else gets there first