[platinumhamburg]

Extend tests and docs to cover new functions and array/map utilities.

Purpose

Linked issue: close #2387

Brief change log

Tests

API and Format

Documentation

[platinumhamburg]

Hi @fresh-borzoni, would you have time to review this PR? It adds new aggregate functions to the aggregation merge engine. Given your work on last_n, your feedback on the function semantics, edge cases, and fit with the existing aggregation framework would be especially helpful. Thanks!

[polyzos]

Thanks @platinumhamburg.

I've added a few comments and suggestions. 👍

[fresh-borzoni]

Thanks @platinumhamburg, looked through. Seema @polyzos already did a good job with the review :)

Couple of thoughts:
FieldCollectAgg, FieldMergeMapAgg and both factories carry the "based on Apache Paimon" header, but they aren't in the Apache Paimon list in the root LICENSE. The other Paimon-derived aggregators are all listed there. Shall we add?

On the binary concat: #4923 adds BinaryArraySplicer, which does that for last_n. It copies a range of a binary array as bytes and shifts the offsets of the variable-length data. At 1000 elements a last_n merge went from 10.9 µs to 0.75 µs for BIGINT, and from 18.3 µs to 5.45 µs for STRING on my laptop.

Non-distinct collect is the same case:

if (spliceable && BinaryArraySplicer.canSplice(accumulatorArray, inputArray)) {
    return new BinaryArraySplicer(elementType)
            .append((BinaryArray) accumulatorArray, 0, accumulatorArray.size())
            .append((BinaryArray) inputArray, 0, inputArray.size())
            .build();
}

with spliceable = BinaryArraySplicer.supports(elementType), as last_n does it.

Can you take a look and share your thoughts? I'm dubious since merge isn't the part that dominates, so eager to align on this.