Skip to content

[C++][Compute] Support decimal32 and decimal64 in more compute kernels #55349

Open

Description

@555aaditya

Describe the enhancement requested

Many compute functions accept decimal128 and decimal256 but raise ArrowNotImplementedError for decimal32 and decimal64. I checked each function on main (27.0.0.dev) with decimal32(5, 1), decimal64(5, 1) and decimal128(5, 1) arrays from pyarrow. The following work for decimal128 and fail for decimal32 and decimal64:

  • Aggregations: min_max, min, max, index, mode, sum, mean, product
  • Grouped aggregations: hash_min_max, hash_sum, hash_list
  • Hashing: unique, value_counts, dictionary_encode
  • Set lookup: is_in, index_in
  • Sorting: sort_indices, rank, select_k_unstable
  • Arithmetic and rounding: abs, negate, sign, round, ceil, floor
  • Selection: choose

Already working for all widths: add, equal, if_else, coalesce, case_when, take, drop_null, count_distinct, hash_distinct. The replace kernels (replace_with_mask, fill_null_forward, fill_null_backward) are covered by #55256 / #55257.

For many of these, the kernel code is already templated over the decimal type and only the registration lists decimal128 and decimal256. For example, hash_min_max registers decimal128(1, 1), decimal256(1, 1) in hash_aggregate.cc, and the scalar min_max registration in aggregate_basic.cc does the same. Those should mostly need the extra registrations plus tests. Others, like the sort comparators and rounding, may need more work.

sum, mean and product need a decision first. Today sum of decimal128(5, 1) returns decimal128(38, 1), widening to the maximum precision of the same type (#35166), while mean and product keep the input type. For decimal32 the maximum precision is only 9, so following the same rule would overflow much sooner. Should sum of decimal32 / decimal64 stay within the input type, as decimal32(9, s) / decimal64(18, s), or promote to decimal128(38, s)?

I'd like to work on this and split it into small PRs, one per kernel family, starting with min_max / hash_min_max. I'll leave sum / mean / product until the output type is agreed here.

Component(s)

C++

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions