All pull requests

Pull requests

• fix: [branch-1.0] revert unsafe partial aggregates after final fallback (#5421)

  area:aggregationarea:expressionsbugcorrectnessrun-spark-4.0-tests
  #6422·andygrove opened Sep 29, 2026Member·

  ·

• fix: keep Spark's cache scan for a relation whose cached plan records observed metrics

  area:scanbug
  #6421·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: [branch-1.0] log partial memory grants at DEBUG and drop the memory usage dump (#6269)

  area:memorybug
  #6419·andygrove opened Sep 29, 2026Member·

  ·

• docs: promote Spark 4.2 from experimental to supported

  documentation
  #6417·andygrove opened Sep 29, 2026Member·

  ·

• fix: add native metrics from every plan a task runs instead of keeping the last

  bug
  #6416·dwsmith1983 opened Sep 29, 2026Contributor·

  ·
  1

• test: run Comet in Spark SQL and Iceberg suites that build their own sessions

  area:Icebergenhancementrun-spark-3.4-testsrun-spark-3.5-testsrun-spark-4.0-testsrun-spark-4.1-teststest
  #6415·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: hash NaN the way Spark does in hash, xxhash64 and native shuffle

  area:aggregationarea:expressionsarea:scanbug
  #6413·andygrove opened Sep 29, 2026Member·

  ·

• fix: report a cached relation's decoded size to the planner, not its compressed size

  bug
  #6412·andygrove opened Sep 29, 2026Member·

  ·
  1

• ci: grant label runs the contents: write the docs job requests

  area:cibuildenhancement
  #6409·andygrove opened Sep 29, 2026Member·

  ·
  1

• docs: correct what the Iceberg split-operator plan moves into AQE

  area:Icebergdocumentation
  #6407·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: match container field ids that the INT96 coercion drops

  area:scanbug
  #6405·dwsmith1983 opened Sep 29, 2026Contributor·

  ·
  1

  4

• deps: use DataFusion main (4a5d580) with Arrow/Parquet 60 and object_store 0.14

  area:expressionsarea:ffiarea:joinsarea:scanarea:shuffledependencies
  #6404·andygrove opened Sep 29, 2026Member·

  ·

• fix: retry a shuffle page allocation that Spark failed after dropping the task's entry

  area:memoryarea:shufflebug
  #6403·dwsmith1983 opened Sep 29, 2026Contributor·

  ·
  1

• feat: write the executor memory usage log to the Spark event log

  enhancement
  #6401·comphead opened Sep 29, 2026Contributor·

  ·
  3

• refactor: move native -0.0 and NaN handling into one float_semantics module

  area:aggregationarea:expressionsarea:scanenhancement
  #6400·andygrove opened Sep 29, 2026Member·Queued
  5

• build: bump the Spark 4.2 test diff to Comet 1.2.0-SNAPSHOT

  buildenhancementrun-spark-4.2-tests
  #6398·andygrove opened Sep 29, 2026Member·Queued
  1

  2

• test: cover IF branch selection and short-circuiting

  area:expressionsenhancementperformance
  #6397·rich7420 opened Sep 29, 2026Contributor·

  ·
  3

• perf: prune unused filter output columns

  enhancementperformance
  #6396·rich7420 opened Sep 29, 2026Contributor·

  ·

• ci: keep a macOS native build cache on main

  buildenhancementrun-macos-tests
  #6395·andygrove opened Sep 29, 2026Member·

  ·
  1

  1

• ci: rebalance the Spark SQL test shards

  buildenhancementrun-spark-4.1-tests
  #6393·andygrove opened Sep 29, 2026Member·Queued
  1

  1

• ci: stop main's cargo build cache from being evicted

  area:Icebergbuildenhancement
  #6392·andygrove opened Sep 29, 2026Member·Queued
  1

  2

• ci: raise the merge queue's build concurrency from 2 to 4

  buildenhancement
  #6391·andygrove opened Sep 29, 2026Member·Queued
  1

  2

• perf: keep the sort-merge join when the forced hash join's build side is too large

  area:Icebergarea:joinsenhancementperformance
  #6384·dwsmith1983 opened Sep 29, 2026Contributor·

  ·

• fix: report row contents to native write statistics trackers

  area:writerbug
  #6383·LinSimon-901101 opened Sep 29, 2026Contributor·

  ·
  1

  1

• fix: restore localReads assertion in AdaptiveQueryExecSuite diffs

  bug
  #6382·mizulun opened Sep 29, 2026Contributor·

  ·
  1