Skip to content

Antalya 26.3 Backport of #98162 - Improve iceberg writes for unity catalog - #2714

Open
xieandrew wants to merge 1 commit into
antalya-26.3from
backports/antalya-26.3/98162
Open

xieandrew wants to merge 1 commit into
antalya-26.3from
backports/antalya-26.3/98162

Conversation

@xieandrew

Copy link
Copy Markdown
Collaborator

Closes #2182.

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Improve iceberg write compatibility for Unity and other REST catalogs. Fixes writing a corrupted manifest list when the existing manifest list written externally has a key_metadata field.

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Unit tests
  • Performance tests
  • Aarch64 tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • CAS (content-addressed storage; Antalya only)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • OAuth (5m)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Workflow [PR], commit [4af4c5a]

@xieandrew

Copy link
Copy Markdown
Collaborator Author

@blau-ai

@blau-ai

blau-ai commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

CI triage for 11d1a51

Verdict: 1 failing check, 0 caused by this PR. Stateless tests (amd_debug, parallel) fails on 5 test timeouts. They are on dataset and network-heavy tests that have nothing to do with Iceberg, and the same tests are timing out right now on unrelated antalya-26.3 backports. All other checks marked "skipping" in the GitHub UI were dropped because of this one failure (Dropped due to previous failure [Stateless tests (amd_debug, parallel)]) or excluded by the arm/tsan/msan tag filter. They didn't run and didn't fail.

Check Result Classification
Stateless tests (amd_debug, parallel) 5 tests timed out (each one failed in both the main run and the retry, giving the "Failed: 10" count) Infra / pre-existing on base, not PR-related
Build (amd_debug/asan/ubsan/binary/release, arm_release), Fast test, Quick functional, Stateless amd_asan 1–4/4, Unit tests asan/ubsan, Docker images ✅ pass —
Integration, Stress, AST fuzzer, BuzzHouse, other stateless suites ⏭ dropped after the failure above Not run, so they say nothing about this PR yet

Stateless tests (amd_debug, parallel)

Failing tests (report):
04033_tpc_ds_q09, 04033_tpc_ds_q14, 03634_autopr_output_bytes_estimation, 00152_insert_different_granularity, 00154_avro. All hit the 600 s test timeout or the client's 300 s receive timeout. In the first pass, 04033_tpc_ds_q24/q34/q67/q23 and 03634_autopr_input_bytes_estimation also timed out but passed on retry.

Why it isn't the PR: the hung-query check at the end of job.log shows queries such as SELECT * FROM test.hits WHERE URL LIKE '%google%' ... stuck inside DNS resolution during a TLS connect:

clickhouse::__poll
clickhouse::__GI___res_context_send
clickhouse::_nss_dns_gethostbyname4_r
clickhouse::getaddrinfo
Poco::Net::DNS::hostByName(...)                      base/poco/Net/src/DNS.cpp:80
Poco::Net::SecureSocketImpl::isLocalHost(...)
Poco::Net::SecureSocketImpl::verifyPeerCertificateImpl(...)
...
DB::EndpointConnectionPool<Poco::Net::HTTPSClientSession>::prepareNewConnection(...)
  • The tests that timed out all read remote datasets over HTTPS (test.hits, TPC-DS tables, avro and granularity fixtures). The runner's DNS stalled and these queries waited until the timeout. That is an infra/network symptom.
  • The PR only touches Iceberg write paths, RestCatalog.cpp, and a single blank line in IKeeper.h. None of these tests exercise those paths.
  • The same failure shows up on unrelated sibling PRs on the same base today:
  • The 2 BROKEN tests (03441_deltalake_*) are expected-broken and don't count toward the failure.

Next step: re-run the failed jobs. No code change is needed on this PR. If it times out again, the problem is the runner's DNS/egress on the antalya-26.3 stateless runners and needs someone with CI access, not a change to this backport. I found no open tracking issue for it, so filing one may be worth it, given that at least three PRs are affected.

Caveat: because of this failure, the integration tests (where the Iceberg/REST-catalog coverage lives), the stress tests and the fuzzers have not run on this PR. Don't take the green checks as proof the backport is correct until a re-run gets past this stage.

Evidence: praktika result_pr.json and job.log for 11d1a51, plus result_pr.json for #2712 (ede57c1) and #2697 (3c607d2).

Comment thread src/Common/ZooKeeper/IKeeper.h Resolved
Signed-off-by: Andrew Xie <dev@xie.is>
@xieandrew
xieandrew force-pushed the backports/antalya-26.3/98162 branch from 11d1a51 to 4af4c5a Compare October 30, 2026 14:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants