fix(store): size the RocksDB memory budget from available memory

[byteayan]

Purpose of the PR

• close [Bug] Store ships rocksdb.total_memory_size of 32 GB, ignores the container memory limit, and Docker users cannot override it #3254

Both shipped application-pd.yml files set rocksdb.total_memory_size: 32000000000, and the pd profile is always included, so the Runtime.maxMemory() fallback in AppConfig.init() never ran. RaftRocksdbOptions splits the value into the WriteBufferManager cache (66%) and the block cache. Both are native LRUCaches, so neither -Xmx nor the container limit bounds them, and a separate 1 GiB raft log cache sits on top. In the run in the issue all three Stores were OOM-killed at a 4 GiB limit once each held about 1 GB of RocksDB data.

Docker users had no clean way to change it. docker-entrypoint.sh builds SPRING_APPLICATION_JSON from five HG_STORE_* keys, and none of them is a rocksdb.* key. Bind-mounting a config file or passing -Drocksdb.total_memory_size=<bytes> in JAVA_OPTS both work, but neither is documented.

As the issue points out, dropping the line from the yml alone would hand RocksDB a budget equal to the heap, which is a second heap-sized allocation on top of the first.

Main Changes

• New RocksdbMemoryBudget in hg-store-core/.../options. An unset, blank or 0 value means auto: half of (physical memory − max heap − MaxDirectMemorySize − 1.5 GiB reserve), clamped to 256 MiB to 32 GB. The reserve covers the raft log cache plus stacks, metaspace and other native memory. Physical memory comes from OperatingSystemMXBean.getTotalMemorySize(), which on JDK 17 reports the cgroup v1/v2 limit inside a container. Older JDK 17 updates may not see a limit set only on a parent cgroup, so the budget also reads the cgroup itself (v2: memory.max from the process's cgroup up to the mount root; v1: hierarchical_memory_limit) and uses the lower value. The membership path is rebased onto the mount's root from /proc/self/mountinfo, so a cgroupfs mounted from a subtree such as /kubepods is walked from the right directory. Every mount that contains the process's cgroup is checked and the lowest limit wins, so mount order doesn't matter. Inside a private cgroup namespace a limit above the container can't be seen, and the deployment guide says to pin the value there. An unset MaxDirectMemorySize is counted as the heap size capped at 1 GiB. The JVM would allow a full heap of direct memory, but the Store's gRPC/Netty buffers don't grow with the heap, and counting a raised heap twice left large bare-metal hosts on the floor. If physical memory can't be read, the previous heap-size default applies.
  • The issue's setup (4 GiB limit, -Xmx1g, -XX:MaxDirectMemorySize=512m) gets 512 MiB, and an 8 GiB limit gets 2.5 GiB. I checked the 4 GiB case with resolve() inside eclipse-temurin:17-jre-jammy under a 4 GiB Docker limit: 536870912 bytes. With -Xmx2g and no direct-memory cap it lands on the 256 MiB floor. The 32 GB cap means no host gets a bigger budget than before. On bare metal the default launcher (2 GiB heap) on a 16 GiB host gets 5.75 GiB; a raised heap gets less (2.75 GiB with -Xmx8g on 16 GiB, 12.75 GiB with -Xmx36g on 64 GiB), so a large node that relied on the fixed 32 GB should pin the value after upgrading.
  • A positive value is used unchanged, with one exception: values under 1 MiB are refused. The write buffer manager gets 2/3 of the budget, so 1 would give a 0-byte manager, which RocksDB treats as no memtable limit at all. 4GB and other non-numeric values now fail at startup with "must be a number of bytes", where before they failed with a bare NumberFormatException. Nothing shipped (configs, compose files, Helm, CI) sets a value under 1 MiB.
  • Startup logs the chosen value and its inputs. It warns when the 256 MiB floor applies (heap plus direct memory already fill the limit), when an explicit value is under 256 MiB, and when an explicit value exceeds the memory left after the heap, direct memory and the reserve. That last case is what an upgraded install with an old mounted application-pd.yml will hit; the Store also warns when it sees exactly 32000000000, the old shipped value, and the deployment guide has an upgrade step (set it to 0, remove it, or set HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE).
• AppConfig.init() uses it. While there I fixed the raft disruptor sizing for budgets under 1 GB: log(0) rounded to a buffer size of 0. It now gives 32. The formula moved to RocksdbMemoryBudget.disruptorBufferSize() so it has a test. The shipped application.yml pins disruptorBufferSize: 1024, so this only matters when that key is absent.
• Both application-pd.yml files ship total_memory_size: 0, which also removes the [Bug] Store ships rocksdb.total_memory_size of 32 GB, ignores the container memory limit, and Docker users cannot override it #3254 TODO from chore: add TODO breadcrumbs for the Helm chart's server workarounds #3260. The test resources keep their explicit values.
• docker-entrypoint.sh accepts HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE. Values must be digits only; anything else exits with code 2. It is added to SPRING_APPLICATION_JSON only when set, so an unset variable leaves the config file or auto-sizing in charge. When set it outranks both the config file and -Drocksdb.total_memory_size (checked against Spring Boot 2.5's property order).
• helm/hugegraph/values-cluster.yaml: the store comment said container limits don't configure the RocksDB budget. It now says how the budget is derived (2.5 GiB for that preset) and how to pin it with store.extraEnv.

In a small container the write buffer manager now flushes memtables much earlier than before, which means more small L0 files and lower write throughput in exchange for staying inside the limit. Both caps are soft, so RocksDB can still go over briefly under heavy writes. I have not measured either effect. Setting the value explicitly restores the old behaviour.

Verifying these changes

• Need tests and can be verified as follows:
  • New RocksdbMemoryBudgetTest (21 tests, added to CoreSuiteTest, which the store-core-test profile runs). It covers:
    • explicit values kept unchanged, and values under 1 MiB refused;
    • unset, blank, 0 and 00 going to auto;
    • refusal of 4GB, -1, 1.5e9 and a value past Long.MAX_VALUE;
    • the 4 GiB and 8 GiB examples above, both clamp bounds, and the unknown-physical fallback;
    • the disruptor size for 512 MiB, 1 GB and 32 GB;
    • the derived write_buffer_size staying at or above RocksDB's 1 MiB minimum, checked through the same HugeConfig validation as startup;
    • unset direct memory counted as at most 1 GiB, and the three bare-metal sizes above;
    • cgroup limits read from fake v1 and v2 trees: a limit set only on the pod, the container's own lower limit, the namespace-root view, v1 and v2 subtree mounts rebased through mountinfo (including an escaped mount point), several mounts in different orders, unlimited values, and an unreadable /proc/self/cgroup.
  • New test-store-docker-entrypoint.sh (9 cases), added to the store job in pd-store-ci.yml. It checks that the rocksdb key is absent when the variable is unset or empty, that bytes and 0 pass through, that app.data-path is still set, and that 4GB, -1, 1.5e9, 1 2 and 1"} are refused. It clears the variable from the caller's environment first, so "unset" really is unset.
  • mvn test -pl hugegraph-store/hg-store-test -P store-core-test on JDK 17: 126 tests, 0 failures. python3 -m unittest discover -s .github/scripts -p 'test_*.py' passes. helm unittest helm/hugegraph: 294 tests pass. mvn editorconfig:check and git diff --check are clean.
  • Not tested: a full Store started in a memory-limited container, and the load from the issue.
  • test-store-docker-entrypoint.sh is mapped to the store suite in .github/scripts/ci-policy.py, so a PR that changes only that script still runs it; the selector test covers the mapping.

Does this PR potentially affect the following parts?

• Dependencies (add/update license info & regenerate_known_dependencies.sh)
• Modify configurations
• The public API
• Other affects (typed here)
• Nope

Documentation Status

• Doc - TODO: required documentation is pending; complete it before merging.
• Doc - Done: documentation is included here or linked below.
• Doc - No Need: no user-visible documentation is affected.

Documentation files in this PR or paired hugegraph-doc PR:

• hugegraph-store/docs/deployment-guide.md: the auto-sizing rule and its inputs, the 1 MiB minimum, the new env var and its precedence in the compose and Kubernetes examples, and the RocksDB block (total_memory_size: 0; the nonexistent block_cache_size key replaced with write_buffer_ratio).
• hugegraph-store/README.md: the env var in the docker run section.
• Paired docs(store): document the auto-sized RocksDB memory budget hugegraph-doc#535 (merge together with this PR): content/{en,cn}/docs/quickstart/hugegraph/hugegraph-hstore.md (the default, the auto-sizing rule and the env var) and content/{en,cn}/docs/guides/hugegraph-docker-cluster.md (the new HG_STORE_* row). The hstore page's "checked against Server master commit" note should move to this PR's merge commit once both land.

Conceptual overview

Concept illustration: derive the RocksDB budget from memory remaining after heap, direct memory and reserve, with clamp bounds and a 1 MiB derived write-buffer floor. Cache budgets remain soft.

[codecov]

Codecov Report

❌ Patch coverage is 81.21212% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 54.62%. Comparing base (92af2f8) to head (535c2d0).
⚠️ Report is 26 commits behind head on master.

Files with missing lines	Patch %	Lines
...e/hugegraph/store/options/RocksdbMemoryBudget.java	81.21%	14 Missing and 17 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3355      +/-   ##
============================================
+ Coverage     54.31%   54.62%   +0.31%     
- Complexity     2250     2350     +100     
============================================
  Files           770      775       +5     
  Lines         67952    68655     +703     
  Branches       9213     9599     +386     
============================================
+ Hits          36905    37502     +597     
- Misses        27470    27605     +135     
+ Partials       3577     3548      -29     

Flag	Coverage Δ
commons	96.37% <ø> (?)
hstore	35.45% <ø> (?)
pd	49.24% <ø> (?)
server_memory	54.15% <ø> (?)
server_rocksdb	42.08% <ø> (?)
store	32.54% <81.21%> (?)

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bitflicker64]

Blocking: yes. Summary: Score 7/10. The container case from #3254 is handled, and the earlier write_buffer_size thread is fixed by writeBufferSize() at 1927fbe. One blocking problem: auto-sizing counts an unset -XX:MaxDirectMemorySize as a second full heap, and the tarball launcher never sets it, so bare-metal installs with a large heap drop from the shipped 32000000000 bytes to the 256 MiB floor on upgrade. Evidence: the head's RocksdbMemoryBudget.java compiled standalone and resolve(null) run on a 16 GiB host with JDK 17.0.20 returned 268435456 for -Xmx8g and 3221225472 for -Xmx8g -XX:MaxDirectMemorySize=512m; start-hugegraph-store.sh, util.sh calc_xmx, the Store Dockerfile, RaftRocksdbOptions and the Helm store template were read at e691489. Only HugeGraph-Server CI and check-license are listed for this head, both queued, so the new Store tests were not seen passing in CI.

[bitflicker64]

Blocking: no. Summary: Score 9/10. The earlier threads are fixed at this head: the derived write_buffer_size is clamped to 1 MiB, and an unset -XX:MaxDirectMemorySize now counts as at most 1 GiB, so a large-heap bare-metal Store no longer drops to the 256 MiB floor. The auto budget fits the #3254 container setup, and an explicit value still overrides it. Evidence: read RocksdbMemoryBudget, AppConfig.init(), RaftRocksdbOptions, RocksDBOptions, the Store launcher and calc_xmx, the Dockerfile, the entrypoint and both Helm values files at 84e8e27; worked through the arithmetic for 4 GiB/-Xmx1g/512m (512 MiB), the Helm cluster preset (2.5 GiB) and 16 GiB/-Xmx8g (2.75 GiB); ran test-store-docker-entrypoint.sh locally (9 passed); CI at this head ran CoreSuiteTest with the new tests (52 tests, 0 failures) and the entrypoint step, and cluster, docker and helm jobs pass. Only codecov/project fails, because the base is 19 commits behind.

[imbajin]

Blocking: yes. Summary: Auto-sizing can still use host memory on supported older JDK 17 builds that miss parent-only cgroup limits. Evidence: exact-head source and cgroup behavior review; see inline comments for the mounted-config upgrade and Store test-selection gaps.

[imbajin]

[P2] Consider compatible readable cgroup mount views instead of returning the first match.

The original mount-root translation finding is fixed at this revision: independent Java 21 synthetic-file checks using the exact resolver, with only Lombok logging stubbed, now find the 4 GiB limit and produce the 512 MiB budget. Fourteen bounded checks pass, but three multi-mount discrepancies remain, confirmed by two independent reviews.

The resolver returns the first matching cgroup mount without considering later readable views. With membership /kubepods/pod1/ctr, an unrelated first mount or a leaf-only first mount followed by a full readable hierarchy can hide a visible 4 GiB ancestor. With a simulated 64 GiB host, 1 GiB heap and 512 MiB direct cap, the budget is then 32,000,000,000 bytes instead of 536,870,912. Reversing mount order succeeds; this reproduces for both v2 and v1.

Please consider compatible readable mount views and their ancestor limits, with order-invariance regressions. Selecting only the most-specific mount can still omit visible ancestors. Multiple mounts are a plausible layout: see the OpenJDK duplicate-mount fix and runc issue #2404.

Evidence boundary: synthetic file fixtures, not real mounts, full Store startup, performance or load measurement. A correct JVM container limit can mask the overbudget path; this is conditional, not a claim that every container is affected. No new numerical score is assigned.

[byteayan]

@imbajin thanks, fixed in 535c2d0. The resolver now reads every cgroup2 mount, or every v1 mount carrying memory, from mountinfo instead of the first one. It skips mounts whose root doesn't contain the process's cgroup, walks each remaining one from its rebased path, and keeps the lowest limit. A leaf-only or unrelated mount listed first can no longer hide a fuller view listed later. New tests run three mount orders for v2 (unrelated, leaf-only and full views) and two for v1 (with a cpu-only mount that's ignored). Each finds the 4 GiB pod limit and, with a 64 GiB host, 1 GiB heap and 512 MiB direct cap, gives 512 MiB. Store core tests: 126 passed.

[bitflicker64]

Blocking: no. Score: 10/10.

Since 33e53ee, 535c2d0 reads every cgroup2 mount, or every v1 mount carrying memory, from mountinfo. It skips mounts whose root does not contain the process cgroup and keeps the lowest limit across the rest. I reran my earlier synthetic case against this head. A leaf-only cgroup2 mount listed before the full hierarchy now gives 4294967296 instead of -1, and the hybrid v2 plus v1 layout still resolves to the v1 limit. I also tried a private namespace with 0::/, a 0::/../sib path, a /.. mount root, the v1 9223372036854771712 sentinel, a nested child cgroup and an escaped mount point. Each gave the expected limit or fell back to the mount root.

Evidence: on a cgroup v2 host (JDK 17.0.20) I ran resolve() inside a transient slice with MemoryMax=4G and an unlimited service, using -Xmx1g -XX:MaxDirectMemorySize=512m. The JVM reported 16473444352, resolve(null) and resolve("0") returned 536870912, and resolve("32000000000") was kept. Locally at 535c2d0, RocksdbMemoryBudgetTest passed 21 tests and CoreSuiteTest passed 62 with 0 failures. CI at this head is still running.

Earlier threads:

• @imbajin, cgroup mount first match: fixed in 535c2d0.
• @imbajin, subtree mount root not rebased: fixed in 33e53ee, still holds.
• @imbajin, entrypoint test not picked by CI policy: fixed in 3fb0da0.
• @imbajin, parent-only limit missed by older JDK 17, and the old application-pd.yml keeping 32 GB: fixed and documented in 03a4d43, with a WARN on 32000000000.
• @imbajin, derived write_buffer_size below 1 MiB: fixed in 1927fbe.
• Mine, unset MaxDirectMemorySize counted as a full heap: fixed in 84e8e27, still holds.

Not covered: a full Store under write load in a memory-limited container.

[imbajin]

Rechecked the multi-mount finding against the updated source, reviewed 2026-10-16. The prior finding is fixed: all applicable mount views are considered, unrelated roots are excluded, and the minimum visible limit wins rather than the first mount. Two independent source reviews found no new blocker in this repair.

I independently replayed the three earlier counterexamples and added reordered v1/v2 mount and containment controls: 40 bounded Java 21 synthetic-file assertions passed. The executable copy retained the exact resolver/sizing code and replaced only Lombok logging with a no-op logger. This was not a full Store/container load test or a performance measurement.

Provisional assessment: 8.5/10, based on the earlier full review plus this verified repair, not a fresh full Store acceptance. Current-head CI and the pending paired documentation still need completion; the author's 126 Store tests remain author-reported rather than my own execution.

[byteayan]

@imbajin thanks for the thorough rounds on this one. Everything you raised is now addressed at 535c2d0: the ancestor and subtree cgroup limits, the multi-mount order, the upgrade step for an old application-pd.yml, and the CONTRACT_SUITES mapping. All 19 checks are green on that commit, and the paired doc PR is apache/hugegraph-doc#535, linked above with Doc - Done. Could you take a final look when you have a moment?

[imbajin]

Blocking: yes. Summary: Automatic sizing can miss the process cgroup limit in a supported cgroup-namespace mount layout and can reserve too little for uncapped direct memory. Either condition can leave the Store with a RocksDB budget above its available container memory. Evidence: exact-head source review of RocksdbMemoryBudget, the default Store launcher and gRPC configuration; see inline comments for the two cases.

[imbajin]

⚠️ Important: This fallback misses the process cgroup in the namespace layout documented by cgroup_namespaces(7): /proc/self/cgroup reports 0::/ while a cgroup2 mount created before the namespace reports root /..; contains rejects that mount, then this code reads only the default mount root. If the namespace root is a child with a tighter memory.max than its parent, the process limit is missed and auto-sizing can allocate an oversized RocksDB budget. Please resolve this mount layout to the process cgroup, or fail safe when it cannot be matched, and add a fixture with an unlimited parent and limited child.

[imbajin]

⚠️ Important: When MaxDirectMemorySize is unset, the JVM permits NIO direct-buffer memory up to Runtime.maxMemory(), but this calculation reserves at most 1 GiB. The default Store launcher does not set a matching direct-memory cap, and the shipped gRPC configuration permits 1000 MB inbound requests, so concurrent direct-buffer use can exceed the amount reserved while the auto-sized RocksDB caches remain available. Please enforce the assumed cap in the default launch configuration or budget against the actual JVM allowance; otherwise direct-buffer-heavy workloads can still exceed the cgroup limit. Evidence: OpenJDK VM.java and hugegraph-store/hg-store-node/src/main/resources/application.yml.

[imbajin]

Updated independent cross-review: 8.0/10, pending focused handling of the namespace mapping case noted in #3355 (comment). I independently reproduced the missing-child-limit case using synthetic proc/mountinfo/cgroup files and the exact resolver source, compiled with only Lombok logging replaced by a no-op logger. Eight new bounded Java 21 assertions passed, and all 40 earlier controls still pass. An inherited mount rooted at /.. with process membership / falls back to the parent mount root and misses the limited child. If the JVM also reports host memory, auto-sizing can overestimate the budget; an accurate JVM-reported limit masks the failure. This is a conditional gap in the new protection, not a demonstrated production OOM or a claim that it regresses the previous fixed default. Please distinguish ambiguous mapping from an unlimited hierarchy and provide narrow safe handling or an explicit-budget diagnostic with a regression fixture. Normalizing /.. alone cannot recover the hidden child path.

The direct-memory finding #3355 (comment) also identifies a real accounting assumption: an independent Java 21 probe with -Xmx2g and unset MaxDirectMemorySize reported a 2 GiB allowance, while this code reserves 1 GiB. This heuristic is explicit in the source. No buffers or Store workload were allocated, so no direct-buffer workload failure or OOM was demonstrated. Counting the default heap-sized allowance is the conservative option; imposing a new runtime direct-memory cap would need separate validation.

Evidence boundary: two independent source reviews, synthetic resolver tests and a lightweight JVM allowance probe; no live namespace, JDK 17, full Store, OOM or performance acceptance. Current review source:

hugegraph/hugegraph-store/hg-store-core/src/main/java/org/apache/hugegraph/store/options/RocksdbMemoryBudget.java

Lines 220 to 237 in 535c2d0

	}
	boolean matched = false;
	for (String[] mount : v2 ? v2Mounts : v1Mounts) {
	if (!contains(mount[0], parts[2])) {
	// A mount of another subtree says nothing about this cgroup
	continue;
	}
	matched = true;
	Path point = Paths.get(mount[1]);
	String relative = rebase(parts[2], mount[0]);
	limit = lowest(limit, v2 ? cgroupV2Limit(point, relative) :
	cgroupV1Limit(point, relative));
	}
	if (!matched) {
	String relative = rebase(parts[2], "/");
	limit = lowest(limit, v2 ? cgroupV2Limit(defaultRoot, relative) :
	cgroupV1Limit(defaultRoot.resolve("memory"), relative));
	}

. Prior 8.5 assessment is superseded by this new evidence on the same revision.

CI status at publication (2026-10-17 01:36 UTC): 19 exact-head checks passed, 2 were skipped, and 0 remain running/queued. Both Codecov checks passed. CI source: https://github.com/apache/hugegraph/actions/runs/38938886440. These reported CI results do not establish coverage of the synthetic namespace-mapping case and are separate from our bounded independent controls.