Skip to content

fix(cpp): complete and adopt bounded lru cache - #967

Open
Rayan-and-beyond wants to merge 5 commits into
apache:developfrom
Rayan-and-beyond:fix/943-cpp-lru-cache
Open

Rayan-and-beyond wants to merge 5 commits into
apache:developfrom
Rayan-and-beyond:fix/943-cpp-lru-cache

Conversation

@Rayan-and-beyond

@Rayan-and-beyond Rayan-and-beyond commented Sep 17, 2026 •

Copy link
Copy Markdown

closes #943

this finishes the existing c++ lru cache contract and puts it on the reader device-node path.

  • fixes the broken lookup apis and adds explicit non-copying pointer/reference access
  • documents caller-side synchronization, reference lifetime, elasticity, and unbounded mode
  • removes the stale metadataquerier cache wiring that used std::mutex as the map type
  • replaces the reader's unbounded shared-arena device cache with a 64-entry lru where each entry owns its arena, so eviction can actually reclaim metadata pages
  • adds focused lru behavior coverage plus a reader regression that fills past capacity, reloads an evicted device, and checks reader metadata memory stays flat

validation:

  • 6/6 focused lrucache gtests pass
  • device-node bounded/reload regression passes
  • full TsFile_Test target builds successfully with -j1 on the constrained vm
  • changed production objects compile and libtsfile links with optional codecs/antlr disabled
  • git diff --check clean

@ColinLeeo
ColinLeeo requested a balanced review from Copilot October 10, 2026 01:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Comment thread cpp/src/common/cache/lru_cache.h Outdated
* Legacy overload retained for source compatibility. Despite its historic
* name it copies; new code should use the pointer overload or getPtr().
*/
bool tryGetRef(const Key& k, Value& vOut) { return tryGetCopy(k, vOut); }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a concrete downstream compatibility requirement for retaining these aliases? I couldn't find any in-repository callers of tryGet(), get(), or the copying tryGetRef(Key, Value&); the production reader already uses tryGetCopy(). Unless downstream compatibility is required, could we remove the compatibility-only aliases, especially this overload? Having tryGetRef either copy or return a cached pointer depending on the output parameter type makes the API easy to misuse.

}
ASSERT_EQ(io_reader.TEST_device_node_cache_size(), capacity);
const int64_t memory_at_capacity = io_reader.TEST_reader_memory_bytes();
ASSERT_GT(memory_at_capacity, 0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we guard the memory snapshot and related assertions with #ifdef ENABLE_MEM_STAT, while keeping the cache-capacity and eviction/reload checks enabled? ENABLE_MEM_STAT=OFF is a supported CMake configuration, and ModStat::get_stat() returns 0 in that build, so this unconditional ASSERT_GT(memory_at_capacity, 0) fails. I reproduced this failure in TsFileReaderTest.DeviceNodeCacheIsBoundedAndReloadsEvictedDevices with memory statistics disabled. The later memory-equality assertions also become vacuous (0 == 0) in this configuration.

@ColinLeeo

Copy link
Copy Markdown
Contributor

Potential follow-up (can be separate from this PR): cache resolved per-series metadata so repeated queries can reuse more than the device's top index node.

A device-node cache hit skips reading/deserializing that node, but load_timeseries_index_for_ssi() still calls load_measurement_index_entry() and then reads/parses the selected timeseries metadata. The measurement lookup reaches MetaIndexNode::binary_search_children(), whose clone(pa_) copies the measurement name into the cached node's arena.

This also leaves a hot-entry memory-growth case: after warming one device, I repeated alloc_ssi() / revert_ssi() 10,000 times, resetting the query arena after each iteration. The cache remained at one entry, while reader memory accounting grew from 1,072 to 84,688 bytes. This allocation behavior predates this PR. Bounding entry count does not prevent an individual retained arena from growing with repeated lookups. The lookup-result copy should use a query-owned arena, or another ownership-safe result representation.

For a subsequent optimization, could we consider a bounded, reader-owned (device, measurement) cache of parsed metadata with an owning handle, building on PreparedSeries / init_prepared()? A hit could bypass measurement-index traversal and repeated metadata I/O/deserialization, while each query keeps its own filter, scan cursor, and decoding buffers. Entries should be loaded lazily, have a memory budget, and share aligned time metadata where appropriate. The miss path would still need the allocation-lifetime fix above.

A repeated-query benchmark tracking metadata lookup/read/parse counts and retained memory would make both the performance benefit and the memory behavior measurable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement][C++] Complete and adopt the existing LRU cache infrastructure

3 participants