fix(cpp): keep aligned value columns row-aligned when a record omits measurements

[ColinLeeo]

Bug

For an aligned (tree-model) device, every value column has to consume exactly one row per time-column row. The C++ writer only advanced the columns that the incoming row actually carried:

• TsFileWriter::write_record_aligned() built its writer list from record.points_, while the time chunk was written for every record;
• TsFileWriter::write_tablet_aligned() only advanced the tablet's own columns.

So a measurement that was missing from a row left its column one row short. Its not-null bitmap then started at the wrong row index, and on read the values were paired with the earliest timestamps of the page while the tail rows came back as NULL; the statistics that RestorableTsFileIOWriter recomputes after a crash described those shifted rows too (so time-filter pruning could drop or return the wrong range).

Java does not have this problem — AlignedChunkGroupWriterImpl#write fills the measurements a row does not carry:

for (Map.Entry<String, ValueChunkWriter> entry : valueChunkWriterMap.entrySet())
  if (!existingMeasurements.contains(entry.getKey())) emptyValueChunkWriters.add(entry.getValue());
...
if (!emptyValueChunkWriters.isEmpty()) writeEmptyDataInOneRow(emptyValueChunkWriters);

Reproducer (same data, same C++ reader; Java-written file on the left, C++-written file on the right) — a device with s1/s2 where even rows only write s1 and odd rows only write s2:

Java                                   C++
time  d1.s1  d1.s2                     time  d1.s1  d1.s2
0     100                              0     100    201
1            201                       1     102    203
2     102                              2     104    205
3            203                       3     106    207
4     104                              4     108    209
5            205                       5
...                                    ...
9            209                       9

The tablet path was broken the same way: a column that first showed up in the second tablet got its values shifted to the beginning of the page.

Fix

Make the aligned invariant hold: every registered measurement of an aligned device advances exactly one row per row written, with a NULL bit when the row carries no value for it.

• TsFileWriter::register_aligned_timeseries() now creates the ValueChunkWriter of an aligned measurement at registration time (ensure_aligned_value_chunk_writer()), so the column takes part from row 0.
• write_record_aligned() iterates the device schema instead of only the record's points and writes NULL rows for the measurements the record does not carry. Values are still written in record order; the mapping is by measurement name, so the order of the points inside a record (and the order of the columns) does not matter.
• write_tablet_aligned() writes NULL rows for the registered measurements a tablet does not carry, and both paths now include those columns in the page-seal lockstep.
• ValuePageWriter::write_null_rows() / ValueChunkWriter::write_null_batch() implement the NULL walk: one zero bit per row, no value bytes, no statistic update, page boundaries split on page_writer_max_point_num_ like write_batch() so the page lists of the time column and of every value column stay in step.
• A tablet that ends exactly at the page capacity now seals the time page and every value page together, including all-NULL columns. This prevents the next sparse record from appending its timestamp to the old page while its missing value starts a new page. Bulk column writes are retained.
• The schema-reference registration overloads release their internally allocated copies on failure. Aligned batch registration validates the complete list and initializes its value writers before publishing the device schema; a failed batch leaves all input schemas owned by the caller and permits a corrected retry.
• Match Java’s aligned registration policy: register the complete measurement list once per device. Every subsequent registration for that device returns E_INVALID_ARG, even before the first row, after an empty or nonempty flush, and after recovery. The non-aligned registration entry point cannot bypass this restriction. The device schema persists across flushes and is rebuilt from recovered chunk metadata, so no separate write-history flag is needed. Ordinary non-aligned devices can still register additional measurements.
• A record that repeats the same measurement now advances that column once (the last point wins). Previously both points were written, which ran the column ahead of the time column.

Behaviour notes / limits

• Callers that previously registered aligned measurements one at a time must now submit the complete vector in one call; the single-schema overload registers a device with exactly one measurement. Aligned registration also refuses an existing non-aligned device.
• A measurement that is registered but never carries a value now shows up as an all-NULL column (count == 0) instead of being absent from the metadata. This matches what the aligned tablet path already did for a column whose values are all NULL.
• Files already written by the buggy writer cannot be repaired: the page only stores "the first N rows are non-null", so the original row positions of those N values are not in the file (for N values inside start_time..end_time there is generally more than one candidate). Those files should be regenerated; the stored chunk statistic contradicting the bitmap-derived range can at least be used to detect them.

Tests

cpp/test/writer/tsfile_writer_test.cc

• AlignedRecordMissingMeasurementsStayRowAligned — sparse records, per-row values/timestamps and per-measurement statistics.
• AlignedRecordMissingMeasurementsAcrossPages — same with page_writer_max_point_num_ = 7 so the NULL padding has to seal pages in lockstep.
• AlignedTabletMissingColumnStaysRowAligned — tablet path, column introduced by the second tablet.
• AlignedRecordDuplicateMeasurementWritesOneRow
• AlignedRecordPointOrderDoesNotMatter — rotated add_point order per row.
• RegistrationStages/AlignedRegistrationTest.MeasurementsAreFixedAtRegistration — 8 combinations of single-schema/batch first registration and registration before writing / after empty flush / after buffered writes / after data flush. Checks both registration entry points, duplicate measurements, rejected schema-reference copies, and exact timestamp/value/NULL read-back.
• InvalidBatches/AlignedRegistrationFailureTest.FailedBatchCanBeRetried — empty, null, duplicate-name, and unsupported-encoding batches fail without publishing a partial device; corrected batches can reuse caller-owned schemas.
• NonAlignedRegistrationRemainsIncremental — ordinary measurements remain extensible, and aligned registration cannot convert an existing device.
• Existing multi-column writer and metadata-reader fixtures now register their aligned columns in one batch.
• PageBoundaries/AlignedTabletRecordBoundaryTest.MissingMeasurementsStayAligned — 24 combinations of tablet sizes 6/7/8/14 with a page capacity of 7, present/absent tablet columns, and no flush / flush after tablet / flush after sparse record. Checks page alignment and reads back every timestamp, value and NULL. Six boundary combinations fail before the follow-up fix; all 24 pass afterward.

cpp/test/file/restorable_tsfile_io_writer_test.cc

• RecoveredAlignedDeviceRejectsMeasurementRegistration — rejects extra and duplicate columns immediately after recovery, including attempts through the non-aligned entry point. Appends to existing columns, checks an originally all-NULL column’s row positions, and permits a new device.
• AlignedTimeseriesRecoverAndWriteNullValue — 10 sparse rows, corrupt tail, RestorableTsFileIOWriter recovery, 10 more sparse rows; asserts count == 10, start/end per measurement and the row positions after recovery.

The original sparse-row reproducers fail against unpatched develop. The 9 registration/recovery policy cases also fail against the previous PR revision and pass with this change.

Default C++ Release build using the repository’s feature defaults (Snappy, LZ4, lzokay, zlib, Zstandard, ANTLR4 and SIMD enabled): 990 tests, 987 passed, 3 skipped, 0 failed. The skipped cases require external fixture configuration. This PR does not change any build feature defaults.

macOS leaks reports 0 leaked bytes across the registration stages, failed-batch retries, non-aligned registrations, and recovered-device regression. Scoped C++ Spotless formatting passes.

[Copilot]

🟡 Changes recommended

Mixed tablet/record page boundaries and duplicate tablet columns can still corrupt aligned column row counts.

3 open findings

• Null writes can misalign value and time page boundaries · New
• Duplicate measurement names advance the same writer twice · New
• Failed schema registration leaks copied measurement schema · New

What changed in this PR

Fixes sparse aligned C++ writes so value columns remain synchronized with timestamps.

Changes:

• Adds NULL padding for omitted record/tablet measurements.
• Initializes aligned value writers during registration.
• Adds alignment and recovery regression tests.

File	Description
cpp/​src/​writer/​tsfile_writer.cc	Implements aligned-column padding and registration checks.
cpp/​src/​writer/​tsfile_writer.h	Declares aligned writer initialization helper.
cpp/​src/​writer/​value_chunk_writer.h	Adds batched NULL-row writing.
cpp/​src/​writer/​value_page_writer.h	Adds page-level NULL advancement.
cpp/​test/​writer/​tsfile_writer_test.cc	Tests sparse records, tablets, pages, duplicates, and registration.
cpp/​test/​file/​restorable_tsfile_io_writer_test.cc	Tests sparse aligned recovery and continued writes.

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🟡 Changes recommended

Late measurement registration remains possible after flushing or recovery because the guard only checks buffered data.

2 open findings

• Track persistent device write history across flushes and recovery · New
• Duplicate measurement names advance the same writer twice

2 resolved since last review

• Null writes can misalign value and time page boundaries
• Failed schema registration leaks copied measurement schema

🧠 Review effort: Balanced

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.