diff --git a/PRIVACY.md b/PRIVACY.md new file mode 100644 index 0000000..bc2194b --- /dev/null +++ b/PRIVACY.md @@ -0,0 +1,33 @@ +# Privacy and data handling + +CSV does not implement an upload service, analytics client, or extension-specific telemetry collection. Normal editable views parse, search, sort, and edit files through the VS Code document and the extension's bundled code. No account or external processing service is required by CSV. + +## Where processing happens + +Normal editable CSV views receive the file's decoded text from VS Code. The extension parses that text with the bundled Papa Parse library, sends table content to its VS Code webview, and applies edits through VS Code's document APIs. Saving in these views uses VS Code's normal file-saving behavior and the selected file encoding. + +The read-only large-file preview supports host-local UTF-8 files and reads bounded blocks directly from the saved file. It does not write the saved file. + +For a normal local workspace, these operations run on your computer. In Remote SSH, WSL, Dev Containers, or another remote workspace, the extension host and files can be on that remote environment. File contents travel through your existing VS Code connection to display and edit them in the webview. “No external processing service” does not mean that a remote workspace's data stays on your local computer. + +The font picker uses the bundled `font-list` package to enumerate fonts available to the extension host through operating-system helpers. It does not use an online font service. + +## Actions that share information outside the table + +- **Clipboard:** Copying writes the selected contents to the system clipboard; pasting reads clipboard contents supplied by the editor. Other applications, clipboard managers, and operating-system clipboard synchronization may have access to that clipboard. +- **Links:** Ctrl/Cmd+click on a recognized link asks VS Code to open its URL using your configured browser or application. Supported schemes include HTTP, HTTPS, FTP, and email links. The destination receives the URL, including any information in its path or query, according to that application's behavior. CSV does not fetch a link merely to render its text. You can disable link interaction with `csv.clickableLinks`. +- **Support:** If you attach a file, screenshot, or log to a GitHub issue, you are sharing it with that issue's audience. CSV does not submit these materials automatically. + +VS Code's Marketplace connections, updates, Settings Sync, remote services, other extensions, and the browser or application used for links have their own data-handling behavior. They are separate from CSV's runtime processing. + +## Logs and stored state + +The extension writes diagnostic messages to the extension host's console. Routine messages include file paths/URIs, row counts, and the row/column coordinates of an edit; routine edit messages omit the cell value. Errors can still contain document or operation details, and a failed external-link operation can log its URL. Treat extension host logs as potentially containing sensitive information, especially before sharing them for support. The log location and retention are controlled by VS Code and may be in a remote environment. + +CSV's code does not send those logs or cell contents to a telemetry endpoint. This statement does not describe or override VS Code's own telemetry, logging, crash reporting, or another extension's behavior. Consult your editor's privacy policy and settings for those services. + +CSV stores per-file view preferences in VS Code workspace state on the extension host, keyed by the file URI: header mode, serial-index visibility, separator overrides, hidden leading rows, manual column widths, frozen rows/columns, and column filters. Valid filter conditions are saved as you type in the filter details row and may contain sensitive CSV values; incomplete or invalid drafts are not saved. Clear all filters removes the saved conditions for that file. Its webview state remembers scroll/selection positions, column widths, row heights, and zoom. View preferences and saved state can themselves contain sensitive data and metadata; they are not intended as full-document backups. VS Code independently manages the document, undo history, unsaved-file backups, autosave, and any platform synchronization you enable. + +## Scope of these statements + +These statements describe this repository's extension and webview implementation. The main data paths are in [`src/CsvEditorProvider.ts`](src/CsvEditorProvider.ts), [`media/main.js`](media/main.js), [`src/commands.ts`](src/commands.ts), and [`src/extension.ts`](src/extension.ts); runtime dependencies are declared in [`package.json`](package.json). They are not a promise about every VS Code component, remote host, dependency version, or installed extension. Review the version you use and your environment's policies when deciding whether it fits your data requirements. diff --git a/README.md b/README.md index 74b27cf..352b7db 100644 --- a/README.md +++ b/README.md @@ -37,13 +37,14 @@ Working with CSV files shouldn’t be a chore. With CSV, you get: - **Add/Delete Columns:** Right-click any cell to add a column left or right, or remove the selected column. - **Add/Delete Rows:** Insert above/below or remove the selected row via context menu. - **Edit Empty CSVs:** Create or open an empty CSV file and start typing immediately. -- **Column Sorting:** Right-click a header and choose A–Z or Z–A. +- **Column Sorting:** Right-click a header and choose A–Z or Z–A. With filters active, sorting rearranges matching rows within their source positions and leaves hidden rows in place. +- **Column Filters:** Right-click a column and choose **Filter this column** at the top of the menu to add a condition to the filter details row. Choose its column and operator and type a value there; matching rows update as you type. **Add filter** and repeated right-clicks add more conditions, including multiple conditions on the same column; every condition combines with AND. Remove a condition individually or use **Clear all filters**. Every column offers Contains, Does Not Contain, Regular Expression, Equals, Does Not Equal, Is Empty and Is Not Empty. Number and date columns also offer >, >=, < and <= comparisons. Operator choices use the inferred type from the original body rows, excluding headers and hidden metadata; other filters do not change it. A text column whose nonblank cells all contain supported dates also offers date comparisons, without changing column colors. Filtering searches the full file, retains source row numbers and remembers valid conditions per file. - **Custom Font Controls:** Choose a font family and optional font-size override, or inherit VS Code defaults. - **In-View Zoom:** Use `Ctrl/Cmd + Mouse Wheel` or `Ctrl/Cmd + +/-/0` to zoom the CSV view without changing global editor zoom. -- **Find & Replace Overlay:** Built-in find/replace bar with match options (case, whole-word, regex), keyboard navigation, and single/all replace actions across the full file (including chunked rows). +- **Find & Replace Overlay:** Built-in find/replace bar with match options (case, whole-word, regex), keyboard navigation, and single/all replace actions across the full file (including chunked rows), or the matching rows while column filters are active. - **Multiline Cell Display:** Cells with embedded newlines render as wrapped multi-line content (with preserved line breaks and matching row height). - **Clickable Links:** URLs in cells are automatically detected and displayed as clickable links. Ctrl/Cmd+click to open them in your browser. -- **Preserved CSV Integrity:** All modifications respect CSV formatting—no unwanted extra characters or formatting issues. +- **Preserved CSV Integrity:** Cell edits, expanding grid pastes, and row/column moves preserve existing field quoting and line endings. Appended values follow the existing column quote style when the column uses one consistently. - **Optimized for Performance:** Designed for medium-sized datasets, ensuring a smooth editing experience without compromising on functionality. - **Large File Support:** Loads big CSVs in chunks so even large datasets open quickly. - **CSV/TSV/TAB/PSV Support:** `.csv`, `.tsv`, `.tab`, and `.psv` files are recognized automatically. Defaults: comma for CSV, tab for TSV/TAB, pipe for PSV. @@ -75,7 +76,11 @@ Cursor (built on VS Code 1.99) and the latest VS Code releases (1.102). - Detail edit: press `Enter` on a selected cell or double‑click to enter a focused edit. Arrow Left/Right move the text caret; Arrow Up goes to start; Arrow Down goes to end. Click outside the cell (or blur) to save. - **Keyboard Navigation:** Use Arrow keys to move between cells when not editing. Use `Tab`/`Shift+Tab` to move horizontally (wrapping across rows as needed). - **Multi-Cell Selection:** Click and drag or use `Shift + Click` to select multiple cells, then copy them as CSV using `Ctrl/Cmd + C`. -- **Find & Replace:** Press `Ctrl/Cmd + F` to open Find, or `Ctrl/Cmd + H` to open Find + Replace. +- **Find & Replace:** Press `Ctrl/Cmd + F` to open Find, or `Ctrl/Cmd + H` to open Find + Replace. With filters active, Find and Replace All operate on the visible filtered table (including its header); hidden metadata and nonmatching rows are excluded. + +Filter comparisons use the column's inferred type. Text matching ignores case. Numbers compare finite decimal/scientific values, including consistently grouped thousands. Boolean equality accepts `true/false`, `t/f`, `yes/no`, `y/n`, `on/off` and `1/0`. Date queries accept explicit numeric dates and English month names, optional times and timezone offsets; date-only or timezone-less values use UTC. Ambiguous numeric dates default to month/day/year (`03/04/2026` means March 4); a leading day above 12 is unambiguous (`13/04/2026` means April 13). Use an ISO date or an explicit month name to avoid ambiguity. Supported timestamps compare exactly, including sub-millisecond fractional seconds. Blank or invalid cell values do not match typed comparisons, including Does Not Equal; use Is Empty to find empty cells. Incomplete or invalid values show an inline error and retain the last valid results until corrected. A blank value leaves that draft condition inactive; Is Empty and Is Not Empty apply without a value. Filter controls retain focus while results update. + +Regular Expression uses [RE2 syntax](https://github.com/google/re2/wiki/Syntax). Plain patterns ignore case; `/pattern/flags` accepts `i`, `m` and `s`, and explicit flags override that default (`/pattern/` is case-sensitive). Lookaround and backreferences are unsupported and rejected by input validation. --- @@ -124,6 +129,7 @@ Per-file (stored by the extension; set via commands): - Serial index column (default `true`) — `CSV: Toggle Serial Index Column` - CSV separator override — `CSV: Change CSV Separator` (or clear it with `CSV: Reset CSV Separator (Inherit)`) - Hide first N rows (inherits `csv.defaultIgnoreRows`) — `CSV: Hide First N Rows`. An explicit `0` overrides a nonzero default; blank input clears the per-file override. +- Column filter conditions — edit in the live filter details row; add from its Add filter button or a column’s context menu and remove individually. Conditions, including any entered CSV text, are stored in VS Code workspace state on the extension host (which may be remote). Clear all filters removes the saved conditions for that file. No CSV rows are removed by filtering. - Frozen leading visible rows/columns — `CSV: Freeze Leading Rows` / `CSV: Freeze Leading Columns`. Enter `0` to unfreeze that axis. The default keeps the first header row frozen; with headers off, no rows freeze until requested. Hidden metadata rows do not count; the virtual append row is never frozen as a leading row. --- @@ -141,7 +147,7 @@ Per-file (stored by the extension; set via commands): - Save the file: `Ctrl/Cmd + S` commits the active cell before saving, including all inserted line breaks. - Global: - Copy selection: `Ctrl/Cmd + C` - - Paste selection: `Ctrl/Cmd + V` (selection mode). Pasting a single value into a selected rectangle fills that rectangle. + - Paste selection: `Ctrl/Cmd + V` (selection mode). Pasting a single value into a selected rectangle fills that rectangle. With filters active, copying/cutting and filling affect visible selected rows; matrix paste advances through matching rows and appends at the physical end when needed. Nonmatching rows remain intact, and saving always includes the full source file. - Zoom in/out/reset: `Ctrl/Cmd + +`, `Ctrl/Cmd + -`, `Ctrl/Cmd + 0` (also `Ctrl/Cmd + Mouse Wheel`) - Find: `Ctrl/Cmd + F` - Replace: `Ctrl/Cmd + H` @@ -197,6 +203,12 @@ npm run compile --- +## Privacy + +CSV processes files through VS Code and bundled libraries, with no extension-specific upload service or telemetry client. Remote workspaces, clipboard operations, links, diagnostic logs, and editor-managed backups have additional data-handling implications. See [Privacy and data handling](PRIVACY.md) for the verified behavior and scope. + +--- + ## Support Have questions, suggestions, or encountered an issue? diff --git a/images/inline-column-filters.jpg b/images/inline-column-filters.jpg new file mode 100644 index 0000000..ce1a86b Binary files /dev/null and b/images/inline-column-filters.jpg differ diff --git a/media/column-filters.js b/media/column-filters.js new file mode 100644 index 0000000..b77b851 --- /dev/null +++ b/media/column-filters.js @@ -0,0 +1,246 @@ +// Inline filter drafts live in this webview; only complete conditions reach the host. +window.createColumnFilterControls = ({ root, metadata, onChange, onLayout, onPending, onActivity }) => { + const columns = Array.isArray(metadata.columns) ? metadata.columns : []; + const maxFilters = Number.isSafeInteger(metadata.maxFilters) ? metadata.maxFilters : 20; + let revision = Number.isSafeInteger(metadata.revision) ? metadata.revision : 0; + let timer = null; + let nextId = 0; + let snapshotRows = []; + let pending = false; + const rows = []; + const make = (tag, text) => { + const element = document.createElement(tag); + if (text !== undefined) element.textContent = text; + return element; + }; + const toolbar = make('div'); + toolbar.className = 'column-filter-toolbar'; + const summary = make('span'); + summary.dataset.filterSummary = ''; + summary.setAttribute('aria-live', 'polite'); + const addButton = make('button', 'Add filter'); + addButton.id = 'addColumnFilter'; + addButton.type = 'button'; + const clearButton = make('button', 'Clear all filters'); + clearButton.id = 'clearAllFilters'; + clearButton.type = 'button'; + const globalError = make('span'); + globalError.dataset.filterError = ''; + globalError.setAttribute('role', 'alert'); + toolbar.append(summary, addButton, clearButton, globalError); + const list = make('div'); + list.className = 'column-filter-conditions'; + root.replaceChildren(toolbar, list); + root.setAttribute('aria-label', 'Column filters'); + const isUnary = operator => operator === 'empty' || operator === 'notEmpty'; + const typedOperator = operator => !['contains', 'notContains', 'regex', 'empty', 'notEmpty'].includes(operator); + const columnFor = column => columns.find(item => item.column === column) || columns[0]; + const setSummary = data => { summary.textContent = `Filtered: ${data.matchedRows ?? 0} of ${data.totalRows ?? 0} rows`; }; + setSummary(metadata); + const updateLayout = () => { + root.hidden = rows.length === 0; + addButton.disabled = !columns.length || rows.length >= maxFilters; + clearButton.disabled = rows.length === 0; + rows.forEach((row, index) => { row.and.textContent = index ? 'AND' : ''; }); + onLayout?.(); + }; + const clearErrors = () => { + globalError.textContent = ''; + rows.forEach(row => { + row.error.textContent = row.incomplete ? 'Finish editing this condition to apply filters.' : row.invalidOperator ? 'Choose an operator supported by this column type.' : ''; + row.value.toggleAttribute('aria-invalid', !!row.invalidOperator); + if (row.invalidOperator) row.value.setAttribute('aria-invalid', 'true'); + }); + }; + const condition = row => { + const column = columnFor(Number(row.column.value)); + const operator = row.operator.value; + if (!column || (!isUnary(operator) && row.value.value.length === 0)) return null; + const filter = { column: column.column, operator, value: isUnary(operator) ? '' : row.value.value }; + if (typedOperator(operator) && column.dataType !== 'string') filter.dataType = column.dataType; + return filter; + }; + const send = () => { + timer = null; + if (rows.some(row => row.composing || row.incomplete)) return; + snapshotRows = rows.filter(row => condition(row)); + onChange({ revision, filters: snapshotRows.map(condition) }); + }; + const changed = () => { + onActivity?.(); + // Invalidate older replies immediately, even before debounce/composition ends. + revision += 1; + pending = !rows.some(row => row.incomplete); + onPending?.(pending); + if (timer !== null) clearTimeout(timer); + timer = null; + clearErrors(); + if (pending && !rows.some(row => row.composing)) timer = setTimeout(send, 100); + updateLayout(); + }; + const fillOperators = (row, preferred) => { + const column = columnFor(Number(row.column.value)); + row.operator.replaceChildren(); + for (const choice of column?.operators || []) { + const option = make('option', choice.label); + option.value = choice.operator; + row.operator.append(option); + } + row.error.textContent = row.incomplete ? 'Finish editing this condition to apply filters.' : ''; + row.value.removeAttribute('aria-invalid'); + row.invalidOperator = !!preferred && !Array.from(row.operator.options).some(option => option.value === preferred); + if (row.invalidOperator) { + const unavailable = make('option', `${preferred} (unsupported for ${column?.dataType === 'string' ? 'text' : column?.dataType || 'text'})`); + unavailable.value = preferred; + unavailable.disabled = true; + row.operator.append(unavailable); + row.error.textContent = 'Choose an operator supported by this column type.'; + row.value.setAttribute('aria-invalid', 'true'); + } + if (preferred) row.operator.value = preferred; + row.value.hidden = isUnary(row.operator.value); + row.value.disabled = isUnary(row.operator.value); + row.type.textContent = column?.dataType === 'string' ? 'text' : column?.dataType || 'text'; + }; + const append = filter => { + const column = columnFor(filter?.column); + if (!column || rows.length >= maxFilters) return null; + const id = ++nextId; + const element = make('div'); + element.className = 'column-filter-row'; + const and = make('span'); + and.className = 'column-filter-and'; + const columnSelect = make('select'); + columnSelect.dataset.filterColumn = ''; + columnSelect.setAttribute('aria-label', `Filter ${id} column`); + for (const item of columns) { + const option = make('option', item.label); + option.value = String(item.column); + columnSelect.append(option); + } + columnSelect.value = String(column.column); + const type = make('span'); + type.className = 'column-filter-type'; + const operator = make('select'); + operator.dataset.filterOperator = ''; + operator.setAttribute('aria-label', `Filter ${id} operator`); + const value = make('input'); + value.type = 'text'; + value.dataset.filterValue = ''; + value.setAttribute('aria-label', `Filter ${id} value`); + value.value = filter?.value || ''; + const remove = make('button', 'Remove'); + remove.type = 'button'; + remove.dataset.removeFilter = ''; + remove.setAttribute('aria-label', `Remove filter ${id}`); + const error = make('span'); + error.id = `column-filter-error-${id}`; + error.className = 'column-filter-error'; + error.setAttribute('role', 'alert'); + value.setAttribute('aria-describedby', error.id); + const row = { element, and, column: columnSelect, type, operator, value, remove, error, composing: false, incomplete: filter?.composing === true }; + fillOperators(row, filter?.operator || 'contains'); + element.append(and, columnSelect, type, operator, value, remove, error); + list.append(element); + rows.push(row); + columnSelect.addEventListener('change', () => { row.incomplete = false; fillOperators(row, operator.value); changed(); }); + operator.addEventListener('change', () => { row.incomplete = false; row.invalidOperator = false; value.hidden = isUnary(operator.value); value.disabled = isUnary(operator.value); changed(); }); + value.addEventListener('input', () => { row.incomplete = false; changed(); }); + value.addEventListener('compositionstart', () => { row.incomplete = false; row.composing = true; changed(); }); + value.addEventListener('compositionend', () => { row.incomplete = false; row.composing = false; changed(); }); + remove.addEventListener('click', () => { + rows.splice(rows.indexOf(row), 1); + element.remove(); + changed(); + }); + return row; + }; + const add = column => { + const row = append({ column, operator: 'contains', value: '' }); + if (!row) return; + changed(); + row.value.focus(); + }; + addButton.addEventListener('click', () => add(Number(rows[rows.length - 1]?.column.value ?? columns[0]?.column))); + clearButton.addEventListener('click', () => { + rows.splice(0).forEach(row => row.element.remove()); + changed(); + }); + const restored = metadata.filterDraft; + for (const filter of restored?.rows || metadata.filters || []) append(filter); + updateLayout(); + if (restored) { + // This state arrived through one controller's refresh ACK, never VS Code storage. + changed(); + setTimeout(() => { + const focus = restored.focus; + const field = rows[focus?.row]?.[focus?.field]; + if (!field || !['column', 'operator', 'value', 'remove'].includes(focus?.field)) return; + field.focus({ preventScroll: true }); + if (focus.field === 'value' && Number.isInteger(focus.selectionStart) && Number.isInteger(focus.selectionEnd)) { + field.setSelectionRange(focus.selectionStart, focus.selectionEnd, focus.selectionDirection || 'none'); + } + }, 0); + } + const captureDraft = () => { + if (!rows.length && !pending) return undefined; + const draft = { + rows: rows.map(row => ({ column: Number(row.column.value), operator: row.operator.value, value: row.value.value, ...((row.composing || row.incomplete) ? { composing: true } : {}) })), + session: metadata.session, + filterColumnsRevision: metadata.filterColumnsRevision, + columns: columns.map(column => ({ column: column.column, label: column.label })) + }; + rows.some((row, index) => { + for (const field of ['column', 'operator', 'value', 'remove']) { + if (row[field] !== document.activeElement) continue; + draft.focus = { row: index, field }; + if (field === 'value') Object.assign(draft.focus, { + selectionStart: row.value.selectionStart, + selectionEnd: row.value.selectionEnd, + selectionDirection: row.value.selectionDirection + }); + return true; + } + return false; + }); + return draft; + }; + return { + add, + captureDraft, + retry: changed, + get latestRevision() { return revision; }, + receive(message) { + if (message.session !== metadata.session || message.revision !== revision) return false; + pending = false; + clearErrors(); + if (Array.isArray(message.columns) && JSON.stringify(message.columns) !== JSON.stringify(columns)) { + columns.length = 0; + for (const column of message.columns) columns.push(column); + for (const row of rows) { + const selected = row.column.value; + row.column.replaceChildren(); + for (const column of columns) { + const option = make('option', column.label); + option.value = String(column.column); + row.column.append(option); + } + row.column.value = selected; + fillOperators(row, row.operator.value); + } + } + if (message.error) globalError.textContent = message.error; + else if (message.errors?.length) globalError.textContent = 'Showing the last valid results.'; + for (const item of message.errors || []) { + const row = snapshotRows[item.index]; + if (!row) continue; + row.error.textContent = item.message; + row.value.setAttribute('aria-invalid', 'true'); + } + if (message.error || message.errors?.length) { setSummary(message); onLayout?.(); return true; } + setSummary(message); + onLayout?.(); + return true; + } + }; +}; diff --git a/media/frozen-panes.js b/media/frozen-panes.js index 8aeb5a6..24db08e 100644 --- a/media/frozen-panes.js +++ b/media/frozen-panes.js @@ -1,5 +1,5 @@ // Layout uses measured dimensions so multiline cells, manual sizes and zoom agree. -window.createFrozenPaneManager = ({ table, firstRow, realRowCount, onNeedRows }) => { +window.createFrozenPaneManager = ({ table, firstRow, realRowCount, rowOrder, topInset, onNeedRows }) => { let rows = 0; let columns = 0; let scheduled = false; @@ -42,16 +42,18 @@ window.createFrozenPaneManager = ({ table, firstRow, realRowCount, onNeedRows }) left += cell.getBoundingClientRect().width; } } - const lastFrozenRow = Math.min(firstRow + rows, realRowCount) - 1; - const firstFrozenCell = rendered.find(({ index }) => index >= firstRow && index <= lastFrozenRow)?.element.cells[0]; + const visibleRows = rowOrder?.length ? rowOrder.filter(row => row < realRowCount) : null; + const frozenRows = visibleRows ? new Set(visibleRows.slice(0, rows)) : null; + const lastFrozenRow = visibleRows ? (visibleRows[Math.min(rows, visibleRows.length) - 1] ?? -1) : Math.min(firstRow + rows, realRowCount) - 1; + const firstFrozenCell = rendered.find(({ index }) => frozenRows ? frozenRows.has(index) : index >= firstRow && index <= lastFrozenRow)?.element.cells[0]; // A collapsed border straddles the sticky edge. Cover its inner half once, // keeping the measured spacing between frozen rows unchanged. const topBorder = firstFrozenCell && getComputedStyle(table).borderCollapse === 'collapse' ? parseFloat(getComputedStyle(firstFrozenCell).borderTopWidth) || 0 : 0; - let top = -topBorder / 2; + let top = (topInset?.() || 0) - topBorder / 2; for (const { element, index } of rendered) { - const frozenRow = index >= firstRow && index <= lastFrozenRow; + const frozenRow = frozenRows ? frozenRows.has(index) : index >= firstRow && index <= lastFrozenRow; const height = element.getBoundingClientRect().height || Math.max(0, ...Array.from(element.cells).map(cell => cell.getBoundingClientRect().height)); for (const cell of Array.from(element.cells)) { const attr = cell.getAttribute('data-col'); diff --git a/media/main.js b/media/main.js index 408ab75..4b634d7 100644 --- a/media/main.js +++ b/media/main.js @@ -6,6 +6,19 @@ try { document.body.focus({ preventScroll: true }); } catch { try { document.bod const vscode = acquireVsCodeApi(); const root = document.getElementById('csv-root'); +let filterMetadata = {}; +try { filterMetadata = JSON.parse(document.getElementById('__csvFilters')?.textContent || '{}'); } catch {} +const filterSession = filterMetadata.session; +let visibleRowOrder = []; +try { visibleRowOrder = JSON.parse(root?.dataset?.roworder || '[]'); } catch {} +let headerSourceRow = Number(root?.dataset?.firstrow || '0'); +let filterEpoch = 0; +let filterSnapshotPending = false; +const adjacentVisibleRow = (row, direction) => { + if (!visibleRowOrder.length) return row + direction; + const index = visibleRowOrder.indexOf(row); + return visibleRowOrder[index + direction] ?? (row >= Number(root?.dataset?.realrowcount || '0') ? row + direction : row); +}; let frozenRowCount = Number(root?.dataset?.frozenrows || '0'); let frozenColumnCount = Number(root?.dataset?.frozencolumns || '0'); const parsePositiveNumber = value => { @@ -166,6 +179,7 @@ const persistState = () => { }; const restoreState = () => { + if (document.activeElement?.closest?.('#filterStatus')) return; try { const st = vscode.getState() || {}; const restoredZoom = parsePositiveNumber(st.zoomScale); @@ -195,7 +209,7 @@ const restoreState = () => { } } if (typeof st.anchorRow === 'number' && typeof st.anchorCol === 'number') { - const tag = (hasHeader && st.anchorRow === 0 ? 'th' : 'td'); + const tag = (hasHeader && st.anchorRow === headerSourceRow ? 'th' : 'td'); let sel = table.querySelector(`${tag}[data-row="${st.anchorRow}"][data-col="${st.anchorCol}"]`); // If not present yet (due to chunking), load chunks until available or exhausted if (!sel && typeof window.__csvLoadNextChunk === 'function') { @@ -253,73 +267,46 @@ const requestRemoteChunk = () => { remoteChunkRequestInFlight = true; remoteChunkRequestedStart = remoteNextChunkStart; remoteChunkRequestSeq += 1; - vscode.postMessage({ type: 'requestChunk', start: remoteNextChunkStart, requestId: remoteChunkRequestSeq }); + vscode.postMessage({ type: 'requestChunk', start: remoteNextChunkStart, requestId: remoteChunkRequestSeq, filterEpoch, session: filterSession }); }; -if (csvChunks.length || remoteHasMoreChunks) { +let chunkObserver = null; +let chunkLoading = false; +const primeChunks = () => { + chunkObserver?.disconnect(); + const last = table.tBodies[0]?.querySelector('tr:last-child'); + if (last && (csvChunks.length || remoteHasMoreChunks)) chunkObserver?.observe(last); +}; +loadNextChunk = () => { const tbody = table.tBodies[0]; - let loading = false; - - loadNextChunk = () => { - if (loading || !tbody) return false; - if (!csvChunks.length) { - requestRemoteChunk(); - return false; - } - loading = true; - try { - const html = csvChunks.shift(); - if (html) { - tbody.insertAdjacentHTML('beforeend', html); - applySizeStateToRenderedCells(); - window.dispatchEvent(new Event('csvChunkLoaded')); - } - if (!csvChunks.length) { - requestRemoteChunk(); - } - return !!html; - } finally { - loading = false; - } - }; - // Expose for restoration logic - window.__csvLoadNextChunk = loadNextChunk; - - nearBottom = () => { - if (!scrollContainer) return false; - const remain = scrollContainer.scrollHeight - scrollContainer.scrollTop - scrollContainer.clientHeight; - return remain < 300; // px threshold - }; - - const io = new IntersectionObserver((entries)=>{ - if (entries[0] && entries[0].isIntersecting) { - loadNextChunk(); - const last = tbody && tbody.querySelector('tr:last-child'); - if (last) io.observe(last); - } - }, { root: scrollContainer || null, rootMargin: '0px 0px 300px 0px' }); - - const prime = () => { - const last = tbody && tbody.querySelector('tr:last-child'); - if (last) { io.observe(last); } - }; - prime(); - - // Fallback: scroll-driven loader to ensure progress even if IO misses - const scrollHandler = () => { - if (!csvChunks.length && !remoteHasMoreChunks) return; - if (nearBottom()) { - // Load until we create headroom or exhaust currently available chunks - let guard = 10; - while (nearBottom() && guard-- > 0) { - if (!loadNextChunk()) break; - } - prime(); - } - }; - if (scrollContainer) scrollContainer.addEventListener('scroll', scrollHandler, { passive: true }); - else window.addEventListener('scroll', scrollHandler, { passive: true }); -} + if (chunkLoading || !tbody) return false; + if (!csvChunks.length) { requestRemoteChunk(); return false; } + chunkLoading = true; + try { + const html = csvChunks.shift(); + if (html) { + tbody.insertAdjacentHTML('beforeend', html); + applySizeStateToRenderedCells(); + window.dispatchEvent(new Event('csvChunkLoaded')); + } + if (!csvChunks.length) requestRemoteChunk(); + primeChunks(); + return !!html; + } finally { chunkLoading = false; } +}; +window.__csvLoadNextChunk = loadNextChunk; +nearBottom = () => !!scrollContainer && scrollContainer.scrollHeight - scrollContainer.scrollTop - scrollContainer.clientHeight < 300; +chunkObserver = new IntersectionObserver(entries => { + if (entries[0]?.isIntersecting) loadNextChunk(); +}, { root: scrollContainer || null, rootMargin: '0px 0px 300px 0px' }); +primeChunks(); +const scrollChunkLoader = () => { + if (!csvChunks.length && !remoteHasMoreChunks) return; + let guard = 10; + while (nearBottom() && guard-- > 0) { if (!loadNextChunk()) break; } + primeChunks(); +}; +(scrollContainer || window).addEventListener('scroll', scrollChunkLoader, { passive: true }); const ensureTargetStep = () => { if (!pendingEnsureTarget) return; @@ -383,7 +370,7 @@ const handleZoomWheel = e => { }; window.addEventListener('wheel', handleZoomWheel, { passive: false }); -const hasHeader = document.querySelector('thead') !== null; +let hasHeader = document.querySelector('thead') !== null; const getCellCoords = cell => ({ row: parseInt(cell.getAttribute('data-row')), col: parseInt(cell.getAttribute('data-col')) }); const clearSelection = () => { currentSelection.forEach(c => c.classList.remove('selected')); currentSelection = []; }; const contextMenu = document.getElementById('contextMenu'); @@ -400,7 +387,10 @@ const showContextMenu = (x, y, row, col) => { const item = (label, cb) => { const d = document.createElement('div'); d.textContent = label; - d.addEventListener('click', () => { cb(); contextMenu.style.display = 'none'; }); + d.addEventListener('click', () => { + if (!filterSnapshotPending) cb(); + contextMenu.style.display = 'none'; + }); contextMenu.appendChild(d); }; const divider = () => { @@ -409,6 +399,10 @@ const showContextMenu = (x, y, row, col) => { d.style.margin = '1px 0'; contextMenu.appendChild(d); }; + if (Number.isInteger(col) && col >= 0) { + item('Filter this column…', () => window.csvColumnFilters?.add(col)); + divider(); + } item('Cut', () => copySelectionToClipboard(true)); item('Copy', () => copySelectionToClipboard()); item('Paste', () => pasteFromClipboard()); @@ -435,9 +429,9 @@ const showContextMenu = (x, y, row, col) => { /* Header-only: SORT functionality */ if (lastContextIsHeader) { - item('Sort: A-Z', () => + item(root?.dataset?.filtered === '1' ? 'Sort matching rows: A-Z' : 'Sort: A-Z', () => vscode.postMessage({ type: 'sortColumn', index: col, ascending: true })); - item('Sort: Z-A', () => + item(root?.dataset?.filtered === '1' ? 'Sort matching rows: Z-A' : 'Sort: Z-A', () => vscode.postMessage({ type: 'sortColumn', index: col, ascending: false })); } @@ -1016,9 +1010,10 @@ const selectRange = (start, end) => { clearSelection(); const minRow = Math.min(start.row, end.row), maxRow = Math.max(start.row, end.row); const minCol = Math.min(start.col, end.col), maxCol = Math.max(start.col, end.col); - for(let r = minRow; r <= maxRow; r++){ + const selectedRows = visibleRowOrder.length ? visibleRowOrder.filter(row => row >= minRow && row <= maxRow) : Array.from({ length: maxRow - minRow + 1 }, (_, index) => minRow + index); + for (const r of selectedRows) { for(let c = minCol; c <= maxCol; c++){ - const selector = (hasHeader && r === 0 ? 'th' : 'td') + '[data-row="'+r+'"][data-col="'+c+'"]'; + const selector = (hasHeader && r === headerSourceRow ? 'th' : 'td') + '[data-row="'+r+'"][data-col="'+c+'"]'; const selCell = table.querySelector(selector); if(selCell){ selCell.classList.add('selected'); currentSelection.push(selCell); } } @@ -1071,7 +1066,8 @@ const getDataSelectionBounds = () => { const maxRow = Math.max(...rows); const minCol = Math.min(...cols); const maxCol = Math.max(...cols); - const expectedCount = (maxRow - minRow + 1) * (maxCol - minCol + 1); + const rowIds = [...new Set(rows)].sort((a, b) => a - b); + const expectedCount = rowIds.length * (maxCol - minCol + 1); const rectangular = keys.size === expectedCount; return { minRow, maxRow, minCol, maxCol, rectangular }; }; @@ -1129,6 +1125,19 @@ const isFindWidgetTarget = target => { const el = getElementTarget(target); return !!(el && el.closest('#findReplaceWidget')); }; +const isGridControlTarget = target => isFindWidgetTarget(target) || !!getElementTarget(target)?.closest('#filterStatus'); +const preventPendingGridAction = event => { + if (!filterSnapshotPending || isGridControlTarget(event.target)) return; + event.preventDefault(); + event.stopImmediatePropagation(); +}; +for (const type of ['keydown', 'copy', 'cut', 'paste']) { + document.addEventListener(type, preventPendingGridAction, true); +} +for (const type of ['mousedown', 'mouseup', 'dblclick', 'contextmenu', 'beforeinput']) { + table.addEventListener(type, preventPendingGridAction, true); +} + const clearFindHighlights = () => { document.querySelectorAll('.highlight, .active-match').forEach(el => { el.classList.remove('highlight'); @@ -1311,6 +1320,7 @@ const replaceInText = (text, replaceAllMatches) => { return text.replace(regex, matched => preserveReplacementCase(replacementText, matched)); }; const replaceCurrentMatch = () => { + if (filterSnapshotPending) return; if (!findMatches.length || findReplaceState.invalidRegex) return; const match = findMatches[currentMatchIndex]; if (!match) { @@ -1331,6 +1341,7 @@ const replaceCurrentMatch = () => { runFind(true); }; const replaceAllMatches = () => { + if (filterSnapshotPending) return; if (findReplaceState.invalidRegex || !findInput.value || !findMatches.length) return; const seen = new Set(); if (!findMatches.length) return; @@ -1498,7 +1509,7 @@ window.addEventListener('csvChunkLoaded', () => { // Capture-phase handler to intercept Cmd/Ctrl + Arrow and move to extremes document.addEventListener('keydown', e => { - if (isFindWidgetTarget(e.target)) { + if (isGridControlTarget(e.target)) { return; } const isArrowKey = (k) => ['ArrowUp','ArrowDown','ArrowLeft','ArrowRight','Up','Down','Left','Right','Home','End'].includes(k); @@ -1531,13 +1542,13 @@ document.addEventListener('keydown', e => { if (ref) { const { row, col } = getCellCoords(ref); if (['ArrowLeft','Left','Home'].includes(e.key)) { - const tag = (hasHeader && row === 0 ? 'th' : 'td'); + const tag = (hasHeader && row === headerSourceRow ? 'th' : 'td'); const rowCells = Array.from(table.querySelectorAll(tag + '[data-row="'+row+'"]')) .filter(el => el.getAttribute('data-col') !== null && el.getAttribute('data-col') !== '-1'); const min = rowCells.reduce((acc, el) => Math.min(acc, parseInt(el.getAttribute('data-col'))), Infinity); target = rowCells.find(el => parseInt(el.getAttribute('data-col')) === min) || ref; } else if (['ArrowRight','Right','End'].includes(e.key)) { - const tag = (hasHeader && row === 0 ? 'th' : 'td'); + const tag = (hasHeader && row === headerSourceRow ? 'th' : 'td'); const rowCells = Array.from(table.querySelectorAll(tag + '[data-row="'+row+'"]')) .filter(el => el.getAttribute('data-col') !== null && el.getAttribute('data-col') !== '-1'); const max = rowCells.reduce((acc, el) => Math.max(acc, parseInt(el.getAttribute('data-col'))), -1); @@ -1575,6 +1586,7 @@ document.addEventListener('keydown', e => { }, true); document.addEventListener('keydown', e => { + if (getElementTarget(e.target)?.closest('#filterStatus')) return; if (maybeHandleZoomShortcut(e)) { return; } @@ -1604,7 +1616,7 @@ document.addEventListener('keydown', e => { closeFindReplace(); return; } - if (isFindWidgetTarget(e.target)) { + if (isGridControlTarget(e.target)) { return; } if ((e.ctrlKey || e.metaKey) && e.key === 'a' && !editingCell) { @@ -1650,22 +1662,22 @@ document.addEventListener('keydown', e => { if (refCell) { const { row, col } = getCellCoords(refCell); if (['ArrowLeft','Left','Home'].includes(e.key)) { - const tag = (hasHeader && row === 0 ? 'th' : 'td'); + const tag = (hasHeader && row === headerSourceRow ? 'th' : 'td'); const rowCells = Array.from(table.querySelectorAll(tag + '[data-row="'+row+'"]')) .filter(el => el.getAttribute('data-col') !== null && el.getAttribute('data-col') !== '-1'); const min = rowCells.reduce((acc, el) => Math.min(acc, parseInt(el.getAttribute('data-col'))), Infinity); target = rowCells.find(el => parseInt(el.getAttribute('data-col')) === min) || refCell; } else if (['ArrowRight','Right','End'].includes(e.key)) { - const tag = (hasHeader && row === 0 ? 'th' : 'td'); + const tag = (hasHeader && row === headerSourceRow ? 'th' : 'td'); const rowCells = Array.from(table.querySelectorAll(tag + '[data-row="'+row+'"]')) .filter(el => el.getAttribute('data-col') !== null && el.getAttribute('data-col') !== '-1'); const max = rowCells.reduce((acc, el) => Math.max(acc, parseInt(el.getAttribute('data-col'))), -1); target = rowCells.find(el => parseInt(el.getAttribute('data-col')) === max) || refCell; } else if (['ArrowUp','Up'].includes(e.key)) { if (hasHeader) { - target = table.querySelector('th[data-row="0"][data-col="'+col+'"]') || refCell; + target = table.querySelector('th[data-row="'+headerSourceRow+'"][data-col="'+col+'"]') || refCell; } else { - target = table.querySelector('td[data-row="0"][data-col="'+col+'"]') || refCell; + target = table.querySelector('td[data-row="'+getFirstDataRow()+'"][data-col="'+col+'"]') || refCell; } } else if (['ArrowDown','Down'].includes(e.key)) { const colCells = Array.from(table.querySelectorAll('td[data-col="'+col+'"]')); @@ -1702,13 +1714,13 @@ document.addEventListener('keydown', e => { let targetRow = coords.row; let targetCol = coords.col + (isBackward ? -1 : 1); if (!isBackward && targetCol > maxCol) { - targetRow += 1; + targetRow = adjacentVisibleRow(targetRow, 1); targetCol = minCol; } else if (isBackward && targetCol < minCol) { if (targetRow <= firstDataRow) { return; } - targetRow -= 1; + targetRow = adjacentVisibleRow(targetRow, -1); targetCol = maxCol; } const nextCell = ensureRenderedCellByCoords(targetRow, targetCol); @@ -1749,14 +1761,14 @@ document.addEventListener('keydown', e => { const { row, col } = getCellCoords(rangeEndCell || anchorCell); let targetRow = row, targetCol = col; switch(e.key){ - case 'ArrowUp': targetRow = row - 1; break; - case 'ArrowDown': targetRow = row + 1; break; + case 'ArrowUp': targetRow = adjacentVisibleRow(row, -1); break; + case 'ArrowDown': targetRow = adjacentVisibleRow(row, 1); break; case 'ArrowLeft': targetCol = col - 1; break; case 'ArrowRight':targetCol = col + 1; break; } if(targetRow < 0 || targetCol < 0) return; - const tag = (hasHeader && targetRow === 0 ? 'th' : 'td'); - const nextCell = table.querySelector(`${tag}[data-row="${targetRow}"][data-col="${targetCol}"]`); + const tag = (hasHeader && targetRow === headerSourceRow ? 'th' : 'td'); + const nextCell = ensureRenderedCellByCoords(targetRow, targetCol); if(nextCell){ e.preventDefault(); rangeEndCell = nextCell; @@ -1772,14 +1784,14 @@ document.addEventListener('keydown', e => { const { row, col } = getCellCoords(anchorCell); let targetRow = row, targetCol = col; switch(e.key){ - case 'ArrowUp': targetRow = row - 1; break; - case 'ArrowDown': targetRow = row + 1; break; + case 'ArrowUp': targetRow = adjacentVisibleRow(row, -1); break; + case 'ArrowDown': targetRow = adjacentVisibleRow(row, 1); break; case 'ArrowLeft': targetCol = col - 1; break; case 'ArrowRight':targetCol = col + 1; break; } if(targetRow < 0 || targetCol < 0) return; - const tag = (hasHeader && targetRow === 0 ? 'th' : 'td'); - const nextCell = table.querySelector(`${tag}[data-row="${targetRow}"][data-col="${targetCol}"]`); + const tag = (hasHeader && targetRow === headerSourceRow ? 'th' : 'td'); + const nextCell = ensureRenderedCellByCoords(targetRow, targetCol); if(nextCell){ e.preventDefault(); clearSelection(); @@ -1800,14 +1812,14 @@ document.addEventListener('keydown', e => { const { row, col } = getCellCoords(editingCell); let targetRow = row, targetCol = col; switch(e.key){ - case 'ArrowUp': targetRow = row - 1; break; - case 'ArrowDown': targetRow = row + 1; break; + case 'ArrowUp': targetRow = adjacentVisibleRow(row, -1); break; + case 'ArrowDown': targetRow = adjacentVisibleRow(row, 1); break; case 'ArrowLeft': targetCol = col - 1; break; case 'ArrowRight':targetCol = col + 1; break; } if (targetRow >= 0 && targetCol >= 0) { - const tag = (hasHeader && targetRow === 0 ? 'th' : 'td'); - const nextCell = table.querySelector(`${tag}[data-row="${targetRow}"][data-col="${targetCol}"]`); + const tag = (hasHeader && targetRow === headerSourceRow ? 'th' : 'td'); + const nextCell = ensureRenderedCellByCoords(targetRow, targetCol); if (nextCell) { const commitAndMove = () => { clearSelection(); @@ -1856,7 +1868,7 @@ document.addEventListener('keydown', e => { } const { row, col } = getCellCoords(editingCell); editingCell.blur(); - const targetRow = row + 1; + const targetRow = adjacentVisibleRow(row, 1); // Editing Enter commits and moves selection down (no auto-edit). const nextCell = ensureRenderedCellByCoords(targetRow, col); if (nextCell) { @@ -1881,13 +1893,13 @@ document.addEventListener('keydown', e => { if (bounds) { targetCol = col + (isBackward ? -1 : 1); if (!isBackward && targetCol > bounds.maxCol) { - targetRow += 1; + targetRow = adjacentVisibleRow(targetRow, 1); targetCol = bounds.minCol; } else if (isBackward && targetCol < bounds.minCol) { if (targetRow <= firstDataRow) { canMove = false; } else { - targetRow -= 1; + targetRow = adjacentVisibleRow(targetRow, -1); targetCol = bounds.maxCol; } } @@ -1926,12 +1938,12 @@ const pasteFromClipboard = () => { }; for (const type of ['copy', 'cut']) { document.addEventListener(type, e => { - if (editingCell || isTextInputTarget(e.target) || isFindWidgetTarget(e.target)) return; + if (editingCell || isTextInputTarget(e.target) || isGridControlTarget(e.target)) return; if (copySelectionToClipboard(type === 'cut')) e.preventDefault(); }); } document.addEventListener('paste', e => { - if (isTextInputTarget(e.target) || isFindWidgetTarget(e.target)) { + if (isTextInputTarget(e.target) || isGridControlTarget(e.target)) { return; } if (editingCell) { @@ -2121,9 +2133,10 @@ table.addEventListener('dblclick', e => { const copySelectionToClipboard = (cut = false) => { const bounds = getDataSelectionBounds(); if (!bounds) return false; - const { minRow, maxRow, minCol, maxCol } = bounds; + const { minCol, maxCol } = bounds; const matrix = []; - for (let row = minRow; row <= maxRow; row++) { + const rowIds = [...new Set(currentSelection.map(getDataCellCoords).filter(Boolean).map(coords => coords.row))].sort((a, b) => a - b); + for (const row of rowIds) { const values = []; for (let col = minCol; col <= maxCol; col++) { const cell = getRenderedCellByCoords(row, col); @@ -2131,21 +2144,24 @@ const copySelectionToClipboard = (cut = false) => { } matrix.push(values); } - vscode.postMessage({ type: 'copyCells', matrix, startRow: minRow, startCol: minCol, cut }); + vscode.postMessage({ type: 'copyCells', matrix, rowIds, startCol: minCol, cut }); return true; }; window.addEventListener('message', event => { const message = event.data; - if (message.type === 'prepareHiddenRowsRefresh') { + if (message.type === 'prepareViewRefresh') { if (editingCell) editingCell.blur(); persistState(); - vscode.postMessage({ type: 'hiddenRowsRefreshReady' }); + vscode.postMessage({ type: 'viewRefreshReady', session: filterSession, filterDraft: window.csvColumnFilters?.captureDraft() }); + } else if (message.type === 'columnFiltersResult') { + applyColumnFiltersResult(message); } else if (message.type === 'frozenPanes') { frozenRowCount = Number(message.rows); frozenColumnCount = Number(message.columns); window.csvFrozenPanes?.update(frozenRowCount, frozenColumnCount); } else if(message.type === 'focus'){ + if (document.activeElement?.closest?.('#filterStatus')) return; if (anchorCell) { try { anchorCell.focus({ preventScroll: true }); } catch { try { anchorCell.focus(); } catch {} } } else { @@ -2154,6 +2170,7 @@ window.addEventListener('message', event => { } else if (message.type === 'chunkData') { const requestId = Number(message.requestId); const start = Number(message.start); + if (message.session !== filterSession || Number(message.filterEpoch) !== filterEpoch) return; if (remoteChunkRequestInFlight) { if (!Number.isInteger(requestId) || requestId !== remoteChunkRequestSeq) { return; @@ -2199,6 +2216,7 @@ window.addEventListener('message', event => { scheduleFind(true); } } else if (message.type === 'pasteApplied') { + if (filterSnapshotPending || document.activeElement?.closest?.('#filterStatus')) return; const startRow = Number(message.startRow); const startCol = Number(message.startCol); const endRow = Number(message.endRow); @@ -2266,10 +2284,11 @@ window.addEventListener('message', event => { // After initial restoreState, if there's a pending edit request, perform it const maybeResumePendingEdit = () => { + if (document.activeElement?.closest?.('#filterStatus')) return; try { const st = vscode.getState() || {}; if (st && typeof st.anchorRow === 'number' && typeof st.anchorCol === 'number' && st.pendingEdit === 'detail') { - const tag = (hasHeader && st.anchorRow === 0 ? 'th' : 'td'); + const tag = (hasHeader && st.anchorRow === headerSourceRow ? 'th' : 'td'); const sel = table.querySelector(`${tag}[data-row="${st.anchorRow}"][data-col="${st.anchorCol}"]`); if (sel) { editCell(sel, undefined, 'detail'); @@ -2297,7 +2316,7 @@ window.addEventListener('blur', () => { window.addEventListener('focus', () => { setTimeout(maybeResumePendingEdit, 0); }, { passive: true }); document.addEventListener('keydown', e => { - if (findReplaceState.open) { + if (findReplaceState.open || isGridControlTarget(e.target)) { return; } if(!editingCell && e.key === 'Escape'){ @@ -2305,16 +2324,121 @@ document.addEventListener('keydown', e => { } }); -// Freeze layout never replaces cells or moves focus, including during a live edit. -if (typeof window.createFrozenPaneManager === 'function') { +// A partial filter redraw keeps the delegated table and focused controls alive. +const initializeFrozenPanes = () => { + window.csvFrozenPanes?.dispose(); + if (typeof window.createFrozenPaneManager !== 'function') return; window.csvFrozenPanes = window.createFrozenPaneManager({ table, firstRow: Number(root?.dataset?.firstrow || '0'), realRowCount: Number(root?.dataset?.realrowcount || '0'), + rowOrder: visibleRowOrder, + topInset: () => document.getElementById('filterStatus')?.getBoundingClientRect().height || 0, onNeedRows: () => loadNextChunk() }); window.csvFrozenPanes.update(frozenRowCount, frozenColumnCount); - window.addEventListener('csvChunkLoaded', () => window.csvFrozenPanes.schedule()); - window.addEventListener('resize', () => window.csvFrozenPanes.schedule()); - window.addEventListener('pagehide', () => window.csvFrozenPanes.dispose(), { once: true }); +}; +initializeFrozenPanes(); +window.addEventListener('csvChunkLoaded', () => window.csvFrozenPanes?.schedule()); +window.addEventListener('resize', () => window.csvFrozenPanes?.schedule()); +window.addEventListener('pagehide', () => window.csvFrozenPanes?.dispose(), { once: true }); +const invalidateFilterFind = () => { + clearTimeout(findDebounce); + latestFindRequestId = ++findRequestSeq; + pendingFindRequests.clear(); + clearFindHighlights(); + findMatches = []; + findMatchKeySet = new Set(); + currentMatchIndex = -1; + updateFindStatus(); + updateFindControls(); +}; +const setFilterPending = pending => { + filterSnapshotPending = pending; + table.toggleAttribute('inert', pending); + table.setAttribute('aria-busy', String(pending)); + if (pending) { + if (editingCell) editingCell.blur(); + invalidateFilterFind(); + } +}; +const submitColumnFilters = snapshot => { + if (editingCell) editingCell.blur(); + vscode.postMessage({ type: 'updateColumnFilters', ...snapshot, session: filterSession }); +}; +const applyColumnFiltersResult = message => { + if (message.session !== filterSession || !window.csvColumnFilters || message.revision !== window.csvColumnFilters.latestRevision) return; + // A newly started draft must reach the host before any replacement render. + if (editingCell) { + editingCell.blur(); + window.csvColumnFilters.retry(); + return; + } + if (!window.csvColumnFilters.receive(message)) return; + const epoch = Number(message.filterEpoch); + if (typeof message.tableHtml === 'string' && Number.isSafeInteger(epoch) && epoch >= 0 && + (!(message.error || message.errors?.length) || epoch !== filterEpoch)) { + const parsed = document.createElement('template'); + parsed.innerHTML = message.tableHtml; + const replacement = parsed.content.querySelector('table'); + if (!replacement) return; + const priorAnchor = anchorCell ? getCellCoords(anchorCell) : null; + clearSelection(); + anchorCell = rangeEndCell = startCell = endCell = null; + isSelecting = false; + selectionMode = 'cell'; + resizeState = reorderState = null; + table.style.cursor = ''; + hideDragIndicator(); + pendingEnsureTarget = null; + invalidateFilterFind(); + chunkObserver?.disconnect(); + remoteChunkRequestInFlight = false; + remoteChunkRequestedStart = -1; + remoteChunkRequestSeq += 1; + filterEpoch = epoch; + table.replaceChildren(...Array.from(replacement.childNodes)); + hasHeader = !!table.tHead; + headerSourceRow = Number(message.firstVisibleRow); + root.dataset.firstrow = String(message.firstVisibleRow); + root.dataset.realrowcount = String(message.realRowCount); + visibleRowOrder.length = 0; + for (const row of Array.isArray(message.visibleRowIds) ? message.visibleRowIds : []) visibleRowOrder.push(row); + root.dataset.roworder = JSON.stringify(visibleRowOrder); + root.dataset.filtered = message.filters?.length ? '1' : '0'; + try { csvChunks = JSON.parse(message.chunksJson || '[]'); } catch { csvChunks = []; } + remoteNextChunkStart = Number.isInteger(message.nextChunkStart) ? message.nextChunkStart : -1; + remoteHasMoreChunks = !!message.hasRemoteChunks && remoteNextChunkStart >= 0; + root.dataset.nextchunkstart = remoteNextChunkStart >= 0 ? String(remoteNextChunkStart) : ''; + root.dataset.hasmorechunks = remoteHasMoreChunks ? '1' : '0'; + if (chunkTemplate) chunkTemplate.textContent = JSON.stringify(csvChunks); + const colors = document.getElementById('__csvColumnColors'); + if (colors && typeof message.colorCss === 'string') colors.textContent = message.colorCss; + if (priorAnchor) { + const cell = getRenderedCellByCoords(priorAnchor.row, priorAnchor.col); + if (cell) { cell.classList.add('selected'); currentSelection.push(cell); anchorCell = rangeEndCell = cell; } + } + applySizeStateToRenderedCells(); + initializeFrozenPanes(); + primeChunks(); + persistState(); + replaceAll.title = message.filters?.length ? 'Replace All in matching filtered rows' : 'Replace All'; + } + setFilterPending(false); + if (findReplaceState.open && findInput.value) runFind(true); +}; +const filterStatus = document.getElementById('filterStatus'); +if (filterStatus && typeof window.createColumnFilterControls === 'function') { + window.csvColumnFilters = window.createColumnFilterControls({ + root: filterStatus, + metadata: filterMetadata, + onChange: submitColumnFilters, + onActivity: () => vscode.postMessage({ type: 'filterDraftChanged', session: filterSession }), + onPending: setFilterPending, + onLayout: () => window.csvFrozenPanes?.schedule() + }); + if (typeof ResizeObserver === 'function') new ResizeObserver(() => window.csvFrozenPanes?.schedule()).observe(filterStatus); } +for (const button of [replaceOne, replaceAll]) button.addEventListener('click', event => { + if (filterSnapshotPending) { event.preventDefault(); event.stopImmediatePropagation(); } +}, true); diff --git a/package-lock.json b/package-lock.json index 2d9813c..6ee3f22 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,7 +9,8 @@ "version": "1.3.0", "dependencies": { "font-list": "^1.5.1", - "papaparse": "^5.5.3" + "papaparse": "^5.5.3", + "re2js": "1.3.3" }, "devDependencies": { "@types/node": "^20.11.19", @@ -4527,6 +4528,12 @@ "node": ">=0.10.0" } }, + "node_modules/re2js": { + "version": "1.3.3", + "resolved": "https://registry.npmjs.org/re2js/-/re2js-1.3.3.tgz", + "integrity": "sha512-s/I5zEAo79SUK0Qw4dpZKpiMwbQ6Gz0KU2NRr7eaO4x/p2g7Vvmn3hdeXDg8VsaUjfj/ora+e9oi27LX/C9+mw==", + "license": "MIT" + }, "node_modules/read": { "version": "1.0.7", "resolved": "https://registry.npmjs.org/read/-/read-1.0.7.tgz", diff --git a/package.json b/package.json index b31b643..f8188c1 100644 --- a/package.json +++ b/package.json @@ -257,7 +257,8 @@ }, "dependencies": { "font-list": "^1.5.1", - "papaparse": "^5.5.3" + "papaparse": "^5.5.3", + "re2js": "1.3.3" }, "devDependencies": { "@types/node": "^20.11.19", diff --git a/src/CsvEditorProvider.ts b/src/CsvEditorProvider.ts index e1429be..158f7db 100644 --- a/src/CsvEditorProvider.ts +++ b/src/CsvEditorProvider.ts @@ -2,6 +2,8 @@ import Papa from 'papaparse'; import * as vscode from 'vscode'; import * as path from 'path'; import { parseClipboardMatrix, serializeClipboardMatrix } from './clipboard'; +import { ColumnFilter, normalizeColumnFilters, compileColumnFilters, visibleSourceRows, remapColumnFilters, getFilterOperatorChoices, validateColumnFilterInput, MAX_COLUMN_FILTERS, MAX_FILTER_VALUE_LENGTH } from './column-filters'; +import { FilterDataType, parseFilterDate } from './filter-values'; type FrozenPanes = { rows: number; columns: number }; @@ -37,7 +39,9 @@ type PasteApplyResult = { plan: PastePlan; }; type ChunkRenderState = { - startAbs: number; + rowIds: number[]; + bodyStart: number; + virtualRow: number; allRows: string[][]; allRowsCount: number; chunkRows: number; @@ -50,6 +54,25 @@ type ChunkRenderState = { isDark: boolean; serialIndexWidthCh: number; }; +type FilterColumn = { column: number; label: string; dataType: FilterDataType; operators: ReturnType }; +type FilterDraft = { + session: string; + filterColumnsRevision: number; + rows: Array<{ column: number; operator: ColumnFilter['operator']; value: string; composing?: boolean }>; + columns: Array<{ column: number; label: string }>; + focus?: { + row: number; + field: 'column' | 'operator' | 'value' | 'remove'; + selectionStart?: number; + selectionEnd?: number; + selectionDirection?: 'forward' | 'backward' | 'none'; + }; +}; +type FilterRender = { + tableHtml: string; chunksJson: string; colorCss: string; nextChunkStart: number; hasRemoteChunks: boolean; + visibleRowIds: number[]; matchedRows: number; totalRows: number; filters: ColumnFilter[]; columns: FilterColumn[]; + firstVisibleRow: number; realRowCount: number; headerEnabled: boolean; chunkState: ChunkRenderState | undefined; +}; type ChunkResponse = { html: string; nextStart: number; @@ -74,6 +97,15 @@ class CsvEditorController { private isDiffContext = false; private currentHeader = false; private chunkRenderState: ChunkRenderState | undefined; + private currentFilterView: FilterRender | undefined; + private latestFilterRevision = 0; + private filterEpoch = 0; + private filterSession = ''; + private filterDraftActivityRevision = 0; + private filterSessionUri = ''; + private filterColumnsRevision = 0; + private pendingFilterDraft: { uri: string; draft: FilterDraft } | undefined; + private failedCellEdits = new Set(); constructor(private readonly context: vscode.ExtensionContext) {} @@ -131,7 +163,16 @@ class CsvEditorController { } }); - webviewPanel.webview.onDidReceiveMessage(e => this.enqueueOperation(async () => { + webviewPanel.webview.onDidReceiveMessage(e => { + if (e.type === 'filterDraftChanged') { + if (e.session === this.filterSession) { this.filterDraftActivityRevision += 1; } + return; + } + if (e.type === 'updateColumnFilters' && e.session === this.filterSession && Number.isSafeInteger(e.revision) && e.revision > this.latestFilterRevision) { + this.latestFilterRevision = e.revision; + } + const receivedFilterDraftRevision = this.filterDraftActivityRevision; + return this.enqueueOperation(async () => { switch (e.type) { case 'editCell': await this.updateDocument(e.row, e.col, e.value); @@ -143,7 +184,7 @@ class CsvEditorController { await this.pasteCells(e.text, e.anchorRow, e.anchorCol, e.selection); break; case 'requestChunk': - await this.requestChunk(e.start, e.requestId); + await this.requestChunk(e.start, e.requestId, e.filterEpoch, e.session); break; case 'findMatches': await this.findMatches(e.requestId, e.query, e.options); @@ -151,7 +192,16 @@ class CsvEditorController { case 'save': await this.handleSave(); break; - case 'hiddenRowsRefreshReady': + case 'updateColumnFilters': + await this.updateColumnFilters(e.revision, e.filters, e.session); + break; + case 'viewRefreshReady': + if (e.session !== this.filterSession) { break; } + if (receivedFilterDraftRevision !== this.filterDraftActivityRevision) { + void webviewPanel.webview.postMessage({ type: 'prepareViewRefresh' }); + break; + } + this.captureFilterDraft(e.filterDraft); this.updateWebviewContent(); break; case 'freezePanesChanged': @@ -161,7 +211,7 @@ class CsvEditorController { await CsvEditorProvider.setColumnSizesForUri(this.context, this.document.uri, e.columnSizes); break; case 'copyCells': - await this.copyCells(e.matrix, e.startRow, e.startCol, e.cut === true); + await this.copyCells(e.matrix, e.rowIds, e.startCol, e.cut === true); break; case 'pasteFromClipboard': await this.pasteCells(await vscode.env.clipboard.readText(), e.anchorRow, e.anchorCol, e.selection); @@ -203,7 +253,8 @@ class CsvEditorController { await this.openLinkExternally(e.url); break; } - })); + }); + }); const changeDocumentSubscription = vscode.workspace.onDidChangeTextDocument(e => { if ( @@ -228,6 +279,7 @@ class CsvEditorController { saveDocumentSubscription.dispose(); CsvEditorProvider.editors = CsvEditorProvider.editors.filter(ed => ed !== this); this.currentWebviewPanel = undefined; + this.pendingFilterDraft = undefined; }); } @@ -425,9 +477,172 @@ class CsvEditorController { ); } - public refreshHiddenRows(): void { + private getFilterColumns(data: string[][]): FilterColumn[] { + const offset = Math.min(Math.max(0, this.getHiddenRows()), data.length); + const header = this.getEffectiveHeader(data, offset); + const body = data.slice(offset + (header ? 1 : 0)); + const count = Math.max(1, data.reduce((max, row) => Math.max(max, row.length), 0)); + return Array.from({ length: count }, (_, column) => { + const values = body.map(row => row[column] ?? ''); + const inferred = this.estimateColumnDataType(values); + let dataType: FilterDataType = inferred === 'integer' || inferred === 'float' ? 'number' + : inferred === 'date' || inferred === 'boolean' ? inferred : 'string'; + if (inferred === 'string') { + const nonblank = values.filter(value => value.trim() !== ''); + if (nonblank.length && nonblank.every(value => parseFilterDate(value) !== undefined)) { + dataType = 'date'; + } + } + return { column, label: (header ? data[offset]?.[column] : '') || `Column ${column + 1}`, dataType, operators: getFilterOperatorChoices(dataType) }; + }); + } + + private filterResponse(): Record { + if (!this.currentFilterView) { return { filterEpoch: this.filterEpoch }; } + const { chunkState, ...payload } = this.currentFilterView; + return { ...payload, filterEpoch: this.filterEpoch }; + } + + private async updateColumnFilters(rawRevision: unknown, rawFilters: unknown, session: unknown): Promise { + const panel = this.currentWebviewPanel; + if (!panel || session !== this.filterSession) { return; } + const reply = (extra: Record) => { + if (session !== this.filterSession || panel !== this.currentWebviewPanel) { return; } + void panel.webview.postMessage({ type: 'columnFiltersResult', session: this.filterSession, revision: rawRevision, ...this.filterResponse(), ...extra }); + }; + if (typeof rawRevision !== 'number' || !Number.isSafeInteger(rawRevision) || rawRevision <= 0) { + reply({ error: 'Filter revisions must be positive safe integers.' }); + return; + } + if (rawRevision !== this.latestFilterRevision || rawRevision <= this.filterEpoch) { return; } + if (this.failedCellEdits.size) { + reply({ error: 'Retry the failed cell edit before changing filters.' }); + return; + } + if (!Array.isArray(rawFilters) || rawFilters.length > MAX_COLUMN_FILTERS) { + reply({ error: `Provide at most ${MAX_COLUMN_FILTERS} filter conditions.` }); + return; + } + const data = this.trimTrailingEmptyRows(Papa.parse(this.document.getText(), { delimiter: this.getSeparator() }).data); + const columns = this.getFilterColumns(data); + const filters: ColumnFilter[] = []; + const errors: Array<{ index: number; message: string }> = []; + rawFilters.forEach((raw: unknown, index: number) => { + if (!raw || typeof raw !== 'object' || Array.isArray(raw)) { + errors.push({ index, message: 'Provide a valid filter condition.' }); + return; + } + const item = raw as Record; + if (Object.keys(item).some(key => !['column', 'operator', 'value', 'dataType'].includes(key)) || + typeof item.column !== 'number' || !Number.isSafeInteger(item.column) || item.column < 0 || item.column >= columns.length || + typeof item.operator !== 'string' || typeof item.value !== 'string' || item.value.length > MAX_FILTER_VALUE_LENGTH || + (item.dataType !== undefined && (typeof item.dataType !== 'string' || !['string', 'number', 'date', 'boolean'].includes(item.dataType)))) { + errors.push({ index, message: 'Provide a valid column, operator, type and filter value.' }); + return; + } + const column = columns[item.column]; + const choice = column.operators.find(operator => operator.operator === item.operator); + if (!choice) { + errors.push({ index, message: `Choose an operator supported by this ${column.dataType} column.` }); + return; + } + const typed = !['contains', 'notContains', 'regex', 'empty', 'notEmpty'].includes(choice.operator); + const dataType = typed && column.dataType !== 'string' ? column.dataType : undefined; + // Empty text boxes remain local drafts, including incomplete typed queries. + if (choice.operator !== 'empty' && choice.operator !== 'notEmpty' && item.value.length === 0) { return; } + const error = validateColumnFilterInput(choice.operator, item.value, dataType); + if (error) { errors.push({ index, message: error }); return; } + filters.push({ column: item.column, operator: choice.operator, value: item.value, ...(dataType ? { dataType } : {}) }); + }); + if (errors.length) { reply({ errors, columns }); return; } + const view = this.buildFilterView(data, filters); + if (session !== this.filterSession || rawRevision !== this.latestFilterRevision) { return; } + const previous = CsvEditorProvider.getColumnFiltersForUri(this.context, this.document.uri); + try { + await CsvEditorProvider.setColumnFiltersForUri(this.context, this.document.uri, filters); + if (session !== this.filterSession || rawRevision !== this.latestFilterRevision) { + await CsvEditorProvider.setColumnFiltersForUri(this.context, this.document.uri, previous); + return; + } + } catch { + reply({ error: 'The filter conditions could not be saved. Try again.' }); + return; + } + this.chunkRenderState = view.chunkState; + this.currentFilterView = view; + this.currentHeader = view.headerEnabled; + this.filterEpoch = rawRevision; + reply({}); + } + + private captureFilterDraft(raw: unknown): void { + this.pendingFilterDraft = undefined; + const columns = this.currentFilterView?.columns; + if (!columns || !raw || typeof raw !== 'object' || Array.isArray(raw)) { return; } + const item = raw as Record; + if (item.session !== this.filterSession || item.filterColumnsRevision !== this.filterColumnsRevision || + !Array.isArray(item.rows) || item.rows.length > MAX_COLUMN_FILTERS || + !Array.isArray(item.columns) || item.columns.length !== columns.length) { return; } + if (!item.columns.every((column: unknown, index: number) => { + if (!column || typeof column !== 'object' || Array.isArray(column)) { return false; } + const identity = column as Record; + return identity.column === columns[index].column && identity.label === columns[index].label; + })) { return; } + const rows: FilterDraft['rows'] = []; + for (const rawRow of item.rows as unknown[]) { + if (!rawRow || typeof rawRow !== 'object' || Array.isArray(rawRow)) { return; } + const row = rawRow as Record; + const operator = getFilterOperatorChoices('number').find(choice => choice.operator === row.operator)?.operator; + if (typeof row.column !== 'number' || !Number.isSafeInteger(row.column) || row.column < 0 || row.column >= columns.length || + !operator || typeof row.value !== 'string' || row.value.length > MAX_FILTER_VALUE_LENGTH || + (row.composing !== undefined && typeof row.composing !== 'boolean')) { return; } + // Preserve blank and invalid queries as drafts; only submission validates their meaning. + rows.push({ column: row.column, operator, value: row.value, ...(row.composing ? { composing: true } : {}) }); + } + const draft: FilterDraft = { + session: this.filterSession, + filterColumnsRevision: this.filterColumnsRevision, + rows, + columns: columns.map(column => ({ column: column.column, label: column.label })) + }; + if (item.focus !== undefined) { + if (!item.focus || typeof item.focus !== 'object' || Array.isArray(item.focus)) { return; } + const focus = item.focus as Record; + if (typeof focus.row !== 'number' || !Number.isSafeInteger(focus.row) || focus.row < 0 || focus.row >= rows.length || + (focus.field !== 'column' && focus.field !== 'operator' && focus.field !== 'value' && focus.field !== 'remove')) { return; } + draft.focus = { row: focus.row, field: focus.field }; + if (focus.selectionStart !== undefined || focus.selectionEnd !== undefined || focus.selectionDirection !== undefined) { + if (focus.field !== 'value' || typeof focus.selectionStart !== 'number' || typeof focus.selectionEnd !== 'number' || + !Number.isSafeInteger(focus.selectionStart) || !Number.isSafeInteger(focus.selectionEnd) || + focus.selectionStart < 0 || focus.selectionEnd < focus.selectionStart || focus.selectionEnd > rows[focus.row].value.length || + (focus.selectionDirection !== 'forward' && focus.selectionDirection !== 'backward' && focus.selectionDirection !== 'none')) { return; } + Object.assign(draft.focus, { selectionStart: focus.selectionStart, selectionEnd: focus.selectionEnd, selectionDirection: focus.selectionDirection }); + } + } + this.pendingFilterDraft = { uri: this.filterSessionUri, draft }; + } + + private consumeFilterDraft(columns: FilterColumn[]): FilterDraft | undefined { + const pending = this.pendingFilterDraft; + this.pendingFilterDraft = undefined; + if (!pending || pending.uri !== this.document.uri.toString() || pending.draft.filterColumnsRevision !== this.filterColumnsRevision || + pending.draft.columns.length !== columns.length || pending.draft.columns.some((column, index) => + column.column !== columns[index].column || column.label !== columns[index].label)) { return undefined; } + return pending.draft; + } + + private async remapFilters(columnOrder: number[]): Promise { + if (columnOrder.length !== this.currentFilterView?.columns.length || columnOrder.some((column, index) => column !== index)) { + this.filterColumnsRevision++; + this.pendingFilterDraft = undefined; + } + await CsvEditorProvider.setColumnFiltersForUri(this.context, this.document.uri, + remapColumnFilters(CsvEditorProvider.getColumnFiltersForUri(this.context, this.document.uri), columnOrder)); + } + + public refreshViewSafely(): void { if (this.currentWebviewPanel?.visible) { - this.currentWebviewPanel.webview.postMessage({ type: 'prepareHiddenRowsRefresh' }); + this.currentWebviewPanel.webview.postMessage({ type: 'prepareViewRefresh' }); } else { this.refresh(); } @@ -455,7 +670,9 @@ class CsvEditorController { private async updateDocument(row: number, col: number, value: string) { this.isUpdatingDocument = true; let structuralChange = false; + let filterMembershipChanged = false; let applied = false; + const editKey = `${row}:${col}`; try { const separator = this.getSeparator(); const oldText = this.document.getText(); @@ -468,10 +685,15 @@ class CsvEditorController { ? String(data[row][col] ?? '') : undefined; + const filters = CsvEditorProvider.getColumnFiltersForUri(this.context, this.document.uri); + const matches = compileColumnFilters(filters); + const previouslyMatched = matches(data[row] ?? []); const { data: nextData, trimmed, createdRow, createdCol } = this.mutateDataForEdit(data, row, col, value); + filterMembershipChanged = filters.length > 0 && previouslyMatched !== matches(nextData[row] ?? []); structuralChange = !!(trimmed || createdRow || createdCol || row >= hadRows || col >= hadColsAtRow); if (!structuralChange && previousValue === value) { + this.failedCellEdits.delete(editKey); return; } @@ -482,12 +704,15 @@ class CsvEditorController { separator, [{ row, col, value: String(value ?? '') }] ); + } else { + newCsvText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(oldText, separator, nextData); } if (newCsvText === undefined) { newCsvText = Papa.unparse(nextData, { delimiter: separator }); } if (newCsvText === oldText) { + this.failedCellEdits.delete(editKey); return; } @@ -499,22 +724,29 @@ class CsvEditorController { const edit = new vscode.WorkspaceEdit(); edit.replace(this.document.uri, fullRange, newCsvText); applied = await vscode.workspace.applyEdit(edit); + } catch (error) { + this.failedCellEdits.add(editKey); + throw error; } finally { this.isUpdatingDocument = false; } if (!applied) { + this.failedCellEdits.add(editKey); return; } + this.failedCellEdits.delete(editKey); - console.log(`CSV: Updated row ${row + 1}, column ${col + 1} to "${value}"`); + console.log(`CSV: Updated row ${row + 1}, column ${col + 1}`); const config = vscode.workspace.getConfiguration('csv', this.document.uri); const clickableLinks = config.get('clickableLinks', true); const rendered = this.formatCellContent(value ?? '', clickableLinks); this.currentWebviewPanel?.webview.postMessage({ type: 'updateCell', row, col, value, rendered }); // Trigger a full re-render if structure may have changed (new row/col created) - if (structuralChange) { + if (filterMembershipChanged) { + this.refreshViewSafely(); + } else if (structuralChange) { try { this.updateWebviewContent(); } catch (e) { console.error('CSV: refresh failed after structural edit', e); } } } @@ -583,16 +815,16 @@ class CsvEditorController { } } - private async copyCells(rawMatrix: unknown, rawStartRow: unknown, rawStartCol: unknown, cut: boolean): Promise { + private async copyCells(rawMatrix: unknown, rawRowIds: unknown, rawStartCol: unknown, cut: boolean): Promise { if (!Array.isArray(rawMatrix) || rawMatrix.length === 0 || !rawMatrix.every(row => Array.isArray(row) && row.length > 0 && row.every(value => typeof value === 'string') )) { return; } const matrix = rawMatrix as string[][]; - const startRow = Number(rawStartRow); + const rowIds = Array.isArray(rawRowIds) ? rawRowIds : []; const startCol = Number(rawStartCol); - if (!Number.isInteger(startRow) || startRow < 0 || !Number.isInteger(startCol) || startCol < 0) { + if (rowIds.length !== matrix.length || new Set(rowIds).size !== rowIds.length || !rowIds.every(row => Number.isInteger(row) && row >= 0) || !Number.isInteger(startCol) || startCol < 0) { return; } const documentVersion = this.document.version; @@ -604,7 +836,7 @@ class CsvEditorController { } if (cut && this.document.version === documentVersion) { await this.replaceCells(matrix.flatMap((row, rowOffset) => row.map((_value, colOffset) => ({ - row: startRow + rowOffset, + row: rowIds[rowOffset], col: startCol + colOffset, value: '' })))); @@ -675,7 +907,8 @@ class CsvEditorController { matrix: string[][], anchorRow: number, anchorCol: number, - selection: PasteSelectionBounds | undefined + selection: PasteSelectionBounds | undefined, + targetRows?: number[] ): PasteApplyResult { const plan = CsvEditorController.computePastePlan(matrix, anchorRow, anchorCol, selection); if (!plan) { @@ -693,6 +926,11 @@ class CsvEditorController { }; } + const rowsToWrite = targetRows ?? Array.from({ length: plan.endRow - plan.startRow + 1 }, (_, index) => plan.startRow + index); + if (rowsToWrite.length) { + plan.startRow = rowsToWrite[0]; + plan.endRow = rowsToWrite[rowsToWrite.length - 1]; + } const updates: Array<{ row: number; col: number; value: string }> = []; let changed = false; let structuralChange = false; @@ -720,7 +958,7 @@ class CsvEditorController { if (plan.fillSelection) { const value = String(matrix[0][0] ?? ''); - for (let row = plan.startRow; row <= plan.endRow; row++) { + for (const row of rowsToWrite) { for (let col = plan.startCol; col <= plan.endCol; col++) { setCellValue(row, col, value); } @@ -728,7 +966,7 @@ class CsvEditorController { } else { for (let r = 0; r < matrix.length; r++) { for (let c = 0; c < matrix[r].length; c++) { - setCellValue(plan.startRow + r, plan.startCol + c, String(matrix[r][c] ?? '')); + setCellValue(rowsToWrite[r], plan.startCol + c, String(matrix[r][c] ?? '')); } } } @@ -762,9 +1000,21 @@ class CsvEditorController { const separator = this.getSeparator(); const oldText = this.document.getText(); const result = Papa.parse(oldText, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; - - const pasteResult = CsvEditorController.applyPasteMatrixToData(data, matrix, anchorRow, anchorCol, selection); + const data = this.trimTrailingEmptyRows(result.data as string[][]); + + const realData = this.trimTrailingEmptyRows(data); + const filters = normalizeColumnFilters(CsvEditorProvider.getColumnFiltersForUri(this.context, this.document.uri), data.reduce((max, row) => Math.max(max, row.length), 0)); + const visibleRows = [...visibleSourceRows(realData, this.getHiddenRows(), this.getEffectiveHeader(realData, this.getHiddenRows()), filters), realData.length]; + const anchorIndex = visibleRows.indexOf(anchorRow); + if (anchorIndex < 0) { + void vscode.window.showInformationMessage('CSV: Select a visible cell and paste again.'); + return; + } + const plan = CsvEditorController.computePastePlan(matrix, anchorRow, anchorCol, selection); + const targetRows = plan?.fillSelection + ? visibleRows.filter(row => row >= plan.startRow && row <= plan.endRow) + : matrix.map((_row, index) => visibleRows[anchorIndex + index] ?? realData.length + anchorIndex + index - visibleRows.length + 1); + const pasteResult = CsvEditorController.applyPasteMatrixToData(data, matrix, anchorRow, anchorCol, selection, targetRows); if (!pasteResult.changed) { return; } @@ -772,6 +1022,8 @@ class CsvEditorController { let newCsvText: string | undefined; if (!pasteResult.structuralChange) { newCsvText = CsvEditorProvider.applyFieldUpdatesPreservingFormat(oldText, separator, pasteResult.updates); + } else { + newCsvText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(oldText, separator, this.trimTrailingEmptyRows(data)); } if (newCsvText === undefined) { newCsvText = Papa.unparse(data, { delimiter: separator }); @@ -810,8 +1062,8 @@ class CsvEditorController { if (start < state.allRowsCount) { const end = Math.min(start + state.chunkRows, state.allRowsCount); const html = state.allRows.slice(start, end).map((row, localR) => { - const absRow = state.startAbs + start + localR; - const displayIdx = start + localR + 1; + const absRow = state.rowIds[start + localR]; + const displayIdx = absRow - state.bodyStart + 1; let cells = ''; for (let cIdx = 0; cIdx < state.numColumns; cIdx++) { const rawValue = row[cIdx] || ''; @@ -835,8 +1087,8 @@ class CsvEditorController { } if (start === state.allRowsCount && state.includeTrailingEmptyRow) { - const virtualAbs = state.startAbs + state.allRowsCount; - const displayIdx = state.allRowsCount + 1; + const virtualAbs = state.virtualRow; + const displayIdx = virtualAbs - state.bodyStart + 1; const idxCell = state.addSerialIndex ? `${displayIdx}` : ''; @@ -847,14 +1099,16 @@ class CsvEditorController { return { html: '', nextStart: -1, done: true }; } - private async requestChunk(rawStart: unknown, requestId: unknown): Promise { - if (!this.currentWebviewPanel || !this.chunkRenderState) { + private async requestChunk(rawStart: unknown, requestId: unknown, filterEpoch: unknown, session: unknown): Promise { + if (!this.currentWebviewPanel || !this.chunkRenderState || filterEpoch !== this.filterEpoch || session !== this.filterSession) { return; } const start = Number(rawStart); if (!Number.isInteger(start) || start < 0) { this.currentWebviewPanel.webview.postMessage({ type: 'chunkData', + session: this.filterSession, + filterEpoch: this.filterEpoch, requestId, start: -1, html: '', @@ -866,6 +1120,8 @@ class CsvEditorController { const response = this.renderChunkFromState(this.chunkRenderState, start); this.currentWebviewPanel.webview.postMessage({ type: 'chunkData', + session: this.filterSession, + filterEpoch: this.filterEpoch, requestId, start, html: response.html, @@ -935,7 +1191,8 @@ class CsvEditorController { const offset = Math.min(Math.max(0, hiddenRows), data.length); const matches: Array<{ row: number; col: number; value: string }> = []; - for (let row = offset; row < data.length; row++) { + const filters = normalizeColumnFilters(CsvEditorProvider.getColumnFiltersForUri(this.context, this.document.uri), data.reduce((max, row) => Math.max(max, row.length), 0)); + for (const row of visibleSourceRows(data, offset, this.getEffectiveHeader(data, hiddenRows), filters)) { const current = data[row] || []; for (let col = 0; col < current.length; col++) { const value = String(current[col] ?? ''); @@ -1016,14 +1273,17 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = Array.from({ length: data.reduce((max, row) => Math.max(max, row.length), 0) }, (_, column) => column); for (const row of data) { if (index > row.length) { while (row.length < index) row.push(''); } row.splice(index, 0, ''); } - const newText = Papa.unparse(data, { delimiter: separator }); + order.splice(index, 0, -1); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, undefined, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1033,6 +1293,7 @@ class CsvEditorController { edit.replace(this.document.uri, fullRange, newText); await vscode.workspace.applyEdit(edit); this.isUpdatingDocument = false; + await this.remapFilters(order); this.updateWebviewContent(); } @@ -1042,7 +1303,8 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = Array.from({ length: data.reduce((max, row) => Math.max(max, row.length), 0) }, (_, column) => column); for (let k = 0; k < count; k++) { for (const row of data) { if (index > row.length) { @@ -1051,7 +1313,9 @@ class CsvEditorController { row.splice(index, 0, ''); } } - const newText = Papa.unparse(data, { delimiter: separator }); + order.splice(index, 0, ...Array(count).fill(-1)); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, undefined, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1061,6 +1325,7 @@ class CsvEditorController { edit.replace(this.document.uri, fullRange, newText); await vscode.workspace.applyEdit(edit); this.isUpdatingDocument = false; + await this.remapFilters(order); this.updateWebviewContent(); } @@ -1069,13 +1334,16 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = Array.from({ length: data.reduce((max, row) => Math.max(max, row.length), 0) }, (_, column) => column); for (const row of data) { if (index < row.length) { row.splice(index, 1); } } - const newText = Papa.unparse(data, { delimiter: separator }); + order.splice(index, 1); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, undefined, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1085,6 +1353,7 @@ class CsvEditorController { edit.replace(this.document.uri, fullRange, newText); await vscode.workspace.applyEdit(edit); this.isUpdatingDocument = false; + await this.remapFilters(order); this.updateWebviewContent(); } @@ -1094,7 +1363,8 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = Array.from({ length: data.reduce((max, row) => Math.max(max, row.length), 0) }, (_, column) => column); const sorted = [...indices].sort((a,b)=>b-a); for (const idx of sorted) { for (const row of data) { @@ -1103,7 +1373,9 @@ class CsvEditorController { } } } - const newText = Papa.unparse(data, { delimiter: separator }); + for (const column of sorted) { order.splice(column, 1); } + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, undefined, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1113,6 +1385,7 @@ class CsvEditorController { edit.replace(this.document.uri, fullRange, newText); await vscode.workspace.applyEdit(edit); this.isUpdatingDocument = false; + await this.remapFilters(order); this.updateWebviewContent(); } @@ -1163,26 +1436,22 @@ class CsvEditorController { return sa.localeCompare(sb, undefined, { sensitivity: 'base' }); }; - body.sort((r1, r2) => { + const filters = normalizeColumnFilters(CsvEditorProvider.getColumnFiltersForUri(this.context, this.document.uri), rows.reduce((max, row) => Math.max(max, row.length), 0)); + const matches = compileColumnFilters(filters); + const matchingSlots = body.map((_row, slot) => slot).filter(slot => matches(body[slot])); + const sortedMatches = matchingSlots.map(slot => body[slot]).sort((r1, r2) => { const diff = cmp(r1[index] ?? '', r2[index] ?? ''); return ascending ? diff : -diff; }); + matchingSlots.forEach((slot, order) => { body[slot] = sortedMatches[order]; }); const prefix = rows.slice(0, offset); const combined = treatHeader ? [...prefix, header, ...body] : [...prefix, ...body]; - // Sanitize before unparse: ensure undefined/null/NaN become empty strings - const sanitized: string[][] = combined.map(r => r.map((v: any) => { - if (v === undefined || v === null) return ''; - const t = typeof v; - if (t === 'number') { - return Number.isNaN(v) ? '' : String(v); - } - const s = String(v); - return s.toLowerCase() === 'nan' ? '' : s; - })); - - const newCsv = Papa.unparse(sanitized, { delimiter: separator }); + const originalIndex = new Map(rows.map((row, position) => [row, position])); + const rowOrder = combined.map(row => originalIndex.get(row)!); + const newCsv = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, combined, rowOrder) + ?? Papa.unparse(combined, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, @@ -1204,14 +1473,17 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = data.map((_row, position) => position); const numColumns = data.reduce((max, r) => Math.max(max, r.length), 0); const newRow = Array(numColumns).fill(''); if (index > data.length) { while (data.length < index) data.push(Array(numColumns).fill('')); } data.splice(index, 0, newRow); - const newText = Papa.unparse(data, { delimiter: separator }); + order.splice(index, 0, -1); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1230,7 +1502,8 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = data.map((_row, position) => position); const numColumns = data.reduce((max, r) => Math.max(max, r.length), 0); for (let k = 0; k < count; k++) { const newRow = Array(numColumns).fill(''); @@ -1239,7 +1512,9 @@ class CsvEditorController { } data.splice(index, 0, newRow); } - const newText = Papa.unparse(data, { delimiter: separator }); + order.splice(index, 0, ...Array(count).fill(-1)); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1257,11 +1532,14 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = data.map((_row, position) => position); if (index < data.length) { data.splice(index, 1); } - const newText = Papa.unparse(data, { delimiter: separator }); + order.splice(index, 1); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1280,14 +1558,17 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); + const order = data.map((_row, position) => position); const sorted = [...indices].sort((a,b)=>b-a); for (const idx of sorted) { if (idx < data.length) { data.splice(idx, 1); } } - const newText = Papa.unparse(data, { delimiter: separator }); + for (const row of sorted) { order.splice(row, 1); } + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, data, order) + ?? Papa.unparse(data, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1349,7 +1630,7 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); const numColumns = data.reduce((max, row) => Math.max(max, row.length), 0); if (numColumns <= 0) return; @@ -1360,13 +1641,13 @@ class CsvEditorController { const reorderedData = data.map(row => { const normalized = Array.from({ length: numColumns }, (_, i) => row[i] ?? ''); const next = columnOrder.map(colIdx => normalized[colIdx] ?? ''); - while (next.length > 0 && next[next.length - 1] === '') { - next.pop(); - } - return next; + // Keep explicit empty fields; omit only positions absent from the source row. + const lastPresent = columnOrder.reduce((last, colIdx, index) => colIdx < row.length ? index : last, -1); + return next.slice(0, lastPresent + 1); }); - const newText = Papa.unparse(reorderedData, { delimiter: separator }); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, reorderedData, undefined, columnOrder) + ?? Papa.unparse(reorderedData, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1375,6 +1656,7 @@ class CsvEditorController { const edit = new vscode.WorkspaceEdit(); edit.replace(this.document.uri, fullRange, newText); await vscode.workspace.applyEdit(edit); + await this.remapFilters(columnOrder); this.updateWebviewContent(); } finally { this.isUpdatingDocument = false; @@ -1387,13 +1669,15 @@ class CsvEditorController { const separator = this.getSeparator(); const text = this.document.getText(); const result = Papa.parse(text, { dynamicTyping: false, delimiter: separator }); - const data = result.data as string[][]; + const data = this.trimTrailingEmptyRows(result.data as string[][]); if (!data.length) return; - const { reordered, changed } = this.reorderByIndices(data, indices, beforeIndex); + const { reordered: rowOrder, changed } = this.reorderByIndices(data.map((_, index) => index), indices, beforeIndex); if (!changed) return; - const newText = Papa.unparse(reordered, { delimiter: separator }); + const reordered = rowOrder.map(index => data[index]); + const newText = CsvEditorProvider.applyStructuralUpdatesPreservingFormat(text, separator, reordered, rowOrder) + ?? Papa.unparse(reordered, { delimiter: separator }); const fullRange = new vscode.Range( 0, 0, this.document.lineCount, @@ -1438,34 +1722,20 @@ class CsvEditorController { const cellPadding = config.get('cellPadding', 4); const data = this.trimTrailingEmptyRows((parsed.data || []) as string[][]); - const treatHeader = this.getEffectiveHeader(data, hiddenRows); - this.currentHeader = treatHeader; - const clickableLinks = config.get('clickableLinks', true); - const configuredColumnColorMode = config.get('columnColorMode', 'type'); - const diffUseThemeForeground = config.get('diffUseThemeForeground', true); - const columnColorMode = CsvEditorController.resolveEffectiveColumnColorMode( - configuredColumnColorMode, - this.isDiffContext, - diffUseThemeForeground - ); - const columnColorPalette = config.get('columnColorPalette', 'default'); - const showTrailingEmptyRow = config.get('showTrailingEmptyRow', true); + const filters = normalizeColumnFilters(CsvEditorProvider.getColumnFiltersForUri(this.context, this.document.uri), data.reduce((max, row) => Math.max(max, row.length), 0)); + const view = this.buildFilterView(data, filters); + const filterDraft = this.consumeFilterDraft(view.columns); + this.currentHeader = view.headerEnabled; + this.chunkRenderState = view.chunkState; + this.currentFilterView = view; + this.latestFilterRevision = 0; + this.filterEpoch = 0; + this.filterSession = this.getNonce(); + this.filterDraftActivityRevision = 0; + this.filterSessionUri = this.document.uri.toString(); const mouseWheelZoomEnabled = config.get('mouseWheelZoom', true); const mouseWheelZoomInvert = config.get('mouseWheelZoomInvert', false); - - const { tableHtml, chunksJson, colorCss, nextChunkStart, hasRemoteChunks, chunkState } = - this.generateTableAndChunks( - data, - treatHeader, - addSerialIndex, - hiddenRows, - clickableLinks, - columnColorMode, - columnColorPalette, - showTrailingEmptyRow, - /* maxSerializedChunks */ 0 - ); - this.chunkRenderState = chunkState; + const { tableHtml, chunksJson, colorCss, nextChunkStart, hasRemoteChunks } = view; const nonce = this.getNonce(); @@ -1485,11 +1755,31 @@ class CsvEditorController { mouseWheelZoomInvert, columnSizes: CsvEditorProvider.getColumnSizesForUri(this.context, this.document.uri), frozenPanes: this.getCurrentFrozenPanes(), - firstVisibleRow: hiddenRows, - realRowCount: data.length + firstVisibleRow: view.firstVisibleRow, + realRowCount: data.length, + visibleRowIds: view.visibleRowIds, + filterMetadata: { session: this.filterSession, filterColumnsRevision: this.filterColumnsRevision, filters, columns: view.columns, matchedRows: view.matchedRows, totalRows: view.totalRows, maxFilters: MAX_COLUMN_FILTERS, revision: 0, ...(filterDraft ? { filterDraft } : {}) } }); } + private buildFilterView(data: string[][], filters: ColumnFilter[]): FilterRender { + const config = vscode.workspace.getConfiguration('csv', this.document.uri); + const hiddenRows = Math.min(Math.max(0, this.getHiddenRows()), data.length); + const headerEnabled = this.getEffectiveHeader(data, hiddenRows); + const columnColorMode = CsvEditorController.resolveEffectiveColumnColorMode( + config.get('columnColorMode', 'type'), this.isDiffContext, config.get('diffUseThemeForeground', true)); + const render = this.generateTableAndChunks(data, headerEnabled, + CsvEditorProvider.getSerialIndexForUri(this.context, this.document.uri), hiddenRows, + config.get('clickableLinks', true), columnColorMode, config.get('columnColorPalette', 'default'), + config.get('showTrailingEmptyRow', true), 0, filters); + const visible = visibleSourceRows(data, hiddenRows, headerEnabled, filters); + return { ...render, filters, columns: this.getFilterColumns(data), headerEnabled, + firstVisibleRow: hiddenRows, realRowCount: data.length, + visibleRowIds: filters.length ? [...visible, data.length] : [], + matchedRows: visible.filter(row => !(headerEnabled && row === hiddenRows)).length, + totalRows: Math.max(0, data.length - hiddenRows - (headerEnabled ? 1 : 0)) }; + } + private generateTableAndChunks( data: string[][], treatHeader: boolean, @@ -1499,7 +1789,8 @@ class CsvEditorController { columnColorMode: string, columnColorPalette: string, showTrailingEmptyRow: boolean, - maxSerializedChunks: number = Number.MAX_SAFE_INTEGER + maxSerializedChunks: number = Number.MAX_SAFE_INTEGER, + filters: ColumnFilter[] = [] ): { tableHtml: string; chunksJson: string; @@ -1525,8 +1816,12 @@ class CsvEditorController { } else { bodyData = data.slice(offset); } + const bodyStart = headerFlag ? offset + 1 : offset; + const matches = compileColumnFilters(filters); + const rowIds = bodyData.map((_row, index) => bodyStart + index).filter(row => matches(data[row])); + bodyData = rowIds.map(row => data[row]); const visibleForWidth = headerFlag ? [headerRow, ...bodyData] : bodyData; - let numColumns = visibleForWidth.reduce((max, row) => Math.max(max, row.length), 0); + let numColumns = data.reduce((max, row) => Math.max(max, row.length), 0); if (numColumns === 0) numColumns = 1; // ensure at least 1 column for the virtual row const columnData = Array.from({ length: numColumns }, (_, i) => bodyData.map(row => row[i] || '')); @@ -1543,7 +1838,7 @@ class CsvEditorController { const BASE_CHUNK_ROWS = 1000; const MAX_CELLS_PER_CHUNK = 20000; const MIN_CHUNK_ROWS = 10; - const allRows = headerFlag ? data.slice(offset + 1) : data.slice(offset); + const allRows = bodyData; const allRowsCount = allRows.length; // preserve total before any truncation const chunkRows = Math.max( MIN_CHUNK_ROWS, @@ -1552,7 +1847,7 @@ class CsvEditorController { // Always keep one editable row for fully empty views; otherwise allow disabling the // trailing virtual row via settings. const includeTrailingEmptyRow = showTrailingEmptyRow || allRowsCount === 0; - const serialIndexMaxDisplay = includeTrailingEmptyRow ? allRowsCount + 1 : allRowsCount; + const serialIndexMaxDisplay = data.length - bodyStart + (includeTrailingEmptyRow ? 1 : 0); const serialIndexWidthCh = Math.max(4, String(Math.max(1, serialIndexMaxDisplay)).length + 1); const chunks: string[] = []; const chunked = allRowsCount > chunkRows; @@ -1569,9 +1864,8 @@ class CsvEditorController { break; } const htmlChunk = allRows.slice(i, i + chunkRows).map((row, localR) => { - const startAbs = headerFlag ? offset + 1 : offset; - const absRow = startAbs + i + localR; - const displayIdx = i + localR + 1; // numbering relative to first visible data row + const absRow = rowIds[i + localR]; + const displayIdx = absRow - bodyStart + 1; // retain source numbering across filtered gaps let cells = ''; for (let cIdx = 0; cIdx < numColumns; cIdx++) { const rawValue = row[cIdx] || ''; @@ -1606,27 +1900,27 @@ class CsvEditorController { }`; for (let i = 0; i < numColumns; i++) { const safe = this.formatCellContent(headerRow[i] || '', clickableLinks); - tableHtml += `${safe}`; + tableHtml += `${safe}`; } tableHtml += ``; const initialBodyRows = chunked ? allRows.slice(0, chunkRows) : allRows; initialBodyRows.forEach((row, r) => { tableHtml += `${ addSerialIndex - ? `${r + 1}` + ? `${rowIds[r] - bodyStart + 1}` : '' }`; for (let i = 0; i < numColumns; i++) { const rawValue = row[i] || ''; const safe = this.formatCellContent(rawValue, clickableLinks); const titleAttr = this.getMultilineCellTitleAttr(rawValue); - tableHtml += `${safe}`; + tableHtml += `${safe}`; } tableHtml += ``; }); if (!chunked && includeTrailingEmptyRow) { - const virtualAbs = offset + 1 + initialBodyRows.length; - const idxCell = addSerialIndex ? `${initialBodyRows.length + 1}` : ''; + const virtualAbs = data.length; + const idxCell = addSerialIndex ? `${virtualAbs - bodyStart + 1}` : ''; const dataCells = Array.from({ length: numColumns }, (_, i) => ``).join(''); tableHtml += `${idxCell}${dataCells}`; } @@ -1637,20 +1931,20 @@ class CsvEditorController { nonHeaderRows.forEach((row, r) => { tableHtml += `${ addSerialIndex - ? `${r + 1}` + ? `${rowIds[r] - bodyStart + 1}` : '' }`; for (let i = 0; i < numColumns; i++) { const rawValue = row[i] || ''; const safe = this.formatCellContent(rawValue, clickableLinks); const titleAttr = this.getMultilineCellTitleAttr(rawValue); - tableHtml += `${safe}`; + tableHtml += `${safe}`; } tableHtml += ``; }); if (!chunked && includeTrailingEmptyRow) { - const virtualAbs = offset + nonHeaderRows.length; - const displayIdx = nonHeaderRows.length + 1; + const virtualAbs = data.length; + const displayIdx = virtualAbs - bodyStart + 1; const idxCell = addSerialIndex ? `${displayIdx}` : ''; const dataCells = Array.from({ length: numColumns }, (_, i) => ``).join(''); tableHtml += `${idxCell}${dataCells}`; @@ -1661,9 +1955,8 @@ class CsvEditorController { // If chunked, append a final chunk with the virtual row so it appears at the end. if (chunked && includeTrailingEmptyRow) { if (nextChunkStart === -1 && serializedChunkCount < safeMaxSerializedChunks) { - const startAbs = headerFlag ? offset + 1 : offset; - const virtualAbs = startAbs + allRowsCount; - const displayIdx = allRowsCount + 1; + const virtualAbs = data.length; + const displayIdx = virtualAbs - bodyStart + 1; const idxCell = addSerialIndex ? `${displayIdx}` : ''; const dataCells = Array.from({ length: numColumns }, (_, i) => ``).join(''); const vrow = `${idxCell}${dataCells}`; @@ -1676,7 +1969,9 @@ class CsvEditorController { const hasRemoteChunks = nextChunkStart >= 0; const chunkState: ChunkRenderState | undefined = chunked ? { - startAbs: headerFlag ? offset + 1 : offset, + rowIds, + bodyStart, + virtualRow: data.length, allRows, allRowsCount, chunkRows, @@ -1747,8 +2042,10 @@ class CsvEditorController { frozenPanes: FrozenPanes; firstVisibleRow: number; realRowCount: number; + visibleRowIds: number[]; + filterMetadata: { session: string; filterColumnsRevision: number; filters: ColumnFilter[]; columns: FilterColumn[]; matchedRows: number; totalRows: number; maxFilters: number; revision: number; filterDraft?: FilterDraft }; }): string { - const { webview, nonce, fontFamily, fontSize, cellPadding, separator, tableHtml, chunksJson, extraColumnColorCss, nextChunkStart, hasRemoteChunks, mouseWheelZoomEnabled, mouseWheelZoomInvert, columnSizes, frozenPanes, firstVisibleRow, realRowCount } = args; + const { webview, nonce, fontFamily, fontSize, cellPadding, separator, tableHtml, chunksJson, extraColumnColorCss, nextChunkStart, hasRemoteChunks, mouseWheelZoomEnabled, mouseWheelZoomInvert, columnSizes, frozenPanes, firstVisibleRow, realRowCount, visibleRowIds, filterMetadata } = args; const themeKind = vscode.window.activeColorTheme.kind; const isDark = themeKind === vscode.ColorThemeKind.Dark || themeKind === vscode.ColorThemeKind.HighContrast; // Build script URI using file path for compatibility (older APIs may lack Uri.joinPath) @@ -1756,6 +2053,8 @@ class CsvEditorController { vscode.Uri.file(path.join(this.context.extensionPath, 'media', 'main.js')) ); + const filterScriptUri = webview.asWebviewUri(vscode.Uri.file(path.join(this.context.extensionPath, 'media', 'column-filters.js'))); + const filterMetadataJson = JSON.stringify(filterMetadata).replace(/[<>&\u2028\u2029]/g, character => '\\u' + character.charCodeAt(0).toString(16).padStart(4, '0')); const frozenPanesScriptUri = webview.asWebviewUri( vscode.Uri.file(path.join(this.context.extensionPath, 'media', 'frozen-panes.js')) ); @@ -2008,16 +2307,29 @@ class CsvEditorController { #csv-root .csv-frozen-row.active-match, #csv-root .csv-frozen-column.active-match { background-image: linear-gradient(var(--vscode-editor-findMatchBackground, transparent), var(--vscode-editor-findMatchBackground, transparent)); } + #filterStatus { position: sticky; top: 0; left: 0; z-index: 6; display: flex; flex-direction: column; gap: 8px; align-items: stretch; max-height: 25vh; overflow: auto; padding: 6px; background: var(--vscode-editorWidget-background, ${isDark ? '#252526' : '#ffffff'}); color: var(--vscode-editorWidget-foreground, inherit); border-bottom: 1px solid var(--vscode-editorWidget-border, ${isDark ? '#555' : '#ccc'}); } + #filterStatus[hidden] { display: none; } + #filterStatus button { color: inherit; background: transparent; border: 1px solid var(--vscode-button-border, currentColor); border-radius: 3px; cursor: pointer; } + .column-filter-toolbar, .column-filter-row { display: flex; flex-wrap: wrap; align-items: center; gap: 6px; } + .column-filter-conditions { display: flex; flex-direction: column; gap: 6px; } + .column-filter-row select, .column-filter-row input { max-width: 100%; min-width: 0; box-sizing: border-box; font: inherit; color: var(--vscode-input-foreground, inherit); background: var(--vscode-input-background, transparent); border: 1px solid var(--vscode-input-border, var(--vscode-editorWidget-border, currentColor)); } + .column-filter-row input { flex: 1 1 18ch; } + .column-filter-error { color: var(--vscode-errorForeground, inherit); flex-basis: 100%; } + .column-filter-type, .column-filter-and { font-size: 0.9em; } + th[data-filter-column="active"]::after { content: " ⏷"; } + #filterStatus button:focus-visible { outline: 2px solid var(--vscode-focusBorder); } /* Per-column computed colors */ - ${extraColumnColorCss} + -
+
+
${tableHtml}
+
@@ -2050,7 +2362,7 @@ class CsvEditorController {
- +