Skip to content

Preserve quoted newlines following backslashes - #935

Open
repzakske wants to merge 1 commit into
andialbrecht:masterfrom
repzakske:codex/preserve-quoted-escaped-newlines
Open

repzakske wants to merge 1 commit into
andialbrecht:masterfrom
repzakske:codex/preserve-quoted-escaped-newlines

Conversation

@repzakske

Copy link
Copy Markdown

Formatting a quoted value containing a backslash followed by LF can remove whitespace and normalize line endings inside that value:

query = "SELECT 'a\\\nb  \r\nc';"
assert sqlparse.format(query) == query

The assertion currently fails: the two spaces after b disappear and its CRLF becomes LF. SPLIT_REGEX uses \\. for quoted escapes, but . does not match LF. Enabling DOTALL keeps the complete quoted value together so the serializer only strips whitespace outside it.

Added 30 regression cases for single/double quotes, LF/CR/CRLF, paired backslashes, and default/whitespace/reindent formatting. Eight fail before the fix. Independent SQLite checks preserve all 36 tested values after the fix; the original formatter changes eight.

  • ran the tests (pytest): 536 passed on each of Python 3.12, 3.13 and 3.14; the baseline's two xfails and one xpass are unchanged
  • all style issues addressed (ruff check sqlparse/)
  • changes are covered by tests
  • changes are documented in the changelog

@carlitogwapito

Copy link
Copy Markdown

Here's a small optional test-only commit, directly on your current head.

It combines keyword uppercasing with whitespace stripping or reindentation, using two quoted values containing backslash/LF, trailing spaces and CRLF. The words select and from inside those values stay lowercase.

Two guards worth keeping: both quoted payloads remain exact, and the outside SELECT/AS transformation really happens. That checks a non-identity formatting operation alongside your existing preservation cases.

On Python 3.14.4, both new cases pass your source. The base transforms the outside keywords but loses both literal payloads. The unchanged keyword-case test passes on both. Focused Ruff checks pass; I haven't run the full suite, dialect/SQL-semantic checks, benchmarks, build or version/OS matrix. No SQL was executed.

To run the added file in your tree: python -m pytest tests/test_keyword_transform_quoted_payloads.py -q.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants