From c95c4dc6241083268ace534d25d19869763dd0bb Mon Sep 17 00:00:00 2001 From: Trevor Gross Date: Mon, 2 Nov 2026 06:56:48 -0600 Subject: [PATCH] input-format: Delegate exact Unicode version to the standard library During discussion about using the new unstable `char::is_xid_start` and `char::is_xid_continue` in the compiler, a point was made that doing so would mean that the Unicode version used to validate identifiers would become tied to whatever std supports. However, we are already using std's version of Unicode in many other places, e.g. via `char::is_whitespace`, so this actually seems reasonable to codify. Thus, add lanugage indicating that the exact Unicode version is delegated to the standard library. This effectively means that T-lang does not need to FCP future Unicode updates either. As also noted on the XID thread, Java provides us some prior art here . --- src/identifiers.md | 4 ++-- src/input-format.md | 6 +++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/src/identifiers.md b/src/identifiers.md index 5abe303ca2..bb71f3ff64 100644 --- a/src/identifiers.md +++ b/src/identifiers.md @@ -85,5 +85,5 @@ It is an error to use the [RESERVED_RAW_IDENTIFIER] token. [proc-macro]: procedural-macros.md [reserved]: keywords.md#reserved-keywords [strict]: keywords.md#strict-keywords -[UAX15]: https://www.unicode.org/reports/tr15/tr15-57.html -[UAX31]: https://www.unicode.org/reports/tr31/tr31-43.html +[UAX15]: https://www.unicode.org/reports/tr15 +[UAX31]: https://www.unicode.org/reports/tr31 diff --git a/src/input-format.md b/src/input-format.md index 88ab3658ba..ad6b93ba6b 100644 --- a/src/input-format.md +++ b/src/input-format.md @@ -21,11 +21,14 @@ r[input.encoding] ## Source encoding r[input.encoding.utf8] -Each source file is interpreted as a sequence of Unicode characters encoded in UTF-8. +Each source file is interpreted as a sequence of Unicode characters encoded in UTF-8. Information on this encoding can be found at . r[input.encoding.invalid] It is an error if the file is not valid UTF-8. +r[input.encoding.unicode-version] +Unicode is an evolving standard, which Rust tracks. The exact version used by the Rust language is delegated to the standard library, documented at [`char::UNICODE_VERSION`]. + r[input.byte-order-mark] ## Byte order mark removal @@ -62,3 +65,4 @@ The resulting sequence of characters is then converted into tokens as described [Crates and source files]: crates-and-source-files.md [shebang]: shebang.md [whitespace]: whitespace.md +[`char::UNICODE_VERSION`]: https://doc.rust-lang.org/nightly/std/primitive.char.html#associatedconstant.UNICODE_VERSION