Syntax Highlighting

Syntax highlighting engine for Kate syntax definitions

Table of contents

1. Introduction
2. Out of scope
3. Syntax definition files
4. Color theme files
5. Build it
6. How to contribute
   • Licensing
   • Tips for contributing to syntax definition files
   • Adding unit tests for a syntax definition
7. Report bug or help to fix them
8. Updating the syntax & themes pages of the kate-editor.org website

Introduction

This is a stand-alone implementation of the Kate syntax highlighting engine. It's meant as a building block for text editors as well as for simple highlighted text rendering (e.g. as HTML), supporting both integration with a custom editor as well as a ready-to-use QSyntaxHighlighter sub-class.

Upstream also provides a [QML API](@ref qml_api); this standalone fork builds the C++ library only.

Out of scope

To not turn this into yet another text editor, the following things are considered out of scope:

• code folding, beyond providing folding range information
• auto completion
• spell checking
• user interface for configuration
• management of text buffers or documents

If you need any of this, check out KTextEditor.

Syntax definition files

This library uses Kate syntax definition files for the actual highlighting, the file format is documented here.

More than 300 syntax definition files are included, that are located in data/syntax/ and have the .xml extension. Some files are generated at compile time, all xml files are available here. Additional ones are picked up from the file system if present, so you can easily extend this by application-specific syntax definitions for example.

To install or test a syntax definition file locally, place it in org.kde.syntax-highlighting/syntax/, which is located in your user directory. Usually it is:

For local user	$HOME/.local/share/org.kde.syntax-highlighting/syntax/
For Flatpak packages	$HOME/.var/app/package-name/data/org.kde.syntax-highlighting/syntax/
For Snap packages	$HOME/snap/package-name/current/.local/share/org.kde.syntax-highlighting/syntax/
On Windows®	%USERPROFILE%\AppData\Local\org.kde.syntax-highlighting\syntax\
On macOS®	$HOME/Library/Application Support/org.kde.syntax-highlighting/syntax/

For more details, see "The Highlight Definition XML Format" (Working with Syntax Highlighting, KDE Documentation).

Also, in data/schema/ there is a script to validate the syntax definition XML files. Use the command validatehl.sh mySyntax.xml.

Color theme files

This library includes the color themes, which are documented here.

The color theme files use the JSON format and are located in data/themes/ with the .theme extension. Additional ones are also picked up from the file system if present, in the org.kde.syntax-highlighting/themes/ folder of your user directory, allowing you to easily add custom color theme files. This location is the same as shown in the table of the previous section, replacing the syntax folder with themes. For more details, see "The Color Themes JSON Format" (Working with Color Themes, KDE Documentation).

The KTextEditor library (used by Kate, Kile and KDevelop, for example) provides a user interface for editing and creating KSyntaxHighlighting color themes, including a tool for exporting and importing the JSON theme files.

Note that in KDE text editors, the KSyntaxHighlighting color themes are used since KDE Frameworks 5.75, released on October 10, 2020. Previously, Kate's color schemes (KConfig based schema config) were used and are now deprecated. The tool utils/schema-converter/ and the script utils/kateschema_to_theme_converter.py convert the old Kate schemas to KSyntaxHighlighting themes.

Also see "Submit a KSyntaxHighlighting Color Theme" (Kate Editor Website).

Build it

This fork starts from upstream KSyntaxHighlighting 6.30.0 and builds the real VSyntaxHighlighting target without KDE Frameworks or ECM. It requires CMake 3.21+, Qt 6.9+ Core/Gui/Network, Python 3 and a compiler supporting C++20 for the syntax indexer. The library and its public consumers require C++17; Core5Compat is not needed by this dependency. Qt 5 is not supported.

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target VSyntaxHighlighting --parallel

Set CMAKE_PREFIX_PATH to your Qt SDK when needed. Native Windows builds stage the matching Qt Core DLL beside the build-time indexer, including the Debug runtime, so catalog generation does not require Qt's bin directory on the build environment's PATH. An embedding project can use add_subdirectory() and link VSyntaxHighlighting; the target supplies the source and generated include directories. Both <Repository> and <KSyntaxHighlighting/Repository> (and the other public CamelCase headers) work. Version, logging and export headers are generated in the build tree. BUILD_SHARED_LIBS selects shared versus static builds; no installed library/package export is provided by this fork.

QRC_SYNTAX defaults to ON and embeds the full indexed/compressed catalog, including checked-in PHP/Twig/DoxygenLua/XSLT/JSONC variants and Python-generated Jinja grammars. With it OFF, cmake --install build installs the indexed catalog under ${CMAKE_INSTALL_DATADIR}/org.kde.syntax-highlighting/syntax-bundled instead. Themes are always embedded. NO_STANDARD_PATHS=ON disables system/user-directory discovery but retains embedded resources and explicit addCustomSearchPath() calls.

Cross-compiling requires KATEHIGHLIGHTINGINDEXER_EXECUTABLE pointing to a native host indexer built from the same sources, with its runtime libraries available on the host. A target executable is never run as a host tool. Xerces-C is optional for build-time schema validation.

The existing Repository::themeFromFile(const QString &) const extension loads a complete snapshot, including Qt resource paths, and returns an invalid theme on read/JSON errors. It does not cache paths. The named Default theme keeps its white background and classic palette, with current Breeze Light operator/raw-string styles and CSV/TSV column colors. GitHub Light-inspired INI/Python/Rust/XML/Odin overrides use that same palette. Its editor colors use current role names; defaultTheme() still selects upstream Breeze Light/Breeze Dark. Rebuild consumers against this engine; binary compatibility with the old fork is not provided.

The standalone build does not configure KDE's CLI, QML, examples or autotest targets. The vtextedit parent provides native editor/theme regression coverage. The contribution and reference-generation instructions below describe the KDE upstream build, not this standalone build.

Regenerating checked-in syntax definitions

Normal builds do not need Perl. The 14 Perl-generated XMLs are checked into data/syntax/: six PHP variants (HTML, CSS, JavaScript, JSX, TypeScript, Mustache), five Twig variants (HTML, CSS, JavaScript, JSX, TypeScript), DoxygenLua, XSLT and JSONC. Python still generates Jinja definitions during the build.

With Perl installed, run this from the syntax-highlighting repository root:

cmake -P data/generators/regenerate-perl-syntax.cmake

This runs the original generators and normalizes their outputs to LF. Do not hand-edit the generated XMLs. The tradeoff is manual refresh: whenever their base grammars, PHP/Twig dependencies, generator scripts or data/generators/xslt.base.xml change (including upstream updates), rerun the command and commit the refreshed XMLs with the input changes. Rebuild the catalog to validate references. When syncing upstream build rules, retain this checked-in-data workflow: restoring in-build Perl generation would reintroduce the dependency and duplicate the checked-in definitions.

How to contribute

KDE uses a GitLab instance at invent.kde.org for code review. The official repository of the KSyntaxHighlighting framework is here.

All the necessary information to send contributions is here.

Licensing

Contributions to KSyntaxHighlighting shall be licensed under MIT.

All files shall contain a proper "SPDX-License-Identifier: MIT" identifier inside a header like:

/*
    SPDX-FileCopyrightText: 2020 Christoph Cullmann <cullmann@kde.org>

    SPDX-License-Identifier: MIT
*/

Tips for contributing to syntax definition files

• If you are modifying an existing syntax definition XML file, you must increase the version number of the language.

• Do not use hard-coded colors, as they may not look good or be illegible in some color themes. Prefer to use the default color styles.

  For more information, see:

  • Available Default Styles (Working with Syntax Highlighting, KDE Documentation)
  • Kate Part (KF5): New Default Styles for better Color Schemes (Kate Editor Website)

• While writing syntax, you can use ksyntaxhighlighter6 (previously named kate-syntax-highlighter) to display the style or others informations associated with each part of text. For example ksyntaxhighlighter6 --output-format=ansi --syntax-trace=format test.cpp.

  On Linux and Unix systems, you can isolate the syntax files to be modified by using the XDG_DATA_DIRS environment variable to point to a folder containing org.kde.syntax-highlighting/syntax/ in which you will put all the files required for a syntax:

  XDG_DATA_DIRS=mydir ksyntaxhighlighter6 ....

• Add test files, these are found in autotests/input/. If you are going to add a new syntax XML file, create a new test file; if you are going to modify a XML file, adds examples to existing test files.

  Then, it is necessary to generate and update the files in autotests/folding/, autotests/html/ and autotests/reference/, which must be included in the patches. Instructions are below.

Adding unit tests for a syntax definition

1. Add an input file into the autotests/input/ folder, lets call it test.<language-extension>.

2. If the file extension is not sufficient to trigger the right syntax definition, you can add an second file testname.<language-extension>.syntax that contains the syntax definition name to enforce the use of the right extension.

3. Do make && make test.

   Note that after adding or modifying something in <source-directory>/autotests/input/, an error will be showed when running make test, because the references in the source directory do not match the ones now generated.

4. Inspect the outputs found in your binary directory autotests/folding.out/, autotests/html.output/ and autotests/output/.

5. If OK, run in the binary folder ./autotests/update-reference-data.sh to copy the results to the right location. That script updates the references in the source directory in autotest/folding/, autotest/html/ and autotest/reference/.

6. Add the result references after the copying to the git.

Report bug or help to fix them

KDE uses Bugzilla to management of bugs at bugs.kde.org. You can see the bugs reported of frameworks-syntax-highlighting here.

Also, you can report a bug here.

However, some users often report bugs related to syntax highlighting in kate/syntax and kile/editor.

Updating the syntax & themes pages of the kate-editor.org website

To update the kate-editor.org/syntax and kate-editor.org/themes websites including the update site & all linked examples/files, please run after successful build & test the following make target:

make update_kate_editor_org

This will clone the kate-editor.org git from invent.kde.org into kate-editor-org inside the build directory and update the needed things.

You can afterwards step into kate-editor-org and commit & push the change after review.

The kate-editor.org webserver will update itself periodically from the repository on invent.kde.org.