D16: Syntax highlighting
Table of Contents
- Status: Decided (2026-09-28)
- Decision: syntect 5.3 with pure Rust regular expressions (
regex-fancy) and its bundled Sublime Text syntax definitions, in thekalem-highlightcrate; tree-sitter is kept in mind for structure-aware features later. - Tasks: T1.6a.2 (pulled forward for T1.4.5)
- Code:
crates/kalem-highlight,examples/highlight_timing.rs
1. Why
| Option | For | Against |
|---|---|---|
| syntect (chosen) | Pure Rust with regex-fancy, so no C library and easy cross-compilation; about 70 bundled languages covering Org's common source block languages; Sublime syntax compatibility, so more languages can be added as files |
Regex-based, slower than tree-sitter; no syntax tree |
| tree-sitter | Incremental, fast, a real syntax tree for folding and text objects | A C grammar per language, compiled into the binary; larger builds; grammars of uneven quality for less common languages |
Highlighting in Kalem colors code in source blocks and plain text files. Breadth matters more than structure there, and a single pure Rust binary matters for every platform (§2.4).
2. Measurements
On an Apple M-series machine, release build, highlighting crates/kalem-core/src/view.rs (1,443 lines of Rust):
| Measure | Result |
|---|---|
| Loading the syntax set | 0.7 ms (lazy) |
| First pass, compiling the grammar's patterns | 133 ms, once per language per session |
| Later passes | 62 µs per line |
| Release binary with the terminal frontend | 5.7 MB (target: 15 MB, §15) |
3. Consequences
- Spans carry a kind (comment, string, keyword…), not colors; each frontend colors kinds from its theme (T1.6.2).
- Highlighting is cached per block and text version. Long blocks and plain text files need line states kept per line, so an edit rehighlights from the changed line (T1.6a.3).
- Org's language names (
sh,emacs-lisp,elisp) map to syntax names inkalem-highlight.