Standard modes and their oracles
Table of Contents
Kalem edits standard formats and never extends them. This part of the book says, for each format, what Kalem reads, shows, edits and writes, and against what it is tested. It is informative: the standards themselves are the normative texts, and each chapter names the one Kalem follows.
These are the formats built into Kalem. Every other format is to be a plugin, compiled Rust run as a WebAssembly component on the same contracts the built-in modes use (Plugins); the plugin runtime is not built yet, so today Kalem opens exactly the formats below, and every other text file as plain text. No file is converted in order to be opened: a format is opened as itself, or as plain text.
1. Three rules
Every standard mode keeps three rules (docs/design_doc2.md, section 2):
- Ranges, never text. A mode reads the file into ranges of the text. Rendering, editing and every command work on those ranges, and an edit replaces only the characters it changes. Kalem never writes a file back from a tree, so the parts you did not edit stay byte for byte as they were, spacing, comments and line endings included.
- No extension. Kalem writes nothing into a standard file that its standard does not define. What a format cannot express is not offered in it: the toolbar and the menus of a LaTeX file have no Org commands, those of a CSV file no formatting.
- Unknown constructs stay visible. What a mode does not understand is shown as the source it is, never hidden or guessed.
2. The modes and their oracles
An oracle is the reference implementation or test suite a mode is checked against. A mode is complete when its oracle agrees with it.
| Format | Standard | Oracle | State |
|---|---|---|---|
| Org | The Org Syntax, as org-element.el of Org 9.7 |
Differential tests against Emacs 30.1 on the corpus and on mutated files (kalem diff-emacs); the exporters against ox.el |
Parser, model, editing, tables and exports implemented |
| LaTeX | LaTeX as the TeX engines accept it | Byte-exact round trip; structure against pandoc's LaTeX reader (kalem diff-pandoc); compiling with a TeX engine; planned, the PDF the authors published, for numbers, references, citations and formulas (kalem diff-pdf) |
Rendered subset, projects, diagnostics and building |
| BibTeX | BibTeX and BibLaTeX data files | The same file read by BibTeX or biber, and by hayagriva | Grid, sorting and field editing |
| CSV | RFC 4180 and the dialects in use | RFC 4180's cases; files written by spreadsheets | Grid, dialect detection and editing |
| Markdown | CommonMark with the GitHub extensions | The CommonMark and GFM specification suites (markdown::spec): 632 of 648 and 23 of 24 examples, the rest explained in the chapter |
Rendered view, editing, tables, front matter, conversion to Org |
| Plain text | The file's own bytes | The file unchanged but for the edits | Highlighting, indentation and line tools |
The chapters that follow take each format in turn. Where Kalem differs from its oracle on purpose, the difference is listed in the format's known differences chapter, and every entry there is a test.