The Kalem Book GitHub

This Book describes Kalem's main branch, which may be ahead of the latest release.

D18: Entity table provenance

  • Status: Decided (owner, 2026-10-06): the renderings are made from standards, not from Org; only the names are Org's
  • Affects: crates/org-syntax/src/tables/entities.rs, the Phase 0 exit criterion "org-syntax 0.1 ready for crates.io"

1. The question

org-syntax is licensed MIT OR Apache-2.0 (D1). Its entity table, 414 entries, is generated by tools/gen-tables.el from Org's org-entities.el. That file is part of GNU Emacs, copyright the Free Software Foundation, under GPL-3.0-or-later.

Each entry has seven columns: the name (alpha), and the LaTeX, LaTeX-math flag, HTML, ASCII, Latin-1 and UTF-8 renderings. Publishing that table in a permissively licensed crate may make the crate a derivative of GPL code.

The other generated tables (character classes, case mappings, scripts) are also produced by running Emacs. They encode Unicode Character Database facts as Emacs applies them, not Emacs source, and carry much less risk.

2. What the parser needs

Column Needed for Could be replaced?
Name Deciding whether \foo is an entity: parsing must match Emacs exactly No: the names are the interface. Most are HTML 4 or LaTeX names.
UTF-8 Showing entities as symbols in the editor Mostly: for HTML-named entities it is the character the HTML standard assigns
LaTeX, math flag, HTML, ASCII, Latin-1 Exporters (Phase 2) and the :latex=/:html=/… properties Emacs reports Yes, by authoring an independent table, at the cost of export output that differs from Emacs in places

3. Options

  1. Keep the table as is, with attribution. This rests on the argument that the table is functional interoperability data rather than a creative work. It is the simplest option, but the risk is the owner's to judge.
  2. Split the table. org-syntax keeps only names and UTF-8 characters, the minimum for parsing and display, sourced from the HTML and Unicode standards where possible. The export renderings go into a separate org-entities crate. That crate is either GPL-3.0-or-later and optional (a binary using it becomes GPL) or authored independently.
  3. Ask for permission. Ask the Org maintainers, and through them the FSF, whether the entity data can be used under a permissive license. This can run in parallel with option 2.
  4. Relicense org-syntax under the GPL. This conflicts with D1 and with the goal of an ecosystem crate.

4. Recommendation

Option 2 now, with option 3 in parallel. Parsing needs only the names, and exporters are Phase 2 work, so the split costs nothing today. If permission is granted, the full table can come back.

5. Until decided

  • The crate stays unpublished; cargo publish --dry-run -p org-syntax succeeds, but nothing is uploaded.
  • The README's "Data provenance" section states where the tables come from.

6. Decision

The owner chose, on 2026-10-06, to make the table from sources that are not Org's, for the binaries as much as for the crate. tools/gen-entities.py writes entities.rs from:

  • the names, crates/org-syntax/data/entity-names.txt: Org's 412, the interface, which decide what the parser reads as an entity, as Org does;
  • for each name, what it stands for: Kalem's own rows (crates/org-syntax/data/entities-kalem.tsv), then LaTeX's log-like operators (\sin, read from latex.ltx), a name LaTeX defines as a math symbol in LaTeX's meaning (\circ is the ring), then the HTML Standard's named character references (WHATWG, CC BY 4.0), then unicode-math's symbol table;
  • for each character, its renderings: LaTeX's math symbols (fontmath.ltx, amssymb.sty, latexsym.sty) before its text encodings (the *enc.dfu files), LaTeX 2ε being under the LPPL; in HTML, the name HTML 4.01 gives the character (→, ∞; of its entity sets, W3C, only which names they hold is read), else the entity's own name when the HTML Standard gives it this character (|), else an ASCII character as itself, else the HTML Standard's shortest name for it (⋯), else a numeric reference (since 2026-10-11: before, a character was named only by the entity's own name, and HTML 4.01's names, which Org writes, were numbers); ASCII and Latin-1 by the Unicode Character Database's decompositions and names, and Kalem's own table of symbols.

No part of Emacs or Org is read. The names stay Org's because parsing must agree with Org's; the parser's differential tests compare only the names. Kalem's rows (44 and two choices: \deg and \sup are the degree and superset signs, as Org documents mean them) are written for it: spaces and marks, emoticons, aliases LaTeX defines through others (\gets, \land), and symbols no standard above gives a command.

What differs from Emacs: the renderings follow the standards, so a few characters differ (\phi is LaTeX's ϕ, \Amacr is Ā), and the ASCII and Latin-1 renderings, Org's own, differ for about 150 names. The export cases compared with Emacs still agree. Of Worg's 287 files, all agree in HTML and Markdown but the two that list Org's entities in a table (org-syntax.org's appendix, the HTML publishing tutorial's symbols), where these characters differ, and so do renderings Org gives that no standard does (\odot as o, \Idot as &idot;). org-syntax may be published as far as this decision goes.