D18: Entity table provenance
Table of Contents
- Status: Decided (owner, 2026-10-06): the renderings are made from standards, not from Org; only the names are Org's
- Affects:
crates/org-syntax/src/tables/entities.rs, the Phase 0 exit criterion "org-syntax0.1 ready for crates.io"
1. The question
org-syntax is licensed MIT OR Apache-2.0 (D1). Its entity table, 414 entries, is generated by tools/gen-tables.el from Org's org-entities.el. That file is part of GNU Emacs, copyright the Free Software Foundation, under GPL-3.0-or-later.
Each entry has seven columns: the name (alpha), and the LaTeX, LaTeX-math flag, HTML, ASCII, Latin-1 and UTF-8 renderings. Publishing that table in a permissively licensed crate may make the crate a derivative of GPL code.
The other generated tables (character classes, case mappings, scripts) are also produced by running Emacs. They encode Unicode Character Database facts as Emacs applies them, not Emacs source, and carry much less risk.
2. What the parser needs
| Column | Needed for | Could be replaced? |
|---|---|---|
| Name | Deciding whether \foo is an entity: parsing must match Emacs exactly |
No: the names are the interface. Most are HTML 4 or LaTeX names. |
| UTF-8 | Showing entities as symbols in the editor | Mostly: for HTML-named entities it is the character the HTML standard assigns |
| LaTeX, math flag, HTML, ASCII, Latin-1 | Exporters (Phase 2) and the :latex=/:html=/… properties Emacs reports |
Yes, by authoring an independent table, at the cost of export output that differs from Emacs in places |
3. Options
- Keep the table as is, with attribution. This rests on the argument that the table is functional interoperability data rather than a creative work. It is the simplest option, but the risk is the owner's to judge.
- Split the table.
org-syntaxkeeps only names and UTF-8 characters, the minimum for parsing and display, sourced from the HTML and Unicode standards where possible. The export renderings go into a separateorg-entitiescrate. That crate is either GPL-3.0-or-later and optional (a binary using it becomes GPL) or authored independently. - Ask for permission. Ask the Org maintainers, and through them the FSF, whether the entity data can be used under a permissive license. This can run in parallel with option 2.
- Relicense
org-syntaxunder the GPL. This conflicts with D1 and with the goal of an ecosystem crate.
4. Recommendation
Option 2 now, with option 3 in parallel. Parsing needs only the names, and exporters are Phase 2 work, so the split costs nothing today. If permission is granted, the full table can come back.
5. Until decided
- The crate stays unpublished;
cargo publish --dry-run -p org-syntaxsucceeds, but nothing is uploaded. - The README's "Data provenance" section states where the tables come from.
6. Decision
The owner chose, on 2026-10-06, to make the table from sources that are
not Org's, for the binaries as much as for the crate.
tools/gen-entities.py writes entities.rs from:
- the names,
crates/org-syntax/data/entity-names.txt: Org's 412, the interface, which decide what the parser reads as an entity, as Org does; - for each name, what it stands for: Kalem's own rows
(
crates/org-syntax/data/entities-kalem.tsv), then LaTeX's log-like operators (\sin, read fromlatex.ltx), a name LaTeX defines as a math symbol in LaTeX's meaning (\circis the ring), then the HTML Standard's named character references (WHATWG, CC BY 4.0), then unicode-math's symbol table; - for each character, its renderings: LaTeX's math symbols
(
fontmath.ltx,amssymb.sty,latexsym.sty) before its text encodings (the*enc.dfufiles), LaTeX 2ε being under the LPPL; in HTML, the name HTML 4.01 gives the character (→,∞; of its entity sets, W3C, only which names they hold is read), else the entity's own name when the HTML Standard gives it this character (|), else an ASCII character as itself, else the HTML Standard's shortest name for it (⋯), else a numeric reference (since 2026-10-11: before, a character was named only by the entity's own name, and HTML 4.01's names, which Org writes, were numbers); ASCII and Latin-1 by the Unicode Character Database's decompositions and names, and Kalem's own table of symbols.
No part of Emacs or Org is read. The names stay Org's because parsing
must agree with Org's; the parser's differential tests compare only the
names. Kalem's rows (44 and two choices: \deg and \sup are the
degree and superset signs, as Org documents mean them) are written for
it: spaces and marks, emoticons, aliases LaTeX defines through others
(\gets, \land), and symbols no standard above gives a command.
What differs from Emacs: the renderings follow the standards, so a few
characters differ (\phi is LaTeX's ϕ, \Amacr is Ā), and the
ASCII and Latin-1 renderings, Org's own, differ for about 150 names.
The export cases compared with Emacs still agree. Of Worg's 287 files,
all agree in HTML and Markdown but the two that list Org's entities in
a table (org-syntax.org's appendix, the HTML publishing tutorial's
symbols), where these characters differ, and so do renderings Org gives
that no standard does (\odot as o, \Idot as &idot;).
org-syntax may be published as far as this decision goes.