Data from examples #
This file defines the data a study reads off a paper's examples; the lookups a study makes on
them are in Data/Examples/Readers.lean. An example is what the paper prints under a number; a Datum is one piece of data the paper
reports with it in the sense of Tonhauser and Matthewson: an expression (the example, or one
alternant of it), the context it is judged in, and the judgment, less the information about the
speakers that papers seldom give. A datum is a row of the Examples component of the
Cross-Linguistic Data Formats of Forkel et al., in version 1.3, and the
sentence-level counterpart of Data/Wordlist/.
Main definitions #
Datum: a piece of data about an example, with the judgment layer (judgment,alternatives,readings) and the paper's own columns (paperFeatures).SourceRef: a reference to a paper, a bibkey with a locator.
Implementation notes #
- Per-paper data lives in
Linglib/Data/Examples/{AuthorYear}.jsonand is compiled byscripts/gen_examples.pyintoLinglib/Data/Examples/{AuthorYear}.lean, declaringnamespace {AuthorYear}.Examples. The JSON keys are the field names, andiduses only the characters of a CLDF identifier.scripts/export_examples_cldf.pywrites the data as a CLDF dataset, which CI validates. - Four JSON keys are the source's record and not fields, since no theorem reads them and every
field is paid for when a kernel
decidereduces a row:translation(Translated_Text, empty for an English example),discourseSegments(the utterances of a discourse),comment, andverified(how the row was last checked against its source). The generator validates them and the CLDF export carries them. languageis a Glottocode, which the generator checks againstlanguages.csv, the language table of the data drawn from Glottolog; it is empty for a constructed string that belongs to no language, such as a pattern of a formal language.glossedTokenspairs each analysed word with its gloss, so the word-by-word alignment of Rule 1 of the Leipzig Glossing Rules holds by construction. CLDF'sLGR_Conformancecolumn is not stored: whether a gloss also aligns morpheme by morpheme (Rule 2) is a property of the pairs.- Translations are into English, the default of CLDF's
Meta_Language_ID, which is therefore not stored. - A discourse is one datum:
primaryTextjoins its utterances with single spaces, and the judgment is of the last utterance in the context of the others. paperFeaturesare the paper's own columns: what the paper states about the datum, such as the cell of its design or the class it assigns. The results a paper prints for an experiment belong inData/Statistics/, and an analysis the formaliser derives belongs in the study. A key repeats when the expression bears the property twice (two indefinite series in one clause), so the list is not a map:feature?reads the first value,featuresevery value.- The file imports no tactic library: every module with example data imports it, and names such
a library brings into scope (mathlib's root
Tree) would reach every study that reads rows. - The lookups and fits on rows are in
Data/Examples/Readers.lean, which only the studies that read rows import, so a new reader rebuilds those studies and not every module of example data.
TODO #
- 86 rows in 21 papers with
discourseSegmentsdo not have their concatenation asprimaryText(thediscoursecolumn ofscripts/export_examples_cldf.py --report).
References #
A Glottolog language identifier, such as "stan1293" for Standard English.
Equations
- Glottocode = String
Instances For
Equations
- instDecidableEqSourceRef.decEq { bibkey := a, paperLabel := a_1 } { bibkey := b, paperLabel := b_1 } = if h : a = b then h ▸ if h : a_1 = b_1 then h ▸ isTrue ⋯ else isFalse ⋯ else isFalse ⋯
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- instReprSourceRef = { reprPrec := instReprSourceRef.repr }
How an example and its judgment were obtained.
- introspection : Method
Constructed and judged by the author.
- elicitation : Method
Elicited from, or judged by, native-speaker consultants.
- corpus : Method
Attested in a text, a corpus or on the web.
- experiment : Method
A stimulus whose judgment an experiment measured.
- description : Method
Cited from a grammar or dictionary that does not say how it was obtained.
Instances For
Equations
- Datum.instDecidableEqMethod x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯
Equations
- Datum.instReprMethod.repr Datum.Method.introspection prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "Datum.Method.introspection")).group prec✝
- Datum.instReprMethod.repr Datum.Method.elicitation prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "Datum.Method.elicitation")).group prec✝
- Datum.instReprMethod.repr Datum.Method.corpus prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "Datum.Method.corpus")).group prec✝
- Datum.instReprMethod.repr Datum.Method.experiment prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "Datum.Method.experiment")).group prec✝
- Datum.instReprMethod.repr Datum.Method.description prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "Datum.Method.description")).group prec✝
Instances For
Equations
- Datum.instReprMethod = { reprPrec := Datum.instReprMethod.repr }
A Datum is a piece of data a paper reports with one of its examples
[TM16]: an expression of a language with its gloss, the context it is judged
in, the judgment, the forms and readings the paper judges with it, and the paper's own
classifications of it. It is the part of a row of a CLDF ExampleTable that theorems read.
- id : String
The
IDcolumn holds a stable identifier keyed to the paper,{authoryear}_{label}. - source : SourceRef
The paper that introduced the example.
- reportedIn : Option SourceRef
The paper whose data file holds the row, when it only reports an example from
source. - method : Option Method
How the example and its judgment were obtained;
nonewhere that is not yet classified. - language : Glottocode
The
Language_IDcolumn holds the Glottocode of the language of the expression; empty for a constructed string of no language. - primaryText : String
The
Primary_Textcolumn holds the expression without judgment marks. - glossedTokens : List (String × String)
The
Analyzed_WordandGlosscolumns paired, each word with its gloss; empty when the source gives no gloss. - context : String
The scenario the source gives for the judgment; empty when it gives none.
- judgment : Judgment
The judgment the source reports for the expression in the context.
- verdict : Option Bool
The truth-value judgment the source reports, where the task is truth in the context rather than the acceptability that
judgmentrecords;nonewhere the source reports none. - alternatives : List (String × Judgment)
The other forms the source judges in the same frame, each a whole sentence, with their judgments.
- readings : List (String × Judgment)
The readings the source distinguishes for the expression, each with its judgment.
- paperFeatures : List (String × String)
The paper's own columns, as key-value pairs.
Instances For
Equations
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- instReprDatum = { reprPrec := instReprDatum.repr }