Levshina et al. (2023): Why We Need a Gradient Approach to Word Order #
This file formalizes the corpus illustrations of [LNAT+23]'s case for describing
word order by continuous measures rather than categorical labels. The paper's two measures of a
binary order are the proportion of one order and the Shannon entropy of the order in bits; the
entropy is the entropy of the Bernoulli law of the order at that proportion
(orderEntropy_eq_binEntropy), so it is 0 when one order is fixed and 1 at an even split,
the worked values of section 4.1.1, and among languages that put the subject first more often
than not it ranks languages in reverse of the proportion (orderEntropy_lt_iff). In the 31
news corpora of Figure 1 the subject precedes the object in most clauses of every language
(subject_usually_first). Figure 3 plots the mutual information between case marking and
syntactic role against order entropy for 30 corpora: the two covary positively over the sample
(covariance_entropy_caseInformation_pos), the corpus of maximal entropy, Lithuanian, has the
most informative case marking (freest_most_case_informative), and the corpus of minimal
entropy has none (most_rigid_least_case_informative). The Russian register study of section
4.1.3 is the population of its 300 annotated clauses under the uniform measure: verb–object
order is less probable in conversation than in fiction (conversation_lt_fiction) and equally
probable in fiction and news (fiction_eq_news).
Implementation notes #
- The rows are the paper's OSF datasets, generated into
Data.WordOrder.Corpus.LevshinaEtAl2023. The entropies of Figure 3 are the dataset's printed values from the paper's own extraction, not recomputed from the counts of Figure 1, which come from a different sample of the same corpora. - The claim of Figure 1 that the proportions form a continuum with no natural cut-off is a visual one without a statistic and is not stated as a theorem; nor is the logistic regression of section 4.1.3, whose descriptive content is the conditional probabilities.
References #
Proportion and entropy of a binary order (sections 1.1 and 4.1.1) #
The proportion of clauses with the subject before the object.
Equations
- LevshinaEtAl2023.proportion r = ↑r.subjectFirst / (↑r.subjectFirst + ↑r.objectFirst)
Instances For
The subject is first in more than half the clauses exactly when it is first in more clauses than the object.
The law of subject–object order in a language's corpus, which puts the subject first with the observed proportion.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The paper's word-order entropy, the Shannon entropy of the order in bits.
Equations
- LevshinaEtAl2023.orderEntropy r = InformationTheory.measureEntropy (LevshinaEtAl2023.orderLaw r) / Real.log 2
Instances For
Word-order entropy is the binary entropy of the proportion, in bits.
A fixed order has entropy 0.
An even split of the two orders has entropy 1.
Among languages that put the subject first at least half the time, entropy ranks them in reverse of the proportion: the closer to an even split, the more variable the order.
The subject–object proportions of Figure 1 #
In every one of the 31 corpora the subject precedes the object in most clauses.
Across the corpora of Figure 1, ranking by entropy is ranking by proportion reversed.
Case marking against order entropy (Figure 3) #
The corpora of Figure 3.
Equations
Instances For
The printed statistics of a corpus.
Equations
Instances For
The entropy of subject–object order in a corpus, in bits.
Equations
- LevshinaEtAl2023.entropy i = ↑(LevshinaEtAl2023.statistics i).entropy1000 / 1000
Instances For
The mutual information between case marking and syntactic role in a corpus, in bits.
Equations
Instances For
Over the 30 corpora, order entropy and case information covary positively, so freer order goes with more informative case marking.
The corpus with the most variable subject–object order, Lithuanian, has the most informative case marking.
The corpus with the most rigid subject–object order has case marking carrying no information about syntactic role.
Register and modality in Russian (section 4.1.3) #
The annotated clauses, as a population.
Equations
Instances For
The order of object and verb in a clause.
Equations
Instances For
The text type of a clause.
Equations
Instances For
The probability of verb–object order in a text type, under the uniform measure on the clauses.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The probability of verb–object order in a text type is the share of its clauses in that order.
Each text type contributes one hundred clauses.
Verb–object order in 39 of the 100 conversational clauses.
Verb–object order in 83 of the 100 fiction clauses.
Verb–object order in 83 of the 100 news clauses.
Verb–object order is less probable in conversation than in fiction.
Verb–object order is equally probable in fiction and in news.