A usable data dictionary connects an identified dataset version to row meaning, field semantics, codes, units and relationship constraints that a recipient can apply without its author.
Substantively revised October 21, 2026. Author interpretation, documented cases and hypothetical examples are distinguished below.
A spreadsheet can open perfectly and still be unusable. The recipient sees an amount column but cannot tell whether it is in dollars or cents. A blank might mean not collected, not applicable or zero. A customer code repeats because each row represents a transaction, yet the recipient treats it as a unique customer table. No file is missing. The missing dependency is the author's interpretation.
Direct answer
Attach a dictionary to an identified dataset version and document what each row represents, what fields mean, their units, missing-value codes, keys and permitted relationships. Include known limitations and the status of any validation. Then ask a recipient to answer a concrete question using that version and dictionary without the author's oral explanation. Successful parsing alone does not establish usable meaning.
This article proposes an Interpretation Contract for a data handover. It concerns the meaning of a dataset, rather than a general guide to importing CSV files or a record of how research was found. The decision is whether another person can make a specified interpretation without silently inventing the assumptions that the original author kept in memory.
Key takeaways
Start with row meaning before writing field definitions. The same identifier can be unique in a customer table and repeat appropriately in a transaction table. A dictionary that lists types but omits row granularity leaves aggregation and joins vulnerable to plausible mistakes.
Document units and missingness explicitly. A numeric value is not a measurement until its scale and meaning are known. Zero, blank, unknown and not applicable should not be collapsed without a justified dataset rule. Label uncertainty instead of supplying a tidy definition that the evidence cannot support.
Treat declared constraints and observed validation as different evidence. A dictionary can declare uniqueness while the delivered file contains duplicates. A validation result applies to a particular version and check. Neither a schema nor a green validation result proves that the underlying observations are correct or suitable for every analysis.
Observation: the author is acting as an undocumented decoder
Teams often discover dictionary gaps through small questions. Which date does this field mean? Does the amount include tax? Why does this identifier appear twice? The author answers quickly, so the file appears well documented. The answers remain in chat or memory, and the next recipient asks the same questions again.
The pattern is an invisible decoding service. The author supplies interpretation at the moment of use, repairing the gap between the column label and the intended meaning. That service can work while the author is available. It becomes an owner bottleneck when a colleague must produce a result independently or when a later version changes the meaning without changing the familiar column name.
A useful dictionary does not attempt to replace all domain expertise. It records the assumptions a competent recipient must not have to guess. The test should therefore use a realistic analytical question, not merely ask whether the recipient likes the documentation. A wrong but reasonable interpretation is valuable evidence of a missing definition.
Definitions and root cause
Row granularity states what one record represents: a person, transaction, observation interval or another defined entity. A field definition states the meaning of a value in that record. A unit specifies its measurement scale. A missing-value convention distinguishes absent or special states. A key identifies records under declared conditions, and a relationship describes how records may be connected to another table.
A dataset version identifies the actual delivered content and its documentation context. A schema expresses structural rules that a tool may validate. A dictionary may include a schema but also needs domain explanations that cannot be inferred from types alone. The string type does not tell a reader whether a code is a postal identifier, a category or a free-text comment.
The root cause is confusing structural readability with semantic completeness. Software can parse a file while a human misreads its units or row meaning. The reverse also occurs: a knowledgeable author can understand an imperfect file because they remember exceptions. A durable handover must make the important interpretation rules explicit and identify where the delivered data does not satisfy them.
Framework: the Interpretation Contract
The Interpretation Contract is an author-proposed method with six components: Version, Record, Measure, Absence, Relation and Demonstration. It is not a formal standard or a promise of scientific validity. It organizes the information needed to answer a particular question without relying on the original author as a decoder.
Version binds the dictionary to a dataset release or stable content identifier. Include the file names and relevant schema or documentation revision. Explain whether a later correction changes values, definitions or both. A dictionary floating beside several similarly named exports can be accurate yet applied to the wrong file.
Record defines one row and the population or process it covers. State whether records are events, current states or snapshots. Include relevant time boundaries and exclusions. A recipient should know whether counting rows counts customers, purchases or repeated measurements before writing an aggregation.
Measure defines each field's meaning, unit, scale and relevant calculation basis. An amount may be a raw observation or a derived value; document that distinction. If a code maps to a category, provide the actual mapping and its version. Do not infer a measurement unit from a plausible value range when the source evidence is missing.
Absence defines blank and special values, including whether unknown, not collected and not applicable are distinguished. Identify any conversion rule applied before analysis. A missing marker that looks like a legitimate value needs special care. The documentation should make clear when zero is a real measured value rather than a convenient replacement for absence.
Relation defines declared keys, reference targets and expected join cardinality. State the relationship in human language as well as any machine-readable constraint. A valid foreign-key match does not by itself explain whether joining two event tables will multiply rows in an analytically intended way. Include known unmatched records and the treatment required for the intended task.
Demonstration gives a recipient a bounded interpretation task and compares their answer with the documented assumptions. Record the dataset version and any validation results used. If they need the author to explain a unit or exclusion, improve the dictionary and repeat that part of the task. The test is not a quiz about the recipient's intelligence; it is an inspection of the documentation's sufficiency.
Three documented approaches to tabular meaning
Frictionless Data's version-one Table Schema specification describes field types, constraints, missing-value markers, primary keys and foreign keys. It explicitly provides a structural vocabulary beyond a collection of column names. Use the version label accurately: this reference is the v1 specification, not a claim about the latest release. Its rules can support validation, but they do not establish the truth of the dataset's measurements. [S1]
The W3C CSV on the Web metadata recommendation defines metadata at several tabular levels, including column datatypes, null values and key relationships. This shows how interpretation can be attached to tabular data rather than left to a filename extension. A vocabulary does not fill in unknown domain definitions, nor does its existence guarantee that every tool implements the relevant metadata. [S2]
UK Data Service's ReShare guidance calls for useful file descriptions, meaningful versioning, row and column explanations, codes, missing-data information and measurement units. That is a reader-oriented documentation expectation, not proof that a particular recipient understands a delivered file. A concrete interpretation exercise remains necessary when the handover depends on another person using those definitions correctly. [S3]
These are documented specifications and repository practices. They support the components of the contract without endorsing the author's framework or the hypothetical case below. A team can use an appropriate standard and still fail the handover if its actual definitions are vague, outdated or attached to the wrong version.
Case study: the transaction table mistaken for a customer table
This case is hypothetical. An analyst delivers a table with 12 rows, 4 customer identifiers and an amount field. Each row is a transaction, and amounts are stored in the smallest currency unit for a single stated currency. Some records have no delivery date because delivery has not occurred. These invented details define the case; they are not a report about a real dataset or a universal encoding recommendation.
The first dictionary lists customer_id as text, amount as integer and delivery_date as date. The recipient is asked how many customers purchased and what the total transaction amount was. They count rows as customers and read the amount as whole currency units. Their calculations are internally consistent with their assumptions, but the assumptions are wrong.
The author revises the Record component to say one row equals one transaction and that customer identifiers can repeat. The Measure component defines the amount's currency and scale. The Absence component explains that a missing delivery date means not yet delivered in this version, rather than a measured zero date or a failed import. The dictionary also states that this interpretation is specific to the delivered process.
A second table contains one row per customer. The revised Relation component identifies its key and the intended many-transactions-to-one-customer join. The analyst runs a version-bound uniqueness check on that customer table and records its result separately from the declared rule. If the check failed, the declaration would remain an intended constraint rather than evidence that the delivered file satisfied it.
The recipient repeats the task on the identified version and explains the aggregation before computing it. The team also gives them one deliberately ambiguous question about deliveries outside the documented period. Instead of guessing, they identify the scope limitation. That is a useful result: a usable dictionary helps a reader recognize what cannot be concluded, not merely produce more confident numbers.
Failure example: replacing every missing value with zero
A hypothetical workflow fills all blank numeric fields with zero so that a calculation runs without errors. The output looks complete. Yet a blank field originally meant that a measurement was not collected, while a zero meant that an observed quantity was absent. The conversion removes a meaningful distinction before the recipient can inspect it.
The failure is not that zero is always an invalid missing marker. It is that the conversion was made without a documented semantic rule. Correct the dictionary, identify the transformed version and decide whether the original distinction can be recovered from the source. If it cannot, state the limitation rather than inventing which zeros were observations and which were substitutions.
Testing the dictionary with a recipient
Choose a question that exercises the dataset's actual risks. A count tests row meaning. A total tests units and exclusions. A join tests keys and cardinality. The task should be small enough that the recipient can explain their assumptions, rather than hide them inside a large notebook whose final number is hard to inspect.
Ask the recipient to point to the definitions used for each decision. If they rely on general domain knowledge, determine whether that knowledge belongs in the documented prerequisites or whether the dictionary should state the rule. Not every gap requires a longer field description. Sometimes a short table-level statement about the population or time interval prevents many column-level misunderstandings.
Inspect errors without immediately coaching the recipient. The point is to learn what the documentation supports on its own. Once a gap is identified, revise the relevant definition and repeat the affected step. Record the change so a later reader can distinguish a corrected explanation from a changed dataset value.
Include a refusal boundary. A dictionary should let a reader say the data does not answer this question because the population, measurement or relationship is not documented. That is better than a plausible analysis built on an invented assumption. Usability includes knowing where interpretation must stop or seek additional evidence.
Tradeoff: completeness can become an obstacle
A dictionary that attempts to explain every possible analytical use can become too long to navigate. Prioritize the meaning needed to interpret the delivered version: row granularity, units, codes, missingness, relationships and known limitations. Link to broader methods or collection documentation where appropriate, while keeping the essential definitions close to the data.
Machine-readable metadata can reduce inconsistent repetition, but it has its own maintenance cost and tool support limits. A human-readable note may be sufficient for a small handover if it is precise and version-bound. A schema is valuable when its constraints will actually be validated or consumed. Choose the format for the recipient's task rather than treating technical formality as evidence of useful meaning.
Counterargument: competent analysts can infer the meaning
Experienced analysts can often identify likely interpretations from value patterns and domain context. That is useful for discovering questions, not a substitute for evidence. An amount column containing values around ten thousand might represent cents, whole units or a scaled index. A plausible guess can survive several calculations before anyone notices the error.
Inference should therefore produce a labeled hypothesis that can be checked against collection or transformation evidence. If the original meaning is genuinely unknown, document that uncertainty. A dictionary should not make the dataset look more complete by converting a reasonable guess into an authoritative definition. The recipient's expertise is best used to inspect the assumptions, not silently supply them.
Common mistakes
Writing definitions that merely repeat column names adds little. Amount means amount does not identify a currency, scale, inclusion rule or calculation basis. Replace circular definitions with the distinction a reader must use, and mark unresolved details instead of burying them under polished prose.
Declaring a key without checking the delivered version can make joins appear safer than they are. Record the intended key and the observed validation separately. A passing uniqueness check is useful evidence for that version, but it does not establish that the key has the domain meaning the analysis assumes.
Reusing a dictionary across changed exports can silently preserve obsolete meanings. Bind it to a version and review changes in field definitions, codes, units and population. A familiar filename is not a version identifier, and an unchanged column label does not prove an unchanged measurement.
Action plan
Select one dataset handover and identify the exact version. Write the table-level row meaning, population, time boundary and exclusions before expanding field descriptions. Add units, code mappings, missing conventions and relationship rules using the available source evidence. Label unresolved meanings explicitly.
Choose a recipient task that requires a count, measurement interpretation or join relevant to the actual use. Have the recipient explain their assumptions using only the delivered documentation and stated prerequisites. Separate documentation gaps from data defects and record each correction against the appropriate version.
Run meaningful structural checks where applicable and retain their version-bound results. Do not present those checks as proof of scientific correctness. Close the handover when the recipient can answer the agreed question and identify the documented limits without the author's private explanation. Keep future definition changes visible alongside data changes.
FAQ
Is a list of column types a data dictionary?
It supplies some structural information but may omit the meaning needed for use. Include row granularity, units, codes, missingness and relationships. A parser's ability to read a field does not establish a recipient's ability to interpret it.
Why define the row before the columns?
Row meaning determines what counts and aggregations represent. Repeated customer identifiers can be correct in a transaction table and invalid in a customer table. Column definitions alone may not reveal which interpretation applies.
Should zero always mean missing?
No. Define missing markers from the actual collection and transformation process. Zero may be a legitimate observation. Replacing absence with zero without a justified rule can erase a distinction the recipient needs for analysis.
Can a schema prove the data is correct?
It can support specified structural checks for an identified version. It cannot by itself establish the truth of observations, the appropriateness of a measurement or the validity of every analytical conclusion drawn from the data.
What should a unit definition include?
State the measurement unit, scale and relevant calculation basis. For amounts, identify the currency and whether values use whole or smaller units. Add inclusion rules where they affect interpretation, rather than expecting the reader to infer them.
How should unknown meanings be documented?
Label them as unresolved and explain the evidence available. A reasonable hypothesis can guide investigation, but it should not become an authoritative field definition until supported. State which intended analyses the uncertainty prevents or limits.
Are primary keys enough to make joins safe?
No. Explain the relationship and expected cardinality, then check relevant constraints on the delivered version. Two structurally valid tables can still produce an unintended analytical multiplication when joined under the wrong assumptions.
Does the dictionary need a version?
Yes, it needs a clear relationship to the delivered dataset version. Definitions, codes and populations can change even when filenames or column labels remain familiar. Record whether a revision changes documentation, values or both.
Must every dictionary be machine readable?
Choose the format for the recipient and task. Machine-readable metadata helps when tools validate or consume it. A precise human-readable document can still be useful, provided essential meanings and version relationships are inspectable.
Which Frictionless specification is cited here?
The cited reference is the version-one Table Schema specification. Identify that scope rather than calling it the latest standard. Its vocabulary is useful for the documented structural concepts without establishing every tool's current implementation behavior.
Can CSVW metadata fill in missing domain knowledge?
No. It provides a vocabulary for expressing metadata, not the evidence needed to discover unknown meanings. The author or data steward must supply supported definitions, and recipients still need tools that implement the relevant features.
How do we test whether a dictionary is usable?
Give a recipient a bounded question and ask them to explain the definitions behind their answer. Inspect where they must guess or ask the author. Revise the relevant documentation and repeat the affected interpretation step.
Should every recipient get the same exercise?
Use an exercise that reflects the intended use and major semantic risks. A count, total or join may be enough for one handover. A different task can require different definitions, prerequisites or limits without invalidating the earlier observation.
What if the recipient correctly refuses to answer?
That can demonstrate useful understanding of a documented limitation. Determine whether the task exceeded the dataset's scope or exposed a gap that can be resolved. Do not reward a confident guess merely because it produces a number.
How is this different from preserving CSV identifiers?
Identifier preservation concerns keeping values intact through import or conversion. This contract concerns the dataset's meaning, including row granularity, units, absence and relationships. Intact identifiers are helpful but cannot supply those definitions alone.
Recommended books
Thinking in Systems by Donella H. Meadows helps explain why a column's meaning depends on its relationships and boundaries. Apply that habit to row granularity and joins. The book is not a tabular metadata specification and should not be cited as evidence for a particular dataset definition.
The Checklist Manifesto by Atul Gawande is useful for constructing a brief interpretation handover pause. The relevant questions are whether the recipient knows the row meaning, units and exceptions. Avoid turning the pause into a long form that repeats column names without clarifying their use.
Site Reliability Engineering, edited by Betsy Beyer, Chris Jones, Jennifer Petoff and Niall Richard Murphy, supplies broader context for observable systems and data integrity. Use that perspective to separate declared constraints from observed checks, while relying on the dataset's actual collection and transformation evidence for semantic claims.
Further exploration
Compare two versions of one dataset and identify changes in values, schema and meaning separately. A new category code, a changed population boundary and a corrected measurement can require different recipient actions. A short, version-bound explanation of those differences often preserves more usable knowledge than a much longer dictionary that presents the latest state without its interpretation history.
Primary evidence sources
Related concepts: systems thinking, wealth structure, and the survival loop. Read the editorial policy and citation guide.