address-errors

address-errors

Generate tunable, realistic UK address recording errors from your own structured addresses, using bundled learned error profiles. No UPRN and no OS data are needed to apply corruption. Keep your own record identifiers and the output is a labelled synthetic benchmark: every corrupted string says which input record it came from, which errors were requested, which were applied and which could not be.

Training a new empirical error profile is a separate, advanced workflow. It needs paired reference and recorded addresses whose correspondence you trust; the supplied OS workflow uses UPRNs for that pairing.

pip install address-errors
address-errors corrupt --components example --seed 1 --output demo.csv

The second line corrupts a bundled, invented town of 261 addresses, so you can see the output before you use any of your own data.

Status, 16 October 2026: version 0.1.1 (documentation and metadata update of 0.1.0; the generator is unchanged).

Contents: What you need · Corrupt your own addresses · Quick example · Controls · Output · Starting from an OS NGD order · House style and rates · What it reproduces · How well it works · Profiles · Training your own profile · Data, licences and disclosure.

What you need

You need Python 3.9 or later. The corruptor uses the standard library only, so pip install address-errors has no dependencies. If address-errors is not found after installing, the environment you installed into is not active; python -m address_errors.cli --help works either way.

Corrupt your own addresses

Make a CSV with these columns, one row per address. Empty cells are fine.

UPRNs are treated as text throughout; never cast IDs to numbers.

Example (my_addresses.csv, invented addresses):

uprn,subname,name,number,streetname,townname,postcode
rec1,FLAT 2,,18,HIGH STREET,EXAMPLETOWN,ZZ1 2CD
rec2,,ROSE COTTAGE,,MILL LANE,EXAMPLETOWN,ZZ1 3EF
rec3,,,7,HIGH STREET,EXAMPLETOWN,ZZ1 2CD

Command line:

address-errors corrupt --components my_addresses.csv --seed 1 --output corrupted.csv

Python:

from address_errors import Profile, Corruptor, Controls, load_components

recs = load_components('my_addresses.csv')            # or example_path() for the invented town
gen = Corruptor(Profile.load(), seed=1)               # pooled rates of the bundled profile
gen.index(recs)
rows = gen.generate([r['uprn'] for r in recs])        # one dict per input record
print(gen.stats)                                      # records, touched, applied and failed edits

source_uprn and assigned_uprn in the output hold your IDs. Join them back to your data to keep any other fields. On the command line, records are taken as residential when classificationcode is empty or starts with R; pass --uprns ids.txt (one ID per line) to choose them yourself.

What changes when you supply less

The corruptor works on whatever you give it, but some mechanisms need information you may not have:

• No parentuprn. Flats are still recognised from subname, but there are no building conventions (all units of a block written the same way) and no parent or sibling label drift.
• No usrn. No neighbour label drift.
• No postcode. No postcode errors, and every record is treated as urban for its rates.
• Collision checks only see your table. collision compares each corrupted dwelling and street with the other rows you supplied. no_match on a sample means "no collision with these rows", not "no collision in the real world"; not_tested means there was no street to compare. Corrupt a full reference (for example an OS extract) when collisions matter to your benchmark.
• Edits that cannot happen are recorded, not hidden. They appear in failed_rules, and gen.stats counts them.

Quick example

The invented town has houses, converted houses, blocks of flats and rural named properties, with record IDs from 900000000001 and postcodes in the non-existent ZZ area.

from address_errors import Profile, Corruptor, Controls, Noise, load_components, example_path

prof = Profile.load()                                     # bundled all-six profile; Profile.load('frozen') for the tested one
town = load_components(example_path())
gen = Corruptor(prof, style='leeds', seed=1)              # Leeds house style and Leeds rates
gen.index(town)
rows = gen.generate([r['uprn'] for r in town])
print(gen.stats)

gen = Corruptor(prof, style='leeds', seed=1, controls=Controls(prevalence=2.0, noise=Noise(rate=0.1)))

The same on the command line, with tuned controls:

address-errors corrupt --components example --style leeds --seed 1 --output demo.csv
echo '{"prevalence": 2, "noise": {"rate": 0.1}}' > controls.json
address-errors corrupt --components example --rates pooled --controls controls.json --output demo_tuned.csv

At the default settings about 6% of the town's records carry a profile error; Controls(prevalence=2.0) gives about 17%. The town is small, so these percentages move between seeds. Use a full address list for anything you report.

Controls

Every default is a no-op: Controls() reproduces the profile. Change one at a time.

Controls(prevalence=2.0)                         # more records touched, same mix
Controls(intensity=1.59)                         # the top of the development band
Controls(composition={'SU7': 3})                 # more dropped sub-units (the main to-parent collision route)
Controls(error_multiplier=3)                     # same volume of variation, three times the identity-shifting errors
Controls(commas=True)                            # match a source whose fields are joined with commas
Controls(sy1_cap=70)                             # a single 70-character field
Controls(noise=Noise(rate=0.2, max_ops=2))       # stress test beyond what councils do

On the command line, pass the same fields as JSON: --controls controls.json.

Output (truth table and audit)

The command line adds record_id (one per output row) and copy, and prints the run's counts (records, touched, applied and failed edits) to standard error.

What the truth columns mean

source_uprn always identifies the input record that was corrupted. That stays true when the corruption removed the information needed to identify it (a dropped flat number leaves the building's address) or produced a string equal to another address. It does not mean the string still describes that property unambiguously. When you score a matcher, use collision and failed_rules to separate such records and report them on their own. assigned_uprn answers a different question, "reproduce the register's label"; the two differ only under label drift.

Starting from an OS NGD order

If you have an OS NGD order (Address GB built address, alternate addresses and Royal Mail tables, as zips), convert it to the components table. UPRNs, parent UPRNs, USRNs and the building hierarchy are carried through:

pip install "address-errors[pipeline]"
address-errors build-reference --os-order OS_data/Wigan --la-gss E08000010 --output ref/wigan
address-errors corrupt --components ref/wigan/components.csv.gz --style leeds --n 50000 --copies 3 --output synthetic.csv.gz

A full reference gives the most complete output: building conventions, label drift and collision checks against every address in the area. --copies makes several independent synthetic sources of the same addresses. Output built from OS data falls under your own OS licence terms.

House style and rates

style is one of prof.styles() (wigan, bradford, leeds, derby, cornwall): how that council writes addresses, with its own error rates. With no style, or for a source the profile has never seen, use rates='pooled' (pooled rates at intensity 1.0, the study's setting for unseen councils) and report results across the intensity band prof.intensity_band() (0.35 to 1.44 for all6; 0.49 to 1.59 for frozen).

Guidance for a new source

• Separators. Generate and score the same string a matcher will see. If your source has separate fields, decide how they are joined before anything else, or set Controls(commas=True) to match. If you compare against real records, check punctuation first: a classifier that wins on commas alone is telling you about your cleaning.
• Intensity. Pooled intensity 1.0 suits a council like the development ones. For a rural or high-error source, measure its touch rate on a labelled sample and set intensity (or prevalence) to match. Report results across prof.intensity_band() either way, and say when your source is outside it.
• Same-building sharing. Results that depend on blocks of flats (sub-unit errors, collisions with the parent building) are the least certain. Report them with Controls(building_conventions=False) as a sensitivity.
• House style. With a few thousand labelled records from your source, you can refit its house style against the profile's core (training your own profile).

What it reproduces

Errors are applied as named mechanisms from the study taxonomy (v2): sub-unit (SU), building name (BN), number (PN), qualifiers (DQ), street (ST), text edits (TX), format (FM), label drift (LB), postcode (PC) and systemic truncation (SY1). Each output follows the study's design:

• Bundles, not single errors. Edits that occur together in real records are drawn together, and only offered to records where every edit can happen (a flat-number edit only lands on flats).
• Context. Rates depend on dwelling type, rural or urban, and the source council.
• Buildings are recorded one way. If one unit of a block is written differently, most of its other units are too, with the same edits. House-style edits (unit and floor wording, component order, brackets) and per-record edits are also shared within a building at the profile's measured concordance.
• House style. Case, commas, which locality, town and county words a source writes, street spelling conventions, floor wording, component order, Royal Mail vs NGD rendering.
• Label drift. A small pooled share of clean strings carries a neighbour's, parent's or sibling's ID.
• Collisions. Edits that make a string equal another address (a dropped flat number that leaves the parent building's address) happen at the real rate.
• Systemic truncation (SY1), applied last, for single-field sources with a length cap.

Also in the package, not realism-checked yet: postcode errors (PC1 to PC5; off with Controls(postcode='clean')), an inline collision check on every record, extra noise (off by default), and source_id, which fixes source-level conventions per synthetic source so that --copies gives each copy its own source.

How well it works

Two profiles are bundled. all6 (the default) is fitted on all six study councils (Wigan, Bradford, Leeds, Basingstoke and Deane, Derby and Cornwall); its check is leave-one-council-out. frozen was fixed before an external test on two councils it had never seen; the external-test results belong to it.

Default all6 profile, leave one council out (refit on five councils, test on the whole sixth with its own house style and pooled rates at intensity 1.0; full table: loco_results.md):

Fitting house style on the joined string removed the comma artefact: Derby's AUC is 0.58 here against 1.00 in the external test. Pooled rates fit typical councils; the quietest (Wigan) and noisiest (Cornwall) need their own intensity. This is not an external test: every fold's profile saw the other five councils.

Frozen profile, on the development councils (held-out buildings in Wigan, Bradford, Leeds and Basingstoke), a gradient-boosting classifier separates real from synthetic records with AUC 0.65 on all records and 0.83 on records with a discrepancy (0.5 means indistinguishable).

Frozen profile, on two councils never used in its development, each run once with pooled rates at intensity 1.0:

Nothing was refitted on either council. What the results mean:

• Rates and collisions transfer to a council like the development ones.
• Punctuation does not transfer. Both AUCs of 1.00 come from commas: the real strings had commas from the study's cleaning that the synthetic ones did not. With commas removed (a post hoc analysis), Derby is harder to tell apart than any development council.
• Same-building sharing is the weakest part, and it varies by council more than the development councils showed.
• Rural, high-error councils need their own intensity. Pooled rates give Cornwall half its real error rate.

Validation detail and code provenance: validation.md.

Profiles

Profile.load()                                          # bundled portable profile, fitted on all six councils
Profile.load('frozen')                                  # bundled portable frozen profile (external-test evidence)
Profile.load('v2')                                      # bundled portable v2
Profile.load().with_style('mysource', 'fit/mysource')   # add a house style you refitted
prof.export_portable('my_portable_profile')             # how the bundled ones were built

Profile v2 keys house style by section, then source; v3 keys it by source, so a source's whole style can be refitted and swapped in on its own, and adds style rates per eligible record, collision modifiers, building-name abbreviations and place text by postcode sector.

The bundled portable profiles contain mechanism IDs, rates, eligibility flags, clustering, label-drift and house-style parameters, and generic vocabulary. They contain no addresses and no UPRNs. Compared with the full study profile they leave out the per-street lookups (locality text and street spellings per USRN) and place names by postcode sector. The v3 profiles also ship building-name words with their short forms (single words, each seen at least 3 times). Without the lookups, locality text a source adds beyond NGD and PAF is not produced, and street spelling conventions are drawn generically. Each profile's PORTABLE.json lists what was removed.

Training your own profile (advanced)

You only need this to learn how a particular source records addresses. It needs your recorded addresses paired with reference addresses you trust. The supplied workflow (stage-records, mine, fit-house-style, refit-profile, realism-check) takes the pairing from UPRN labels against an OS NGD reference. Any reliable pairing would serve in principle, but the scripts expect UPRNs.

pip install "address-errors[pipeline]"     # staging, mining and refitting
pip install "address-errors[evaluate]"     # the real-vs-synthetic realism check

Full guide: pipeline.md.

Data, licences and disclosure

• Rates are discrepancies under the council's UPRN label, not verified error rates.

• Council records used, all under the Open Government Licence v3.0:

  • Wigan, Bradford, Leeds: published under OGL; profile development.
  • Basingstoke and Deane: released by the council under OGL v3; profile development.
  • Derby: the council confirms that information it owns is available for reuse under OGL, with no separate reuse request needed (FOI938460154, 18 August 2026); external test and the all6 refit.
  • Cornwall: the council states that its Council Tax dataset is open data under OGL; external test and the all6 refit.

  The bundled profiles hold aggregated parameters only; Basingstoke and Deane's council-level house style is not bundled.

• The study uses these records to estimate address discrepancy patterns and to evaluate linkage. Third-party address rights, and publication of outputs derived using OS reference data, remain subject to clarification under the project's OS agreement.

• Contains public sector information licensed under the Open Government Licence v3.0 (https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/), from Wigan, Bradford, Leeds, Basingstoke and Deane, Derby and Cornwall councils.

• Contains OS data © Crown copyright and database rights 2026; Royal Mail data © Royal Mail copyright and database right 2026; GeoPlace data © Local Government Information House Limited copyright and database right 2026.

• The example town is invented: its addresses, record IDs and ZZ postcodes do not exist.

• Address-level outputs you build from OS data fall under your own OS licence terms.