Skip to content

Add Scotland's second homes and empty dwellings, QEP payment methods and Bands I and J (chronicle#435) - #436

Draft
juaristi22 wants to merge 2 commits into
mainfrom
uk-scotland-second-homes-435
Draft

juaristi22 wants to merge 2 commits into
mainfrom
uk-scotland-second-homes-435

Conversation

@juaristi22

@juaristi22 juaristi22 commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #435 (item 1) and #420 (items 12 and 13), the published facts Microcosm's next data PR (PolicyEngine/microcosm#1095) reads that are not yet in Chronicle. Still to come:

Five packages add 2,219 facts. Each comes from an official publication under the Open Government Licence v3.0, written by a generator script from pinned bytes; no microdata is read or stored.

Scotland's second homes and empty dwellings (#435 item 1)

scotgov-second-homes-empty-properties-2025 (396 facts), from the Scottish Government's Second homes and empty properties in September 2025 (published 16 December 2025). It ports all six tables of the web table, for Scotland and its 32 council areas, at September 2024 and September 2025:

  • second homes;
  • empty properties (empty for more than 6 months and liable for council tax);
  • long-term empty properties (12 months or more);
  • long-term empty properties with a discount below 10% or an increase;
  • unoccupied exemptions;
  • second homes plus empty properties.

Microcosm levels Scotland's council tax bills to the liability of every chargeable dwelling. These counts let it take out the second homes and empty dwellings, which no survey household lives in.

Semantics:

  • Each class is its own concept, scotgov.council_tax.<class>, quoting the cover sheet's definition and the workbook notes on it.
  • Each year is month YYYY-09 ("Data presented for September each year"), the month scotgov-council-tax-bands-2024 and -2025 give CTAXBASE's chargeable dwellings.
  • Second homes and empty properties are chargeable_dwelling, unoccupied exemptions exempt_dwelling. The provenance class is administrative (the issue's admin_aggregate).
  • gov.scot's caveats travel with the facts. Before the guidance was strengthened in September 2024, empty homes "will therefore be overstated in such instances" (note 10). South Ayrshire still includes "a very small number" of short-term empties (note 11), and Na h-Eileanan Siar's 2024 figures "are not as up to date as they would normally be" (note 13).

Parity:

  • Councils add to Scotland in every table and year but one misprinted cell. For long-term empty properties with a discount below 10% or an increase at September 2025, the Scotland row prints 25,073; its councils add to 25,703, the figure the edition's own text gives. The cell is ported as printed, and the generator fails if gov.scot corrects it.
  • Classes add up. Second homes plus empty properties is the total in every area and year, and the subsets nest.
  • No revisions. The September 2024 edition prints the same cells for every year to 2024.
  • Year-on-year changes reproduce. The key points' changes from 2024 match the packaged cells: second homes −679, empty properties +915, long-term empty +741, unoccupied exemptions −2,790.

At September 2024, Scotland had 21,606 second homes and 43,538 empty properties, of which 31,596 were long-term empty.

Not in the package:

Payment methods by region (#420 item 12)

Three packages from DESNZ's quarterly domestic energy price statistics (updated 29 September 2026), for a Direct Debit, standard credit and prepayment draw in Microcosm (PolicyEngine/uk-energy-price-cap-jan-2027#4):

  • desnz-qep-electricity-payment-methods-2026 (627 facts): table 2.4.2, standard electricity;
  • desnz-qep-time-of-use-electricity-payment-methods-2026 (627): table 2.4.3, Economy 7 and other time of use electricity;
  • desnz-qep-gas-payment-methods-2026 (561): table 2.5.2, gas.

They give the percentage of domestic customers paying by standard credit, Direct Debit and prepayment. Coverage is each quarter from April–June 2024 to April–June 2026, plus the annual sheets for 2024 and 2025.

  • Filters: payment method (standard_credit, direct_debit, prepayment), fuel, and the metering arrangement (standard, economy_7_and_time_of_use), which keeps tables 2.4.2 and 2.4.3 apart.
  • Regions: the 14 former PES areas, under the desnz:qep:pes ids and value ids the QEP unit cost packages already use. Gas uses them too, "for consistency with the electricity tables". The electricity tables add England and Wales, Scotland, Great Britain, Northern Ireland and the United Kingdom; the gas table stops at Great Britain.
  • Register: the label register gains the four nation rows these packages share.
  • Shares, not counts: each region's three shares are rounded percentages that add to 100 within 2 points. The survey of suppliers is "uplifted to Ofgem's total customer numbers".
  • Annual figures: table 2.4.3's annual figures "are averages of the rounded quarterly figures".
  • Not ported: earlier periods, the fixed-tariff sheets, and the supplier-type tables 2.4.1 and 2.5.1.

Bands I and J (#420 item 13)

scotgov-council-tax-high-value-bands-consultation-2026 (8 facts), from the Scottish Government's Council Tax High-Value Property Bands (Mansion Tax): consultation (6 July 2026), Figures 2 and 3. It holds the published low and high estimates for Scotland:

  • Band I (more than £1 million and up to £2 million): 12,680 to 16,740 properties at 1 April 2026 values, and £9.13m to £12.05m a year of additional revenue at the paper's illustrative rates on 2026-27 Band D charges.
  • Band J (more than £2 million): 760 to 1,070 properties, and £2.74m to £3.85m a year.

How the facts are typed:

  • Filters: band is council_tax_band (I, J) and the bound is statistic (low_estimate, high_estimate).
  • Properties are an observation at month 2026-04.
  • Revenue is a source_projection for fiscal year 2026-27, as the OBR's council tax surcharge costing is.
  • Provenance: both are model_output. The paper calls its figures "illustrative".

The chart labels print the low estimates for Band I and J, then the high ones. The tests check that order against the paper's own totals: "approximately 15,000 properties" and "between £12 million and £16 million per year".

Checks

  • Generators: --check reports each generator's committed YAML up to date.
  • Tests:
    • the three package test files (31);
    • test_dimension_value_register.py (7);
    • test_issue_376_one_label_per_value.py (48);
    • test_chronicle_bundle.py with the whole-bundle test deselected (34);
    • test_chronicle_boundaries.py (48);
    • test_issue_270_qep_energy_prices.py (17).
  • Register: the static check finds no problems.
  • Bundle pins move by the packages' own counts:
    • facts 2,245,294 (+2,219), packages 506, tables 448;
    • desnz 354,399, scotgov 7,058;
    • households 680,028, dwellings 708,936, government 14,075.
  • Unchanged: periods (610) and semantic duplicate keys (3,930). The period, entity and source pins each sum to the fact count. The whole-bundle test runs in CI.
  • R2: all five raw files are uploaded with chronicle publish-raw and read back with chronicle verify-raw.

🤖 Generated with Claude Code

juaristi22 and others added 2 commits November 10, 2026 16:42
…nicle#435 item 1)

The Scottish Government's "Second homes and empty properties in September 2025" web table
(16 December 2025), from the councils' council tax base (CTAXBASE) returns: all six tables,
for Scotland and its 32 council areas at September 2024 and September 2025, 396 facts. For
Microcosm's Scottish council tax level (PolicyEngine/microcosm#1095), which takes these
dwellings out of the liability no survey household lives in.

- Classes, each its own concept: second homes; empty properties (more than 6 months, liable
  for council tax); long-term empty (12 months or more); long-term empty with a discount
  below 10% or an increase; unoccupied exemptions; second homes plus empty properties.
- Month YYYY-09, as the CTAXBASE chargeable dwellings packages stamp September. Second homes
  and empty properties are chargeable_dwelling, unoccupied exemptions exempt_dwelling.
- The cover sheet's definitions and the workbook notes on each class travel in the facts'
  evidence notes: the overstated empty homes before 2024 (note 10), South Ayrshire (11),
  Na h-Eileanan Siar's 2024 figures (13) and the second homes premium powers (14).
- Parity, asserted by the generator and the tests: councils add to Scotland except one
  misprinted cell (long-term empty with a discount below 10% or an increase, September
  2025: Scotland prints 25,073, the councils and the edition's text give 25,703; ported as
  printed); second homes plus empty is the total; the subsets nest.
- The workbook is pinned and in R2 (publish-raw, verify-raw).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent methods and Bands I and J

Microcosm's next data PR (PolicyEngine/microcosm#1095) needs these two #420 items too, so
they ride chronicle#435's package PR.

- Item 12: DESNZ Quarterly Energy Prices tables 2.4.2 (standard electricity), 2.4.3
  (Economy 7 and other time of use) and 2.5.2 (gas), updated 29 September 2026. The
  percentage of domestic customers paying by standard credit, Direct Debit and prepayment,
  by former PES region and nation, each quarter from April 2024 to June 2026 and in 2024
  and 2025 (627, 627 and 561 facts). Regions reuse the unit cost packages' desnz:qep:pes
  ids and value ids; the register lists the England and Wales, Scotland, Great Britain and
  Northern Ireland rows these packages now share.
- Item 13: the Scottish Government's Band I and J consultation (6 July 2026), Figures 2 and
  3: the low and high estimates of properties at 1 April 2026 values (observation) and of
  additional revenue per year (source_projection), model_output, 8 facts. The chart labels'
  order is checked against the paper's own totals. Registers of Scotland's sales over £1m
  and £2m are not here: the RoS page refused the download (HTTP 403).
- Bundle pins: 2,245,294 facts, 506 packages, 448 tables; periods (610) and semantic
  duplicate keys (3,930) unchanged.
- All four raw files are pinned and in R2 (publish-raw, verify-raw).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22 juaristi22 changed the title Add Scotland's second homes and empty dwellings by council area (chronicle#435 item 1) Add Scotland's second homes and empty dwellings, QEP payment methods and Bands I and J (chronicle#435) Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant