CommunityAI
A privacy-safe community-intelligence layer for every neighbourhood in England, built on open data alone and engineered so each claim survives due diligence: clients join it to their own data, in their own environment, and never hand over a single person's record.
- Period
- 2026–present
- Role
- Chief Technology Officer (Designate)
- Scale
- Organisation and platform
- Context
- Leadership, Product
- I1Purpose + procurement
- I2Product + delivery
- I3Evidence + governance
- O1Decision criteria
- O2Target architecture
- O3Implementation roadmap
Chief Technology Officer (Designate)
- Open-data lakehouse across 40-plus national sources covering the 33,755 England LSOAs, with all 237 code commits to date (July 2026) authored by me.
- Honest validation on real data: wellbeing R squared 0.998 to minus 0.03, and life expectancy 0.997 to 0.675, once random CV is replaced by leave-region-out spatial block CV; every number re-runnable from the open pipeline (DuckDB, PySAL/esda, xgboost, shap).
- Orthogonality report computed on 33,755 LSOAs: PM2.5 84.7 per cent and NO2 82.5 per cent orthogonal to the deprivation baseline (genuinely incremental), against bad-health 39 and disability 49 per cent (restatement), each verdict measured and reported with its fallback share.
- Unified STAC catalogue (ML-Model extension) plus DCAT v3 export with per-feature licence governance and the data, processing and model versioning triple, served through a FastAPI API, a Python SDK and an MCP server for in-environment federated joins.
This public case records my technical mandate and the reproducible open-data architecture I built, not ownership of the platform. The community-index method pre-dates me and the founding team is collaborative. Named counterparties, client-sensitive work and gated demonstrations are deliberately withheld, described only by sector, and the platform's wider intellectual property sits with the venture and remains unsettled. The validation is predictive and honest, not a causal claim: the model shows that deprivation tracks life expectancy out of region, it does not estimate what changing any single indicator would do.
A mission-led venture wants to sell a community-intelligence layer, a signal about what each neighbourhood needs, to commercial commissioners and funders across grocery retail, consumer brands and media. The credibility problem is twofold and it matters because these buyers run sceptical due diligence: does the signal add anything a client's own deprivation and demographic model does not already contain, and can it be trusted without ever handling personal data? A prior partner's method looked excellent yet was largely an artefact, having predicted a deprivation index from its own components with no spatial cross-validation. The task was to turn a good idea into a dependable, auditable, privacy-safe platform whose every number is reproducible, built on clean open data.
My contribution
As the technical lead on the CTO track I set the technology and data architecture and personally wrote it: all 237 code commits to date (July 2026) are mine. I built an Apache Iceberg ACID medallion lakehouse (bronze, silver, gold) ingesting 40-plus national open sources across the 33,755 England LSOAs, with a three-way reproducibility contract binding every result: Iceberg snapshots version the data, a run id plus code hash and config hash version the processing, and MLflow together with a STAC ML-Model Item versions the models. I wrote a unified STAC catalogue with a DCAT v3 export and per-feature licence governance, and an honest spatial-ML suite: leave-region-out spatial block cross-validation, split and Mondrian conformal prediction, TreeSHAP explanations, a decorrelated geodemographic typology built to be demonstrably independent of deprivation, and an orthogonality measurement that scores, per signal, how much is genuinely incremental to a deprivation baseline rather than a restatement of it. I served the layer through a FastAPI API, a Python SDK and an MCP server so a client's own systems and agents join it to their data inside their own environment.
Decisions and constraints
Decisions I made
- Refuse to ship inflated accuracy. Report subjective wellbeing R squared falling from 0.998 under random cross-validation to minus 0.03 under leave-region-out spatial CV, naming the honest negative rather than hiding it, because spatial cross-validation is the only scorer that does not let a model peek across neighbouring areas.
- Build strictly on licence-clean open data with explicit claim boundaries, context rather than observed behaviour, and catalogue every dataset and model in STAC and DCAT so the whole pipeline is reproducible and audit-ready from the open stack.
- Serve federated: publish aggregate area features and let clients join on geography inside their own clean room. Never ingest personal data, and drop the ethnicity proxy as a proxy-discrimination hazard that added no predictive skill.
- Measure incrementality before selling it. Lead with the signals that are genuinely orthogonal to deprivation, air quality at roughly 85 per cent orthogonal, and demote signals that merely re-price it, bad-health and disability share, to modifiers rather than headline novelty.
Operating constraint
A collaborative founding team and an early-stage, IP-encumbered platform. The community-index method pre-dates my involvement, so clean-IP provenance had to be re-established through an open-data rebuild and my own model code. Area-level modelled data only, with no personal data ingested at any point, and only licence-clean sources (OGL, CC-BY, CC0, CDLA-Permissive) permitted in the served product.
Claim boundary: This public case records my technical mandate and the reproducible open-data architecture I built, not ownership of the platform. The community-index method pre-dates me and the founding team is collaborative. Named counterparties, client-sensitive work and gated demonstrations are deliberately withheld, described only by sector, and the platform's wider intellectual property sits with the venture and remains unsettled. The validation is predictive and honest, not a causal claim: the model shows that deprivation tracks life expectancy out of region, it does not estimate what changing any single indicator would do.
Evidence
- Open-data lakehouse across 40-plus national sources covering the 33,755 England LSOAs, with all 237 code commits to date (July 2026) authored by me.
- Honest validation on real data: wellbeing R squared 0.998 to minus 0.03, and life expectancy 0.997 to 0.675, once random CV is replaced by leave-region-out spatial block CV; every number re-runnable from the open pipeline (DuckDB, PySAL/esda, xgboost, shap).
- Orthogonality report computed on 33,755 LSOAs: PM2.5 84.7 per cent and NO2 82.5 per cent orthogonal to the deprivation baseline (genuinely incremental), against bad-health 39 and disability 49 per cent (restatement), each verdict measured and reported with its fallback share.
- Unified STAC catalogue (ML-Model extension) plus DCAT v3 export with per-feature licence governance and the data, processing and model versioning triple, served through a FastAPI API, a Python SDK and an MCP server for in-environment federated joins.