Add new 2011 Super Output Areas

[dracos]

ONS released new Super Output Area boundaries after the 2011 census. MapIt currently holds the 2001 boundaries only.
It may be worth considering this within a scheme to switch MapIt to using dates rather than the less-flexible/easy-to-understand generations, and also within #127.

[huwd]

Can I confirm the status of this, is there a plan to add the 2011 areas?

We'd be using the mappit API and have just realised that it's not in line with the latest versions!

[jacksonj04]

@huwd: it's unlikely to be in the next couple of weeks, but it's certainly something we want to be done by the end of the summer at the latest.

[huwd]

Great news! Thanks for letting me know.

[huwd]

Love the new design on mapit.mysociety.org (congrats to all involved).

I see 2001 Lower and Middle Super outputs - Is a 2011 refresh (with the smallest OA areas) still on the cards?

I'd be happy to explore getting relevant vectors and perhaps a pull request if I could have a quick chat with someone about repository layout?

[dracos]

Yes! I've already worked out where all the vectors are:
England and Wales Full Extent Lower: http://geoportal.statistics.gov.uk/datasets/da831f80764346889837c72508f046fa_1
England and Wales Full Extent Middle: http://geoportal.statistics.gov.uk/datasets/826dc85fb600440889480f4d9dbb1a24_1
England and Wales Full Clipped Lower: http://geoportal.statistics.gov.uk/datasets/da831f80764346889837c72508f046fa_0
England and Wales Full Clipped Middle: http://geoportal.statistics.gov.uk/datasets/826dc85fb600440889480f4d9dbb1a24_0
England and Wales Generalised (20m) Lower: http://geoportal.statistics.gov.uk/datasets/da831f80764346889837c72508f046fa_2
England and Wales Generalised Middle: http://geoportal.statistics.gov.uk/datasets/826dc85fb600440889480f4d9dbb1a24_2
England and Wales Super Generalised (200m) Lower: http://geoportal.statistics.gov.uk/datasets/da831f80764346889837c72508f046fa_3
England and Wales Super Generalised Middle: http://geoportal.statistics.gov.uk/datasets/826dc85fb600440889480f4d9dbb1a24_3
Scotland Output Areas: https://www.nrscotland.gov.uk/statistics-and-data/geography/our-products/census-datasets/2011-census/2011-boundaries plus https://www.nrscotland.gov.uk/statistics-and-data/geography/our-products/census-datasets/2011-census/2011-indexes if we want to map Data/Intermediate Zones (whichI think are the same as Super Output Areas)
NI: http://www.nisra.gov.uk/deprivation/super_output_areas.htm

E&W Output Areas are at http://geoportal.statistics.gov.uk/datasets/09b8a48426e3482ebbc0b0c49985c0fb_1 (full extent) http://geoportal.statistics.gov.uk/datasets/09b8a48426e3482ebbc0b0c49985c0fb_0 (full clipped) http://geoportal.statistics.gov.uk/datasets/09b8a48426e3482ebbc0b0c49985c0fb_2 (generalised 20m clipped) though we don't have the 2001 of those.

I don't know whether we want to load in the generalised ones or not, given we can simplify_tolerance any boundary anyway. But I imagine they are different in some way.

[huwd]

Any update?

[dracos]

Hi @huwd – sorry, I thought that you said you were going to be looking into it in your previous message, and I provided the data sources I had handy to help you. I haven't had the time to test how well the data imports into a MapIt installation, or the correct incantation of import to run, if you wanted to do that that would be great. Otherwise, while I am working on MapIt things at the moment, I can't guarantee when I'd be able to look at this.

[huwd]

@dracos so I did!
Do you have a quick primer on how you've used past iterations of these out anywhere or should I just dig in and see if it makes it past the tests?

[dracos]

Looking at my old email, it looks like the https://github.com/mysociety/mapit/blob/master/mapit_gb/management/commands/mapit_UK_import_soa.py script was created for this purpose, it was run four times for each combination of Lower/Middle Full/Generalized SOA.

However, I think nowadays the generic mapit_import script should handle importing these areas fine, if provided with the correct parameters. Hopefully its help is comprehensive enough, let me know if anything's unclear :) Hmm, apart from the country code, if England+Wales are supplied in one file, but that could be fixed manually afterwards, probably easier than creating a new management command to work out the right country (though that should be easy enough, first letter of the code).

If you had the full Lower England/Wales output file, then it would include arguments like: --country_code E --area_type_code OLF --name_type_code S --code_type gss
I don't know what the --name_field or --code_field would be without looking at the source (ogrinfo will be useful for that).

[huwd]

Much appreciated!

[dracos]

These have now been added in #344, sorry for the delay.