Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds draft validation procedure #6

Merged
merged 2 commits into from
Mar 11, 2024

Conversation

alpha-beta-soup
Copy link
Member

@alpha-beta-soup alpha-beta-soup commented Mar 10, 2024

This is based strongly on ALUM, but also tempered by experience with validation in Northland.

I didn't actually know that ALUM had this process when doing this for Northland, so it's encouraging that we developed a pretty similar process. James and I took a stratified random sample of 1,000, and split it in half. We used MapAccuracy to do it efficiently. Where we deviated from ALUM is that we quite explicitly didn't do a validation or confusion matrix, but rather a "consistency" check where a land use was considered either "consistent" or "inconsistent" with aerial imagery. e.g. a field is not consistent with heavy industry, but various forms of agriculture are all consistent with a field, but we didn't assess anything stricter than "consistency". We did this because we only had ~4 year old aerial imagery and no local knowledge to go on. We would have preferred to have tasked others with validation, but we lacked any resource to do this.

For the framework, I think we ought to aim for actual validation, but bear in mind the principle of "practicality".

@alpha-beta-soup alpha-beta-soup added the enhancement New feature or request label Mar 10, 2024
@alpha-beta-soup alpha-beta-soup added this to the draft-framework milestone Mar 10, 2024
@brandonnodnarb brandonnodnarb merged commit c07411d into main Mar 11, 2024
1 check passed
@alpha-beta-soup alpha-beta-soup deleted the framework/procedure/validation branch March 14, 2024 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants