Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is based strongly on ALUM, but also tempered by experience with validation in Northland.
I didn't actually know that ALUM had this process when doing this for Northland, so it's encouraging that we developed a pretty similar process. James and I took a stratified random sample of 1,000, and split it in half. We used MapAccuracy to do it efficiently. Where we deviated from ALUM is that we quite explicitly didn't do a validation or confusion matrix, but rather a "consistency" check where a land use was considered either "consistent" or "inconsistent" with aerial imagery. e.g. a field is not consistent with heavy industry, but various forms of agriculture are all consistent with a field, but we didn't assess anything stricter than "consistency". We did this because we only had ~4 year old aerial imagery and no local knowledge to go on. We would have preferred to have tasked others with validation, but we lacked any resource to do this.
For the framework, I think we ought to aim for actual validation, but bear in mind the principle of "practicality".