Document Type
Article
Publication Title
Oikos
Publisher
Wiley
Rights and Access Note
This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. In addition, no permission is required from the rights-holder(s) for educational uses. For other uses, you need to obtain permission from the rights-holder(s).
Publication Date
12-2012
Publisher location
Amsterdam, Netherlands
First Page
1
Last Page
6
Issue Number
3
Volume Number
122
Abstract/ Summary
Distribution models are used to predict the likelihood of occurrence or abundance of a species at locations where census data are not available. An integral part of modelling is the testing of model performance. We compared different schemes and measures for testing model performance using 79 species from the North American Breeding Bird Survey. The four testing schemes we compared featured increasing independence between test and training data: resubstitution, random data hold-out and two spatially segregated data hold-out designs. The different testing measures also addressed different levels of information content in the dependent variable: regression R2 for absolute abundance, squared correlation coefficient r2 for relative abundance and AUC/Somer’s D for presence/absence. We found that higher levels of independence between test and training data lead to lower assessments of prediction accuracy. Even for data collected independently, spatial autocorrelation leads to dependence between random hold-out test data and training data, and thus to inflated measures of model performance. While there is a general awareness of the importance of autocorrelation to model building and hypothesis testing, its consequences via violation of independence between training and testing data have not been addressed systematically and comprehensively before. Furthermore, increasing information content (from correctly classifying presence/absence, to predicting relative abundance, to predicting absolute abundance) leads to decreasing predictive performance. The current tests for presence/absence distribution models are typically overly optimistic because a) the test and training data are not independent and b) the correct classification of presence/absence has a relatively low information content and thus capability to address ecological and conservation questions compared to a prediction of abundance. Meaningful evaluation of model performance requires testing on spatially independent data, if the intended application of the model is to predict into new geographic or climatic space, which arguably is the case for most applications of distribution models.
Repository Citation
Bahn, Volker and McGill, Brian, "Testing the Predictive Performance of Distribution Models" (2012). Publications. 117.
https://digitalcommons.library.umaine.edu/mitchellcenter_pubs/117
Citation/Publisher Attribution
Bahn, V., & McGill, B.J. 2013. Testing the predictive performance of distribution models. Oikos 2013 122(3):321-331.
Publisher Statement
© 2012 The Authors
DOI
10.1111/j.1600-0706.2012.00299.x
Version
post-print (i.e. final draft post-refereeing with all author corrections and edits)
Rights Statement
