A Framework of Quality Assessment Methods for Crowdsourced Geographic Information: a Systematic Literature Review
Mar 12th, 2017 by GIScienceHD
Crowdsourced Geographic Information (CGI) has emerged as a potential source of geographic information for different domains. Despite advantages associated with it, such information lacks quality assurance, since it is provided by different individuals. Several authors have investigated different approaches to assess CGI quality. Some of the existing methods have been summarized in different classification schemas. However, there are little works that propose a framework of the methods used to assess the quality of CGI in absence of authoritative data, yet some deal with specific cases e.g. for intrinsic OSM quality analysis (Barron et al. 2013) or for conceptual VGI (Ballatore & Zipf 2015). Based on a systematic literature review (similar to the approach of Steiger et al. 2015 for Twitter Analytics), in a recently accepted paper (Degrossi et al. 2017) we propose a framework of methods for assessing the quality of CGI without authoritative data. We identified 8 types of methods that can be used to assess the quality of different VGI sources. These are:
- Geographic context,
- Redundancy of volunteered contribution,
- Scoring volunteers’ contribution,
- Automatic location checking,
- Spatiotemporal clustering,
- Volunteer’s profile & reputation,
- Identify/correct by crowd and
- Historical data analysis.
In our framework, we categorize the methods according to
(i) the CGI source,
(ii) the type of reference data,
(iii) the approach employed and
(iv) the temporality of the method.
Unlike the works in the literature, here we provide a basis of methods that can be employed in each CGI source. By developing this framework, we aim at assisting the quality assessment in new and existing crowdsourcing-based platforms. The assessment is an important step in all CGI sources since the information comes from unknown sources and with unknown quality. In order to use the framework, three factors should be consider, i.e. the CGI source, the approach chosen, the reference data available, and the characteristics of the dataset. Besides this, the scientific community can also benefit from the results of our framework because it provides an overview of existing methods, but also possibilities for future research directions.
Degrossi, L.C.; Albuquerque, J.P.d.; Rocha, R.d.S; Zipf, A. (2017): A Framework of Quality Assessment Methods for Crowdsourced Geographic Information: a Systematic Literature Review. ISCRAM 2017. 14th International Conference on Information Systems for Crisis Response And Management. Albi, France. (accepted)
Ballatore, A. and Zipf, A. (2015): A Conceptual Quality Framework for Volunteered Geographic Information. COSIT 2015 - CONFERENCE ON SPATIAL INFORMATION THEORY XII, Santa Fe, NM, USA, October 12-16, 2015, Proceedings (pp. 89–107). Springer International Publishing. http://doi.org/10.1007/978-3-319-23374-1_5
Barron, C., Neis, P. & Zipf, A. (2013): A Comprehensive Framework for Intrinsic OpenStreetMap Quality Analysis. , Transactions in GIS, 18(6), 877–895. DOI: 10.1111/tgis.12073.
Steiger, E., de Albuquerque, J. P., & Zipf, A. (2015). An Advanced Systematic Literature Review on Spatiotemporal Analyses of Twitter Data. Transactions in GIS, 19(6), 809–834. http://doi.org/10.1111/tgis.12132