Detecting OSM Building Facades with Graffiti Artwork Based on Street View Images and Social Media using Deep Learning
Feb 5th, 2020 by GIScienceHD
As a recognized type of art, graffiti is a cultural asset and an important aspect of a city’s aesthetics. As such, graffiti is associated with social and commercial vibrancy and is known to attract tourists. However, positional uncertainty and incompleteness are current issues of open geo-datasets containing graffiti data. In a newly published paper, we present an deep learning approach towards detecting building facades with graffiti artwork based on the automatic interpretation of images from Google Street View (GSV).
It starts with the identification of geo-tagged photos of graffiti artwork posted on the photo sharing media Flickr. GSV images are then extracted from the surroundings of these photos and interpreted by a customized, i.e. transfer learned, convolutional neural network (CNN). The compass heading of the GSV images classified as containing graffiti artwork and the possible positions of their acquisition are considered for scoring building facades according to their potential of containing the artwork observable in the GSV images. More than 36,000 GSV images and 5,000 facades from buildings represented in OpenStreetMap were processed and evaluated for a case study in London (UK).
Precision and recall rates were computed for different facade score thresholds. False-positive errors are caused mostly by advertisements and scribblings on the building facades as well as by movable objects containing graffiti artwork and obstructing the facades. However, considering higher scores as threshold for detecting facades containing graffiti leads to the perfect precision rate.
Our approach can be applied for identifying previously unmapped graffiti artwork and for assisting map contributors interested in the topic. Furthermore, researchers interested on the spatial correlations between graffiti artwork and socio-economic factors can profit from our open-access code and results.
Novack T, Vorbeck L, Lorei H, Zipf A. (2020): Towards Detecting Building Facades with Graffiti Artwork Based on Street View Images. ISPRS International Journal of Geo-Information. 2020; 9(2):98.
