human_computation

1. g a b c d e f h SC WY Our similarity measure for rock art, explicitly assumes that the human computation step can meaningfully extract the essential shape of the petroglyphs. It is non-obvious that this is true. One might imagine that the subjective notions of the humans in the loop might overwhelm any true object similarity that exists between the petroglyphs. We designed a detailed experiment to test this. We found eight petroglyphs which can objectively group into four pairs. We asked two volunteers, who we denote SC and WY, to annotate the eight petroglyphs using our petroannotator tool. The volunteers were computer science grad students, without any special knowledge or training in either anthropology or art. Neither of the volunteers met the other, or was allowed to see the others work. The figure above shows the eight original petroglyphs, and the transcribed skeletons produced by the two volunteers. Note that there is significant variation between the two volunteers output. The most notable of these is in the two deer denoted e and f. User WY transcribed these as stick figures, however user SC transcribed them as outlines.

2. However we obviously need our similarity measure to be robust to such human factors. We can test to see if it is by hierarchically clustering all sixteen transcribed petroglyphs. The results are very reassuring. At the lowest branches of the dendrogram, each of the two transcribed versions of an original petroglyph are linked with each other. Likewise, at the next higher level of the dendrogram, both versions of the birds (lizards/deer/sheep) join together in a sub-tree before merging with the rest of the animals. This dendrogram shows that our distance measure is very robust to human variability.