Is my new tracker really better than yours?

The problem of visual tracking evaluation is sporting an abundance of performance measures, which are used by various authors, and largely suffers from lack of consensus about which measures should be preferred. This is hampering the cross-paper tracker comparison and faster advancement of the field. In this paper we provide a critical analysis of the popular measures and evaluate them experimentally by a large-scale tracking experiment. We also analyze various visualizations of the performance measures. We show that several measures are equivalent from the point of information they provide for tracker comparison and, crucially, that some are more brittle than the others. Based on our analysis we narrow down the specter of measures to only a few complementary ones, thus pushing towards homogenization of the tracker evaluation methodology.