Research
An empirical comparison of spatial scan statistics for outbreak detection
-
Correspondence: Daniel B Neill neill@cs.cmu.edu
HJ Heinz III College, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA
International Journal of Health Geographics 2009, 8:20 doi:10.1186/1476-072X-8-20
Published: 16 April 2009Abstract
Background
The spatial scan statistic is a widely used statistical method for the automatic detection of disease clusters from syndromic data. Recent work in the disease surveillance community has proposed many variants of Kulldorff's original spatial scan statistic, including expectation-based Poisson and Gaussian statistics, and incorporates a variety of time series analysis methods to obtain expected counts. We evaluate the detection performance of twelve variants of spatial scan, using synthetic outbreaks injected into four real-world public health datasets.
Results
The relative performance of methods varies substantially depending on the size of
the injected outbreak, the average daily count of the background data, and whether
seasonal and day-of-week trends are present. The expectation-based Poisson (EBP) method
achieves high performance across a wide range of datasets and outbreak sizes, making
it useful in typical detection scenarios where the outbreak characteristics are not
known. Kulldorff's statistic outperforms EBP for small outbreaks in datasets with
high average daily counts, but has extremely poor detection power for outbreaks affecting
more than 
of the monitored locations. Randomization testing did not improve detection power
for the four datasets considered, is computationally expensive, and can lead to high
false positive rates.
Conclusion
Our results suggest four main conclusions. First, spatial scan methods should be evaluated for a variety of different datasets and outbreak characteristics, since focusing only on a single scenario may give a misleading picture of which methods perform best. Second, we recommend the use of the expectation-based Poisson statistic rather than the traditional Kulldorff statistic when large outbreaks are of potential interest, or when average daily counts are low. Third, adjusting for seasonal and day-of-week trends can significantly improve performance in datasets where these trends are present. Finally, we recommend discontinuing the use of randomization testing in the spatial scan framework when sufficient historical data is available for empirical calibration of likelihood ratio scores.