Automated Biosurveillance Data from England and Wales, 1991–2011

Doyo G. Enki, Angela Noufaily, Paul H. Garthwaite, Nick J. Andrews, André Charlett, Chris Lane, and C. Paddy Farrington

Author affiliations: Author affiliations: Open University, Milton Keynes, UK (D.G. Enki, A. Noufaily, P.H. Garthwaite, C.P. Farrington); Health Protection Agency, London, UK (N.J. Andrews, A. Charlett, C. Lane)

Abstract

Outbreak detection systems for use with very large multiple surveillance databases must be suited both to the data available and to the requirements of full automation. To inform the development of more effective outbreak detection algorithms, we analyzed 20 years of data (1991–2011) from a large laboratory surveillance database used for outbreak detection in England and Wales. The data relate to 3,303 distinct types of infectious pathogens, with a frequency range spanning 6 orders of magnitude. Several hundred organism types were reported each week. We describe the diversity of seasonal patterns, trends, artifacts, and extra-Poisson variability to which an effective multiple laboratory-based outbreak detection system must adjust. We provide empirical information to guide the selection of simple statistical models for automated surveillance of multiple organisms, in the light of the key requirements of such outbreak detection systems, namely, robustness, flexibility, and sensitivity.

The past decade has witnessed much interest in real-time outbreak detection methods for infectious diseases, driven by worries about the possibility of large-scale bioterrorism, public concern about emerging and reemerging infections, and the increased availability of computerized data (1–3). More prosaically, outbreaks of commonly occurring pathogens, notably, those causing infectious intestinal disease, remain a serious public health issue, causing an appreciable number of deaths and imposing a substantial drain on public health resources in many countries (4,5).
In England and Wales, automated laboratory surveillance of infectious diseases has been undertaken since the early 1990s. Laboratory surveillance is based on counts of laboratory isolates of infectious pathogens, usually classified for epidemiologic purposes by subtype or phage type. The organism reports come mainly from samples sent to hospital laboratories or to specialist laboratories when additional typing is required, as for salmonellae.
This automated system was designed to supplement the frontline investigator-led outbreak detection methods used by national and regional epidemiologists, with the primary aim of identifying geographically distributed outbreaks that may have escaped local detection. In a typical week, several hundred different pathogens are reported; the automated system provides a back-up and the assurance that the entire database is routinely scanned. The output comprises a short list of organisms with potential outbreaks for review, ranked according to an exceedance score that measures the degree of statistical aberrance. The statistical methodology of the system was described previously (6) and has since been applied in Scotland (7) and in several other European countries (8).
Much research on statistical methods of prospective outbreak detection has been aimed at identifying unusual clusters of 1 syndrome or disease (9–12), and some work has focused on multivariate surveillance methods (12). However, little research has been directed toward developing outbreak detection methods that are suited to large, multiple surveillance systems involving thousands of different organisms, such as the system used in England and Wales.
We are reviewing the statistical methods used in the England and Wales system. The first stage of this review, reported here, has been to carry out a detailed analysis of the data accumulated over the 2 decades since 1991. We aimed to document some of the generic features of surveillance data and their imperfections across the range of organisms of interest and to identify the key problems confronting automated outbreak detection systems. Specifically, we endeavored to answer 2 key questions: How diverse are the patterns displayed by the range of organisms monitored? How complex must a statistical algorithm be to handle this diversity?