Item request has been placed! ×
Item request cannot be made. ×
loading  Processing Request

Trade-offs between individual and ensemble forecasts of an emerging infectious disease.

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
  • Additional Information
    • Source:
      Publisher: Nature Pub. Group Country of Publication: England NLM ID: 101528555 Publication Model: Electronic Cited Medium: Internet ISSN: 2041-1723 (Electronic) Linking ISSN: 20411723 NLM ISO Abbreviation: Nat Commun Subsets: MEDLINE
    • Publication Information:
      Original Publication: [London] : Nature Pub. Group
    • Subject Terms:
    • Abstract:
      Probabilistic forecasts play an indispensable role in answering questions about the spread of newly emerged pathogens. However, uncertainties about the epidemiology of emerging pathogens can make it difficult to choose among alternative model structures and assumptions. To assess the potential for uncertainties about emerging pathogens to affect forecasts of their spread, we evaluated the performance 16 forecasting models in the context of the 2015-2016 Zika epidemic in Colombia. Each model featured a different combination of assumptions about human mobility, spatiotemporal variation in transmission potential, and the number of virus introductions. We found that which model assumptions had the most ensemble weight changed through time. We additionally identified a trade-off whereby some individual models outperformed ensemble models early in the epidemic, but on average the ensembles outperformed all individual models. Our results suggest that multiple models spanning uncertainty across alternative assumptions are necessary to obtain robust forecasts for emerging infectious diseases.
      (© 2021. The Author(s).)
    • References:
      Jones, K. E. et al. Global trends in emerging infectious diseases. Nature 451, 990–993 (2008). (PMID: 18288193596058010.1038/nature06536)
      Smith, K. F. et al. Global rise in human infectious disease outbreaks. J. R. Soc. Interface 11, 20140950 (2014). (PMID: 25401184422391910.1098/rsif.2014.0950)
      Bedford, J. et al. A new twenty-first century science for effective epidemic response. Nature 575, 130–136 (2019). (PMID: 31695207709533410.1038/s41586-019-1717-y)
      Iacono, G. L. et al. Using modelling to disentangle the relative contributions of zoonotic and anthroponotic transmission: the case of lassa fever. PLOS Negl. Trop. Dis. 9, 1–13 (2015). (PMID: 10.1371/journal.pntd.0003398)
      Quinn, T. C. Global burden of the HIV pandemic. Lancet 348, 99–106 (1996). (PMID: 867672610.1016/S0140-6736(96)01029-X)
      Perkins, T. A., Siraj, A. S., Ruktanonchai, C. W., Kraemer, M. U. G. & Tatem, A. J. Model-based projections of zika virus infections in childbearing women in the americas. Nat. Microbiol. 1, 16126 (2016). (PMID: 2756226010.1038/nmicrobiol.2016.126)
      Kraemer, M. U. G. et al. Spread of yellow fever virus outbreak in angola and the democratic republic of the congo 2015-2016: a modelling study. Lancet Infect. Dis. 17, 330–338 (2017). (PMID: 28017559533254210.1016/S1473-3099(16)30513-8)
      Metcalf, C. J. E. & Lessler, J. Opportunities and challenges in modeling emerging infectious diseases. Science 357, 149–152 (2017). (PMID: 28706037677607510.1126/science.aam8335)
      Lloyd-Smith, J. O., Schreiber, S. J., Kopp, P. E. & Getz, W. M. Superspreading and the effect of individual variation on disease emergence. Nature 438, 355–359 (2005). (PMID: 16292310709498110.1038/nature04153)
      Wesolowski, A. et al. Impact of human mobility on the emergence of dengue epidemics in Pakistan. Proc. Natl Acad. Sci. USA 112, 11887–11892 (2015). (PMID: 26351662458684710.1073/pnas.1504964112)
      Kraemer, M. U. G. et al. Utilizing general human movement models to predict the spread of emerging infectious diseases in resource poor settings. Sci. Rep. 9, 5151 (2019). (PMID: 30914669643571610.1038/s41598-019-41192-3)
      Mordecai, E. A. et al. Detecting the impact of temperature on transmission of zika, dengue, and chikungunya using mechanistic models. PLOS Negl. Trop. Dis. 11, 1–18 (2017). (PMID: 10.1371/journal.pntd.0005568)
      Nikolay, B. et al. Transmission of Nipah virus—14 years of investigations in Bangladesh. N. Engl. J. Med. 380, 1804–1814 (2019). (PMID: 31067370654736910.1056/NEJMoa1805376)
      Dudas, G., Carvalho, L. M., Rambaut, A. & Bedford, T. Mers-cov spillover at the camel-human interface. eLife 7, e31257 (2018). (PMID: 29336306577782410.7554/eLife.31257)
      Shea, K. et al. Harnessing multiple models for outbreak management. Science 368, 577–579 (2020). (PMID: 3238170310.1126/science.abb9934)
      Johansson, M. A. et al. An open challenge to advance probabilistic forecasting for dengue epidemics. Proc. Natl Acad. Sci. USA 116, 24268–24274 (2019). (PMID: 31712420688382910.1073/pnas.1909865116)
      Reich, N. G. et al. Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. PLOS Comput. Biol. 15, 1–19 (2019). (PMID: 10.1371/journal.pcbi.1007486)
      Reich, N. G. et al. A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States. Proc. Natl Acad. Sci. USA 116, 3146–3154 (2019). (PMID: 30647115638666510.1073/pnas.1812594116)
      McGowan, C. J. et al. Collaborative efforts to forecast seasonal influenza in the United States, 2015–2016. Sci. Rep. 9, 683 (2019). (PMID: 30679458634610510.1038/s41598-018-36361-9)
      Johnson, L. R. et al. Phenomenological forecasting of disease incidence using heteroskedastic Gaussian processes: a dengue case study. Ann. Appl. Stat. 12, 27–66 (2018). (PMID: 10.1214/17-AOAS1090)
      Del Valle, S. Y. et al. Summary results of the 2014-2015 DARPA Chikungunya challenge. BMC Infect. Dis. 18, 245 (2018). (PMID: 29843621597567310.1186/s12879-018-3124-7)
      Viboud, C. et al. The rapid Ebola forecasting challenge: synthesis and lessons learnt. Epidemics 22, 13–21 (2018). (PMID: 2895841410.1016/j.epidem.2017.08.002)
      ZIKAVAT Collaboration, et al. Preliminary results of models to predict areas in the Americas with increased likelihood of Zika virus transmission in 2017. Preprint at bioRxiv https://doi.org/10.1101/187591 (2017).
      Shea, K. et al. Covid-19 reopening strategies at the county level in the face of uncertainty: multiple models for outbreak decision support. Preprint at medRxiv https://doi.org/10.1101/2020.11.03.20225409 (2020).
      George, D. B. et al. Technology to advance infectious disease forecasting for outbreak management. Nat. Commun. 10, 3932 (2019). (PMID: 31477707671869210.1038/s41467-019-11901-7)
      Ray, E. L. et al. Ensemble forecasts of coronavirus disease 2019 (Covid-19) in the U.S. Preprint at medRxiv https://doi.org/10.1101/2020.08.19.20177493 (2020).
      Brooks, L. C., Farrow, D. C., Hyun, S., Tibshirani, R. J. & Rosenfeld, R. Nonmechanistic forecasts of seasonal influenza with iterative one-week-ahead distributions. PLOS Comput. Biol. 14, 1–29 (2018). (PMID: 10.1371/journal.pcbi.1006134)
      Chowell, G. et al. Real-time forecasting of epidemic trajectories using computational dynamic ensembles. Epidemics 30, 100379 (2020). (PMID: 10.1016/j.epidem.2019.100379)
      Siraj, A. S. et al. Spatiotemporal incidence of zika and associated environmental drivers for the 2015-2016 epidemic in Colombia. Sci. Data 5, 180073 (2018). (PMID: 29688216591428610.1038/sdata.2018.73)
      Kraemer, M. U. G. et al. The effect of human mobility and control measures on the covid-19 epidemic in china. Science 368, 493–497 (2020). (PMID: 3221364710.1126/science.abb4218)
      Black, A. et al. Genomic epidemiology supports multiple introductions and cryptic transmission of zika virus in Colombia. BMC Infect. Dis. 19, 963 (2019). (PMID: 31718580685289710.1186/s12879-019-4566-2)
      Rosenfeld, R. The EM Algorithm. http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/11761-s97/WWW/tex/EM.ps (1997).
      Dietze, M. C. Ecological Forecasting. (Princeton University Press, 2017).
      DeFelice, N. B., Little, E., Campbell, S. R. & Shaman, J. Ensemble forecast of human West Nile virus cases and mosquito infection rates. Nat. Commun. 8, 14592 (2017). (PMID: 28233783533310610.1038/ncomms14592)
      Yang, W., Karspeck, A. & Shaman, J. Comparison of filtering methods for the modeling and retrospective forecasting of influenza epidemics. PLOS Comput. Biol. 10, 1–15 (2014). (PMID: 10.1371/journal.pcbi.1003583)
      Baker, R. E., Yang, W., Vecchi, G. A., Metcalf, C. J. E. & Grenfell, B. T. Susceptible supply limits the role of climate in the early Sars-Cov-2 pandemic. Science 369, 315–319 (2020). (PMID: 32423996724336210.1126/science.abc2535)
      Cauchemez, S. et al. Local and regional spread of chikungunya fever in the Americas. Eur. Surveill. 19, 20854–20854 (2014). (PMID: 10.2807/1560-7917.ES2014.19.28.20854)
      Johansson, M. A., Powers, A. M., Pesik, N., Cohen, N. J. & Staples, J. E. Nowcasting the spread of chikungunya virus in the Americas. PLoS ONE 9, 1–8 (2014). (PMID: 10.1371/journal.pone.0104915)
      Moore, S. M. et al. Local and regional dynamics of chikungunya virus transmission in Colombia: the role of mismatched spatial heterogeneity. BMC Med. 16, 152 (2018). (PMID: 30157921611637510.1186/s12916-018-1127-2)
      Lai, S. et al. Seasonal and interannual risks of dengue introduction from south-east Asia into china, 2005–2015. PLOS Negl. Trop. Dis. 12, 1–16 (2018). (PMID: 10.1371/journal.pntd.0006743)
      Lindström, T., Tildesley, M. & Webb, C. A Bayesian ensemble approach for epidemiological projections. PLOS Comput. Biol. 11, 1–30 (2015). (PMID: 10.1371/journal.pcbi.1004187)
      Yamana, T. K., Kandula, S. & Shaman, J. Superensemble forecasts of dengue outbreaks. J. R. Soc. Interface 13, 20160410 (2016). (PMID: 27733698509520810.1098/rsif.2016.0410)
      McAndrew, T. & Reich, N. G. Adaptively stacking ensembles for influenza forecasting with incomplete data. https://forecasters.org/blog/2021/04/09/challenges-in-training-ensembles-to-forecast-covid-19-cases-and-deaths-in-the-united-states/ (2020).
      Ray, E. L. et al. Challenges in training ensembles to forecast covid-19 cases and deaths in the united states. Int. Inst. Forecasters (2021).
      Perkins, T. A. et al. Estimating unobserved Sars-Cov-2 infections in the united states. Proc. Natl Acad. Sci. USA 117, 22597–22602 (2020). (PMID: 32826332748672510.1073/pnas.2005476117)
      McAndrew, T., Wattanachit, N., Gibson, G. C. & Reich, N. G. Aggregating predictions from experts: a review of statistical methods, experiments, and applications. WIREs Comput. Stat. 13, e1514 (2021).
      Bodner, K., Fortin, M.-J. & Molnár, P. K. Making predictive modelling art: accurate, reliable, and transparent. Ecosphere 11, e03160 (2020). (PMID: 10.1002/ecs2.3160)
      Li, S.-L. et al. Essential information: Uncertainty and optimal control of Ebola outbreaks. Proc. Natl Acad. Sci. USA 114, 5659–5664 (2017). (PMID: 28507121546589910.1073/pnas.1617482114)
      Pei, S. & Shaman, J. Counteracting structural errors in ensemble forecast of influenza outbreaks. Nat. Commun. 8, 925 (2017). (PMID: 29030543564063710.1038/s41467-017-01033-1)
      Allard, A., Althouse, B. M., Hébert-Dufresne, L. & Scarpino, S. V. The risk of sustained sexual transmission of zika is underestimated. PLOS Pathog. 13, 1–12 (2017). (PMID: 10.1371/journal.ppat.1006633)
      Marinović, A. B., Swaan, C., van Steenbergen, J. & Kretzschmar, M. Quantifying reporting timeliness to improve outbreak control. Emerg. Infect. Dis. J. 21, 209 (2015). (PMID: 10.3201/eid2102.130504)
      Keeling, M. J. & Rohani, P. Modeling Infectious Diseases in Humans and Animals. (Princeton University Press, 2008).
      Figueiredo, L. T., Cavalcante, S. M. & Simões, M. C. Dengue serologic survey of schoolchildren in rio de janeiro, brazil, in 1986 and 1987. Bull. Pan Am. Health Organ. 24, 217–225 (1990). (PMID: 2379025)
      Lim, J. T., Han, Y., Lee Dickens, B. S., Ng, L. C. & Cook, A. R. Time varying methods to infer extremes in dengue transmission dynamics. PLOS Comput. Biol. 16, 1–19 (2020). (PMID: 10.1371/journal.pcbi.1008279)
      Sun, H. et al. Prevalent Eurasian avian-like H1N1 swine influenza virus with 2009 pandemic viral genes facilitating human infection. Proc. Natl Acad. Sci. USA 117, 17204–17210 (2020).
      Buckee, C. O. et al. Aggregated mobility data could help fight covid-19. Science 368, 145–146 (2020). (PMID: 3220545810.1126/science.abb8021)
      Bogoch, I. I. et al. Potential for zika virus introduction and transmission in resource-limited countries in Africa and the Asia-pacific region: a modelling study. Lancet Infect. Dis. 16, 1237–1245 (2016). (PMID: 27593584508642310.1016/S1473-3099(16)30270-5)
      Kraemer, M. U. G. et al. The global distribution of the arbovirus vectors Aedes aegypti and Ae. albopictus. eLife 4, e08347 (2015). (PMID: 26126267449361610.7554/eLife.08347)
      Wesolowski, A., Eagle, N., Noor, A. M., Snow, R. W. & Buckee, C. O. Heterogeneous mobile phone ownership and usage patterns in Kenya. PLoS ONE 7, 1–6 (2012). (PMID: 10.1371/journal.pone.0035319)
      Xia, Y., Bjørnstad, O. N. & Grenfell, B. T. Measles metapopulation dynamics: a gravity model for epidemiological coupling and dynamics. Am. Nat. 164, 267–281 (2004). (PMID: 1527884910.1086/422341)
      Oidtman, R. J. et al. Inter-annual variation in seasonal dengue epidemics driven by multiple interacting factors in Guangzhou, China. Nat. Commun. 10, 1148, https://doi.org/10.1038/s41467-019-09035-x (2019).
      Perkins, T. A., Metcalf, C. J. E., Grenfell, B. T. & Tatem, A. J. Estimating drivers of autochthonous transmission of chikungunya virus in its invasion of the Americas. PLOS Curr. Outbreaks (2017).
      Siraj, A. S. et al. Temperature modulates dengue virus epidemic growth rates through its effects on reproduction numbers and generation intervals. PLOS Negl. Trop. Dis. 11, 1–19 (2017). (PMID: 10.1371/journal.pntd.0005797)
      Simini, F., González, M. C., Maritan, A. & Barabási, A.-L. A universal model for mobility and migration patterns. Nature 484, 96–100 (2012). (PMID: 2236754010.1038/nature10856)
      Nordhaus, W. D. Geography and macroeconomics: new data and new findings. Proc. Natl Acad. Sci. USA 103, 3510–3517 (2006). (PMID: 16473945136368310.1073/pnas.0509842103)
      Arulampalam, M. S., Maskell, S., Gordon, N. & Clapp, T. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50, 174–188 (2002). (PMID: 10.1109/78.978374)
      Gneiting, T., Balabdaoui, F. & Raftery, A. Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. 69, 243–268 (2007). (PMID: 10.1111/j.1467-9868.2007.00587.x)
      Pei, S., Kandula, S., Yang, W. & Shaman, J. Forecasting the spatial transmission of influenza in the United States. Proc. Natl Acad. Sci. USA 115, 2752–2757 (2018). (PMID: 29483256585650810.1073/pnas.1708856115)
    • Grant Information:
      United Kingdom WT_ Wellcome Trust; 220414/Z/20/Z United Kingdom WT_ Wellcome Trust
    • Molecular Sequence:
      Dryad 10.5061/dryad.83nj1
    • Publication Date:
      Date Created: 20210911 Date Completed: 20211014 Latest Revision: 20230206
    • Publication Date:
      20240105
    • Accession Number:
      PMC8433472
    • Accession Number:
      10.1038/s41467-021-25695-0
    • Accession Number:
      34508077