Stopping criteria for prioritised screening

The stopping criteria offers a reliable method to decide when to stop screening during machine learning-prioritised screening for a systematic review, or for document review in ediscovery.

Such a method is vital in order to use machine learning safely (by estimating the risk of missing documents). Our method describes statistical confidence levels that any given level of recall has not been achieved. As such, it offers a robust data-driven method to decide when to stop, and a clear and transparent way of communicating the risks of stopping at any given point. Read vignette(“stopping-criteria”) for more details on how this works in the package, or check out the original paper for a full description of the criteria, including theory and evaluations.

References & Source code

Buscar, short for Biased Urn based Stopping Criteria for technology Assisted Review, is an R and Python package, and a web app, implementing the stopping criteria described in

Callaghan, Max, and Finn Müller-Hansen. 2020. “Statistical Stopping Criteria for Automated Screening in Systematic Reviews.” Systematic Reviews. https://doi.org/10.21203/rs.2.18218/v2.

• BuscarR R package
• BuscarPy Python package
• Buscar Web application