Crowdsourcing and machine learning: finding lost trials together


Workshop session 8: Oct 25 Concurrent Session E

Workshop category: 

  • Innovative solutions to challenges of evidence production
Date and Location


Tuesday 25 October 2016 - 14:00 to 15:30


Contact persons and facilitators

Contact person:


Noel-Storr A1, Thomas J2, Dooley G3, Mavergames C4, McDonald S5
1 Cochrane Dementia and Cognitive Improvement Group, Oxford University, United Kingdom
2 EPPI-Centre, UCL. 'Transform' project; CQIMG, United Kingdom
3 Metaxis Ltd, United Kingdom
4 Cochrane Central Executive Team, Germany
5 Cochrane Australia, Australia
Target audience

Is your workshop restricted to a specific audience or open to all Colloquium participants? : 


Target audience: 

Information Specialists, review authors, innovators in review production

Level of difficulty: 

Type of workshop

Length of workshop: 

90 Minutes


1. To outline the technical and methodological features of a pipeline that identifies research for systematic reviews – as soon as it is published.
2. For participants to have the opportunity to try some of the technologies described.
3. To promote discussion about the methodological and procedural issues this work raises.

Description: It is currently very time-consuming to find relevant studies for inclusion in systematic reviews. With ever-increasing volumes of research published each year, the problem of the data deluge is increasing. This workshop will open with a short introduction to two of the technologies developed as part of the ‘Transform’ project: i.e. the ‘evidence pipeline’ and ‘Cochrane Crowd’ platforms. Centralised searches will be conducted, and new citations fed into a machine learning platform. Automatic categorisation will determine the probability that a given citation is a randomized controlled trial and to which review/review group it is relevant. The Cochrane Crowd will then confirm/disconfirm the machine judgements creating a globally unique register of trials.

After a short introduction to the above technologies and processes, this workshop will facilitate the guided engagement of participants in trialling the online platforms. It will use this experience to kick-start a facilitated discussion regarding the methodological and organisational issues the new technologies raise. This is a Project Transform Team workshop.