Construction of an Artificial Intelligence system for evaluating scientific evidence in health
Evidence-Based Medicine. GRADE Approach. Systematic Reviews as Topic. Machine Learning. Deep Learning.
Medical practice involves constant decision-making processes. For these actions to be more assertive, the conscious use of literature is fundamental. However, the exponential growth of published scientific articles exceeds human capacity for interpretation, requiring innovative approaches. With this in mind, the aim of this study was to explore the use of artificial intelligence (AI) in the analysis of medical literature and to develop and evaluate a system for semi-automated use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to classifying systematic reviews by level of evidence. Firstly, a scoping review was conducted to identify the state of the art of using AI to analyze medical literature. Subsequently, a functional AI-based system was developed to identify the domains of imprecision, risk of bias, inconsistency and methodological quality of the review and then classify it by level of evidence in real scenarios. The evaluation of the application of the system was carried out on a sample of Cochrane systematic reviews. The results were one published scoping review, a second article in the publication phase and a registered digital product. The system obtained an overall accuracy of 63.2% and a kappa of 0.44 (moderate) for GRADE classification. These results were higher than those found in the literature, demonstrating that the system can be used to semiautomate the GRADE approach, as an auxiliary tool for human evaluators, seeking greater speed and a reduction in inconsistencies. Further work should be carried out to expand its use in different scenarios and adapt the dimensions and criteria for the GRADE approach based on the results. The latest AI technologies will be indispensable in these new directions.