MetaBayesDTA (v1.5.2)

FULL RELEASE - MetaBayesDTA has now left BETA

Bayesian meta-analysis of diagnostic test accuracy data, with or without a gold standard


This is an extension of the frequentist version of the app, MetaDTA, which is described in this paper: Patel A, Cooper NJ, Freeman SC, Sutton AJ. Graphical enhancements to summary receiver operating characteristic plots to facilitate the analysis and reporting of meta-analysis of diagnostic test accuracy data. Research Synthesis Methods 2020, https://doi.org/10.1002/jrsm.1439. which can be accessed at MetaDTA version 2.1.3


Which builds on the previous version as described in the paper: Freeman SC, Kerby CR, Patel A, Cooper NJ, Quinn T, Sutton AJ. Development of an interactive web-based tool to conduct and interrogate meta-analysis of diagnostic test accuracy studies: MetaDTA. BMC Medical Research Methodology 2019; 19: 81 which can be accessed at MetaDTA version 1.27.

If you use outputs or screenshots generated from MetaBayesDTA, please cite these papers, as well as the following paper for MetaBayesDTA itself: Cerullo E, Sutton AJ, Jones HE, Wu O, Quinn T, Cooper NJ. MetaBayesDTA: codeless Bayesian meta-analysis of test accuracy, with or without a gold standard


Enzo Cerullo, Suzanne Freeman, Clareece Nevill, Amit Patel, Terry Quinn, Alex Sutton, Nicola Cooper, Olivia Wu, Tom Morris, Ryan Field, Janion Nevill

For feedback/questions about this app please email the CRSU Team at apps@crsu.org.uk .

App powered by Rshiny with statistical analyses performed using Stan

( https://mc-stan.org/ )


An interactive primer on diagnostic test accuracy can be found at:

https://crsu.shinyapps.io/diagprimer/
Funding and Support Acknowledgement:

MetaBayesDTA is part of the Complex Reviews Synthesis Unit (CRSU) suite of evidence synthesis apps. The development of these apps is currently funded (majority) and overseen by the Evidence Synthesis Group @ CRSU (NIHR153934). The CRSU Evidence Synthesis Group is one of the groups funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis Programme. Further details of other funders and support, current and past, can be found on our GitHub page . The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

More information about the UK NIHR Complex Reviews Synthesis Unit (CRSU) can be found on our website


THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

User Guide


Please click on the link before for a YouTube video (with subtitles) which gives a tutorial of MetaBayesDTA: https://www.youtube.com/watch?v=UqouZ7EQc1w&t=36s&ab_channel=ESMARConf


We also recommend reading the associated paper for this application here: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-023-01910-y


The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2 ( https://training.cochrane.org/handbook-diagnostic-test-accuracy/current ) is an excellent resource for understanding the models utilised in this app. A guide to reproducing some of the examples in the handbook can be found here: Download Cochrane Guide


In addition to the above, more in-depth tutorials with examples may be coming soon, and will be posted here

Default maximum file size is 5MB

File options



Download example datasets

Standard Example
Quality Assessment Example
Covariate Example
Quality Assessment and Covariate Example

Please select a file to upload


The file should contain at least six columns. Labelling of columns is case sensitive.

The first column should be labelled author and contain the name of the study author. The author name must be unique for each study.

The second column should be labelled year and contain the year of publication.

The third column should be labelled TP and contain the number of patients with a true positive test result.

The fourth column should be labelled FN and contain the number of patients with a false negative test result.

The fifth column should be labelled FP and contain the number of patients with a false positive test result.

The sixth column should be labelled TN and contain the number of patients with a true negative test result.


Including quality assessment data (optional)

To allow the quality assessment results from the QUADAS-2 tool to be incorporated into the plots an additional seven columns are required.

The seventh column should be labelled rob_PS , representing the risk of bias in terms of the patient selection.

The eighth column should be labelled rob_IT , representing the risk of bias in terms of the index test.

The ninth column should be labelled rob_RS , representing the risk of bias in terms of the reference standard.

The tenth column should be labelled rob_FT , representing the risk of bias in terms of the flow and timing.

The eleventh column should be labelled ac_PS , representing the applicability concerns in terms of the patient selection.

The twelfth column should be labelled ac_IT , representing the applicability concerns in terms of the index test.

The thirteenth column should be labelled ac_RS , representing the applicability concerns in terms of the reference standard.

These columns should contain the numbers 1, 2 or 3 which represent low, high or unclear risk of bias/applicability concerncs respectively.


For information about the QUADAS-2 tool and how to use it please visit:

https://www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-2/

Including covariates (optional)

If any covariates are to be added to the file, they should be included as the last columns in the file. If quality assessment data is not included in the file the covariates should be entered starting at the seventh column. If quality assessment data is included in the file the covariate data should be entered starting at the fourteenth column. Multiple covariates can be entered.


Note: Excel files should be saved in csv format and the separator option 'comma' selected for upload.

The default dataset, pre-loaded on the 'Data for Analysis' tab will be used for analysis if no file is selected. The 'Data for Analysis' tab will automatically update once a file is successfully loaded.

The default datasets can be downloaded using the buttons in the sidebar and used as templates to enter your own data.

Please add '.cat' to the end of the column names of categorical or discrete covariates.

Please add '.cts' to the end of the column names of any continuous covariates.

If reference test information is available (which is mandatory for meta-analysis without a gold standard), please add this as a categorical covariate named 'reference.cat'


Sensitivity analysis

To ensure the correct studies are excluded from sensitivity analyses please ensure that study data rows are ordered by the 'author' column alphabetically from A to Z prior to uploading to MetaBayesDTA (Excel can do this easily).


The default dataset uses data from a systematic review investigating the accuracy of an informant-based questionnaire, for detection of all cause dementia in adults. The dataset consists of thirteen studies assessing the use of the IQCODE (Informant Questionnaire on Cognitive Decline in the Elderly) tool for identifying adults with dementia within a secondary care setting.

The IQCODE tool contains a number of questions which are scored on a five point scale. The IQCODE tool has a number of different variants, depending on how many questions are asked. The questions are based on the performance of everyday tasks related to cognitive function. These are then rated on a scale of 1-5. The final score is an average score for each question. The IQCODE tool is only a screening tool and does not offer a definitive diagnosis of dementia.

Under the 'Select example dataset' option there are four different datasets to choose from. The default is the 'Standard' dataset, which includes the author and year of each study along with the true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN). The other options add data onto this 'Standard' dataset and highlight how datasets with quality assessment scores and/or covariates should be displayed.

With this dataset there are three different covariates. The first being the country in which each individual study was conducted. The second is the threshold used in each individual study. In this case if an individuals final score was higher than the threshold the individual was classified as having dementia and would require further diagnosis. The final covariate is labelled as 'IQCODE' and indicates which variant of the tool was used in each individual study. The variants are identified by the number of questions used in the questionnaire. There are three different variants the 16-item, 26-item and 32-item.



Priors






Table of prior distributions

NOTE: The default priors are:

For logit sensitivities and specificities - normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.05, 0.95)

For between-study SD's of logit sensitivities and specificities - truncated (at 0) normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.03, 2.25)

For between-study correlation between study-specific logit sensitivities and specificities - LKJ(2) prior, equivalent to 95% prior interval of (-0.8, 0.8)


Plot of prior distributions


















Note: If the presence of zero's for two of TP, FN, FP and TN causes sensitivity or specificity to be calculated as 0/0 than an error message will appear.




Note: Arrows to the right of the column headings can be used to sort data into ascending or descending order.

N is the total number of individuals in each study ( N = TP + FN + FP + TN )

Sens is the sensitivity, which is the probability of a positive test result given that the patient has the disease ( Sens = TP / [TP + FN] )

Spec is the specificity, which is the probability of a negative test result given that the patient does not have the disease ( Spec = TN / [TN + FP] )

Weight_Sens is the precentage study weight of sensitvitiy, calculated using methods by Burke et al.

Weight_Spec is the percentage study weight of sensitivity, calculated using methods by Burke et al.


Below are the parameter values required by Cochrane's RevMan software to construct plots in the ROC space for users who wish to include the analysis results as part of a Cochrane review.
Download Table

sROC plot

NOTE: The dotted line represents the 95% prediction region from the bivariate model; the greyed out area represents the 95% credible region from the bivariate model

; the lighter hollow diamond represents the summary estimate from the bivariate model for the sensitivity analysis



Download Plot

Note: The numbers in brackets represent 95% posterior intervals.



Sensitivity analysis



Download Plot

Note: The numbers in brackets represent 95% posterior intervals.


Priors:






Table of prior distributions

NOTE: The default priors are:

For continuous meta-regression:

For logit sensitivity and specificity intercepts - normal distribution with mean 0 and SD of 1.5, equivalent to a 95% prior interval of (-2.97, 2.9)

For logit sensitivity and specificity coefficients - normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (-1.96, 1.96)

For between-study SD's of logit sensitivities and specificities - truncated (at 0) normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.03, 2.25)

For between-study correlation between study-specific logit sensitivities and specificities - LKJ(2) prior, equivalent to 95% prior interval of (-0.8, 0.8)

For categorical meta-regression:

For logit sensitivities and specificities - normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.05, 0.95) on the probability scale

For between-study SD's of logit sensitivities and specificities - truncated (at 0) normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.03, 2.25)

For between-study correlation between study-specific logit sensitivities and specificities - LKJ(2) prior, equivalent to 95% prior interval of (-0.8, 0.8)

Plot of prior distributions










Note: If the presence of zero's for two of TP, FN, FP and TN causes sensitivity or specificity to be calculated as 0/0 than an error message will appear.



Note: Arrows to the right of the column headings can be used to sort data into ascending or descending order.

N is the total number of individuals in each study ( N = TP + FN + FP + TN )

Sens is the sensitivity, which is the probability of a positive test result given that the patient has the disease ( Sens = TP / [TP + FN] )

Spec is the specificity, which is the probability of a negative test result given that the patient does not have the disease ( Spec = TN / [TN + FP] )

Weight_Sens is the precentage study weight of sensitvitiy, calculated using methods by Burke et al.

Weight_Spec is the percentage study weight of sensitivity, calculated using methods by Burke et al.


Sampler diagnostics


R-hat statistics


                    

                    
                      
                      Download Table
                    
                    
NOTE:

                    
If there are any R-hat values > 1.05, try:

                    
Increasing the adapt_delta above the default of 0.80,

                    
Increasing the number of iterations above the default of 1500,

                    
Using more informative priors

                    
 

                    
If there are any divergent transitions, try:

                    
Increasing the adapt_delta above the default of 0.80,

                    
Using more informative priors

                    
 

                    
If there are any iterations that have exceeded the maximum treedepth, try increasing the treedepth above the default of 10

                    

                    
Posterior density plots

                    
                    

                    

                    
Trace plots

                    
                    

                  

                

              

            

            

              

                
                

                  

                    
Meta-regression - sROC plot

                    

                    
                    

                    
NOTE: The dotted line(s) represent the 95% prediction region from the bivariate model; 
       the greyed out area(s) represent the 95% credible region from the bivariate model

                    

                    

                  

                  

                    
Meta-regression - accuracy vs covariate plot

                    

                    
                    

                    

                    

                    

                  

                

              

            

          

          

            

              

                
                

                  

                    

                    

                    

                      

                        
                        
                      

                    

                    

                      
                      

                        

                          

                        

                      

                      
                      
                    

                    
Priors:

                    

                      
                      

                        

                          

                        

                      

                      
                      
                    

                    

                    

                    
                    
                    
                    

                    

                    
Table of prior distributions

                    

                    
NOTE: The default priors are:

                    
For logit sensitivities and specificities - normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.05, 0.95) on the probability scale

                    
For between-study SD's of logit sensitivities and specificities - truncated (at 0) normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.03, 2.25) 

                    
For between-study correlation between study-specific logit sensitivities and specificities - LKJ(2) prior, equivalent to 95% prior interval of (-0.8, 0.8)

                    
Plot of prior distributions

                    
                    

                    

                    
                    

                  

                  

                    

                    
                    
                    
                    

                    

                    

                    

                    
                    

                  

                  

                    

                    
Note: If the presence of zero's for two of TP, FN, FP and TN causes sensitivity or specificity to be calculated
                                                                     as 0/0 than an error message will appear.

                    

                    
                    

                    

                    

                    
Note: Arrows to the right of the column headings can be used to sort data into ascending or descending order. 

                    
N is  the total number of individuals in each study ( N = TP + FN + FP + TN )

                    
Sens is the sensitivity, which is the probability of a positive test result given that
                                                                     the patient has the disease ( Sens = TP / [TP + FN] )

                    
Spec is the specificity, which is the probability of a negative test result given that
                                                                     the patient does not have the disease ( Spec = TN / [TN + FP] )

                    
Weight_Sens is the precentage study weight of sensitvitiy, calculated using methods by Burke et al.

                    
Weight_Spec is the percentage study weight of sensitivity, calculated using methods by Burke et al.

                    

                  

                  
                  

                    

                    

                    
R-hat statistics

                    

                    

                    
                      
                      Download Table
                    
                    
NOTE:

                    
If there are any R-hat values > 1.05, try:

                    
Increasing the adapt_delta above the default of 0.80,

                    
Increasing the number of iterations above the default of 1500,

                    
Using more informative priors

                    
 

                    
If there are any divergent transitions, try:

                    
Increasing the adapt_delta above the default of 0.80,

                    
Using more informative priors

                    
 

                    
If there are any iterations that have exceeded the maximum treedepth, try increasing the treedepth above the default of 10

                    

                    
Posterior density plots

                    
                    

                    

                    
Trace plots

                    
                    

                  

                

              

            

            

              

                
                

                  

                    
Subgroup analysis - sROC plot

                    
                    

                    
NOTE: The dotted lines represent the 95% prediction region from the bivariate model; 
       the greyed out areas represent the 95% credible region from the bivariate model

                    

                    

                  

                  

                    
Subgroup analysis - Accuracy vs subgroup plot

                    
                    

                    
NOTE: The vertical bars represent the 95% credible intervals for each subgroup from the bivariate model; 
       the black points represent the study-specific data points for sensitivity (left panel) and specificity (right panel)
       for each subgroup

                    

                    

                  

                

              

            

          

          

            

              

                
                

                  

                    

                      
                      

                        

                          

                        

                      

                      
                      
                    

                    

                    

                    

                    
Priors

                    

                      

                        
                        
                      

                    

                    

                      
                      

                        

                          

                        

                      

                      
                      
                    

                    

                    

                    
                    
                    
                    

                    

                    

                    

                    

                    

                    
Table of prior distributions

                    

                    
Priors for index test parameters:

                    

                    
Priors for reference test parameters (test-specific):

                    

                    
Priors for reference test parameters (shared):

                    

                    
Prior for prevelance:

                    

                    
NOTE: The default priors are:

                    
For logit sensitivities and specificities - normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.05, 0.95) on the probability scale 

                    
For between-study SD's of logit sensitivities and specificities - truncated (at 0) normal distribution with mean 0 and SD of 1, equivalent to a 95% prior interval of (0.03, 2.25) 

                    
For between-study correlation between study-specific logit sensitivities and specificities - LKJ(2) prior, equivalent to 95% prior interval of (-0.8, 0.8)

                    
For study specific prevalences - Beta(1,1), equivalent to a 95% prior interval of (0.03, 0.97)

                    

                    
Plot of prior distributions

                    
                    

                  

                  

                    

                    
                    
                    
                    

                    

                    

                    

                    

                    

                    

                    
                    

                    

                    

                    

                      

                        
                        
                      

                    

                    

                      

                    

                    

                    
                    

                    

                    

                    

                    

                  

                  

                    

                    
Note: If the presence of zero's for two of TP, FN, FP and TN causes sensitivity or specificity to be calculated
                                     as 0/0 than an error message will appear.

                    

                    

                    
                    

                    

                    

                    
Note: Arrows to the right of the column headings can be used to sort data into ascending or descending order. 

                    
N is  the total number of individuals in each study ( N = TP + FN + FP + TN )

                    
Sens is the sensitivity, which is the probability of a positive test result given that 
                                     the patient has the disease ( Sens = TP / [TP + FN] )

                    
Spec is the specificity, which is the probability of a negative test result given that
                                     the patient does not have the disease ( Spec = TN / [TN + FP] )

                    
Weight_Sens is the precentage study weight of sensitvitiy, calculated using methods by Burke et al.

                    
Weight_Spec is the percentage study weight of sensitivity, calculated using methods by Burke et al.

                    

                  

                  

                    

                  

                  

                    

                  

                

              

            

            

              

                
                

                  

                    
Latent class model - sROC plot

                    
                    

                    

                    

                    

                    

                  

                

              

            

          

          

            

              
References

              

              

                Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. 
                    Bivariate analysis of sensitivity andspecificity produces informative summary
                    measures in diagnostic reviews. Journal of Clinical Epidemiology, 2005
              

              

                Carolyn M. Rutter and Constantine A. Gatsonis. “A hierarchical regression approach to meta-
                analysis of diagnostic test accuracy evaluations”. In: Statistics in Medicine (2001). issn: 02776715.
                doi: 10.1002/sim.942
              

              

                Harbord R. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007;8:239-251
              

              

                Haitao Chu, Sining Chen, and Thomas A. Louis. “Random effects models in a meta-analysis
                of the accuracy of two diagnostic tests without a gold standard”. In: Journal of the American
                Statistical Association (2009). issn: 01621459. doi: 10.1198/jasa.2009.0017
              

              

                 J. Menten, M. Boelaert, and E. Lesaffre. “Bayesian meta-analysis of diagnostic tests allowing for
                 imperfect reference standards”. In: Statistics in Medicine (2013). issn: 10970258. doi: 10.1002/
                 sim.5959.
              

              

                Burke DL, Ensor J, Snell KI, van der Windt D, Riley RD. Guidance for deriving and presenting percentage
                        study weights in meta-analysis o
                            tagList(f test accuracy studies. Research synthesis methods. 2018 Jun.
              

              

                Harrison, J.K., Fearon, P., Noel-Storr, A.H., McShane, R., Stott, D.J. and Quinn, T.J., 2015. Informant 
                        Questionnaire on Cognitive Decline in the Elderly (IQCODE) for the diagnosis of dementia within a secondary 
                        care setting. Cochrane database of systematic reviews, (3).
              

              

                Partlett C, Takwoingi Y. Meta-analysis of test accuracy studies in R: a summary of user-written
                      programs and step-by-step guide to using glmer. Version 1.0. August 2016.
              

              

                QUADAS-2 tool
              

              

              

                Bob Carpenter et al. “Stan: A probabilistic programming language”. In: Journal of Statistical
                Software (2017). issn: 15487660. doi: 10.18637/jss.v076.i01.
              

              

                 Stan Modeling Language Users Guide and Reference Manual. https://mc-stan.org/docs/2_
                 25/reference-manual/. 2020.
              

              
Code

              

                The R code for the app is available on GitHub here