The reliability of maternal audit instruments to assign cause of death in maternal deaths review process: a systematic review and meta-analysis

Cahyanti R.D., Widyawati W., Hakimi M.

Doctoral Program, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Obstetrics and Gynecology Department, Faculty of Medicine, Diponegoro University, Semarang, Indonesia; Pediatric and Maternity Nursing Department, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Obstetrics and Gynecology Department, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia


Background: Maternal Death Reviews (MDR) can assist in formulating prevention strategies to reduce maternal mortality. To support MDR, an adequate MDR instrument is required to accurately identify the underlying causes of maternal deaths. We conducted a systematic review and meta-analysis to determine the reliability of maternal death instruments for conducting the MDR process. Method: Three databases: PubMed, ProQuest and EBSCO were systematically searched to identify related research articles published between January 2004 and July 2019. The review and meta-analysis involved identification of measurement tools to conduct MDR in all or part of maternal audit. Eligibiliy and quality of studies were evaluated using the Modified Quality Appraisal of Diagnostic Reliability (QAREL) Checklist: Reliability Studies. Results: Overall, 242 articles were identified. Six articles examining the instrument used for MDR in 4 countries (4 articles on verbal autopsy (VA) and 2 articles on facility-based MDR) were included. None of studies identified reliability in evaluation instruments assessing maternal audit cycle as a comprehensive approach. The pooled kappa for the MDR instruments was 0.72 (95%CI:0.43–0.99; p < 0.001) with considerable heterogeneity (I2 = 96.19%; p < 0.001). Subgroup analysis of MDR instruments showed pooled kappa in VA of 0.89 (95%CI:0.52–1.25) and facility-based MDR of 0.48 (95%CI:0.15–0.82). Meta-regression analysis tended to show the high heterogeneity was likely associated with sample sizes, regions, and year of publications. Conclusions: The MDR instruments appear feasible. Variation of the instruments suggest the need for judicious selection of MDR instruments by considering the study population and assessment during the target periods. © 2021, The Author(s).

Facility-based MDR; Maternal death review; MDR instrument; Verbal autopsy


BMC Pregnancy and Childbirth

Publisher: BioMed Central Ltd

Volume 21, Issue 1, Art No 380, Page – , Page Count

Journal Link:

doi: 10.1186/s12884-021-03840-3

Issn: 14712393

Type: All Open Access, Gold, Green


Kumar, S., Kumar, N., Vivekadhish, S., Millennium development goals (MDGs) to sustainable development goals (SDGs): addressing unfinished agenda and strengthening sustainable development and partnership (2016) Indian J Community Med, 41 (1), pp. 1-4; Cha, S., The impact of the worldwide Millennium Development Goals campaign on maternal and under-five child mortality reduction: “Where did the worldwide campaign work most effectively? (2017) Glob Health Action, 10 (1), p. 1; Alkema, L., Chou, D., Hogan, D., Zhang, S., Moller, A.B., Gemmill, A., Global, regional, and national levels and trends in maternal mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the UN maternal mortality estimation inter-agency group (2016) Lancet., 387 (10017), pp. 462-474; Nov 7] (2017) Available From, ,; Smith, H., Ameh, C., Roos, N., Mathai, M., van den Broek, N., Implementing maternal death surveillance and response: a review of lessons from country case studies (2017) BMC Pregnancy Childbirth, 17 (1), pp. 1-11. , COI: 1:CAS:528:DC%2BC2sXitlOqu7w%3D; Say, L., Chou, D., Gemmill, A., Tunçalp, Ö., Moller, A.B., Daniels, J., Global causes of maternal death: a WHO systematic analysis (2014) Lancet Glob Health, 2 (6), pp. 323-333; van den Akker, T., Nair, M., Goedhart, M., Schutte, J., Schaap, T., Knight, M., Maternal mortality: direct or indirect has become irrelevant (2017) Lancet Glob Health, 5 (12), pp. e1181-e1182; Lewis, G., The cultural environment behind successful maternal death and morbidity reviews (2014) BJOG., 121, pp. 24-31; Hussein, J., Hirose, A., Owolabi, O., Imamura, M., Kanguru, L., Okonofua, F., Maternal death and obstetric care audits in Nigeria: a systematic review of barriers and enabling factors in the provision of emergency care (2016) Reprod Health, 13 (1), pp. 1-11; Mahmood, M.A., Mufidah, I., Scroggs, S., Siddiqui, A.R., Raheel, H., Wibdarminto, K., Root-cause analysis of persistently high maternal mortality in a Rural District of Indonesia: role of clinical care quality and health services organizational factors (2018) Biomed Res Int, 2018, pp. 1-11; (2013) Maternal Death Surveillance and Response: Technical guidance information for action to prevent maternal death, pp. 1-128. ,;jsessionid=D388980896CFC3023542CF8784FB2D12?sequence=1, WHO, Geneva: [cited 2018 Nov 1]. Available from:;jsessionid=D388980896CFC3023542CF8784FB2D12?sequence=1; (2011) Maternal death audit as a tool reducing maternal mortality. HNPNotes, p. 10. ,, [cited 2018 Oct 29]. Available from:; Pattinson, R.C., Bergh, A.M., Implementing recommendations arising from confidential enquiries into maternal deaths (2008) Best Pract Res Clin Obstet Gynaecol, 22 (3), pp. 477-487; Pattinson, R., Say, L., Makin, J., Bastos, M., Critical incident audit and feedback to improve perinatal and maternal mortality and morbidity (review) (2011) Cocharane Database Syst Rev Cochrane Libr, 4, pp. 4-15; Geller, S.E., Koch, A.R., Martin, N.J., Prentice, P., Rosenberg, D., Comparing two review processes for determination of preventability of maternal mortality in Illinois (2015) Matern Child Health J, 19 (12), pp. 2621-2626; Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P.A., The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration (2009) BMJ., 339, p. b2700; Sun, S., Meta-analysis of Cohen’s kappa (2011) Heal Serv Outcomes Res Methodol, 11 (3-4), pp. 145-163; Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D.M., Vandermeer, B., Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews (2017) BMC Med Res Methodol, 17 (1), pp. 1-12; Nussbaumer-Streit, B., Klerings, I., Dobrescu, A.I., Persad, E., Stevens, A., Garritty, C., Kamel, C., Gartlehner, G., Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study (2020) J Clin Epidemiol, 118, pp. 42-54; Lucas, N.P., Macaskill, P., Irwig, L., Bogduk, N., The development of a quality appraisal tool for studies of diagnostic reliability (QAREL) (2010) J Clin Epidemiol, 63 (8), pp. 854-861; Lucas, N., Macaskill, P., Irwig, L., Moran, R., Rickards, L., Turner, R., The reliability of a quality appraisal tool for studies of diagnostic reliability (QAREL) (2013) BMC Med Res Methodol, 13 (1), p. 1; Landis, J.R., Koch, G.G., The measurement of observer agreement for categorical data (1977) Biometrics., 33 (1), pp. 159-174; Fottrell, E., Byass, P., Ouedraogo, T.W., Tamini, C., Gbangou, A., Sombié, I., Högberg, U., Graham, W.J., Revealing the burden of maternal mortality: a probabilistic model for determining pregnancy-related causes of death from verbal autopsies (2007) Popul Health Metrics, 5 (1), pp. 1-9; Midhet, F., Validating the verbal autopsy questionnaire for maternal mortality in Pakistan (2008) Int J Health Sci (Qassim), 2 (1), pp. 91-96; Montgomery, A.L., Morris, S.K., Kumar, R., Jotkar, R., Mony, P., Bassani, D.G., Capturing the context of maternal deaths from verbal autopsies: A reliability study of the Maternal Data Extraction Tool (M-DET). Castro A, editor (2011) PLoS One, 6 (2). , COI: 1:CAS:528:DC%2BC3MXitFyhtrk%3D; Montgomery, A.L., Morris, S.K., Bassani, D.G., Kumar, R., Jotkar, R., Jha, P., Factors associated with physician agreement and coding choices of cause of death using verbal autopsies for 1130 maternal deaths in India (2012) PLoS One, 7 (3). , COI: 1:CAS:528:DC%2BC38XlsVagt7g%3D; Owolabi, H., Ameh, C.A., Bar-Zeev, S., Adaji, S., Kachale, F., van den Broek, N., Establishing cause of maternal death in Malawi via facility-based review and application of the ICD-MM classification (2014) BJOG., 121, pp. 95-101; Mgawadere, F., Unkels, R., van den Broek, N., Assigning cause of maternal death: a comparison of findings by a facility-based review team, an expert panel using the new ICD-MM cause classification and a computer-based program (InterVA-4) (2016) BJOG An Int J Obstet Gynaecol, 123 (10), pp. 1647-1653; (2012) ICD-10 to deaths during pregnancy, childbirth and the puerperium: ICD-MM. WHO Library, 129, pp. 30-33. ,, [cited 2019 Feb 21]. Available from:; Hadley, M.B., Tuba, M., Local problems; local solutions: an innovative approach to investigating and addressing causes of maternal deaths in Zambia’s Copperbelt (2011) Reprod Health, 8 (1), p. 17; Mitchell, C., Lawton, E., Morton, C., McCain, C., Holtby, S., Main, E., California pregnancy-associated mortality review: mixed methods approach for improved case identification, cause of death analyses and translation of findings (2014) Matern Child Health J, 18 (3), pp. 518-526; Merali, H.S., Lipsitz, S., Hevelone, N., Gawande, A.A., Lashoher, A., Agrawal, P., Audit-identified avoidable factors in maternal and perinatal deaths in low resource settings: a systematic review (2014) BMC Pregnancy Childbirth, 14 (280), pp. 2-12; Mgawadere, F., Unkels, R., Kazembe, A., van den Broek, N., Factors associated with maternal mortality in Malawi: application of the three delays model (2017) BMC Pregnancy Childbirth, 17 (1), pp. 1-9; Osungbade, K.O., Ige, O.K., Public health perspectives of preeclampsia in developing countries: implication for health system strengthening (2011) J Pregnancy, 2011, pp. 1-6; Lusambili, A., Jepkosgei, J., Nzinga, J., English, M., What do we know about maternal and perinatal mortality and morbidity audits in sub-Saharan Africa? A scoping literature review (2019) Int J Hum Rights Healthc, 12 (3), pp. 192-207; Bullivant, J., Corbett-nolan, A., (2010) Clinical audit: a simple guide for NHS Boards & partners, pp. 1-31. ,, Healthcare Quality Improvememt Partnership, London: [cited 2018 Dec 29]. Available from:; Pirkle, C.M., Dumont, A., Traore, M., Zunzunegui, M.-V., Validity and reliability of criterion based clinical audit to assess obstetrical quality of care in West Africa (2012) BMC Pregnancy Childbirth, 12 (1), pp. 118-129; (2002) Principles for best practice in clinical audit, pp. 13-200. ,, Radcliffe Medical Press, Oxon: [cited 2018 Apr 21]. Available from:

Indexed by Scopus

Leave a Comment