Table 3 MMAT data evaluation of included empirical studies [37]

Qualitative studies

Is the qualitative approach appropriate to answer the research question?

Are the qualitative data collection methods adequate to address the research question?

Are the findings adequately derived from the data?

Is the interpretation of results sufficiently substantiated by data?

Is there coherence between qualitative data sources, collection, analysis, and interpretation?

Boet et al. (2013) [15] – qualitative component

No

No

Can’t tell

No

No

Boet et al. (2016) [16]

Yes

Yes

Yes

Yes

Yes

Quick (2016) [42] – qualitative component

Yes

No

Can’t tell

Yes

Yes

Quantitative RCTs

Is randomization appropriately performed?

Are the groups comparable at baseline?

Are there complete outcome data?

Are outcome assessors blinded to the intervention provided?

Did the participants adhere to the assigned intervention?

Andrews et al. (2019) [43]

Can’t tell

Can’t tell

Yes

No

Yes

Boet et al. (2013) [15] – RCT component

Yes

Yes

Yes

Yes

Yes

Ha (2020) [44]

No

Yes

Yes

No

Yes

Ha & Lim (2018) [45]

Can’t tell

Can’t tell

Yes

No

Yes

Kim & De Gange (2018) [46]

Can’t tell

Yes

Yes

No

Yes

Kündig et al. (2020) [47]

Can’t tell

Can’t tell

Yes

Can’t tell

Yes

Oikawa et al. (2016) [48]

No

No

Yes

Yes

Yes

Rueda-Medina et al. (2020) [49]

No

Yes

Yes

No

Yes

Rueda-Medina et al. (2021) [50]

No

Yes

Yes

No

Yes

Quantitative non-RCTs

Are the representatives of the target population appropriate?

Are measurements appropriate regarding both the outcome and intervention (or exposure)?

Are there complete outcome data?

Are the confounders accounted for in the design and analysis?

During the study period, is the intervention administered (or exposure occurred) as intended?

Kang & Yu (2018) [51]

Yes

Yes

Yes

Can’t tell

Yes

Lee et al. (2020) [52]

Yes

Yes

Yes

Can’t tell

Yes

Na & Roh (2021) [53]

Yes

Yes

Yes

Can’t tell

Yes

Paige et al. (2021) [54]

Yes

Yes

Can’t tell

Yes

Schreiber et al. (2020) [55]

Yes

No

Yes

Can’t tell

Yes

Tutticci et al. (2017) [56]

Yes

Yes

No

Can’t tell

Yes

Quantitative descriptive studies

Is the sampling strategy relevant to address the research question?

Is the sample representative of the target population?

Are the measurements appropriate?

Is the risk of nonresponse bias low?

Is the statistical analysis appropriate to answer the research question?

Curtis et al. (2016) [57]

Yes

Yes

No

Can’t tell

No

Quick (2016) [42]– quantitative component

Yes

Yes

Yes

Yes

Yes

Mixed methods studies

Is there an adequate rationale for using a mixed methods design to address the research question?

Are the different components of the study effectively integrated to answer the research question?

Are the outputs of the integration of qualitative and quantitative components adequately interpreted?

Are divergence and inconsistencies between quantitative and qualitative results adequately addressed?

Do the different components of the study adhere to the quality criteria of each tradition of the methods involved?

Boet et al. (2013) [15]

No

No

No

Yes

No (RCT – Yes // Qualitative – No)

Quick (2016) [42]

Yes

Yes

Yes

Yes

No (RCT – Yes // Qualitative – No)