Skip to main content

Table 4 Risk of bias assessment of included studies

From: Warfarin use and stroke, bleeding and mortality risk in patients with end stage renal disease and atrial fibrillation: a systematic review and meta-analysis

Included Study

Judgment

Description

Chan 2009 [30]

 Bias due to confounding

Moderate

Analyses were adjusted for risk factors for stroke. These critically important domains were adjusted for using Cox regression analysis, and confirmed by propensity score adjusted analysis.

 Bias in selection of participants into the study

Serious

Study “excluded patients with <90 d of study enrollment” so there may be some selection bias. “Patient outcomes were followed from the date of analysis initiation”, and drug exposure status was determined in the first 90 days of dialysis. Start of follow up and start of intervention do not coincide for all participants.

 Bias in measurement of interventions

Low

Intervention status well defined and based on information collected at the time of intervention.

 Bias due to departures from intended interventions

Low

The primary analysis was intention-to-treat whereby patients were not re-classified, two validation analyses with censoring and time-varying Cox model were used to account for departures from intended interventions. “Similar results were noted when patients were censored when they changed their warfarin, clopidogrel or aspirin prescription after study enrollment.”

 Bias due to missing data

Low

Variables were identified from computerized medical results, so data were reasonably complete.

 Bias in measurement of outcomes

Low

Outcomes were identified from the diagnoses obtained from hospital discharge summaries or medical records. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Lai 2009 [32]

 Bias due to confounding

Critical

For the stroke outcome, study did not adjust for congestive heart failure. For the bleeding outcome, study did not adjust for liver disease, alcohol use and bleeding history. Adjusted analysis was not applied in comparing incidence of stroke and major bleeding episodes. Confounding inherently not controllable.

 Bias in selection of participants into the study

Serious

All participants who would have been eligible for the target trial were included (“no patients were excluded from the analysis”). Study included prevalent users, so a potential important amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

Serious

Intervention status well defined but the intervention status was based on current use vs no use determined retrospectively. It may be determined in a way that could have been affected by knowledge of the outcome.

 Bias due to departures from intended interventions

Serious

Bias due to departure from the intended intervention is expected, and is not adjusted for in the analyses.

 Bias due to missing data

No information

Medical charts of eligible patients treated in a medical center were reviewed. No information about loss to follow up or missing data.

 Bias in measurement of outcomes

Low

Outcomes were identified from the medical charts. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Wizemann 2010 [33]

 Bias due to confounding

Serious

Study adjusted for risk factors in Cox regression analysis, but not confirmed by additional analyses.

 Bias in selection of participants into the study

Critical

Study included patients who had pre-existing (i.e. a history of) AF at enrollment and patients who subsequently hospitalized with the diagnosis of AF. Study included prevalent users, so a substantial amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

Moderate

Intervention is well defined, but some aspects of the assignments of intervention status were determined retrospectively since self-reported medication use status may be subject to recall bias.

 Bias due to departures from intended interventions

No information

Study did not describe the analytical method in detail.

 Bias due to missing data

No information

Study was based on an international, observational study of HD facilities. No information about loss to follow up or missing data.

 Bias in measurement of outcomes

Moderate

Outcomes were identified from hospitalization or death records. The outcome measure may be minimally influenced by knowledge of the intervention received by study participants, since study was retrospective in nature.

 Bias in selection of the reported results

Serious

The outcome measurements and analyses were not clearly described. Study reported outcomes by age subgroups rather than the entire cohort. Study highlighted significant result in the highest risk age subgroup in the abstract.

Winkelmayer 2011 [22]

 Bias due to confounding

Moderate

Study adjusted for risk factors included in the CHADS2 and HAS-BLED score in the time-fixed Cox regression analysis, and confirmed using propensity score-adjusted analyses which yielded similar results.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of intervention and start of intervention coincide for all subjects (30 days after AF hospital discharge).

 Bias in measurement of interventions

Low

Intervention is well defined, and based solely on information collected at the time of intervention in the regional hospital discharge abstract and drug claims databases.

 Bias due to departures from intended interventions

Low

Study used intention-to-treat in which patients were “only censored for end of database” in the primary analysis. Study also used as-treated analysis which “censored patients at treatment cross-over”.

 Bias due to missing data

Low

Study obtained data from the national patient registry and regional healthcare claims database which contains information on relevant ESRD patients. Data were reasonably complete.

 Bias in measurement of outcomes

Low

Outcomes were identified using previously validated claims-based algorithm. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Olesen 2012 [23]

 Bias due to confounding

Serious

Study adjusted for risk factors included in the CHA2DS2-VASc and HAS-BLED score in the time-dependent Cox regression analysis, but not confirmed by additional analyses.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of intervention and start of intervention coincide for all subjects. “The baseline assessment and follow-up period began 7 days after discharge.”

 Bias in measurement of interventions

Low

Intervention is well defined, and based solely on information collected at the time of intervention in the national patient registry.

 Bias due to departures from intended interventions

Serious

Study uses time-dependent analysis, so bias due to departures from intended interventions is expected. Study did not adjust for switches, co-intervention or problems with implementation fidelity in the analyses.

 Bias due to missing data

Low

Study obtained data from the national patient registry which contains information on all residents. Data were reasonably complete.

 Bias in measurement of outcomes

Low

Outcomes were identified using claims-based algorithm. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Khalid 2013 [34]

 Bias due to confounding

Critical

Analyses did not adjust for important confounders such as hypertension, cerebrovascular disease, diabetes.

 Bias in selection of participants into the study

Critical

Participants re-started warfarin after experiencing bleeding outcomes, so their risk profile may be different. Study included prevalent users, so a potential important amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

Moderate

Intervention was not well defined. Not clear when the start of intervention was.

 Bias due to departures from intended interventions

Serious

Treatment could be discontinued due to experiencing bleeding outcomes. Study did not address bias due to departures from intended interventions.

 Bias due to missing data

No information

 

 Bias in measurement of outcomes

Low

Intervention not well defined, but were “adjudicated by two blind reviewers”.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Wakasugi 2014 [29]

 Bias due to confounding

Critical

For the stroke outcome, analysis was adjusted for CHADS2 score and matched by propensity score. For the major bleeding and all-cause mortality outcomes, analyses were not adjusted.

 Bias in selection of participants into the study

Serious

All participants who would have been eligible for the target trial were included. Study included prevalent users, so a potential important amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

Low

Intervention is well defined, and based solely on information collected at the time of intervention since study was conducted prospectively.

 Bias due to departures from intended interventions

Low

“The primary analysis was intention-to-treat in which patients who started using warfarin after study enrollment were not reclassified”, so the specified comparison relates to initiation of intervention regardless of whether it is continued. To account for possible longitudinal changes in drug prescription over time, study also repeated the analysis in which patients were censored when warfarin use changed.

 Bias due to missing data

No information

Medical charts of eligible patients treated in a medical center were reviewed. No information about loss to follow up or missing data.

 Bias in measurement of outcomes

Low

Outcomes were identified from the medical charts. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Bonde 2014 [24]

 Bias due to confounding

Serious

Analyses were adjusted for risk factors included in the CHA2DS2-VASc and HAS-BLED score with age as a continuous covariate. These critically important domains were adjusted for using Cox regression analysis, but not confirmed by additional analyses.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of follow up and start of intervention coincide for all subjects (follow-up began 7 days after discharge).

 Bias in measurement of interventions

Moderate

Intervention is well defined, but assignment of intervention status by “dividing the number of tablets dispensed with the estimated daily dosage” was determined retrospectively.

 Bias due to departures from intended interventions

Serious

“Treatment is often discontinued in terminal patients” so switches and discontinuation of treatment were apparent. Such departures from intended intervention may be a result of outcomes of interest. Study used time-dependent, as-treated Cox regression analysis, but did not adjust for such departures appropriately.

 Bias due to missing data

Low

Accurate data on all patients actively treated for end-stage CKD with RRT recorded in national registry, so data were reasonably complete.

 Bias in measurement of outcomes

Low

Outcomes were identified from ICD-8 or ICD-10 codes and the same method of outcome assessment was applied across intervention groups.

 Bias in selection of the reported results

Serious

Study stratified and analyzed results based on CHA2DS2-VASc score subgroups. Main result reported the high risk subgroup, which appears to be reported on the basis of significant result.

Carrero 2014 [25]

 Bias due to confounding

Moderate

Study accounted for the most important confounders and confirmed results by propensity score-matched analysis. Residual confounding is inherent in observational studies.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of intervention and start of intervention coincide for all subjects (starts with warfarin prescription at discharge).

 Bias in measurement of interventions

Low

Intervention is well defined, and based solely on information collected at the time of intervention.

 Bias due to departures from intended interventions

Low

“Warfarin use vs no warfarin use was considered as a time-fixed binary variable throughout the follow-up period.” Specified comparison relates to initiation of intervention regardless of whether it is continued as in intention-to-treat (ITT) analysis in target trial.

 Bias due to missing data

Low

“The use of unique personal identification number for all Swedish citizens and continuously updated national registries on death date, cause of death and emigration allow a virtually complete follow-up” with no loss to follow up.

 Bias in measurement of outcomes

Low

Outcomes were identified from the National Inpatient Registry and the same method of outcome assessment was applied across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Chen 2014 [26]

 Bias due to confounding

Moderate

Analyses were adjusted for risk factors for stroke. These critically important domains were adjusted for using propensity score matched analysis. Ischemic stroke/TIA history was excluded from the study population. Residual confounding is inherent in observational studies.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of intervention and start of intervention coincide for all subjects.

 Bias in measurement of interventions

Low

Intervention status well defined and based on information collected at the time of intervention.

 Bias due to departures from intended interventions

Low

The primary analysis was intention-to-treat, thus the specified comparison relates to initiation of intervention regardless of whether it is continued.

 Bias due to missing data

Low

Variables were identified from the universal national health insurance program, so data were reasonably complete.

 Bias in measurement of outcomes

Low

Outcomes were identified from the ICD diagnosis codes. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Friberg 2014 [13]

 Bias due to confounding

Moderate

Analyses were adjusted for risk factors for stroke and bleeding. These critically important domains were adjusted for using propensity score adjusted analysis. Residual confounding is inherent in observational studies.

 Bias in selection of participants into the study

Serious

Study included prevalent users (taking warfarin at baseline i.e. 5 months before, and up to 1 month after the index AF diagnosis), so an important amount of follow-up time is missing for prevalent users from analyses. Start of follow up and start of intervention do not coincide for all subjects (time at risk for survival analyses started on Day +14 after index AF diagnosis).

 Bias in measurement of interventions

Low

Intervention status well defined and based on information collected at the time of intervention.

 Bias due to departures from intended interventions

Low

The primary analysis was intention-to-treat, thus the specified comparison relates to initiation of intervention regardless of whether it is continued.

 Bias due to missing data

Low

Variables were identified from the national patient registry, so data were reasonably complete.

Bias in measurement of outcomes

Low

Outcomes were identified from the ICD diagnosis codes. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Shah 2014 [27]

 Bias due to confounding

Moderate

Study adjusted for risk factors included in the CHADS2 and HAS-BLED score in the time-fixed Cox regression analysis, and confirmed using propensity score-adjusted analyses which yielded similar results.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of follow up and start of intervention coincide for all subjects (30 days after AF hospital discharge).

 Bias in measurement of interventions

Low

Intervention is well defined, and based solely on information collected at the time of intervention in the regional hospital discharge abstract and drug claims databases.

 Bias due to departures from intended interventions

Low

Study uses time-fixed, intention-to-treat analysis, so the specified comparison relates to initiation of intervention regardless of whether it is continued.

 Bias due to missing data

Low

Study obtained data from the regional hospital and drug claims databases which contains information on all residents in the region. Data were reasonably complete.

 Bias in measurement of outcomes

Low

Outcomes were identified using claims-based algorithm. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Genovesi 2015 [31]

 Bias due to confounding

Moderate

Analyses were adjusted for risk factors for stroke and bleeding. These critically important domains were adjusted for using multivariate Cox regression analysis followed by propensity score matched analysis. Residual confounding is inherent in observational studies.

 Bias in selection of participants into the study

Serious

Study included prevalent users (taking anticoagulant at recruitment) and incident users (starting anticoagulant within 2 weeks following recruitment), so a potential important amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

Low

Intervention status well defined and based on information collected at the time of intervention.

 Bias due to departures from intended interventions

Low

The primary analysis was intention-to-treat, thus the specified comparison relates to initiation of intervention regardless of whether it is continued.

 Bias due to missing data

No information

Patients from 10 hemodialysis center were prospectively followed-up for 2 years. No information about loss to follow up or missing data.

 Bias in measurement of outcomes

Low

Outcomes were identified from medical charts using clinical diagnosis criteria. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Chan KE 2015 [20]

 Bias due to confounding

Moderate

Study accounted for the most important confounders and “mitigated potential bias from unmeasured factors by performing a matched analysis which supported the main findings of the study”. Residual confounding is inherent in observational studies.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of follow up and start of intervention coincide for all subjects (followed from the time the initiated anticoagulant treatment de novo).

 Bias in measurement of interventions

Low

Intervention status well defined and based on information collected at the time of intervention.

 Bias due to departures from intended interventions

Serious

Patient was censored with discontinuation of treatment medications. Study used Poisson regression which does not take into account of the potentially varying rate (hazard) ratio for the effect of intervention.

 Bias due to missing data

Low

“All subjects registered in the database are followed longitudinally, where data parameters are actively collected,” so data were reasonably complete.

 Bias in measurement of outcomes

Low

The methods of outcome assessment were comparable across intervention groups. Bleeding outcomes were adjudicated retrospectively, but “the adjudicator was blinded to patient, treatment group and outcome”.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Chan PH 2015 [21]

 Bias due to confounding

Serious

Study did not list variables adjusted in the multivariate Cox regression analysis. Results were not confirmed by additional analyses.

 Bias in selection of participants into the study

Critical

Study included patients who had pre-existing AF at enrollment. Study included prevalent users, so a substantial amount of follow-up time is missing for prevalent users from analyses. Study excluded patients with incomplete follow-up data, so there may be some selection bias.

 Bias in measurement of interventions

Low

Intervention is well defined, and based on medical records and discharge summaries at the time of medication prescription.

 Bias due to departures from intended interventions

Serious

Study compared current users vs non-users. Bias due to departure from the intended intervention is expected, but not adjusted for in the analysis.

 Bias due to missing data

Low

Study data were retrieved from the medical records and discharge summaries from the territory-wide information network of all public hospitals in the region. Study excluded patients with incomplete clinical and/or follow-up data, so data were reasonably complete.

 Bias in measurement of outcomes

Moderate

Outcomes were identified from medical records. The outcome measure may be minimally influenced by knowledge of the intervention received by study participants, since study was retrospective in nature.

 Bias in selection of the reported results

Moderate

The statistical analyses were not clearly described. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Shen 2015 [28]

 Bias due to confounding

Moderate

Study used Cox regression analysis in the propensity score matched analysis. History of stroke was not included in the analysis. Unmeasured confounding is inherent in observational studies.

 Bias in selection of participants into the study

Low

All participants who would have been eligible for the target trial were included. Start of intervention and start of intervention coincide for all subjects (30 days after AF hospital discharge).

 Bias in measurement of interventions

Low

Intervention is well defined, and based solely on information collected at the time of intervention in the national patient registry.

 Bias due to departures from intended interventions

Low

Study used intention-to-treat and as-treated analyses, and adjusted for selection bias using inverse probability of treatment and censoring-weighted (IPTW) analysis.

 Bias due to missing data

Low

Study obtained data from the national patient registry which contain information on all ESRD patients. Data were reasonably complete.

 Bias in measurement of outcomes

Low

Outcomes were identified using previously validated claims-based algorithm. The methods of outcome assessment were comparable across intervention groups.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were clearly defined. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Wang 2015 [14]

 Bias due to confounding

Serious

Study reported adjusted multivariate analyses for outcomes (all predictors with P < 0.10 listed). Confounding inherently not controllable. Results not confirmed by additional analyses.

 Bias in selection of participants into the study

Critical

Study included prevalent users, so a substantial amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

Low

Intervention is well defined, and based on medical records and discharge summaries at the time of medication prescription.

 Bias due to departures from intended interventions

Serious

Study compared current users vs non-users. Bias due to departure from the intended intervention is expected, but not adjusted for in the analysis.

 Bias due to missing data

No information

Study data were retrieved from the medical records in a hospital. No information about loss to follow up or missing data.

 Bias in measurement of outcomes

Moderate

Outcomes were identified from medical records. The outcome measure may be minimally influenced by knowledge of the intervention received by study participants, since study was retrospective in nature.

 Bias in selection of the reported results

Serious

Study analyzed multiple outcomes and only reported results for predictors with significant results (P < 0.10) rather than a priori model. There is a high risk of selective reporting from among multiple analyses.

Yodogawa 2015 [35]

 Bias due to confounding

Serious

Study adjusted for CHADS2 in Cox regression analysis, but not confirmed by additional analyses.

 Bias in selection of participants into the study

Critical

Study included patients who had pre-existing AF at enrollment. Study included prevalent users, so a substantial amount of follow-up time is missing for prevalent users from analyses. Study excluded patients with <6 months life expectancy, so the association may be attenuated since the high risk patients were excluded from analysis.

 Bias in measurement of interventions

Moderate

Intervention is well defined, but some aspects of the assignments of intervention status were determined retrospectively since self-reported medication use status may be subject to recall bias.

 Bias due to departures from intended interventions

No information

Study did not describe the analytical method in detail.

 Bias due to missing data

Low

Study was based on a retrospective, observational study on medical records of patients treated in a hospital. Study excluded patients without follow-up data, so data were reasonably complete.

 Bias in measurement of outcomes

Moderate

Outcomes were identified from medical records. The outcome measure may be minimally influenced by knowledge of the intervention received by study participants, since study was retrospective in nature.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were not clearly described. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Findlay 2016 [15]

 Bias due to confounding

Critical

Study used Kaplan-Meier survival curve to analyze outcomes between warfarin and no warfarin group, and did not use adjusted analysis to control for confounding.

 Bias in selection of participants into the study

Critical

Study included patients who had pre-existing AF at enrollment. Study included prevalent users, so a substantial amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

No information

Study did not provide information about measurement of interventions.

 Bias due to departures from intended interventions

Critical

Study did not describe or adjust for any bias due to departures from intended interventions.

 Bias due to missing data

No information

Study did not provide information about missing data.

 Bias in measurement of outcomes

Moderate

Outcomes were identified from the electronic patient record, and was reviewed by 2 independent clinicians. The outcome measure may be minimally influenced by knowledge of the intervention received by study participants, since study was retrospective in nature.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were not clearly described. There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.

Tanaka 2016 [16]

 Bias due to confounding

Critical

Study used Kaplan-Meier survival curve to analyze outcomes between warfarin and no warfarin group, and did not use adjusted analysis to control for confounding.

 Bias in selection of participants into the study

Critical

Study included patients who had pre-existing AF at enrollment. Study included prevalent users, so a substantial amount of follow-up time is missing for prevalent users from analyses.

 Bias in measurement of interventions

Low

Intervention is well defined, and based on records collected during the period of dialysis initiation. Since study was prospective, assume no recall bias.

 Bias due to departures from intended interventions

Critical

Study did not describe or adjust for any bias due to departures from intended interventions.

 Bias due to missing data

Low

Study excluded patients lost to follow-up, so data were reasonably complete.

 Bias in measurement of outcomes

Moderate

Outcomes were identified from survey slips that were sent to the dialysis facilities. The outcome measure may be minimally influenced by knowledge of the intervention received by study participants.

 Bias in selection of the reported results

Moderate

The outcome measurements and analyses were not clearly described.There is no clear indication of selection of the reported analysis from among multiple analyses or multiple subgroups.