How Brim Delivered 100% Sensitivity Identifying Stroke Cases After Surgery

When researchers at Vanderbilt University Medical Center set out to study postoperative complications following mitral valve surgery, they needed to build a clean and complete dataset. One of the most critical variables in their study was whether a patient experienced a stroke between surgery and hospital discharge.

A research fellow in the Department of Cardiac Surgery led the abstraction effort. His team had already manually confirmed 40 stroke cases in a cohort of 1,263 patients. The challenge: Could Brim identify those cases (and more) using only clinical notes?

‍

Key Takeaways

‍

• Brim achieved 100% sensitivity on a chart review case study identifying strokes after mitral valve surgery.
• Not all projects will be able to attain 100% sensitivity due to their problem formulation, complexity and data availability.
• High sensitivity often generates some false positives. Brim makes it faster to review a cohort by linking every variable to the evidence in the chart.
• Human-in-the-loop AI chart abstraction is likely to be both faster and more accurate than humans or AI alone.

‍

Brim's AI-Guided Chart Abstraction Achieved 100% Sensitivity

‍

Using a variable definition focused on postoperative stroke, Brim surfaced 189 patients likely to have experienced a stroke. Among those:

• All 40 of the manually confirmed stroke cases were included.
• 0 confirmed cases were missed.

‍

“Yes, that would be accurate… All of our 40 confirmed cases were among those 189.”
—Dr. Awab Ahmad, VUMC

‍

Achieving 100% recall for known cases gave the team confidence to proceed with a full abstraction pass, starting with Brim’s results to confirm which strokes occurred in the correct time window.

We do want to note that this case was unique; not all projects will attain 100% sensitivity due to several factors. Often the problem formulation doesn't enable high sensitivity because a logical formulation doesn't match how a human would review the same data. In addition, automatic abstraction systems can only work with available data, which is sometimes flawed or incomplete.

‍

Sensitivity vs. Specificity in Medical Research

‍

The variable search strategy prioritized sensitivity over specificity. This meant that some patients in the Brim-generated cohort had strokes that occurred outside the target window (e.g., prior to surgery). In clinical contexts like this, it’s most important to catch all cases by starting with a broad set of potential matches and then refining.

This case also illustrates why we believe human-in-the-loop AI chart abstraction will be faster and more accurate than either AI or human abstraction alone. AI can apply rules too rigidly for nuanced medical situations, and human review is accurate but time-intensive. An AI filter followed by human review allows for high accuracy with much faster human review.

To facilitate human-in-the-loop review, Brim's tools also surfaced evidence of stroke for each of these patients in the context of the chart, allowing quick review and elimination of False Positives.

‍

Why This Matters

‍

In clinical research, especially with rare but serious events, missing even one relevant case can skew findings. Brim’s ability to identify every known case using only unstructured text available in the chart helped this team move faster and with more confidence.

Cases like this are part of the reason we think PIs and medical researchers need Brim, but part of the value of Brim is that it can be used in applications across an institution. Brim is applying this combination of speed and accuracy to chart review problems in clinical workflows, medical research, clinical trial matching, and more. Learn more and request a demo here.

‍