Health Data Collection Is Often Substandard

Most clinical registries that collect statistics on patient outcomes are substandard and lacking in features that could make them useful for patients, doctors and policy makers, according to a study.

The investigation, by researchers from the Johns Hopkins University School of Medicine, was published in the Journal of Healthcare Quality. The study revealed poor data monitoring and reporting, which hurt efforts to study the disease, guide patients in their choice of treatments, and formulate effective health policies.

“Our results highlight the acute need to improve the way clinical outcomes data are collected and reported,” said senior investigator Marty Makary, M.D., M.P.H., professor of surgery at Johns Hopkins. “Failure to measure and accurately track patient outcomes remains one of the greatest problems in modern health care, curtailing our ability to understand disease, evaluate treatments and make the health-care industry a value-driven marketplace.”

Clinical registries are databases of patient outcomes developed and maintained by medical organizations and medical specialty groups.

To evaluate the quality of clinical registries, Makary and colleagues say they created “a registry of registries” to study the way the health care industry measures its performance. “We found it’s the Wild West,” Makary said. “With a few notable exceptions, most registries are underdeveloped, underfunded and often are not based on sound scientific methodology.”

According to a news release from Johns Hopkins, the investigators looked at 153 U.S. clinical registries containing health service and disease outcomes data. On average, a registry contained information on more than 160,000 patients treated across more than 1,600 hospitals.

Less than one-quarter of registries adjusted their results for differences in disease complexity — information statistically reflective of disparities in illness severity and socio-economic status among patients treated across hospitals. Unadjusted data, the researchers say, could be misleading and should be interpreted with great caution.

Less than one-fifth of registries contained independently entered data — information entered by clinicians other than the ones involved in care. Independently entered data is an important principle in eliminating the well-established bias of self-reported data, the researchers say.

The investigators also said, though, that several organizations had excellent registries, with carefully analyzed statistics that were reported and audited in a meaningful way. These organizations included the United Network for Organ Sharing; the National Surgical Quality Improvement Program, maintained by the American College of Surgeons; and the National Cardiovascular Data Registry of the American College of Cardiology.