Data Mining: Diamonds in the data

[rwingo]

http://www.fcw.com/article94909-06-19-06-Print

At this moment, public health officials are poring over terabytes of health care data to detect the first signs of a possible pandemic flu outbreak, bioterrorism attack or other contagion. The Centers for Disease Control and Prevention began a biosurveillance program in 2003, but advances in information exchange standards and concerns about pandemic flu have accelerated its national implementation.

The federal initiative, called BioSense, analyzes existing health care records, such as diagnoses, laboratory test results, physician visits and hospitalizations. The results help public health officials discover where an event is occurring and decide when to intervene with vaccines or quarantines. The CDC works with regional hospital systems to create secure connections between their health care databases and the federal database. The data does not contain patient names, medical numbers or personal identifiers, CDC officials said.

Like the CDC, Medicaid agencies, NASA and many other government agencies have begun to employ software to look for meaningful patterns in large volumes of data. Their searches have various purposes, such as pinpointing criminal activity, improving customer service and detecting fraud, waste and abuse. Some call this activity data surveillance, others call it data mining, and still others prefer the term data analysis. Whatever such searches are called, they usually require federal agencies to strike a balance between observing behaviors and violating privacy...