The Main Hospital Algorithm Targets Black Patients
Computers can do many tasks faster than humans, but they tend to copy our mistakes – after all, they learned this by watching us. The researchers found that the main algorithm used in hospitals is biased towards black patients because it reflects pre-existing racial differences in health.
The researchers did not want to name and shame the specific software or the health care system in which it is used, so we cannot tell you a specific hospital to avoid. But they note that it is “a widely used algorithm, typical of this industry-wide approach and affecting millions of patients.”
Hospitals use this algorithm to determine which patients are most ill and prioritize them for care, including dedicated nurses and faster access to primary care appointments. But it turns out that one of the main metrics he uses to determine who is sick is simply who has borne the most health care costs in the recent past.
So it means that if you get sick but can’t take time off from work to see a doctor when you really should have done so, you will appear to the system as if you are less sick. Or, if you are a black patient and your doctor does not take your complaints seriously, you will also seem less sick to the algorithm. The computer assumes that the people who get the most help are the people who need help the most, but in reality these numbers just don’t match. The researchers found that a black patient had to be significantly worse than a white peer to receive an assessment that recommended them for a coordinated care program.
There is no immediate conclusion for patients, but research has identified problems with these algorithms. Researchers are working with the creators of this algorithm to reprogram it to avoid making preconceived assumptions; so far, they have been able to reduce bias by what they say by 84%.
After all, as the authors point out, when you have enough data to work with, you can program an algorithm to counteract some of the biases created by humans. The problem is that programmers usually don’t do this, and so computers learn to copy the world they live in.