Letter to the Hon. Nancy Pelosi, Speaker of the House, the Hon. Kevin McCarthy, House Minority Leader, the Hon. Mitch McConnell, Senate Majority Leader, and the Hon. Charles Schumer, Senate Minority Leader - Rep. Clarke, Sen. Wyden Lead Letter to Prevent AI Bias in COVID-19 Response

Letter

Dear Speaker Pelosi, Leader McCarthy, Leader McConnell, and Leader Schumer:

The coronavirus crisis is accelerating the deployment of artificial intelligence (AI) across our society. There may be beneficial uses of AI in the context of combatting COVID-19, but without sufficient testing, embedded bias in these systems could perpetuate great harm and inequity. While AI comes to conclusions based on algorithms, the outputs can unintentionally reflect the biases of its programmers or the data sets used to train the systems. Accordingly, it is imperative that any sensitive AI programs deployed or developed with federal dollars during this pandemic are vetted to guard against algorithmic bias.

In the coming months, it is inevitable that AI will play a key role in monitoring the spread of COVID-19 among individuals, predicting future outbreaks, and perhaps even allocating scarce health care resources. Already, algorithms are being used to identify high-risk patients. If the pandemic intensifies and hospitals experience shortages of ventilators or other key supplies, it is conceivable that such risk indexes could be used to prioritize care. Even before COVID-19, there were examples of biased AI resulting in patients of color being offered less care than white patients. Accordingly, amid this lethal pandemic, our failure to enact safeguards against algorithmic bias in sensitive AI systems -- such as those used to produce health care assessments -- is literally a matter of life and death.

The economic consequences of COVID-19 also heighten the urgency of confronting algorithmic bias in other contexts. Two of the most frequently cited examples of AI systems perpetuating bias are employment screening applications which discriminate against elderly individuals and women, and loan origination systems which offer less favorable terms to people of color or fail to include them in loan opportunities altogether. While our country potentially faces the highest unemployment rate since the Great Depression and small businesses desperately seek capital, preventing automated discrimination in employment and lending is critical, and it falls upon us to ensure essential safeguards are in place.

We are not alone in these concerns. Leading advocacy organizations such as EqualAI have called for Congress to "mandate that recipients of stimulus funding that utilize AI for essential services and determinations provide confirmation that they've checked for bias against protected classes (gender, race, socio-economic class, etc.)." Meanwhile, a recent white paper from the National Security Commission on Artificial Intelligence recommends "ensur[ing] that federally funded computing tools created and fielded to mitigate the COVID-19 pandemic are developed with a sensitivity to and account for potential bias and, at a minimum, do not introduce additional unfairness into healthcare delivery and outcomes."

Fortunately, these challenges are surmountable without creating inordinate work or obstacles. AI developers can begin to address unintentional bias in these systems by, at a minimum, simply vetting their code and conducting testing for the purpose of removing or reducing bias. Accordingly, we urge you to include language in forthcoming stimulus legislation requiring:

Any health care provider receiving funding in the package to only deploy AI systems in medical decision-making contexts once it provides written assurances that bias tests have been performed; and
Any business with annual gross receipts of $50,000,000 or greater in 2019 receiving funding in the package to provide a statement that bias tests have been performed on any algorithms they use to automate or partially automate activities (such as employment and lending determinations) which have historically been impacted by discriminatory practices.

In the context of COVID-19, AI can be a force for good. However, without meaningful oversight, AI-facilitated algorithmic bias could also exasperate the demographic and socioeconomic inequities of this pandemic. Only Congress can ensure the possibilities of AI are not overshadowed by its perils. Thank you for your consideration of this request