Organizations are increasingly utilizing machine-learning models to allocate scarce resources or opportunities. As an illustration, such models can assist firms screen resumes to decide on job interview candidates or aid hospitals in rating kidney transplant patients based on their likelihood of survival.
When deploying a model, users typically strive to make sure its predictions are fair by reducing bias. This often involves techniques like adjusting the contains a model uses to make decisions or calibrating the scores it generates.
Nevertheless, researchers from MIT and Northeastern University argue that these fairness methods usually are not sufficient to deal with structural injustices and inherent uncertainties. In a recent paper, they show how randomizing a model’s decisions in a structured way can improve fairness in certain situations.
For instance, if multiple firms use the identical machine-learning model to rank job interview candidates deterministically — with none randomization — then one deserving individual might be the bottom-ranked candidate for each job, perhaps attributable to how the model weighs answers provided in an internet form. Introducing randomization right into a model’s decisions could prevent one worthy person or group from all the time being denied a scarce resource, like a job interview.
Through their evaluation, the researchers found that randomization might be especially useful when a model’s decisions involve uncertainty or when the identical group consistently receives negative decisions.
They present a framework one could use to introduce a certain amount of randomization right into a model’s decisions by allocating resources through a weighted lottery. This method, which a person can tailor to suit their situation, can improve fairness without hurting the efficiency or accuracy of a model.
“Even if you happen to could make fair predictions, do you have to be deciding these social allocations of scarce resources or opportunities strictly off scores or rankings? As things scale, and we see increasingly more opportunities being decided by these algorithms, the inherent uncertainties in these scores might be amplified. We show that fairness may require some form of randomization,” says Shomik Jain, a graduate student within the Institute for Data, Systems, and Society (IDSS) and lead writer of the paper.
Jain is joined on the paper by Kathleen Creel, assistant professor of philosophy and computer science at Northeastern University; and senior writer Ashia Wilson, the Lister Brothers Profession Development Professor within the Department of Electrical Engineering and Computer Science and a principal investigator within the Laboratory for Information and Decision Systems (LIDS). The research will probably be presented on the International Conference on Machine Learning.
Considering claims
This work builds off a previous paper during which the researchers explored harms that may occur when one uses deterministic systems at scale. They found that using a machine-learning model to deterministically allocate resources can amplify inequalities that exist in training data, which may reinforce bias and systemic inequality.
“Randomization is a really useful concept in statistics, and to our delight, satisfies the fairness demands coming from each a systemic and individual standpoint,” Wilson says.
In this paper, they explored the query of when randomization can improve fairness. They framed their evaluation across the ideas of philosopher John Broome, who wrote in regards to the value of using lotteries to award scarce resources in a way that honors all claims of people.
An individual’s claim to a scarce resource, like a kidney transplant, can stem from merit, deservingness, or need. As an illustration, everyone has a right to life, and their claims on a kidney transplant may stem from that right, Wilson explains.
“If you acknowledge that folks have different claims to those scarce resources, fairness goes to require that we respect all claims of people. If we all the time give someone with a stronger claim the resource, is that fair?” Jain says.
That form of deterministic allocation could cause systemic exclusion or exacerbate patterned inequality, which occurs when receiving one allocation increases a person’s likelihood of receiving future allocations. As well as, machine-learning models could make mistakes, and a deterministic approach could cause the identical mistake to be repeated.
Randomization can overcome these problems, but that doesn’t mean all decisions a model makes needs to be randomized equally.
Structured randomization
The researchers use a weighted lottery to regulate the extent of randomization based on the quantity of uncertainty involved within the model’s decision-making. A call that’s less certain should incorporate more randomization.
“In kidney allocation, normally the planning is around projected lifespan, and that’s deeply uncertain. If two patients are only five years apart, it becomes so much harder to measure. We would like to leverage that level of uncertainty to tailor the randomization,” Wilson says.
The researchers used statistical uncertainty quantification methods to find out how much randomization is required in several situations. They show that calibrated randomization can result in fairer outcomes for people without significantly affecting the utility, or effectiveness, of the model.
“There may be a balance available between overall utility and respecting the rights of the individuals who’re receiving a scarce resource, but oftentimes the tradeoff is comparatively small,” says Wilson.
Nevertheless, the researchers emphasize there are situations where randomizing decisions wouldn’t improve fairness and will harm individuals, corresponding to in criminal justice contexts.
But there might be other areas where randomization can improve fairness, corresponding to college admissions, and the researchers plan to review other use cases in future work. Additionally they wish to explore how randomization can affect other aspects, corresponding to competition or prices, and the way it might be used to enhance the robustness of machine-learning models.
“We hope our paper is a primary move toward illustrating that there is perhaps a profit to randomization. We’re offering randomization as a tool. How much you will wish to do it will be as much as all of the stakeholders within the allocation to come to a decision. And, after all, how they determine is one other research query all together,” says Wilson.
