![\"\"](\"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2019\/01\/FAT_AI_Site_01_2019_1400x788-1024x576.png\")
<\/p>\n<\/p>\n
Researchers from Microsoft Research will present a series of studies and insights relating to fairness in machine learning systems and allocations at the FAT* Conference (opens in new tab)<\/span><\/a>\u2014the new flagship conference for fairness, accountability, and transparency in socio-technical systems\u2014to be held from January 29\u201331 in Atlanta, Georgia.<\/p>\n Presented across four papers and covering a broad spectrum of domains, the research is a reflection of the resolute commitment Microsoft Research has made to fairness in automated systems that shape human experience as they become more rapidly adopted in a growing number of contexts in society.<\/p>\n In “Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting,”<\/a> Maria De-Arteaga (opens in new tab)<\/span><\/a>, Alexey Romanov (opens in new tab)<\/span><\/a>, Hanna Wallach<\/a>, Jennifer Chayes<\/a>, Christian Borgs<\/a>, Alexandra Chouldechova (opens in new tab)<\/span><\/a>, Sahin Geyik (opens in new tab)<\/span><\/a>, Krishnaram Kenthapadi (opens in new tab)<\/span><\/a>, and Adam Kalai<\/a> look closely at presumptions and realities regarding gender bias in occupation classification, shedding light on risks inherent in using machine learning in high-stakes settings, as well as on the difficulties that arise when trying to promote fairness by scrubbing explicit gender indicators, such as first names and pronouns, from online bios.<\/p>\n Online recruiting and automated hiring are an enormously impactful societal domain in which the use of machine learning is increasingly popular\u2014and in which unfair practices can lead to unexpected and undesirable consequences. Maintaining an online professional presence has become indispensable for people\u2019s careers, and the data making up that presence often ends up in automated decision-making systems that advertise open positions and recruit candidates for jobs and other professional opportunities. To execute these tasks, a system must be able to accurately assess people\u2019s current occupations, skills, interests, and, more subjective but no less real, their potential.<\/p>\n \u201cAutomated decision-making systems are playing an increasingly active role in shaping our lives\u2014and their predictions today even go as far as to affect the world we will live in tomorrow,\u201d said Hanna Wallach, Principal Researcher at Microsoft Research New York City<\/a>. \u201cFor example, machine learning is becoming increasingly popular in online recruiting and automated hiring contexts. Many of us have had jobs or other professional opportunities automatically suggested to us based on our online professional presences, and we were curious how much these recommendations might be affected by something like our genders.\u201d<\/p>\n The researchers created a new dataset of hundreds of thousands of online biographies and were able to show that occupation classifiers exhibit significant true positive rate (TPR) gender gaps when using three di\ufb00erent semantic representations\u2014bag-of-words, word embeddings, and deep recurrent neural networks. They were also able to show that the correlation between these TPR gender gaps and existing gender imbalances in occupations may compound the imbalances. They performed simulations demonstrating that imbalances are especially problematic if people repeatedly encounter occupation classifiers because they cause underrepresented genders to become even further underrepresented.<\/p>\n The researchers observed that because biographies are typically written in the third person by their subjects (or people familiar with their subjects) and because pronouns are often gendered in English, they were able to extract subjects\u2019 (likely) self-identified binary genders from the biographies. But they took pains to point out that a binary model of gender is a simplification that fails to capture important aspects of gender and erases people who do not fit within its assumptions.<\/p>\n \u201cWe found that when explicit gender indicators\u2014such as first names and pronouns\u2014are present, machine learning classifiers trained to predict people\u2019s occupations do much worse at correctly predicting the occupations of women in stereotypically male professions and men in stereotypically female professions,\u201d said Wallach.<\/p>\n Even when such gender indicators are scrubbed, these performance differences, though less pronounced, remain. In addition to the realization that scrubbing explicit gender indicators isn\u2019t enough to remove gender bias from occupation classifiers, the researchers discovered that even in the absence of such indicators, TPR gender gaps are correlated with existing gender imbalances in occupations. That is, occupation classifiers may in fact exacerbate existing gender imbalances.<\/p>\n \u201cThese findings have very real implications in that they suggest that machine learning classifiers trained to predict people\u2019s occupations may compound or worsen existing gender imbalances in some occupations,\u201d said Wallach.<\/p>\n The findings also suggested that there are di\ufb00erences between men\u2019s and women\u2019s online biographies other than explicit gender indicators, perhaps because of the varying ways that men and women present themselves or their having di\ufb00erent specializations within various occupations.<\/p>\n \u201cOur paper highlights both the risks of using machine learning in a high-stakes setting and the difficulties inherent in trying to promote fairness by \u2018scrubbing\u2019 sensitive attributes, such as gender,\u201d Wallach said.<\/p>\n Although the researchers focused on gender bias, they noted that other biases, such as those involving race or socioeconomic status, may also be present in occupation classification or in other tasks related to online recruiting and automated hiring.<\/p>\n In a world in which personal data drives more and more decision-making, both consequential and routine, there is a growing interest in the ways in which such data-driven decision-making has the potential to reinforce or amplify injustices. In “Access to Population-Level Signaling as a Source of Inequality,<\/a>” Nicole Immorlica<\/a>, Katrina Ligett (opens in new tab)<\/span><\/a>, and Juba Ziani (opens in new tab)<\/span><\/a> examine the idea of fairness through an economic lens, finding that disparity in the data available to unbiased decision-makers\u2014optimizing determinations to fit their specific needs\u2014results in one population gaining a significant advantage over another.<\/p>\n The researchers studied access to population-level signaling as a source of bias in outcomes. Population-level strategic signalers can serve as advocates for owners of personal data by filtering or noising data in hopes of improving individuals’ prospects by making it more challenging for decision-makers to distinguish between high- and low-quality candidates. An example is high schools that, to increase the chances their students will be admitted to prestigious universities, inflate grades, refrain from releasing data on class rankings, and provide glowing recommendation letters for more than just the top students.<\/p>\n The sophistication of the signaling that a school might engage in\u2014how strategic the school is in its data reporting versus how revealing it is (simply reporting the information it collects on its students directly to a university)\u2014makes an enormous difference in outcomes. As expected, strategic schools with accurate information about their students have a significant advantage over revealing schools\u2014and strategic schools get more of their students, including unqualified ones, admitted by a university.<\/p>\n \u201cOne of the many sources of unfairness is that disadvantaged groups often lack the ability to signal their collective quality to decision-makers, meaning each individual must prove their worth on their own merits,\u201d said Principal Researcher Nicole Immorlica of Microsoft Research New England and New York City<\/a>. \u201cIn comparison, members of advantaged groups are often lumped together, causing individuals to acquire the average quality of their peers in the mind of the decision-maker.\u201d<\/p>\n The researchers go on to derive an optimal signaling scheme for a high school and demonstrate that disparities in ability to signal strategically can constitute a significant source of inequality. The researchers also examine the potential for standardized tests to ameliorate the problem, concluding it is limited in its ability to address strategic signaling inequities and may even exacerbate these inequities in some settings.<\/p>\n \u201cBy looking at fairness through an economic lens, we can uncover purely structural sources of unfairness that persist even when unbiased decision-makers act only to maximize their own benefit,\u201d said Immorlica.<\/p>\n In “The Disparate Effects of Strategic Manipulation,<\/a>” Lily Hu (opens in new tab)<\/span><\/a>, Nicole Immorlica<\/a>, and Jenn Wortman Vaughan<\/a> show how the expanding realm of algorithmic decision-making can change the way that individuals present themselves to obtain an algorithm\u2019s approval and how this can lead to increased social stratification.<\/p>\n \u201cWe study an aspect of algorithmic fairness that has received relatively little attention: the disparities that can arise from different populations\u2019 differing abilities to strategically manipulate the way that they appear in order to be classified a certain way,\u201d said Jenn Wortman Vaughan, Senior Researcher at Microsoft Research New York City<\/a>. \u201cTake the example of college admissions. Suppose that admissions decisions incorporate SAT scores as a feature. Knowing that SAT scores impact decisions will prompt students who have the means to do so to boost their scores, say by taking SAT prep courses.\u201d<\/p>\n As Lily Hu (opens in new tab)<\/span><\/a>, a research intern from Harvard and lead author on the paper, put it, \u201cClassifiers don\u2019t just evaluate their subjects, but can animate them, as well.\u201d That is, the very existence of a classifier causes people to react. This becomes a problem when not every person has equal access to resources like test prep classes in the example of college admissions or interview coaching in the domain of automated hiring. Even when an algorithm draws on features that seem to reflect individual merit, these metrics can be skewed to favor those who are more readily able to alter their features.<\/p>\n The researchers believe their work highlights a likely consequence of the expansion of algorithmic decision-making in a world that is marked by deep social inequalities. They demonstrate that the design of classification systems can grant undue rewards to those who appear more meritorious under a particular conception of merit while justifying exclusions of those who have failed to meet those standards. These consequences serve to exacerbate existing inequalities.<\/p>\n \u201cOur game theoretic analysis shows how the relative advantage of privileged groups can be perpetuated in settings like this and that this problem is not so easy to fix,\u201d explained Wortman Vaughan. \u201cFor example, coming back to the college admissions example, we show that providing subsidies on SAT test prep courses to disadvantaged groups can have the counterintuitive effect of making those students worse off since it allows the bar for admissions to be set higher.\u201d<\/p>\n \u201cIt is important to study the impacts of interventions in stylized models in order to illuminate the potential pitfalls,\u201d added fellow researcher Nicole Immorlica.<\/p>\n “Fair Allocation through Competitive Equilibrium from Generic Incomes<\/a>,” by Moshe Babaioff<\/a>, Noam Nisan (opens in new tab)<\/span><\/a>, and Inbal Talgam-Cohen (opens in new tab)<\/span><\/a>, examines an underexplored area of theory\u2014that of notions of fairness as applied to the allocation of indivisible items among players possessing different entitlements in settings without money. Imagine a scenario in which there are two food banks catering to populations of different sizes with different needs and that the two food banks must divide between each other a donation of food items. What will constitute a \u201cfair\u201d allocation of the available items?<\/p>\n These scenarios arise frequently in the context of real-life allocation decisions, such as allocating donations to food banks, allocating courses to students, distributing shifts among workers, and even sharing computational resources across a university or company. The researchers sought to develop notions of fairness that apply to these types of settings and opted for an approach that would study fairness through the prism of competitive market equilibrium, even for cases in which entitlements differ.<\/p>\n Focusing on market equilibrium theory for the Fisher market model, the researchers developed new fairness notions through a classic connection to competitive equilibrium. The first notion generalizes to the case of unequal entitlements and indivisible goods the well-known procedure of dividing a cake fairly between two kids: The first kid cuts the cake, and the second picks a piece. The second notion ensures that when we cannot give both what they deserve, we at least give as much as possible to the one that got less than he should.<\/p>\n \u201cOur paper shows that for allocation of goods, market equilibrium ensures some attractive fairness properties even when people have different entitlements for the goods,\u201d said Moshe Babaioff, Senior Researcher at Microsoft Research<\/a>. \u201cAnd although such market equilibria might fail to exist for some entitlements, we show that this is a knife\u2019s-edge phenomenon that disappears once entitlements are slightly perturbed.\u201d<\/p>\n In addition to the papers previewed here, there are many other exciting happenings at FAT* 2019. On the first day of the conference, Microsoft Research attendees, along with researchers from Spotify and Carnegie Mellon University, will be giving a tutorial titled “Challenges of Incorporating Algorithmic Fairness into Industry Practice (opens in new tab)<\/span><\/a>.” The tutorial draws on semi-structured interviews, a survey of machine learning practitioners, and the presenters\u2019 own practical experiences to provide an overview of the organizational and technical challenges that occur when translating research on fairness into practice.<\/p>\n These efforts reflect Microsoft Research\u2019s commitment to fairness, accountability, transparency, and ethics in AI and machine learning systems. The FATE research group<\/a> at Microsoft studies the complex social implications of AI, machine learning, data science, large-scale experimentation, and increasing automation. A relatively new group, FATE is working on collaborative research projects that address these larger issues, including interpretability. FATE publishes across a variety of disciplines, including machine learning, information retrieval, sociology, algorithmic economics, political science, science and technology studies, and human-computer interaction.<\/p>\nBias in bios<\/h3>\n
Sending signals<\/h3>\n
Strategic manipulation<\/h3>\n
Allocating the indivisible<\/h3>\n
Don\u2019t miss the tutorial<\/h3>\n
![\"\"](\"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2019\/01\/FAT_AI_Site_01_2019_1400x788.png\")
","byline":"","formattedDate":"January 29, 2019","formattedExcerpt":"Researchers from Microsoft Research will present a series of studies and insights relating to fairness in machine learning systems and allocations at the FAT* Conference\u2014the new flagship conference for fairness, accountability, and transparency in socio-technical systems\u2014to be held from January 29\u201331 in Atlanta, Georgia. Presented…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/563847","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/37074"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=563847"}],"version-history":[{"count":7,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/563847\/revisions"}],"predecessor-version":[{"id":564261,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/563847\/revisions\/564261"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/564036"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=563847"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=563847"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=563847"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=563847"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=563847"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=563847"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=563847"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=563847"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=563847"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=563847"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=563847"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}