Affirmative action policies for university admissions
A study based on data from Chile
In many countries around the world, at the end of the last year of high school, students apply for admission to university programs – and, in the process, take a set of standardized tests on various subjects, such as mathematics or language. These tests are meant to measure the students’ aptitude for university studies. In other words, good performance on tests is seen as an indicator that the student will perform well at university. Naturally, when there are more applicants for a university program than available positions in the program, only the best applicants are admitted. The applicants for a position are typically ranked in terms of an admission score, which is calculated from test scores and/or high school grades with a simple formula -- and the applicants with highest admission score are admitted.
However, an admission score is not the only criterion for university admission. Universities may want to have a diverse body of students -- or may have a social role in addition to their educational one, such as promoting gender equality and social mobility. For this reason, universities often enact affirmative action policies addressed at increasing the number of admitted applicants from disadvantaged backgrounds. As an example, universities might adhere to quota for disadvantaged applicants, which require that a specific number of them are admitted.
Each such affirmative action policy aims to achieve two goals. On one hand, the policy should select applicants who will do well at their university studies. For this reason, they should typically ensure that, ceteris paribus, applicants with higher scores are admitted to university over applicants with lower admission scores. On the other hand, the policy should ensure that enough applicants from disadvantaged backgrounds are selected. Unfortunately, however, in practice such applicants often do not achieve as high admission scores as applicants from privileged backgrounds. For example, students from families of lower income may lack the financial resources to prepare well for the standardized tests – and thus be at disadvantage compared to students from wealthier families. Therefore, the two aforementioned goals are often competing in practice. For this reason, an affirmative action policy should aim to strike a good balance between the two goals: admit a good number of applicants from disadvantaged backgrounds, but without severely compromising the quality of admitted applicants.
Is it possible to design admission policies with a good balance between the two goals? And, are certain admission policies better than others in achieving this balance?
Aiming to answer these questions, we recently conducted a case study with data from university admissions in Chile (Mathioudakis, Castillo, et.al., 2019). The case study was supported by the HUMAINT EU project, led by the European Commission’s Joint Research Centre in Seville, Spain. In the study, we measured the relative advantage or disadvantage of certain applicant groups over others; and compared different affirmative action policies. In what follows, we provide a brief overview of the study.
University admissions in Chile
In Chile, the admission to undergraduate programs in all major universities is based on the students’ performance in standardized tests, administered by a public entity, on language, mathematics, natural sciences, and human sciences, as well as their grades in high school.
Each university announces the programs it offers (such as engineering, medicine, or law), and the formula for its admission score. For instance, at one university, the engineering program calculates the admission score as a weighted sum, where high school grades account for 30%, language for 10%, mathematics for 45%, natural sciences for 15%, and human sciences for 0%. At the same university, the law program instead uses 40% for high school grades, 25% for language, 10% for mathematics, 0% for natural sciences, and 25% for human sciences. After learning their results in the tests, applicants rank university programs by decreasing order of preference. Then, applicants applying to each program are ranked by admission score and the top ones are admitted.
In our study, we analyzed anonymized, national-level data of students who took the standardized tests in 2017. This dataset includes the gender, high school type (public or private), income decile, grades in high school, and results in standardized tests of all applicants. Moreover, we focused on one of the most sought-after engineering programs in one of the largest universities. For the engineering program, we have access to data for about all admitted applicants from 2010 to 2014. Most importantly, the engineering-program data contains the grades obtained by admitted students during their first year at the university.
An initial exploration of the data led to two key observations.
(Observation 1) The high school grades and test scores of applicants of the engineering program vary with income level. This is shown in the figure on the left. Each plot in the figure shows the distribution of applicant scores for high school grades and standardized tests. Two distributions are shown in each plot: one for high income students (who come from families of the higher 50% of national income distribution) and one for lower income students. As it is easy to see, applicants of higher income generally have better grades at high school and achieve better scores at standardized tests than those of lower income. It appears, therefore, that students from low-income families are disadvantaged compared to ones from high income families.
(Observation 2) From the available data, we can use an applicant’s grades and test scores to make an approximate prediction about his or her performance at university. Using standard statistical techniques, we identified the best linear formula that predicted the performance of an applicant at university from the applicant’s grades and scores. We used this linear formula as our proposed admission score and compared its values to the actual performance of applicants who were admitted to the engineering school, during their first year of studies. The admission score and actual university performance of applicants are shown in the figure on the right, corresponding to the horizontal (x) and vertical (y) axis, respectively. The predicted performance at university for different admission score values is shown with a straight line. The figure shows that the linear formula identifies a clear pattern in the data: applicants with higher admission score generally perform better in their first year of university studies.
Exploring admission policies
Having identified an admission score that predicts university performance, how could we design affirmative action policies that achieve the two goals discussed earlier, namely, to increase the number of admitted applicants from disadvantaged (low-income) backgrounds without compromising the (predicted) performance of admitted applicants at university? One idea would be to adjust the formula we use for the admission score, to favor disadvantaged applicants. For example, if disadvantaged applicants generally do better at the language test than the math test, we could adjust the admission score so as to put more weight on language – even if our data tell us that some other test, e.g., for math, is more important for the engineering school for which the policy is designed. Another idea would be to directly give a bonus score to disadvantaged applicants, so as to admit a desired quota of them. Each of the above ideas can be used to design different affirmative action policies – let us refer to them here as the adjusted-admission-score and bonus/quota policies.
Our data analysis showed that the bonus/quota policy leads to better results than the adjusted-admission-score policy. In particular, adjusting the admission score formula usually does not change the proportions of admitted disadvantaged and privileged applicants, as the top-performing privileged applicants tend to do better at all tests than the top-performing disadvantaged applicants; and even when the two policies lead to the same number of admitted disadvantaged applicants, the quality of applicants admitted under the bonus/quota policy is better.
In our ongoing research, we study more elaborate types of affirmative-action policy, which would allow us to target multiple attributes of applicants (e.g., both income level and gender). We believe that the results of our research, but also the methodology developed therein, will be helpful to policy makers who wish to make use data to design better affirmative action policies for university admissions, but also for other cases, such as distribution of welfare benefits.
Mathioudakis, M., Castillo, C., Barnabo, G., & Celis, S. (2019). Affirmative Action Policies for Top-k Candidates Selection, With an Application to the Design of Policies for University Admissions. arXiv preprint arXiv:1905.09947.