Painting a New Landscape: Michael Bastedo shows that information is key to more equitable college admissions 

Bastedo, who directs the Center for the Study of Higher and Postsecondary Education (CSHPE), has been researching admissions practices at colleges and universities since 2010. He was particularly concerned about the effects of admissions practices on low-income students, who represent only about 4 percent of the enrollment at selective colleges.  While studying two flagship public universities, he observed and participated in admissions training, in addition to interviewing 60 seasonal readers and admissions officers. Through this work, he developed research questions about whether cognitive biases might be affecting some students’ chances at getting into a college, and he proceeded to investigate this in more detail.

Bastedo was curious about a specific concept in social science. It’s called “correspondence bias,” and it can lead someone to believe that a person’s actions have to do with their personality or personal attributes, when in actuality, behaviors and choices are situational. “The correspondence bias is relevant to admissions,” explains Bastedo, “because holistic review is meant to consider someone’s achievements as they relate to their communities, families, high schools, and neighborhoods. If someone is falling into this bias, it is difficult to make good holistic decisions.”

Partnering with CSHPE alum Nicholas Bowman, Bastedo designed an experiment to test whether providing more robust data on one’s community and high school would alter admissions officers’ decisions, even when the application itself was unchanged. He posited that offering more community context could reduce correspondence bias, and thus increase the chances of admission for people from disadvantaged backgrounds into competitive universities. “In my experiments,” he says, "I found that admissions officers were about 25 percent more likely to admit a low-income applicant if they had better data on the applicant’s high school and community.”

In 2014, Bastedo read a paper written by a researcher at The College Board, and reached out to him via email.  Through their subsequent correspondence, he learned that The College Board was asking similar questions about admissions practices. The College Board has vast data stores that are not traditionally accessible to academic researchers, as well as a network of admissions offices that could implement any new intervention. Their partnership with Bastedo began when he was invited to their headquarters in Washington, D.C. to present his own research. “My goal in giving that presentation,” he says, “was to communicate that there was only one organization that could improve the system, and it was them. They had both the research resources to provide data on all high schools as well as the relationships with the colleges that use their data.” Bastedo was officially asked to consult with The College Board as they determined how to move forward with creating a system to display this information in a user-friendly way. The result was Landscape.

While typical college applications include information about a student’s grades, test scores, and extracurricular activities, they do not contain good measures of a student’s relative opportunities within their high school or neighborhood communities. Landscape’s dashboard attempts to paint a more well-rounded picture of an applicant by displaying basic high school data about a school’s locale—whether city, suburban, town, or rural—size, percentage of students on free/reduced lunch programs, and student Advanced Placement coursework opportunities. It also shows how an applicant’s SAT score compares to others attending the same high school.

These data points also include six neighborhood indicators that researchers say are related to educational outcomes. These include community-wide college attendance, household structure, median family income, housing stability (including rates of home ownership, vacancy rates, and housing turnover), education attainment levels, and the predicted probability of neighborhood crimes like robbery, homicide, larceny, and motor vehicle theft. The indicators are averaged and placed on a 1-100 percentile scale to provide a snapshot of the level of challenge in the community from which an applicant applies. A higher number indicates a greater level of challenge when it comes to educational opportunities and outcomes as compared to all other communities in Landscape. The data come from sources like the U.S. Census Bureau, the National Student Clearinghouse, and Location, Inc.

After its development, the new tool had to be piloted to measure its outcomes and ensure its usability. Bastedo is working with College Board researchers to evaluate several pilot phases. At one early pilot university, thousands of applicants were reviewed using an early version of Landscape, and there was a “large jump” in admissions officers’ admissions of underserved students. The university was already taking a holistic approach, so the tool simply allowed them to put the applicants’ communities in focus to allow for a clearer snapshot of an applicant’s background.

The team then expanded the pilot to eight schools, and found that these schools also admitted more low-income students, particularly those from “non-feeder high schools” that admissions officers did not know particularly well. They now have data on about 50 pilot schools that used Landscape in 2018-19. Bastedo says, “Preliminary results from that pilot group show about a three-point increase in the percentage of disadvantaged students who were admitted.” The team continues to follow the work taking place in these schools. “It’s working well so far,” Bastedo adds. “We don’t expect to be able to move the needle in a huge way, but we hope it continues to go in the right direction.” A Michigan Daily article confirmed that the University of Michigan was one of the 50 initial universities to use this dashboard, and that it will continue to use Landscape in its admissions decision-making processes.

One essential attribute of Landscape is that it helps ensure consistent decision making. Admissions officers are not familiar with all schools, and there will be discrepancies between officers’ knowledge sets. In fact, research shows that admissions officers lack quality high school information for about 25 percent of all applications. The Landscape dashboard provides information that is both consistent and clear, so that admissions officers can fully consider what students achieve in the context of the places they live and learn. Landscape is not used to decide who gets in and who doesn’t. It only helps officers give more students from more places a fair look.

SOE doctoral candidates Kristen M. Glasener and Jandi L. Kelly collaborated with Bastedo and Bowman on some of the research related to Landscape. They compared decisions of admissions officers who were randomly given access to information similar to what’s available in Landscape. In the end, they found that admissions officers who were given the detailed information were 13-14 percentage points more likely to admit a student from an underserved community than admissions officers who had less information—even if that information still included details about the student’s socioeconomic status and high school context. The team also discovered that admissions officers did, indeed, lack consistency when it came to the definition of holistic application review, and that their decisions could be affected by many factors, including whether officers are employed by their alma maters.

Their results suggest that an overview of contextualized information can “provide a meaningful benefit to both applicants and admissions staff,” as the team wrote. In short, since the dashboard information shifted admissions decisions, it enabled a more diverse student body to be admitted.

The team’s next line of inquiry is to investigate the ways in which training can impact the ways Landscape could be used alongside current practices. As they report, “training and norming are crucial in an admissions office to ensure that there is high reliability in admissions decisions across admissions officers. We need to know more about how admissions decisions may change once admissions officers were trained on how to use Landscape in their particular office. This will also tell us more about how the data elements are interpreted by admissions officers, and which elements are perceived to be more useful than others.” Bastedo explains that this is a question about “the black box,” explaining that he hopes to start a dialogue about how we define and communicate what is meant by holistic review.  He plans to “study more deeply about how admissions officers use the dashboard,” he says. “We gave them the data, more low-income students are getting in, so now we want to uncover what’s happening inside the black box.”

Ultimately, he believes that this discussion should be led and reconstructed by admissions professionals, professional organizations, and the colleges they serve. Their combined efforts may be crucial to gaining an understanding about why there is still great stratification across higher education, and it may become a first step toward more consistent admissions practices that better serve our highest ideals for fair, just, and equitable access to selective colleges.

Landscape continues to expand. As of fall 2019, it has been provided for free to approximately 150 institutions, and plans are for it to be accessible to students, parents, and counselors next year. It is expected to be broadly available to colleges and universities by 2020. So far, over 90 percent of its users have reported that Landscape makes it easier to incorporate a student’s context during the application review process. “It is a huge improvement over the data that admissions officers had before,” Bastedo says, “and we’re seeing real effects on the admission of more low-income students.  I’m really excited for the future of this work.”