Summary List Placement
Areeb Khan had spent years preparing for the New York Bar Exam. In the weeks leading up to the October 5 test date, he was studying 10 hours a day to cram for the certification that would bring him one step closer to a career as a tenants’ rights advocate.
But passing the test wasn’t the only thing on Khan’s mind: He was worried about the software that would be watching his every move from his computer’s webcam.
New York, like many other states, selected the software company ExamSoft to administer the Bar Exam remotely amid the COVID-19 pandemic. The company’s product uses facial recognition to verify test takers’ identities and monitor their behavior during the test to flag potential cheating.
But Khan had discovered an issue with ExamSoft’s software: When he tried to take a mandatory mock exam on the platform, the software wouldn’t let him log on because it couldn’t detect his face due to “poor lighting.”
He tried taking off his glasses and turning on his bedroom’s ceiling lights, then shining a desk lamp directly at his face, then moving to his bathroom and standing under bright overhead lights, all to no avail. Khan had read reports that facial recognition software is worse at identifying people of color — and now it appeared that this flaw was affecting him. While the software eventually allowed him log in after he spent hours on the phone with tech support, it remained glitchy for the duration of the test.
While the software recognized Khan’s face when the time came to take the actual exam — which spanned two days at six hours a day — he says he isn’t sure how he would have made up for lost time if it had locked him out. The whole experience shook him.
“I think I just look like any Brown dude who has a beard and wears glasses,” Khan told Insider. “Now I might not be able to take the exam for something that has nothing to do with anything except the color of my skin. That was pretty frustrating and extremely anxiety inducing.”
Khan is one of millions of people who took tests using remote exam software for the first time this year. Schools across the world hastily adopted the software to stay on schedule with testing at the onset of COVID-19 but now, a year into the pandemic, students are increasingly raising concerns about the technology’s effectiveness, while experts and privacy advocates urge schools to use other forms of assessment that don’t hinge on surveillance software.
Privacy activist group Fight for the Future has led a campaign to pressure schools and textbook makers to abandon their contracts with e-proctoring companies, with more than 20 other digital rights activist groups signing on.
“It’s beginning to look like e-proctoring is going to be one of the knee-jerk embarrassments of the COVID-19 pandemic,” Fight for the Future campaigns and communications director Lia Holland told Insider.
In interviews with Insider, students and privacy experts said the software set unhealthy expectations around being surveilled at home and that glitches made high-stakes situations needlessly stressful. That unreliability is compounded by concerns that face-tracking technology is more likely to malfunction when used by people of color or people with disabilities.
Test proctoring software companies’ executives and spokespeople, meanwhile, insist that their technology is equally effective at recognizing students with darker skin or disabilities and that e-proctoring serves a crucial function as schools continue operate remotely.
Still, some institutions are already starting to end the contracts with test software companies that they first inked at the start of the pandemic.
The software brings additional anxiety to an already stressful experience
Kyle Fowler, another recent law school graduate, says he didn’t have any trouble with ExamSoft’s facial recognition when he took the New York Bar Exam in October. But the software’s AI-powered surveillance still put him on edge as he prepared for the test.
Shifting in his seat, moving his head too rapidly, or looking away from his screen during the six-hour test were all behaviors that the ExamSoft could flag as suspicious activity, Fowler was told. He prayed that his neighbors wouldn’t play loud music during the test, as audible background noise could also cause the software to flag him as suspicious.
On top of that, Fowler noticed that ExamSoft’s interface was glitchy and took valuable time to navigate. And if the software did flag his behavior as suspicious, he would have no way of knowing during the exam itself which was “a huge part of the frustration and anxiety,” he said.
“It’s a timed exam, and your career depends on it. So when the software is slow, it significantly impacts like your mental stability and confidence in that moment to get the things out that you need to,” Fowler told Business Insider. “I definitely looked up at least once, and I know I like moved around in my chair because I was just getting so uncomfortable.”
In July, students taking the Michigan Bar Exam through ExamSoft reported that the test crashed, temporarily delaying their completion of the exam. ExamSoft later said the crash was due to a “sophisticated cyberattack.” (In a statement, the Michigan Bar said the matter was being investigated by law enforcement and declined to comment further).
ExamSoft spokesperson Nici Sandberg said in a statement to Business Insider that the company is confident that its software effectively and fairly detects cheating, adding that all suspicious activity flagged by its AI is subsequently reviewed by a human “to remediate against any possible erroneous technology partiality.”
When asked about concerns about differences in how its software recognized users because of their race, Sandberg said the technology is equally effective when tracking white people and people of color, and added that ExamSoft employs humans to review “anomalies” and possible errors made by the algorithm.
“ExamSoft does not believe that our product disproportionately harms individuals of color,” Sandberg said, and that its “non-biased” system has delivered exams to “hundreds of thousands of exam-takers of color” to-date this year already.
The year brought booming business for e-proctoring firms
Companies selling remote exam software have seen a rapid expansion in their businesses that began with school shutdowns at the start of the pandemic last spring and ballooned in the fall.
ExamSoft grew its number of clients from 1,500 in 2019 to over 2,000 today. The company, which was previously backed by private equity, was acquired by the education software company Turnitin in October for an undisclosed amount.
Proctorio, a different remote exam company, facilitated over 20 million exams in 2020 — a fourfold increase from the 6 million exams it administered the year prior, CEO and founder Mike Olsen told Business Insider. Olsen is the company’s owner, and Proctorio has not pursued any outside funding.
While ExamSoft employs humans to review “anomalies” flagged by its algorithms, Proctorio’s platform flags “suspicious” activity and then leaves it to teachers or professors themselves to review the footage and determine whether to penalize students.
“The faculty, they make the decision. It’s not a third party. And that’s the way it should be,” Olsen said. “Maybe nine times [flagged activity] is a student sitting there saying, ‘I gotta pee, I’ll be right back.’ And most times faculty are going to look at that and say, ‘That’s fine.'”
While both ExamSoft and Proctorio rely on cheating-detection software in some form, another remote software maker, ProctorU, takes a different approach: It pays human contractors to monitor students through their webcams while they take an exam and manually flag suspicious activity. More than 1,000 colleges around the world are using the product this year, the firm said, as well as institutions that administer standardized tests including the LSAT and GRE.
But privacy advocates say tools that record students in their homes and monitor their behavior normalize a new, unprecedented form of surveillance.
“Our concern is that these apps create a really unfair and dangerous expectation that students have to give up their personal and private information in order to get an education,” Fight for the Future campaign director Caitlin Seeley George told Business Insider. “They’re basically stalkerware.”
Fight for the Future and other activist groups are now pressuring one of Proctorio’s biggest clients, McGraw Hill, to cut ties with the e-proctoring company. McGraw Hill currently integrates Proctorio’s software in its online test-taking tools.
In a statement to Insider, McGraw Hill spokesperson Tyler Reed said the company is mulling activists’ concerns and hasn’t made a decision, adding that it had a “productive and interesting meeting” with Fight for the Future and other activist groups in January.
“Instructors and institutions need tools to maintain academic integrity in an online learning environment, and the pandemic has only heightened this need. We are aware of the questions that have been raised about remote proctoring technology,” Reed said. “We’re continuing to look into the issues they raised in a thoughtful and deliberate manner.”
There’s mounting backlash from students and faculty
Some students and teachers have called on their schools to ban remote exam software, citing privacy concerns as well as accusations that the software can discriminate against people of color. Several professors across the US have called for boycotts of the software, Vice reported, and at least two schools including Cabrillo College and the University of Illinois have said they will no longer use remote test proctoring after their contracts with software firms end.
A group of faculty at University of California, Santa Barbara raised concerns about invasions of privacy and technical flaws in exam proctoring software last spring, calling on the university to end its relationship with ProctorU.
In response to the UC Santa Barbara faculty letter, ProctorU sent a cease-and-desist letter accusing the faculty of “defamation.”
Students have also pushed back against the software, in some cases successfully pressuring their classes to abandon it. Kareem Taha, a sophomore at the University of Oregon, told Insider that he emailed his professor with concerns about ProctorU, arguing that he was uncomfortable being recorded in his home by software monitored by a third party. While he isn’t sure if his email made a difference, that professor has switched to using Zoom videoconferencing during tests due to technical issues with ProctorU.
“It’s repulsive that this type of testing software is endorsed by our administration,” Taha told Business Insider. “Students are forced to choose between their privacy and their education.”
In a statement to Insider, ProctorU CEO Scott McFarland defended the company’s product and said its use of human proctors is more reliable than artificial intelligence.
“Having a live proctor and a recorded test session has allowed our proctors to advocate for students, verifying their special circumstances, corroborating or proving that students were not cheating when, for example, a cat jumps on a keyboard or a baby in another room starts to cry or a student sneezes repeatedly,” McFarland said.
In response to privacy-focused criticism of Proctorio, CEO Mike Olsen emphasized that Proctorio only shares recordings of students with faculty and noted that its software gives schools the option to delete stored videos after a set amount of time.
The University of Illinois at Urbana-Champaign announced in January that its contract with Proctorio will come to an end after the Summer 2021 term, adding that the university was aware of student feedback about “issues related to accessibility, privacy, data security and equity.”
“Because [the contract] was procured through an emergency action, it was never intended, nor would it be permitted, to be a long-term agreement,” the university’s vice provost Kevin Pitts said in a statement to Insider.
Olsen told Insider in an interview that he doesn’t think many more schools will follow suit. “I don’t think it’s going to be a long-term problem for us or the [e-proctoring] industry in general,” he said. After the interview, Olsen sent an emailed statement adding that, “in reviewing the concerns raised by University of Illinois, many of their issues were addressable through routine use of the software and training.”
University of Michigan professor Shobita Parthasarathy, who co-authored a report in August recommending a ban of facial recognition in schools, told Insider that remote exam software can disproportionately affect students of color because facial recognition algorithms are shown to misidentify them at a higher rate than white people.
The problem may be further compounded by existing biases among humans, Parthasarathy added: If the software incorrectly flags a student of color for suspicious behavior, a biased human reviewer might be more likely to side with the system.
“These kinds of surveillance technologies, while they have the veneer of objectivity, they actually tend to get used to reinforce bias and discrimination,” she said.
How schools can move beyond e-proctoring software
Remote proctoring software saw such a surge during the pandemic because it presented a solution to an unexpected problem and allowed traditional exams to continue even when everyone was forced to be remote.
But opponents argue that there has to be a better way.
“The negative impacts of implementing these kinds of technologies could outweigh the benefits,” Parthasarathy said.
To avoid pitting students against the algorithms or unknown proctors when flagged for cheating remotely, schools could have people known to the test-takers review videos (similar to Proctorio’s approach) since there’s already a relationship there.
“We can already see that students are feeling policed and they feel great anxiety and stress,” Parthasarathy said. “I feel like trust and engagement have to be what we prioritize, rather than disciplining.”
Fight for the Future’s petition calls on professors to “find ways to assess student performance without using invasive surveillance.” While it did not recommend specific alternatives, others have suggested that traditional test-taking simply isn’t a realistic method of evaluating students when they’re learning from home, citing examples of students managing to cheat even while using remote proctoring software.
Parthasarathy said that, in her own classes, she’s attempting to find ways to evaluate students learning remotely other than traditional exams.
She added that, in her own experience, she’s noticed many other educators have opted to do the same in favor of different kinds of projects. Some have stopped using the software altogether, she said: “I think that people are starting to abandon it.”