New York’s Chosen AI Keeps Users From Collecting Unemployment
ID.me’s facial recognition tool was supposed to help administer unemployment securely. Users say the tech has barred them from their accounts — and their paychecks.
This article was originally published in NYS Focus. Read it here.
SIGNING UP FOR unemployment benefits in New York state is supposed to be easy. If you don’t already have one, you create an ny.gov account, says a video released by the New York Department of Labor, which you can use to register for direct deposit and apply for benefits. After a few more steps, you can file your claim.
Before users can go any further, they might get a text, email, or letter telling them to sign up for a separate platform called id.me. The dol’s instruction video doesn’t provide any other details about the service. But if New Yorkers don’t use it properly, they might not get their benefits.
id.me, a private company, deploys facial recognition and a fleet of human customer service agents to confirm applicants’ identities. New York spent $4.7 million to contract with the firm during the pandemic, with the goal of speeding up processing times for the surge in unemployment insurance claims.
Two years after implementation, records indicate that New Yorkers can find id.me extremely difficult to navigate. According to complaints filed with the Federal Trade Commission and the Better Business Bureau, obtained by New York Focus, many applicants have trouble using the platform’s facial recognition tool. Some have waited weeks — and even months — to hear back from the company. In part in response to issues with id.me, the National Center for Law and Economic Justice filed a complaint last month with the federal Department of Labor, while the American Civil Liberties Union and the New York Civil Liberties Union announced Friday they would sue for access to documents about its work with the company.
Issues with id.me’s service may have forced some users into devastating economic circumstances, the complaints allege.
“I’ve [been] submitting a request for over 1 year now saying I need help unlocking my account because I have a new number,” said one Brooklyn resident in a June 2021 complaint to the ftc. “I have a daughter & I will be put out by next month.”
“id.me for nys unemployment hung up [the] phone on me when I said I was hearing impaired and she was complaining about the way I sound,” read another ftc complaint submitted that September from Endicott, New York.
About 100 people in New York have filed ftc complaints about the company, which New York Focus obtained through a public records request. Even more New York residents have filed complaints with the Better Business Bureau.
“For almost six weeks I have unsuccessfully attempted to comply with id.me requirements,” read a bbb complaint filed from Utica in September 2021. “I am desperately seeking help. I am pretty much penniless and cannot afford for this to keep dragging on.”
It’s not clear whether the dol thinks there’s a problem. New York deployed id.me at the beginning of 2021, and the system is still in use today. A November comptroller’s report found that the department has failed to track critical metrics related to the software’s performance. The dol declined to comment on the complaints New York Focus found, citing confidentiality.
Meanwhile, the dol commissioner, Roberta Reardon, has defended the department’s approach to processing unemployment applications. id.me says that its service complies with federal guidelines surrounding digital identity verification. Still, critics argue that the state has essentially outsourced its responsibilities to a private company that now has outsized influence over whether people receive their benefits — with limited avenues for holding the company accountable.
“Like with many tech solutions that we saw during the pandemic, the problem is that we’ve rolled them out for one thing, and then they’re in place and they end up staying forever,” said Caitlin Seeley George, the managing director of the digital rights organization Fight for the Future. “In the crux of the moment, we did not do a good enough job of assessing the bigger picture impacts and concerns and privacy challenges with these tech solutions.”
ONE CHALLENGE IS the system’s biometric software. id.me uses two forms of biometrics as anti-fraud measures: face verification, which checks that a “selfie” taken by an applicant matches a photo of their government id, and facial recognition, which checks whether someone’s face matches any images that have previously been uploaded to the company’s database of identities.
Critics point out that facial recognition has significant accuracy problems, especially when it comes to people of color and women. Its platform can also be more difficult to use for people with certain disabilities, people who are older, and people who don’t have access to a computer.
“There’s a lot of evidence documenting that the technology is error prone. It has racially biased effects, and it’s not accessible to a lot of people,” said Terry Ding, staff attorney at the New York Civil Liberties Union.
In the past, id.me has said that about 10 to 15 percent of people have trouble with its self-service facial verification system and need to go through its customer service agents, dubbed “Trusted Referees.” But in New York, that percentage was roughly double, at 25 percent, or just over 180,000 people, according to the comptroller’s November report.
In the past, id.me has said that about 10 to 15 percent of people have trouble with its self-service facial verification system and need to go through its customer service agents, dubbed “trusted referees.” But in New York, that percentage was roughly double, at 25 percent, or just over 180,000 people, according to the comptroller’s November report.
Asked for comment, id.me told New York Focus that its usage rates vary across its partners, but that its overall “self-serve” pass rates have improved six percent in the past year.
Users have reported several other problems, including a wonky interface, challenges with documentation, and limited accessibility in languages other than English, according to an August analysis published by the National Center for Law and Economic Justice based on interviews with organizations that serve immigrant communities.
THE STATE INSISTS that it needs tech like id.me to handle the influx in benefits applications as a result of the pandemic. After shutdowns put thousands of New Yorkers out of work and federal legislation made more people eligible for unemployment, the Department of Labor found itself overwhelmed with applications for assistance.
It’s not clear how the labor department vetted id.me’s service, though. The department told the comptroller’s office that it considered multiple vendors, but chose id.me because it was already used by other states, a spokesperson for the comptroller’s office told New York Focus. The department then said the id.me contract was an emergency procurement, and that the deal was later extended by the comptroller’s office as a “single source” contract.
id.me hasn’t just confounded users trying to get benefits for the first time — it has also complicated access to existing benefits. During the second summer of the pandemic, some New Yorkers found their benefits suddenly stopped, according to Ciara Farrell, a volunteer attorney with the New York Legal Assistance Group.
The labor department told New York Focus that it has gradually introduced the software. Between February and June of 2021, the office only directed people flagged for fraud or “certain identity flags” to id.me. Then, until the middle of last July, every claimant was required to go through the service. Now, “only claimants who are flagged for identity issues or flagged for fraud are sent to id.me.”
After the Internal Revenue Service announced that it would contract with id.me in November of 2021, the company faced a surge in scrutiny. Privacy experts pointed to concerns about the company’s use of facial recognition, while others pointed to technical issues that the other states using id.me had already encountered. Federal officials started investigating the company’s claims about stopping fraud.
To quell the backlash, id.me announced in February 2022 that it would give government partners like states the option to turn off the facial recognition requirement for users and give people the choice to manually verify their identity with human customer service agents. Today, New Yorkers can now verify themselves through self-service, video chat, or in person.
But the company’s human customer service capacity may be limited. The Verge reported last year that agents were overwhelmed, and former employees told Insider that data security at the company was lax. Complaints filed with the ftc and bbb both reflect trouble accessing trustee referees — and customer service issues that stretched into 2022. id.me has defended its record on wait times since the pandemic, and said that its staffing fluctuates based on “forecasted volume.”
Federal inquiries indicate that it’s not clear how well id.me’s software actually stops fraud. A report commissioned by the House Oversight Committee — which studied id.me’s federal contract with the irs — found that the company had “overstated its capacity to conduct identity verification services.”
Nor is it apparent whether New York measures how well id.me’s systems actually work. When asked about id.me’s performance metrics, the labor department told New York Focus to file a public records request; the department was non-responsive when the aclu filed a request for accuracy information. id.me said it studies reinstatement rates — instances where its staffers identified fraud, but then conducted subsequent investigations that proved otherwise — but did not share what those rates were.
The comptroller’s office told New York Focus that it has raised the question of performance data to the labor department. Without data, the office said it’s difficult to determine whether people failing to complete the process represented fraudsters who were deterred or genuine applicants who were “discouraged by the process.”
“It’s ludicrous that we trust these companies to self-report data, when they can end up being a bottleneck to services people need in an emergency,” says Albert Fox Cahn, the executive director of the Surveillance Technology Oversight Project. “I don’t think any agency or vendors should be allowed to police themselves when it comes to their performance rate.”
Today, filings suggest that the number of people having issues with id.me verification for unemployment has declined, in part because fewer people are applying for unemployment. id.me is planning to open 46 self-service retail kiosk locations, in partnership with identity verification service Sterling Identity, throughout New York.
But problems persist. Some New Yorkers are learning that they now need to use the service to move forward with claims of identity fraud that they filed with the dol several years ago, Farrell, from nylag, told New York Focus.
In April, new federal Department of Labor guidance warned that states must ensure their fraud-prevention tech choices “do not create barriers that prevent or limit access” with discriminatory effects.
And in May, the National Center for Law and Economic Justice filed a Title vi complaint with the federal dol, pointing to two individuals who had issues with the service. The non-profit is requesting that, among other demands, the federal agency force id.me to offer far better translation services — or have the New York dol replace the id.me with another service. (id.me called the complaint “deeply flawed.”)
According to Anjana Malhotra, a senior attorney at the National Center for Law and Economic Justice, the state’s labor department has lost sight of individual impact.
“What the Department of Labor in New York State is forgetting about in its kind of zeal to find fraud,” she said, “is the really serious human cost and toll of these private programs that have a vested interest in identifying fraud.”
