In a series of tweets, David Heinemeier Hansson, a partner at software development firm Basecamp, claimed that he was approved for 20 times the credit limit that his wife received — even though they file joint tax returns and she actually has a better credit score. The posts got a lot of attention, with Apple co-founder Steve Wozniak even weighing in.
Now the New York Department of Financial Services has opened a probe into the allegations.
“The department will be conducting an investigation to determine whether New York law was violated and ensure all consumers are treated equally regardless of sex,” said a spokesman for Linda Lacewell, the superintendent of the NY DFS. “Any algorithm that, intentionally or not, results in discriminatory treatment of women or any other protected class of people violates New York law.”
Goldman spokesman Andrew Williams defended the company’s credit decisions, saying they “are based on a customer’s creditworthiness and not on factors like gender, race, age, sexual orientation or any other basis prohibited by law.”
But Hansson countered that the lender’s statement doesn’t explain Goldman’s move after his tweets went viral.
“As soon as this became a PR issue, they immediately bumped up her credit limit without asking for any additional documentation,” he said, according to Bloomberg. “My belief isn’t there was some nefarious person wanting to discriminate. But that doesn’t matter. How do you know there isn’t an issue with the machine-learning algo when no one can explain how this decision was made?”
One silver lining: The controversy has resulted in an internal review, with Hansson hopeful that it will lead to a wider discussion on black-box algorithms, which have come under fire for instances of bias. Even Washington D.C. lawmakers have taken notice, with Sen. Elizabeth Warren, a 2020 Presidential candidate, telling federal regulators that the government “will have to take action to ensure that anti-discrimination laws keep up with innovation.”
“Goldman and Apple are delegating credit assessment to a black box,” noted Hansson. “It’s not a gender-discrimination intent but it is a gender-discrimination outcome.”