The average person might not see it but mathematicians see the beauty and power of math in our everyday lives. It can make lives beautiful but at the same time, it can also destroy it. Mathematician and former Wall Street analyst Cathy O'Neil believes with all her heart that math is beautiful. However, she also began to see the power of math to destroy and promote evil in the world.
In her book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, she explores and explains how well-intentioned mathematical models can actually deepen inequality and division.
She used Tay as an example. Tay was the artificially intelligent bot Microsoft launched earlier this year. Described as an average teenage American female, she was to take in information, process it, and learn how to speak like the humans.
Tay was a success and a fast learner, too fast that within a few hours, she was spewing remarks like a bigot, especially against feminists. She cursed at them and wished them dead in hell. If that was not enough, she began her political rants accusing Bush of 9/11 and praising Hitler saying the German strongman would've made a better job. Less than 24 hours before her debut, Microsoft shut her down.
That's what happens when algorithms go wrong, O'Neil said. When recidivism models make its way into the mathematical algorithms, it can create destructive feedback loops.
For example, if a student from a less well-to-do background applies for a loan, the lending algorithm armed with this model might tag him as very risky and deny the loan he needs. Because of that, he has to get out of school, which was supposed to be his ticket out of poverty. This increases the poverty and the vulnerability of the less privileged.
She further explained that this is not the only area where the power of math is abused. When she worked as a quantitative analyst in Wall Street, she recalled how math was used to hide corrupt practices in finance. That was her first experience how the power of math was used as a weapon.
She said that math deserves more than being abused and used to scare people. She added that it should not be used to make people believe blindly just because they have a math flavor to it.
