Is your AI system discriminating without knowing it?: The paradox between fairness and privacy
Much has been written about algorithms that discriminate against protected groups, but always in the case that the protected group's representing sensitive variable is in the data set. However, in many if not most real-world situations, this sensitive variable is not in the data set. So how to detect potential discrimination? Today, you can't.
very interesting, i am involved in these type of discussions for facial analysis technology using ML/AI. In the USA, i think the sensitive data is in the data set, often incorrectly weighted/highlighted, and insufficiently training the algorithm. This is only my novice hypothesis at this point. I am of the opinion that the broadest data, most diverse/disparate data, and ultimately, most sensitive data SHOULD be included in the data sets for training algorithm purposes. and the sooner the better. statistical bias and potential discrimination hence trends to zero. Another hypothesis: in the case of facial recognition, the algorithm should be trained with at least 7.53 billion faces (1 of each person in the world) and possibly more for each face at a different angle and different lighting scenarios. But that should be standardized, so each face doesn't get more data than the next face. Then, you could never claim bias or discriminatin - it is what it is at that point. Not practical, just a simplified hypothesis/vision that would take forever to test and there are probably more practical ways to accomplish the same goal.
