Adversarial machine learning is the study of the attacks on machine learning algorithms, and of the defenses against such attacks.[1] A survey from May 2020 exposes the fact that practitioners report a dire need for better protecting machine learning systems in industrial applications.[2]
Most machine learning techniques are mostly designed to work on specific problem sets, under the assumption that the training and test data are generated from the same statistical distribution (IID). However, this assumption is often dangerously violated in practical high-stake applications, where users may intentionally supply fabricated data that violates the statistical assumption.
^Kianpour, Mazaher; Wen, Shao-Fang (2020). "Timing Attacks on Machine Learning: State of the Art". Intelligent Systems and Applications. Advances in Intelligent Systems and Computing. Vol. 1037. pp. 111–125. doi:10.1007/978-3-030-29516-5_10. ISBN978-3-030-29515-8. S2CID201705926.