TY - GEN
T1 - A Methodology for Non-Functional Property Evaluation of Machine Learning Models
AU - Anisetti, Marco
AU - Ardagna, Claudio A.
AU - Damiani, Ernesto
AU - Panero, Paolo G.
N1 - Funding Information:
This work has received funding from CONCORDIA, the Cybersecurity Competence Network supported by the European Union's Horizon 2020 Research and Innovation program under grant agreement No 830927.
Funding Information:
This work has received funding from CONCORDIA, the Cyberse-curity Competence Network supported by the European Union’s Horizon 2020 Research and Innovation program under grant agreement No 830927.
Publisher Copyright:
© 2020 ACM.
PY - 2020/11/2
Y1 - 2020/11/2
N2 - The pervasive diffusion of Machine Learning (ML) in many critical domains and application scenarios has revolutionized implementation and working of modern IT systems. The behavior of modern systems often depends on the behavior of ML models, which are treated as black boxes, thus making automated decisions based on inference unpredictable. In this context, there is an increasing need of verifying the non-functional properties of ML models, such as, fairness and privacy, to the aim of providing certified ML-based applications and services. In this paper, we propose a methodology based on Multi-Armed Bandit for evaluating non-functional properties of ML models. Our methodology adopts Thompson sampling, Monte Carlo Simulation, and Value Remaining. An experimental evaluation in a real-world scenario is presented to prove the applicability of our approach in evaluating the fairness of different ML models.
AB - The pervasive diffusion of Machine Learning (ML) in many critical domains and application scenarios has revolutionized implementation and working of modern IT systems. The behavior of modern systems often depends on the behavior of ML models, which are treated as black boxes, thus making automated decisions based on inference unpredictable. In this context, there is an increasing need of verifying the non-functional properties of ML models, such as, fairness and privacy, to the aim of providing certified ML-based applications and services. In this paper, we propose a methodology based on Multi-Armed Bandit for evaluating non-functional properties of ML models. Our methodology adopts Thompson sampling, Monte Carlo Simulation, and Value Remaining. An experimental evaluation in a real-world scenario is presented to prove the applicability of our approach in evaluating the fairness of different ML models.
KW - Machine Learning Assurance
KW - Multiarmed bandit
KW - Non-functional properties
UR - http://www.scopus.com/inward/record.url?scp=85097877491&partnerID=8YFLogxK
U2 - 10.1145/3415958.3433101
DO - 10.1145/3415958.3433101
M3 - Conference contribution
AN - SCOPUS:85097877491
T3 - Proceedings of the 12th International Conference on Management of Digital EcoSystems, MEDES 2020
SP - 38
EP - 45
BT - Proceedings of the 12th International Conference on Management of Digital EcoSystems, MEDES 2020
T2 - 12th International Conference on Management of Digital EcoSystems, MEDES 2020
Y2 - 2 November 2020 through 4 November 2020
ER -