Systematically Landing Machine Learning onto Market-Scale Mobile Malware Detection [journal]

Three computers with locks in a circle illustration

Journal

IEEE Transactions on Parallel and Distributed Systems - December 21, 2020

Authors

Liangyi Gong, Hao Lin, Zhenhua Li, Feng Qian (assistant professor), Yang Li, Xiaobo Ma, Yunhao Liu

Abstract

Despite being crucial to today's mobile ecosystem, app markets have meanwhile become a natural, convenient malware delivery channel as they actually “lend credibility” to malicious apps. In the past few years, machine learning (ML) techniques have been widely explored for automated, robust malware detection, but till now we have not seen an ML-based malware detection solution applied at market scales. To systematically understand the real-world challenges, we conduct a collaborative study with T-Market, a popular Android app market that offers us large-scale ground-truth data. Our study illustrates that the key to successfully developing such systems is multifold, including feature selection and encoding, feature engineering and exposure, app analysis speed and efficacy, developer and user engagement, as well as ML model evolution. Failure in any of the above aspects could lead to “wooden barrel effect” of the whole system. This paper presents our judicious design choices and first-hand deployment experiences in building a practical ML-powered malware detection system. It has been operational at T-Market, using a single commodity server to check ∼10K apps every day, and has achieved an overall precision of 98.9% and recall of 98.1% with an average per-app scan time of 1.1 minutes.

Link to full paper

Systematically Landing Machine Learning onto Market-Scale Mobile Malware Detection

Keywords

security, machine learning