Single-stroke language-agnostic keylogging using stereo-microphones and domain specific machine learning
Sashank is a second year PhD student in Information Assurance at Northeastern University. He focuses on mobile security specifically on the impact of smartphone sensors on user privacy.
Mobile phones are equipped with an increasingly large number of precise and sophisticated sensors. This raises the risk of direct and indirect privacy breaches. We investigated the feasibility of keystroke inference of user taps on a soft keyboard using the stereoscopic microphones on an Android smartphone. We developed algorithms for sensor-signals processing and domain specific machine learning to infer key taps using a combination of stereo-microphones and gyroscopes. While previous studies focused on larger key sizes and repetitive attempts, we showed that by focusing on the specifics of the keyboard and creating machine learning models and algorithms based on keyboard areas combined with adequate filtering, it is possible to achieve an accuracy of 90% - 94% for much smaller key sizes in a single attempt. In this talk, I will present our approach and findings along with some techniques to mitigate this kind of attack.