Friday, October 7

Technology of Keyboard Sniffer

Suppose you sit close to me, and you can hear my keyboard stroke. Can you tell what I am typing from the sound of keyboard stroke?

"That is IMPOSSIBLE!", you say.

Assume we are using MSN Messenger now, although we are sitting close, we don't want to break the quiet. So you ask me a question from the MSN Messenger:
"Is today Thanksgiving?"

Then you hear I hit the keyboard 3 times follow by a big stroke, before the message actually send to your computer (because of the God-damned network delay, maybe), you know I am typing:
"yes" and Enter.

or
"yep" and Enter.

If you hear I hit the keyboard only twice and Enter, then it must be
"no" and Enter

At the beginning you said it is IMPOSSIBLE to tell from the sound of keyborad stroke, but now you are doing great!

Combined with the context and a dictionary and the word pattern, Keyboard sound recognition is possible. Sept 18, CNN reported Computer eavesdropping yields 96 percent accuracy rate, I can imagine the technology behind it.

First of all, the sound of keyboard stroke for different key are not exactly the same. When people hit the "Enter" key, usually people hit it hard, so the sound is different from others. Also, the "Space" is very different from others too.
People use different strength in different fingers: index finger, middle finger, ring finger, little finger. Also, left hand and right hand have different strength. It is possible for a computer using a good algorithm to tell the difference between them because the volumn will be different. According to CNN's report, the computer had an accuracy of about 60 percent for characters and 20 percent for words.

Second, The pattern of words can help to identify the typing characters. "After spelling and grammar checks were deployed, the accuracy for individual letters jumped to 70 percent and words to 50 percent."

To Be Continued...

Labels: ,

1 Comments:

At October 11, 2005 6:19 AM, Blogger Unknown said...

Yush:
我看了你的blog,挑点儿刺:
1. 文章开始举的msn的例子不合适。
"Is today Thanksgiving?"
"yes"/"no"
问句并不是已知,也同样是“猜”出来的。

2. 漏了最终结果
In the end, it could accurately detect 96 percent of characters and 88 percent of words.

3. 击键声音的不同,原因还可能是:各键在键盘上的位置不同,因此机械特性不同,声音频率上就有区别。

4. 由于各键的空间位置不同,击键声音及附近物体的回声传到话筒的传输路径不同,话筒接收到的“直达”声音和回声的关系也可以作为一个重要特征。

这项研究所采用的技术应该跟语音识别相同,也是hidden markov model。IBM的先期研究对应于“特定人语音识别”,Berkeley的则对应于“非特定人语音识别”。最终达到的精确度有点儿令人吃惊。
采用的技术肯定不是“简单的字典匹配”。

Ben's Reply:
我想的也差不多
我还没有写到最终结果,因为写到现在那个地方,就应该插入“恢复法庭文档”的例子,但是我没找到它,所以就停下来了。

HMM呢还是NN? 我觉得这个技术突兀的地方,并不在于这一点,所以在我的blog中只用一句话就带过去了:
It is possible for a computer using a good algorithm to tell the difference
我觉得,
1,居然每个键的声音还不一样,这是以前想不到的(手指力量、机械特性)
2,根据声音还分不出来的时候,用词典甚至语法来补充。After spelling and grammar checks were deployed, the accuracy for individual letters jumped
当然不是简单的字典匹配,而是把原本看来毫不相关的字典匹配引入到这个项目中,以及引入到“恢复法庭文本”之中,令我豁然开朗。至于HMM和NN,本来就应该是在这一行(pattern recognition)的,所以没有什么奇怪。

第一个问题,问句并不是“听”出来的,只有答复是听出来的。我只是先举个例子,表示在特定上下文环境中,你真的能听出击键的内容。

 

<< Home