Skip to content

Tools for capturing and analysing keyboard input paired with microphone capture 🎤⌨️

License

Notifications You must be signed in to change notification settings

francoamato/kbd-audio

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

kbd-audio Build Status

Description

This is a collection of command-line and GUI tools for capturing and analyzing audio data.

Keytap

The most interesting tool is called keytap - it can guess pressed keyboard keys only by analyzing the audio captured from the computer's microphone.

Check this blog post for more details:

Keytap: description and some random thoughts

Video: short demo of Keytap in action

Keytap2

The keytap2 tool is another interesting tool for recovering text from audio. It does not require training data - instead it uses statistical information about the frequencies of the letters and n-grams in the English language. The tool is still in development, but you can see a short demonstration here:

Video: Keytap2 - recovering text from typing sound (7:50)

CTF: can you guess the text being typed?

Build instructions

Dependencies:

  • SDL2 - used to capture audio and to open GUI windows libsdl

    [Ubuntu]
    $ sudo apt install libsdl2-dev
    
    [Mac OS with brew]
    $ brew install sdl2  
    
  • FFTW3 (optional) - some of the helper tools perform Fourier transformations fftw

Linux and Mac OS

git clone https://github.com/ggerganov/kbd-audio
cd kbd-audio
git submodule update --init
mkdir build && cd build
cmake ..
make

Windows

(todo, PRs welcome)

Tools

Short summary of the available tools. If the status of the tool is not stable, expect problems and non-optimal results.

Name Type Status
record text stable
record-full text stable
play text stable
play-full text stable
view-gui gui stable
view-full-gui gui stable
keytap text stable
keytap-gui gui stable
keytap2 text development
keytap2-gui gui development
- extra -
guess_qp text experiment
guess_qp2 text experiment
key_detector text experiment
scale text experiment
subreak text experiment
key_average_gui gui experiment

Tool details

  • record-full

    Record audio to a raw binary file on disk

    ./record-full output.kbd [-cN]
    

  • play-full

    Playback a recording captured via the record-full tool

    ./play-full input.kbd [-pN]
    

  • record

    Record audio only while typing. Useful for collecting training data for keytap

    ./record output.kbd [-cN]
    

  • play

    Playback a recording created via the record tool

    ./play input.kbd [-pN]
    

  • keytap

    Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool.

    ./keytap input0.kbd [input1.kbd] [input2.kbd] ... [-cN] [-pF] [-tF]
    

  • keytap-gui

    Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool. GUI version.

    ./keytap-gui input0.kbd [input1.kbd] [input2.kbd] ... [-cN]
    

    **Live demo (WebAssembly threads required) **

    keytap-gui


  • keytap2-gui (work in progress)

    Detect pressed keys via microphone audio capture. Uses statistical information (n-gram frequencies) about the language. No training data is required. The 'recording.kbd' input file has to be generated via the record-full tool and contains the audio data that will be analyzed. The 'n-gram.txt' file has to contain n-gram probabilities for the corresponding language.

    ./keytap2-gui recording.kbd n-gram.txt
    

    keytap2-gui


  • view-full-gui

    Visualize waveforms recorded with the record-full tool. Can also playback the audio data.

    ./view-full-gui input.kbd
    

    view-full-gui


  • view-gui

    Visualize training data recorded with the record tool. Can also playback the audio data.

    ./view-gui input.kbd
    

    view-full-gui


Feedback

Any feedback about the performance of the tools is highly appreciated. Please drop a comment here.

About

Tools for capturing and analysing keyboard input paired with microphone capture 🎤⌨️

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 95.7%
  • CMake 3.6%
  • Shell 0.7%