Segregation of the vocal stream into three drums (Bass Drum, Snare Drum and Hi Hat) is achieved using machine learning techniques, so that the user has few tweaks to do to start to control the drums. Two segregation modes are available, the first uses a beat box model made with the recordings of beat-boxers and behaves well for microphone control, while the other tries to separate the incoming sounds (more intelligent but less predictable), and is more efficient for processing complex polyphonic recordings.
The system is implemented as a VST plug-in (Signal Analysis, Onset/Offset Detection, Vocal Hits Segregation) coded in C++ with CLAM Library for Audio and Music library. It works using a frame size of 1024 samples and is fast enough for real-time performance.
NOTE: BillaBoop is currently only really suitable for use in the Plogue Bidule modular framework to control synthesizers, effects, and samplers and the download includes a Bidule layout. They plan to improve the compatibility with other plug-in hosts in the future.
