New Types of Instruments

[nosuchtim]

I've been experimenting with and interested in new types of instruments for a few decades now, and I've been asked by the KVR folks to post some thoughts and information; this is my first post about that topic.

Advances in technology allow new types of instruments to be created, either as evolved versions of existing instruments or completely new types of instruments. In subsequent postings, I'm going to describe some of the new possibilities that exist in instrument design, using my own work and the work of others as examples. These possibilities extend beyond music into visuals, and beyond performance into casual uses for entertainment and interactive installations. These are some of the technologies have seen major advancements recently and can be particularly useful in instruments:

- pressure-sensing surfaces
- haptic feedback
- depth cameras
- GPUs
- open source

My own experience in recent decades has involved building interactive installations deployed at festivals, events, and museums. My installations are specifically designed to be "casual instruments", analagous to "casual games" that are a large and popular category of computer-based games. The appeal of casual games and instruments is that they require very few if any instructions, so people can immediately enjoy them without any prior experience, training, or practice. At the same time, an ideal casual game or instrument has a depth that reveals itself and can be explored as people gain more experience with it. In casual games the depth is discovered by exploring strategies. In casual instruments the depth is discovered by exploring expressiveness and creativity, things which may be new to someone who hasn't had the time or training to learn a traditional instrument. By having no barrier to entry and providing immediately entertaining interaction, a casual instrument can be a portal into finding personal expressiveness and creativity. But the first goal of a casual instrument, like a casual game, is to be fun and entertaining.

Burning Man has been a particularly inspiring event for me. Unlike the "real world" where gallery space and other opportunities are limited and curated, Burning Man has virtually unlimited space and uncurated opportunity, so most of an artist's effort can be spent on creating things rather than marketing and finding opportunities for deployment.

While KVR has historically focused on music, visuals have become an important and expected part of our musical experience, and the technologies for audio and video have advanced and evolved together. For example, modular synthesis in the eurorack world now covers both audio and video, with many things shared and leveraged between those two worlds. In future postings I'll explore all types of instruments - visuals-only, music-only, and visual music instruments that do both simultaneously. I'll provide examples from my own work as well as the work of others, showing what technology is making possible.

...Tim...

[birdsoffire]

This is a fascinating topic. As someone who has owned a Roli Seaboard block and a Linnstrument (sold both) I hope to learn more about this subject. Hopefully the responses will be better than the one above. I owned a Steiner Synthacon in the 70s. Nyle added a piezo electric transducer on the top for natural feeling vibrato like he had on his EVI it really felt like playing a string instrument. Anything that can give you more musical expression easily can be a gateway to creativity.

[Ben [KVR]]

Welcome Tim!

As the newest member of the "KVR Experts", Tim is going to be posting interesting topics here in the KVR Experts forum as and when he has something of interest on his mind. Please be nice and respectful!

Here's his bio:

Tim Thompson is a software engineer, musician, and interactive installation artist. His wide-ranging artistic work over four decades includes a programming language for MIDI, interactive installations at Burning Man and other festivals, musical performances with Playstation dance pads and QWERTY keyboards, and realtime video looping and processing with a handheld security camera. Recently, Tim has focused on the expressive potential of three-dimensional input in casual instruments for visual music, using devices such as the Microsoft Kinect (in the original Space Palette) and the Sensel Morph (in the newer Space Palette Pro).

His website is https://timthompson.com

[whyterabbyt]

While KVR has historically focused on music, visuals have become an important and expected part of our musical experience, and the technologies for audio and video have advanced and evolved together.

So can we get that video forum then, Ben?

Very interested in what gets posted here by Tim. I remember poking at Keykit years ago. Must have a look at it again, since it looks like its running on the Pi. Wonder if it can be made to work on a Nebulae module...

[vurt]

interesting!

[_leras]

I love playing with interactive installations! Thank you on behalf of all the people who get to experience your installations.

It's a very interesting topic for sure. From Bjorks Reaktable through to linnstrumet, and the Roland d beam there have been lots of different approaches to controlling multiple dimensions of a synth through different interfaces.

I think perhaps one of the hardest challenges must be, when related to synths, finding patches that can be played expressively, and then maybe even having a set of them that work with a particular set of parameters. NI kinda led the way to having 8 'macro' controls, but they get used for different things every time.

MPE and the higher level of per note control, and this integration with the CLAP standard probably makes the back end for this type of thing at the moment. Have you tried using these?

The pedantic part of me sees this more as interfaces for synths, or even for specific synth patches, than new instruments per se. But then a traditional instrument is a physical interface to a single physical preset, so why can't a custom interface for a single synth patch be considered an instrument.

I think the biggest challenge is creating patches that can really make use of multi dimensional input in an expressive way. The interface also needs to have an expressive range. For real instruments the wider level of expression is somewhat linked to how easy, or hard, an instrument is to play.

[nix808]

+1
on visuals and video forum,
I'm a visual artist also

My unusual controller is midi guitar,
where some experience with Ruby language allows me some manipulative agility,
making my input device adaptively bent to me,
a new instrument(Fender Mad Catz Squier, Roland GK3, customizations)

I also made a guitar with pneumatic keys, the 'air guitar'

[Winstontaneous]

Welcome Tim, great to have you here! I have a background in community arts and sonic/visual installation but in recent years am "just" making music. I love the idea of "casual instruments" done well and created an (analog) workshop guiding people through experiencing the harmonic series with common household objects and their voices.

Curious about how you've seen "non-experts" from different cultural backgrounds experience things like consonance/dissonance/noise and say, non-tempered or pentatonic scales vs. Western tonality. Also your background/interests have parallels with Bret Victor, who I discovered after buying an Alesis Ion synth he designed.

I'd love a Visuals forum, can't wait for the "Best video reverb?" and "'Shrooms vs. acid" threads!

[aagnello]

Here's a fundamentally different type of instrument. Leon Gruenbaum was granted a patent a while ago for an intervallic keyboard. When you hit a key, you don't get a specific note but instead get a note some interval away from the note that you played. It's a different way to create melodies.

https://www.youtube.com/watch?v=1RS4giQHAUg&t=2s

[nosuchtim]

... a traditional instrument is a physical interface to a single physical preset, so why can't a custom interface for a single synth patch be considered an instrument.

Exactly. And extending that, why can't a custom interface control a curated collection of patches which are meant to be played together simultaneously. Like an orchestra. A conductor could be considered to be playing an orchestra as an instrument. A typical conductor doesn't directly control the pitches being played by the members of the orchestra, but with certain limitations and automation, new instruments can enable that kind of control. At least, I think so, and my Space Palette Pro instrument is an attempt in that direction; playing it does feel like conducting an orchestra.

...Tim...

[nosuchtim]

Curious about how you've seen "non-experts" from different cultural backgrounds experience things like consonance/dissonance/noise and say, non-tempered or pentatonic scales vs. Western tonality.

An instrument often imposes contraints (scale, etc) that fit a particular cultural sound, so non-experts often naturally produce sounds that fit that culture, no matter what their background. Whether they like it or not is up to personal taste and experience. It's interesting that GeoShred (an instrument app for iOS) has become very popular in India because of the sounds and features they've put in that make it possible and easy to produce Indian melodies with their bends and slides:

https://asianlite.com/2022/columns/lite ... -in-store/

...Tim...

[nosuchtim]

Here's a fundamentally different type of instrument. Leon Gruenbaum was granted a patent a while ago for an intervallic keyboard.

Great example! His original instrument was called the Samchillian Tip Tip Tip Cheeepeeeee, which has a Wikipedia page: https://en.wikipedia.org/wiki/Samchillian . The patent is now expired, so this brilliant technique can be implemented freely, and it's been showing up in other instruments and plugins - here's one in Max for Live: https://www.soundmanufacture.net/devices/reboard

The Samchillian won 3rd place in the Guthman Musical Instrument Competition in 2011. The Guthman competition is a wonderful resource for people interested in new instruments. Here are this year's finalists: https://guthman.gatech.edu/2024-finalists . I'm particularly fond of the Sonògraf entry, by the Playmodes collective in Spain that also produced my favorite iPad app, Beatsurfing2.

...Tim...

[FranklyFlawless]

It is a pleasure to encounter you here Tim. I am already quite familiar of your work due to your contributions to the LinnStrument's firmware and the Space Palette Pro using multiple Sensel Morphs.

One of the "casual instruments" I am aware of is the Skoogmusic Skoog. Otherwise, for "new types of instruments", John Pascuzzi maintains the Oddmusic website, and I have my own list of MPE hardware controllers I maintain as well, categorized based on their interfaces. I am highly confident you would appreciate that last particular detail.