ai-jam-js

A.I. Jam

An interactive A.I. jam session.

ABOUT

This interface uses the frontend from the A.I. Duet experiment, combined with the Magenta MIDI interface to recreate the experience of the award-winning Magenta 2016 NIPS Demo in your browser.

CREDITS

Original interface built by Yotam Mann with friends on the Magenta and Creative Lab teams at Google using TensorFlow, Tone.js and open-source tools from the Magenta project.

GETTING STARTED

Pre-built Version

Install Flask and Magenta (v0.1.15 or greater).

From the the ai-jam-js directory launch the interface from the command line:

sh RUN_DEMO.sh

When the script is run, the following pre-trained models will be automatically downloaded to the ai-jam-js directory:

• Attention RNN
• Pianoroll RNN-NADE
• Performance RNN
• Drum Kit RNN

Development

If you'd like to make modifications to this code, first make sure you have Node.js 6 or above installed. You can then install of the dependencies of the project and build the code by typing the following in the terminal:

cd static
npm install
node_modules/.bin/webpack

Adding a -w flag to the final command will cause webpack to continuously re-compile the JavaScript code as you make changes.

MIDI SUPPORT

The A.I. Jam supports MIDI keyboard input using Web Midi API and the WebMIDI library.

MIDI input is routed from the browser to MIDI ports "magenta_piano_in" and "magenta_drums_in", on which two instances of the magenta_midi binary are listening; one for piano and the other for drums. Both of these instances output responses to the "magenta_out" port, which the browser is listening to for playback.

The two magenta_midi instances are kept in sync by a running midi_clock binary that outputs a metronome using MIDI control change messages on the "magenta_clock" port. You can listen to the metronome by pressing the Z key while the browser is in focus.

If you'd like to use your own device or software for synthesis, you can simply mute the browser tab and route these MIDI ports appropriately to your device.

PIANO KEYBOARD

The piano can also be controlled from your computer keyboard thanks to Audiokeys. The center row of the keyboard is the white keys.

CONTROLS

You can change various settings for the models and interactions using the buttons at the top left of the screen. Additionally, you may use the keyboard shortcuts below.

Key	Action
Z	Toggles the metronome.
Q	Toggles between piano and drums.
LEFT/RIGHT	Cycles through available models.
UP/DOWN	Adjusts sampling 'temperature'. Higher temperatures sound more random.
SPACE	Toggles looping of AI sequence.
M	Mutates AI sequence.
0-9	Sets AI response duration (in bars). 0 matches your input.
SHIFT + 0-9	Sets input sequence duration (in bars). 0 matches your input.
DELETE	Stops current AI playback.
X	Toggles "solo mode", which stops AI from listening to inputs.

AUDIO SAMPLES

Audio synthesized from FluidR3_GM.sf2 (Creative Commons Attribution 3.0).