Jason d’Eon (Dalhousie, Vector), Sri Harsha Dumpala (Dalhousie, Vector), Chandramouli Sastry (Dalhousie, Vector), Daniel Oore (IICSI, MUN), Mengyu Yang (U of T), Sageev Oore (Dalhousie, Vector), “A Speech-Based Music Composition Tool With Transformer” and “Musical Speech: A Transformer-based Composition Tool,” NeurIPS 2020 (34th Conference and Workshop on Neural Information Processing Systems)

BREAK DOWN: http://dani.oore.ca/moocow/

The speech in this video clip was fed into an AI transformer-based composition tool, which output raw MIDI files that were then used to trigger all the digital instrument sounds heard.

The initial fragment of regular speech at the beginning of the video below was fed into an AI transformer-based composition tool, which output raw MIDI files that were then used to trigger all the instrument sounds heard, and which was edited together with the video to create this piece: