Yakub and Mendez ES 25 Final Project

Final Implementation Details

Our final implementation, in terms of hardware, changed a bit from conception to now. The main difference is that there is no start/stop button (ran out of room on the teensy in terms of touch pins and did not have time to implement a button) and there is an optometer plugged in to an analog read that codes for different octaves. Otherwise, the original 7 notes and sharp/flat switches are maintained.

On the software side, the pre-programmed songs play upon initial run of the program to reflect the lack of start/stop button. Programming of the song was facilitated by a python library that turned midi files into midi objects. The objects were then parsed for the information needed by the teensy usbMIDI library, namely note numbers and timings. As, due to memory issues, the entire song would not play, a python script was run to split the sequences of notes and timings into 103 arrays of size 10. This worked much better. The code is copied below this blog post.

Reflections

We definitely learned how to extend existing circuit designs to suit our needs. Furthermore, we learned more about MIDI file structure. Notes in MIDI run from 1-127 mapping C0 to C9. It is also divided into a series of tracks and events that control the sequencing of the notes. The timing is done with ticks, which are relative timings based on a resolution. The higher the resolution the more sampling. Knowing this structure was crucial in writing the scripts that made the arrays to run the songs.

In our implementation, we employed knowledge on circuitry, teensy, and compositional fundamentals to create our project. More practical application with the circuitry and with composition writing material would have helped.

Division of Work

Julio mostly did the implementation of the keyboard as well as writing the code to turn the compositions into runnable code. Sasha wrote the compositions for this as well as helped with some aspects of the code.

import midi
import serial
import time
# Class Needs:
# take midi input
# output text file of note sequences
# output text file of note durations


class Song(object):
def __init__(self, midi_file):
self.midi_file = midi_file
self.pattern = midi.read_midifile(self.midi_file)

def get_bpm(self):
for track in self.pattern:
for event in track:
if type(event) is midi.events.SetTempoEvent:
return event.get_bpm()
break
break


def get_timings_and_notes(self, bpm):
#MIDI does things timing relative ticks
#Find how much time per tick using resolution
#MIDI timings in microseconds but we need milliseconds
milli_per_tick = (((60 * 1000000) / bpm)/self.pattern.resolution)/1000
timings = list()
notes = list()
for track in self.pattern:
for event in track:
if type(event) is midi.events.NoteOnEvent:
notes.append(event.data[0])
if type(event) is midi.events.NoteOffEvent:
timings.append(event.tick * milli_per_tick)
return timings,notes


def get_file(self):
self.pattern = midi.read_midifile(self.midi_file)
timings,notes = self.get_timings_and_notes(float(self.get_bpm()))
filename = self.midi_file[self.midi_file.rfind('/')+1:self.midi_file.rfind('.')] + str(0) + ".txt"
f = open(filename, "w+")
array_num = 0
f.write("int notes_1" + str(array_num) + "[] = ")
nums = str()
for x in range(len(notes)):
nums += str(notes[x]) + ","
if x == 0:
f.write('{')
# nums += str(notes[x]) + ","
if (x + 1) % 10 == 0:
array_num += 1
f.write(nums[:-1] + '};\nint notes_1' + str(array_num) + '[] =')
nums = str()
f.write('{')

f.write('\n')
array_num = 0
f.write("float times_1" + str(array_num) + "[] = ")
times = str()
for x in range(len(timings)):
times += str(timings[x]) + ","
if x == 0:
f.write('{')
# times += str(timings[x]) + ","
if (x + 1) % 10 == 0:
array_num += 1
f.write(times[:-1] + '};\nfloat times_1' + str(array_num) + '[] =')
times = str()
f.write('{')
f.close()

def main():
song_1 = Song("/Users/juliomendez/Downloads/es_track1.mid")
song_1.get_file()
timings, notes = song_1.get_timings_and_notes(song_1.get_bpm())
ser = serial.Serial('/dev/cu.usbmodem3136991', 9600)
for x in range(len(timings)):
ser.write(timings[x])
time.sleep(.001)
ser.write(notes[x])


f = open("play.txt", "w+")
for x in range(103):
f.write("play(notes_1"+str(x)+", times_1" + str(x) + ");\n")

if __name__ == '__main__':
main()


Adding Functionality

The main idea is to make  a teensy-based keyboard with pre-programmed songs that are theoretically playable. This is what Sasha’s score looked like.

As you can see, the range here is pretty large. The score goes from C0 to G above C7. The current program builds off of the lab on teensy. It has all of the wiring of the lab representing notes C,D,E,F,G,A,B and an additional two pins for flat and sharp, enabling a chromatic scale. The only problem is that this assumed that the composition would only be in one octave.

I thus had to find a way to represent a difference in pitch. I experimented with using the different current readings given by pressing the touchpins. This didn’t produce a very stable difference, however. Maddie thus helped me figure out the wiring for a potentiometer. The wiring is shown below:

Essentially, the potentiometer has three prongs that are connected into the breadboard. One side is connected to ground, the other to the 3.3V pin on the teensy through a resistor. The middle connects to an Analog in pin on the teensy. The dial on the potentiometer allows us to control the amount of current that is read by the Analog in. This has an effective range of 0-1024 so it is fairly straightforward to divide it into around 10 bins that map to 10 octaves. That is to say that 0-100 map to octave 0, 101-200 to octave 1, etc. Theoretically, one could shift the dial as they run through the song to represent notes not in the octave C4 to B4.

Current Challenge

The programming of the song into teensy has been difficult. I managed to use a python library that abstracted MIDI files into objects. This allowed me to more easily parse the midi file for note values and timings. I need these to play sounds via the usbMIDI.sendNoteOn and usbMIDI.sendNoteOff functions on teensy. Initially, I attempted to read these via the Serial console via a CSV file, but this produced IO errors that were difficult to solve. As such, I tried transferring the notes and timings as two different, rather large arrays directly into the program. This code seems to work (it’ll play around 7 notes, for example, but may be limited by the memory of the device. I am currently debugging and looking into better buffering schemes to see if this solves the issue.

For the first track on the EP, I experimented with a few different compositional techniques: to inform the way that I set-up the large-scale horizontal hemiolas—one of the most potent ways to successfully generate rhythmic processes in a one-voice composition—I found inspiration in the way György Ligeti approached this problem in compositions like his Musica ricercata, as well as works like Continuum and the piano etudes. Naturally, because this essentially a “player-piano” composition, I also turned to the work of Conlon Nancarrow, whose specialty it was to write music for this instrumentation. Going further with the pianistic conception of my idiom, I deliberately chose an arpeggio as one the core elements of this piece for that reason. I wasn’t sure of the Teensy’s range, so I’m ready to modify the composition if it turns out that measures like 25 are out of range.

Introduction:

Both of us have backgrounds in programming (Julio as a concentrator, Sasha as someone who took CS50 last semester) and in music (Sasha as a concentrator/composer, Julio as an amateur musician). We decided this would be a unique opportunity to collaborate and supplement each other’s skill sets in working toward an end product.

We settled on a project revolving around the Teensyduino/Arduino IDE. Essentially, we will write two programs to be used with Teensyduinos. One will be a modification of “touch_sensing.cpp” that will support accidentals; wire “keys” will also potentially be upgraded piezo “keys” via soldering.

The second program will treat the wires as “play/pause,” “stop,” “next track,” and so on—essentially, physical controls for a media player. The player will play songs transcribed from musical notation to code that works with the Teensyduino “piano” and will form a sort of EP.

The composition aspect of this project comes into play largely as an application of the first program; the second program will then become a neat way to showcase said compositions, and, by extension, the success of the first program’s implementation.

Background:

Not counting necessary prior training in programming and specifically C/C++, the knowledge needed for our project has stemmed largely from the lab in which we were taught to use the Teensyduino hardware/Arduino IDE in conjunction with Max.

We plan to make the EP very eclectic so as to showcase the possibilities of recording with our Teensyduino instrument. Until we work out the details of the implementation of our modification of “touch_sensing.cpp” it is difficult to say for sure, but the tentative plan is to make 3-4 original tracks covering something like classical, disco, hip-hop, and jazz.