I Played The First-ever Eye-controlled Instrument At The Sydney Opera House

I Played The First-ever Eye-controlled Instrument At The Sydney Opera House

aac Assistive Technology augmentative communication cerebral palsy Eye Gaze music

• Jess IrwinWoman with cerebral palsy, avid '80s rock fan, founder of DARE2BU

As a result of a complicated birth I have severe cerebral palsy. My condition affects my fine motor skills and causes my body to endure involuntary muscle spasms. Despite my physical disability I have always been an outgoing and creative person. Throughout my life I have been known to overcome barriers and push myself to achieve the next step at every turn. This is a tough gig sometimes, however when you don't fit into the so-called box, you strive to become a game changer. My past is full of critics who grossly underestimated my capabilities, but I just convinced myself to keep pushing forward and change their perception and help spread awareness.

For me, being independent in my business life means so much.

I can remember how difficult it was progressing through school not knowing what the future held due to having a severe physical disability. Today while illustrating what is possible to younger generations I hope to build their confidence and allow them to challenge their own barriers.

Through the use of technology, I am able to be fully independent in my work environment, from using a Macbook for my graphic design and website development, to securing my SLR digital camera to my wheelchair. For me, being independent in my business life means so much.

I have always been surrounded by music. I found that by immersing myself in beats and rhythms decreased my physical pain and lessened the severity of my spasms. One of the reasons I took up music photography was to satisfy a strong urge I had to join my musician friends on stage. Until recently I was forcing my inner musician to be satisfied with just doing event photography, knowing I could never physically pick up an instrument and start playing.

A few years ago, I was lucky to cross paths with Dr Jordan Nguyen at a conference. He was delivering a presentation about his mind-controlled wheelchair. At the time, Jordan was predominantly working with quadriplegics. I wasn't aware that he had never conversed with a non-verbal person with Cerebral Palsy before. He later confessed after our meeting his perspective drastically changed when he realised there was a whole different level of people with disabilities he could work towards assisting.

As time went on Jordan and I became really good friends. Just like me, Jordan has what it takes to break new ground and create new technology that allows others to push forward in their lives. Having very similar goals to Jordan I started becoming involved in his social business Psykinetic where I have a strong sense of belonging. The team at Psykinetic have a common sense of purpose and want to create new technologies purely to empower people in the same physical situation as me.

We went to a few music gigs where Jordan witnessed firsthand how much I wanted to perform like my rock and roll musician friend, Steve Balbi, from Mi-Sex. Every time I watched Steve on the stage, I could hear my inner musician screaming "You need to play music and perform!" However, my logical side just dismissed that idea as a result of my physical limitations. Well... that was until Jordan decided to find some way to get me on stage and enable me to physically play an instrument.

One morning I received a phone call from Jordan asking for my participation on a new project... without hesitation I said "yes". The idea was for me to play classical music at the Sydney Opera House alongside the Australia Piano Quartet, with the first ever eye-controlled instrument from Psykinetic. I hadn't used eye control technology before and knew absolutely nothing about classical music, and I was sitting there thinking how are we going to make this happen, it is classical music at the Opera House?! That is a massive thing to accomplish, however I knew with Jordan's and my determination we just might make this dream happen.

It was very strange because this iconic building was where I had watched some of my favourite musicians play in the past. Then, suddenly, I was on the same stage.

I picked up the software really quickly. However I still needed to be taught how to play classical music on the instrument with a seemingly impossible deadline of four weeks.

Time to bring in the Australia Piano Quartet. James Wannan took the lead as my music teacher. Playing at the Opera House with the piano quartet was a very surreal feeling. It was very strange because this iconic building was where I had watched some of my favourite musicians play in the past. Then, suddenly, I was on the same stage. If a standing ovation is anything to go by, I think we pulled it off.

So what is next for me? Do we have time for me to list everything? I would love to continue my work with the Australia Piano Quartet and grow my music skills, and spread awareness to what is possible with this type of technology from Psykinetic. My desire is to assist in making their technology more available so more people like myself have the opportunity to explore what is actually possible.

Music with the Mind: The Brain-Computer-Music-Interface

Music with the Mind: The Brain-Computer-Music-Interface

als Assistive Technology Brain-Computer Interface (BCI) disability mnd music

http://www.gizmag.com/music-with-the-mind-brain-computer-music-interface/18489/

By Kent Sutherland and Max Sutherland
April 28, 2011

 

The BCMI lets you create music using nothing more than eye movement and brainwaves

 

Imagine a Wii that lets you play a musical instrument with your brain without touching strings or a keyboard. That's exactly what this "proof of concept" brain-computer-music-interface (BCMI) is designed to do – it uses brain waves and eye movement to sound musical notes, so even a person with "locked-in-syndrome" could participate in creative activity analogous to learning to play a musical instrument. Developed by a team headed by Eduardo Miranda, a composer and computer music specialist from the UK's University of Plymouth, the BCMI can be set up on a laptop computer for under $3,500 (including the computer). For people who are disabled, assistive technology usually aims at day-to-day functioning and largely ignores the unique aspect of being a human – creativity. This is different.

The Brain Computer Interface as an assistive technology

"Creativity - like human life itself - begins in darkness."– Julia Cameron
No-one wants to even think about it but imagine a car crash or a stroke left you totally paralyzed and your only active movements were eye movements, facial gestures and minimal head movements. If you still retain full cognitive capacity, you would have what is called locked-in syndrome, a fate some might regard worse than death. For any person with a disability, one of the biggest obstacles is that people simply assume that if your body doesn't work, then your brain is probably not capable of much either. How much worse is this for the person isolated by locked-in syndrome?

Historically, assistive technologies have relied on the person being able to maneuver at least one part of their body. For example, an Augmented Communication Device may require them to press buttons on a keyboard that has pre-designated questions, statements or responses. These devices can be adapted in order for the buttons to be pressed with a finger, a toe, or a metal-pointer attached to their head. Pretty impressive. But what about people with locked-in syndrome who aren't capable of such motor function other than eye movements? Most of the technology has been simply passing them by.
Technology in the form of the brain computer interface (BCI) provides hope for these and many other people because we no longer have to imagine being able to use our thoughts to control a wheelchair or a communication device. In the past decade this technology has moved increasingly from fantasy into a reality.

In 2007, Mike Hanlon wrote in Gizmag about "The first commercially available Brain Computer Interface" and pointed out how work in the area was focused on enabling paralyzed humans to communicate far more freely, but noted the potential to enhance everyone was not that far away. He was right. Within the last five years we have moved from the ability to point with the mind to a thought controlled cursor. And we have moved from driving wheelchairs with brainwaves to driving a car controlled by mind power.

The brain-computer-music-interface

This latest development has thrust the BCI into the world of music and creativity where, in this, its first use, the brain computer musical interface promises to enhance life immensely for those with a most severe disability, locked-in syndrome.
This is the brainchild of a team headed up by Eduardo Miranda, and the Plymouth BCMI Project [PDF]. The system is not yet wireless, but uses a laptop computer, related software, 3 electrodes and an EEG amplifier and can be built for under US$3,500.

Using brainwaves a person can almost immediately produce a full range of musical notes from this device by simply looking intently at one of four icons. These four icons are responsible for sounding pitch, rhythm, and controlling the strength and speed of the notes. Like learning to play a musical instrument, playing music with this device requires skill and learning. As the scientists note, however, this can be an attractive attribute.

With minimal practice in this proof of concept test, the person with locked-in syndrome rapidly demonstrated skill at playing and found it an enjoyable experience.
Check out what such a device can do when output from it is hooked into a piano keyboard. A practiced person has the potential to play masterful music using nothing but his or her brainwaves.

A whole new medium for creativity

Assistive technologies have made life easier for millions of people with disabilities around the globe. We have technology that can help people at home and at work; help them to communicate; help them with mobility. In fact you could say we've got technology for almost everything important to a person's life, right? But until now, these technologies largely ignored the most unique aspect of being a human – creativity.

In the grand scheme of life, you probably wouldn't say that cooking dinner for yourself or getting yourself out of bed in the morning were the things you were most proud of achieving. People want to be unique, innovative, and admired for their talents. Why else would we write books, design cars, or start our own companies? It's in our nature to create. The BCMI promises to give a whole new medium for creativity because it can be used by anyone almost regardless of any physical disability. Inside each one of us is the untapped potential to be the next Beethoven without the agony of studying music theory or learning the piano. All you need is a brain.