-->
Tampilkan postingan dengan label augmentative communication. Tampilkan semua postingan
Tampilkan postingan dengan label augmentative communication. Tampilkan semua postingan

Kamis, 16 Maret 2017

The Entrepreneur with the $100 Million Plan to Link Brains to Computers

The Entrepreneur with the $100 Million Plan to Link Brains to Computers

  • ILLUSTRATION BY KEITH RANKIN
  • Rewriting Life

    Via MIT Tech Review

    https://www.technologyreview.com/s/603771/the-entrepreneur-with-the-100-million-plan-to-link-brains-to-computers/






    Tech big shots are charging into neuroscience, but do they even have a clue?
    Entrepreneur Bryan Johnson says he wanted to become very rich in order to do something great for humankind.
    Last year Johnson, founder of the online payments company Braintree, starting making news when he threw $100 million behind Kernel, a startup he founded to enhance human intelligence by developing brain implants capable of linking people’s thoughts to computers.
    Johnson isn’t alone in believing that “neurotechnology” could be the next big thing. To many in Silicon Valley, the brain looks like an unconquered frontier whose importance dwarfs any achievement made in computing or the Web.
    According to neuroscientists, several figures from the tech sector are currently scouring labs across the U.S. for technology that might fuse human and artificial intelligence. In addition to Johnson, Elon Musk has been teasing a project called “neural lace,” which he said at a 2016 conference will lead to “symbiosis with machines.” And Mark Zuckerberg declared in a 2015 Q&A that people will one day be able to share “full sensory and emotional experiences,” not just photos. Facebook has been hiring neuroscientists for an undisclosed project at Building 8, its secretive hardware division.
    As these people see it, computing keeps achieving new heights, but our ability to interface with silicon is stuck in the keyboard era. Even when speaking to a computer program like Alexa or Siri, you can convey at most about 40 bits per second of information and only for short bursts. Compare that to data transfer records of a trillion bits per second along a fiber-optic cable.
    “Ridiculously slow,” Musk complained.
    But it turns out that connecting to the brain isn’t so easy. Six months after launching Kernel amid a media blitz, Johnson says he’s dropped his initial plans for a “memory implant,” switched scientific advisors, hired a new team, and decided to instead invest in developing a more general-purpose technology for recording and stimulating the brain using electrodes.
    Johnson says the switch-up is part of trying something new. “If you look at the key contributing technologies of society, the ones with the most impact, like rockets, the Internet, biology—there was a transition point from academia to the private sector, and for the most part neuroscience hasn’t made that jump,” says Johnson. “The most critical element is timing, when is the right time to pursue this.”
    Memory implants
    After making a fortune selling Braintree to eBay for $800 million in 2013, Johnson, now 39, reportedly sought the advice of nearly 200 people on how to invest his new wealth. He settled on neurotechnology and, last August, he announced he’d create Kernel and build the first neural prosthetic for human intelligence enhancement.
    But Johnson’s business plan was extremely vague; one scientist called it “metaphysical.” Kernel’s website was plastered with book-jacket-like endorsements from scientific celebrities including J. Craig Venter and Tim O’Reilly, extolling his “great” and “serious” commitment to understanding human intelligence, not to mention the impressive $100 million he later promised to invest in Kernel.
    Bryan Johnson
    The reality is that interfacing with the brain is tough: electronics irritate its tissue and stop working after a while, and no one will get brain surgery just in order to send an e-mail. What’s more, even if you can communicate with the brain, you might not know what it is saying.
    “Billionaires entering the broader neurotechnology field are very optimistic and may overlook details of the problem, which is we are far away from meaningfully understanding the brain,” says Konrad Kording, a Northwestern University neuroscientist who has advised Johnson. “But neurotechnology allows you to work on the most interesting questions in the universe while potentially making money, and so that is exciting.”
    Johnson’s persona is part buttoned-down Mormon missionary (he once was one), part hard-driving door-to-door credit-processing salesman (he was that too), but now, with his new wealth, he’s also taken on the mantle of a technology prophet. At a 2016 startup conference in Silicon Valley, he showed up with his hair unbrushed, wearing a T-shirt with holes in it, and gave a wide-ranging lecture on human tool use from prehistory into the present, arguing that now “our very existence is programmable” through biology and machine interfaces.
    Kernel’s original technology was a memory prosthesis, developed by Theodore Berger of the University of Southern California, who until recently was also the company’s chief scientific officer. Berger’s technology (see “10 Breakthrough Technologies: Memory Implants”) is a way of recording memories of rats and monkeys, storing these patterns on a computer chip, and re-delivering them to the hippocampus. One version of the setup, Berger says, has been tested in a handful of human patients undergoing brain surgery for other reasons.
    But a mere six months after starting Kernel, Berger is no longer part of the company, and memory implants are no longer part of Kernel’s near-term plans. Johnson and Berger both confirmed the separation.
    Berger’s vision, according to several people, was too complex, too speculative, and too far from becoming a medical reality, while Johnson hoped to see a return on his investment sometime soon. “They have a new direction, but we’re still talking,” says Berger. “The basic reason is it was going to take too long. It’s one thing to think about this and quite another to do it.”
    Johnson says he concluded that Berger’s work “is really interesting, but not an entry point” into a commercially viable business.
    Brain interface
    By last November, Johnson was already exploring a pivot for his company, meeting with Christian Wentz, head of a small Cambridge startup, Kendall Research Systems, that sells equipment for recording in the neurons of mice and other animals. The company spun out of the laboratory of Edward Boyden, a professor at MIT who invents new ways of analyzing brain tissue.  
    In February, Johnson acquired Wentz’s company (for an undisclosed sum) and with it brought in a new team, including Wentz and Adam Marblestone, a noted theorist of both the limitations and possibilities of brain interfaces, who will become chief scientific officer. Both are former Boyden lab members, as are two other Kernel scientists, Caroline Moore-Kochlacs and Jake Bernstein.

    Johnson says Kernel will now develop a “generalized human electrophysiology platform”—that is, a flexible way of measuring the electrical impulses from many neurons at once, and stimulating them, too. The eventual objective is to use such electronics to treat major diseases, like depression or Alzheimer’s. “It’s for clinical use,” he says. “We are a for-profit company.”  
    Wentz says as part of the acquisition he and Johnson agreed that much more R&D on brain interfaces will probably be needed. “We have a very sober view of what can and can’t be done,” Wentz says. “We are not naïve.” He calls Kernel’s effort a “15-year endeavor,” although he adds that “we want to do in that period what has been done in the last 100 years.”
    With the pivot, Johnson is effectively jumping on an opportunity created by the Brain Initiative, an Obama-era project which plowed money into new schemes for recording neurons. That influx of cash has spurred the formation of several other startups, including Paradromics and Cortera, also developing novel hardware for collecting brain signals. As part of the government brain project, the defense R&D agency DARPA says it is close to announcing $60 million in contracts under a program to create a “high-fidelity” brain interface able to simultaneously record from one million neurons (the current record is about 200) and stimulate 100,000 at a time.
    “It’s time for neuroscience to graduate from academia to a general neuroscience platform,” says Johnson. With such a technology “a whole range of new applications—a lot of white space—would open up.”
    Johnson declined to describe the specifics of Kernel’s technological approach to connecting with the brain, as did Boyden and Wentz. However, the team members have been working on well-identified problems. Wentz has been involved with developing electronics for high-speed reading of data emitted by wireless implants. Already, the flow of information that can be collected from a mouse’s brain in real time outruns what a laptop computer can handle. The team also needs a way to interface with the human brain. Boyden’s lab has worked on several concepts to do so, including needle-shaped probes with tiny electrodes etched onto their surface. Another idea is to record neural activity by threading tiny optical fibers through the brain’s capillaries, an idea roughly similar to Musk’s neural lace.

    More sophisticated means of reading and writing to the brain are seen as potential ways to treat psychiatric disorders. Under a concept that Boyden calls “brain coprocessors,” it may be possible to create closed-loop systems that detect certain brain signals—say, those associated with depression—and shock the brain to reverse them. Some surgeons and doctors funded by another DARPA program are in the early stages of determining whether serious mental conditions can be treated in this way (see “A Shocking Way to Fix the Brain”).
    Boyden says Johnson’s $100 million makes a big difference to how he and his students view the entrepreneur’s goals. “A lot of neurotechnology has come and gone. But one thing is that it’s very expensive,” he says. “The inventing is expensive, the clinical work is expensive. It’s not easy. And here is someone putting money into the game.”

    Selasa, 14 Maret 2017

    Penn State Start-up giving voice to the voiceless in Centre County (PA)

    Penn State Start-up giving voice to the voiceless in Centre County (PA)

    PHOTO BY ALS Association and Prize4Life Mary Elizabeth McCulloch, founder and CEO of Project Vive, and Trip Miller, CFO, demonstrate how their product, the Voz Box, works. The Voz Box uses small movements from the knee, wrist or finger of the user to construct full sentences.

    PHOTO BY ALS Association and Prize4Life Mary Elizabeth McCulloch, founder and CEO of Project Vive, and Trip Miller, CFO, demonstrate how their product, the Voz Box, works. The Voz Box uses small movements from the knee, wrist or finger of the user to construct full sentences.
    BELLEFONTE — Those who cannot speak can have their voice heard through biomedical speech devices, and one project in Centre County is making the option more affordable.
    Project Vive, pronounced “vee-vay,” which is Spanish for “to live,” is one of the first startup businesses to graduate from Happy Valley LaunchBox and was created by a recent Penn State and Happy Valley LaunchBox graduate, Mary Elizabeth McCulloch, of Milesburg.
    “Our mission is to make communication devices accessible to those who need them,”McCulloch said when she gave a presentation on her project at a recent meeting of the Centre County Board of Commissioners. “We’re really interested in creating something that’s low cost, that you can actually buy without going through insurance claims.”
    McCulloch is a 2016 Penn State biomedical engineering graduate and was in the first class of the Happy Valley LaunchBox startup business incubator located in downtown State College. She said at the Feb. 28 meeting that thanks to a crowdsourcing campaign, enough money — $13,600 to be exact — has been raised to fund 13 devices to help people overcome non-verbal communication barriers.
    These communication barriers are often a result of cerebral palsy or other related disabilities. More than 3.4 million people in the world have a form of cerebral palsy that causes complex communication needs, McCulloch said.
    After McCulloch graduated from Bald Eagle Area High School in 2010, she said she decided to do an exchange year. For that exchange year, she went to Ecuador and worked in a special needs orphanage with children and adults who had cerebral palsy, and could not speak.
    “There was no assistive technology, no devices,” McCulloch said.” We had some devices that were donated from the United States through different organizations, [but] they came with no instructions of how they worked; what happens when they break. I had a really sad story with a little number clicker device that if you typed the 1, it would say “hello,” and within a month of a boy using it — and he started to get really good at it — it broke.”
    That experience led her on the path of biomedical engineering, and it was during her time at Penn State that she started working on the idea of a low-cost speech-generating device.
    One thing makes Project Vive’s assistive technologies stand out from the competition is that they are much easier to use, McCulloch said. Some assistive technologies today use touch screens or keyboards with large icons, which makes them difficult to use for people who have limited control of their muscles, and they can require a lot of training to use. 
    All of the devices use movement to speech technology that uses a customizable menu-based system so that words and phrases can be pre-programmed, McCulloch said. The one device that Project Vive has created, the Voz Box, is connected to a glove that the user wears and is controlled by small movements from the knee, wrist or finger to construct full sentences, versus a keyboard, and uses a Bluetooth-linked speaker to amplify words. Other devices offered by Project Vive include eyeglasses for eye control, which allows the user to blink to convey what they want to communicate, a wearable elbow sensor that acts as a selection method for what the user wants to communicate when they raise their elbow, and a knee movement sensor that works when the user kicks or presses with their foot.
    The cost is another big factor in what sets Project Vive’s assistive technologies apart. Other devices, such as ones that use LCD screens, can cost in the ballpark of $5,000, and eye-tracking devices can cost around $15,000, McCulloch said. The products Project Vive will offer will cost under $500.
    According to Commissioner Mark Higgins, Project Vive is the type of startup business that the county would like to see flourish in one of its business incubators, such as the upcoming Bellefonte SpringBoard incubator. 
    Although the products are not on the market yet, McCulloch said she hopes they will be in six months.
    For more information about Project Vive, visit www.projectvive.com.

    Senin, 13 Maret 2017

    'Voice banks' step in to keep chronically ill patients from falling silent

    'Voice banks' step in to keep chronically ill patients from falling silent


    For thousands of Minnesotans with neurodegenerative diseases, losing one's ability to speak is one of the most devastating consequences.

    Every so often, Robert White breaks into song and serenades his 13-year-old son Kieran with a tender Irish lullaby, “That Little Boy of Mine,” taught to him by his father.
    But the deep-throated voice that used to fill his family’s living room in West St. Paul now quivers and shakes. There are times when White, who has amyotrophic lateral sclerosis, or ALS, a terminal disease that impairs his motor skills, can barely finish the chorus before his voice dips to a hoarsened whisper.
    For White, and thousands of other Minnesotans afflicted with neurodegenerative diseases, losing one’s ability to speak is one of the most devastating consequences.
    Now, however, researchers have found a way to preserve the unique essence of a human voice — in all its idiosyncratic nuance and power — for people with incurable and often debilitating illnesses.
    Using new voice database technology, University of Minnesota speech pathologists can record people saying hundreds of sentences and phrases, break them down into phonetic units, and then reconstruct a personalized voice that can be used on a speech-generating device. The end result of this process, known as “voice banking,” is a voice that is nearly identical to the person’s original, healthy voice.
    While the technology is still new, many people feel they are in a race against time to store their original voices before they become unrecognizable to their families and friends. Even the simple act of saying “I love you” can be too much of a strain for people in the later stages of ALS or other neurodegenerative illnesses.
    “This is about preserving a person’s essential dignity, rather than having to depend on a canned synthetic voice that many find dehumanizing,” said Dr. H. Timothy Bunnell, director of the speech-language laboratory in Wilmington, Del., that pioneered the voice-banking technology about a decade ago.
    But the process of preserving a voice is an emotional one for families. It comes with the recognition that a loved one is dying, and that even a carefully reconstructed voice — built over a period of weeks or months — will never be an exact match to a person’s original speech. Even the most sophisticated audio technology cannot recapture the spontaneous emotion of someone who bursts into laughter, or the gentle timbre of a parent’s voice as they sing to a child, researchers say.
    ‘It’s bittersweet’
    On a recent morning, Wendy Eickhoff, 49, struggled to contain her emotions as she began the arduous process of recording more than 1,600 preselected sentences at a speech laboratory tucked in a brick building on the University of Minnesota’s East Bank campus.
    Last October, Eickhoff was diagnosed with a rare form of ALS known as bulbar onset, which first attacks the muscles that move the tongue, mouth and vocal cords. It was a particularly brutal diagnosis for Eickhoff, a self-described “talker” who works as a technology relationship manager at Wells Fargo. Slurred speech and weakened tongue muscles were among her first symptoms.
    “It was devastating because I have always relished talking,” she said.
    As Eickhoff rattled off sentences in a soundproof room, including lines from the “The Velveteen Rabbit” and “The Wizard of Oz,” her daughter Logan, 21, watched intensely through a window in an adjoining room. With a tinge of sadness, Logan expressed hope that voice banking would preserve the unique character of her mother’s speech, from her high-pitched laugh to the gentle way she called out “Good morning sunshine!” to her daughter each morning.
    At one point, Logan broke down in tears after a speech pathologist calmly informed her that her mother’s recorded voice would survive her death and would always be available to the family.
    “It’s difficult, really difficult,” said Logan, who is pursuing a degree in linguistics. “I don’t want to have to acknowledge that in the potentially near future that my mom won’t be able to speak. But I also feel lucky, because this means that even after she’s gone, I’ll be able to hear her voice. It’s bittersweet.”
    Those who have been through the process speak of it in glowing terms.
    When Robert White finished recording the last of 1,610 sentences last month, after five grueling sessions, he threw up his hands in victory and handed out assorted chocolates to the speech-language pathologists and young students who helped him through the process.
    “There is a real sense of accomplishment when you get to the end,” said his wife, Argerie White. “I mean, how wonderful would it be, if Bob ever loses his voice, that we can still hear him sing?”
    Sophisticated technology
    Until recently, people like the Whites and the Eickhoffs relied on interactive whiteboards or robotic synthesizers, similar to that used by renowned physicist Stephen Hawking, to communicate with the world after illnesses had taken their speech. But voice-banking technology has become so sophisticated that it can now cull through thousands of phonetic syllables and sort them in a way that replicates the pattern of a person’s speech. The recording process emphasizes the repetition of vowels, because the medley of “ohs” and “ahs” are largely what make a person’s voice distinctive, researchers say.
    While the technology is still being tested and enhanced, that has not dissuaded the regional chapter of the ALS Association from encouraging people with the disease to take advantage of the technology while their voices are still strong. The voice-banking service is now available at university speech clinics throughout the Upper Midwest, from Fargo to Duluth. The cost of creating a voice can run from a few hundred dollars to more than $1,000, depending on the provider. In Minnesota and the Dakotas, the cost for those with ALS is covered by the regional ALS Association.
    “It’s crazy-amazing how well this works,” said Rebecca Lulai, a clinical supervisor and speech-language pathologist at the University of Minnesota.
    Pete Klinkhammer, who remembers the exact minute he was diagnosed with ALS (11:47 a.m. on June 14, 2013), was among the first to test voice-banking in Minnesota and is now an outspoken proponent. Klinkhammer, 54, a gregarious former social worker from Albertville and bulky ex-college football player, has a bellowing laugh and a penchant for off-color jokes and puns. The technology enables him to keep speaking through an iPad, often late in the evening, when his throat muscles cramp and his voice starts to slur.
    During a recent visit with his 28-year-old daughter Chelsea, Klinkhammer pulled out his iPad and began tapping out a few of his favorite puns as his daughter watched with amusement.
    “A guy just threw milk at me. How dairy!” he wrote, throwing his head back in laughter. The words “I love you, Chelsea,” came out of a small speaker attached to his iPad with virtually the same inflection as his regular voice.
    “That is definitely your voice, Dad,” gushed Chelsea, hugging him.
    “This illness can come to define you because it takes away so many outward physical aspects of your being,” Klinkhammer said. “But voice banking epitomizes the idea that this illness can’t take everything away, because it can’t take away your voice.”
     Via http://www.startribune.com/new-technology-helps-minnesotans-with-neurological-disorders-preserve-their-voices/415954614/

    Rabu, 01 Maret 2017

    I Played The First-ever Eye-controlled Instrument At The Sydney Opera House


    As a result of a complicated birth I have severe cerebral palsy. My condition affects my fine motor skills and causes my body to endure involuntary muscle spasms. Despite my physical disability I have always been an outgoing and creative person. Throughout my life I have been known to overcome barriers and push myself to achieve the next step at every turn. This is a tough gig sometimes, however when you don't fit into the so-called box, you strive to become a game changer. My past is full of critics who grossly underestimated my capabilities, but I just convinced myself to keep pushing forward and change their perception and help spread awareness.
    For me, being independent in my business life means so much.
    I can remember how difficult it was progressing through school not knowing what the future held due to having a severe physical disability. Today while illustrating what is possible to younger generations I hope to build their confidence and allow them to challenge their own barriers.
    Through the use of technology, I am able to be fully independent in my work environment, from using a Macbook for my graphic design and website development, to securing my SLR digital camera to my wheelchair. For me, being independent in my business life means so much.
    I have always been surrounded by music. I found that by immersing myself in beats and rhythms decreased my physical pain and lessened the severity of my spasms. One of the reasons I took up music photography was to satisfy a strong urge I had to join my musician friends on stage. Until recently I was forcing my inner musician to be satisfied with just doing event photography, knowing I could never physically pick up an instrument and start playing.

    A few years ago, I was lucky to cross paths with Dr Jordan Nguyen at a conference. He was delivering a presentation about his mind-controlled wheelchair. At the time, Jordan was predominantly working with quadriplegics. I wasn't aware that he had never conversed with a non-verbal person with Cerebral Palsy before. He later confessed after our meeting his perspective drastically changed when he realised there was a whole different level of people with disabilities he could work towards assisting.
    As time went on Jordan and I became really good friends. Just like me, Jordan has what it takes to break new ground and create new technology that allows others to push forward in their lives. Having very similar goals to Jordan I started becoming involved in his social business Psykinetic where I have a strong sense of belonging. The team at Psykinetic have a common sense of purpose and want to create new technologies purely to empower people in the same physical situation as me.
    We went to a few music gigs where Jordan witnessed firsthand how much I wanted to perform like my rock and roll musician friend, Steve Balbi, from Mi-Sex. Every time I watched Steve on the stage, I could hear my inner musician screaming "You need to play music and perform!" However, my logical side just dismissed that idea as a result of my physical limitations. Well... that was until Jordan decided to find some way to get me on stage and enable me to physically play an instrument.
    One morning I received a phone call from Jordan asking for my participation on a new project... without hesitation I said "yes". The idea was for me to play classical music at the Sydney Opera House alongside the Australia Piano Quartet, with the first ever eye-controlled instrument from Psykinetic. I hadn't used eye control technology before and knew absolutely nothing about classical music, and I was sitting there thinking how are we going to make this happen, it is classical music at the Opera House?! That is a massive thing to accomplish, however I knew with Jordan's and my determination we just might make this dream happen.
    It was very strange because this iconic building was where I had watched some of my favourite musicians play in the past. Then, suddenly, I was on the same stage.
    I picked up the software really quickly. However I still needed to be taught how to play classical music on the instrument with a seemingly impossible deadline of four weeks.
    Time to bring in the Australia Piano Quartet. James Wannan took the lead as my music teacher. Playing at the Opera House with the piano quartet was a very surreal feeling. It was very strange because this iconic building was where I had watched some of my favourite musicians play in the past. Then, suddenly, I was on the same stage. If a standing ovation is anything to go by, I think we pulled it off.
    So what is next for me? Do we have time for me to list everything? I would love to continue my work with the Australia Piano Quartet and grow my music skills, and spread awareness to what is possible with this type of technology from Psykinetic. My desire is to assist in making their technology more available so more people like myself have the opportunity to explore what is actually possible.
    Unlocking the potential of eye tracking technology

    Unlocking the potential of eye tracking technology

    Via Tech Crunch https://techcrunch.com/2017/02/19/unlocking-the-potential-of-eye-tracking

    The concept of measuring and responding to human eye motion, or eye tracking, isn’t new, but the past year saw a rising interest in the technology. There have been a slew of acquisitions of eye tracking startups by large firms and the rollout of several devices and software that support eye tracking.
    “Eye tracking sensors provide two main benefits,” says Oscar Werner, vice president of the eye tracking company Tobii Tech. “First, it makes a device aware of what the user is interested in at any given point in time. And second, it provides an additional way to interact with content, without taking anything else away. That means it increases the communication bandwidth between the user and the device.”
    There’s a chance that soon eye tracking will be a standard feature of a new generation of smartphones, laptops and desktop monitors setting the stage for a huge reëvaluation of the way we communicate with devices—or how they communicate with us.
    “In the past year eye tracking technology moved from being a promising technology to being adopted in commercial products in a wide array of consumer segments simultaneously,” Werner says.
    Dominic Porco, chief executive officer at Impax Media, a digital advertising company, says less expensive and more potent hardware; new open source software platforms; and new easier and faster ways of obtaining data to train algorithm models have driven the progress in eye tracking technology.
    “Companies like NVIDIA have launched products with more powerful GPUs at competitive prices, accelerating the image recognition speeds,” Porco says.
    Porco adds that popular crowd-sourcing marketplaces such as Amazon Mechanical Turk have enabled the collection of larger and broader datasets to train recognition algorithms. “These developments have accelerated progress in eye tracking technology significantly, allowing researchers and developers to go faster through their cycles of experimentation and implementation.”
    But any technology won’t grow unless it can fulfill specific demands and use cases. And in the case of eye tracking, there seems to be no shortage.

    Virtual Reality

    Businessman in virtual computer room.
    In a push to create a more immersive experience, VR headset companies are making large investments in eye tracking technology. In fact, in many ways, eye tracking is seen as the technology complementing VR.
    “VR is about immersion,” Tobii’s Werner says. “But a VR headset without eye tracking will assume that I am speaking to the person in front of my forehead. It is approximating my area of interest to the direction of my forehead. We all know this is not true. Our real interest is where I am looking, and there is often a difference between where I look and the direction of my head. VR headsets need to take your gaze into account be become truly immersive.”
    Eye tracking technology is key to foveated rendering, a technique where only the portion of the image that lands on the fovea (the part of the retina that can see significant detail) is rendered in full quality.
    With foveated rendering, Werner says, there will be a 30 to 70 percent decrease in the amount of pixels drawn, a processing power saving that can translate to higher framerates and the ability to achieve high quality output with 4k headsets as opposed to the 24k levelneeded to meet natural human vision level.
    Also, eye tracking technology will make it possible to reduce graphics distortion caused from not taking eye position account when rendering VR graphics, Werner adds.
    Fove, a kickstarter-funded project, is the first VR headset to have embedded eye-tracking. Others are not far behind. In the past months, Google and Facebook acquired eye tracking startups Eyefluence and Eye Tribe respectively, and are expected to embed the technology in their future products.
    And SMI, a leader in eye tracking technology, has initiated several partnerships and projects to bring eye tracking to both standalone VR head-mounted displays and smartphone slot-ins.
    Eye tracking will also be part of the upcoming Khronos VR API, an open standard under development which has garnered the support of Oculus, Google, NVIDIA and others.
    “The view that eye tracking will be a key part of second generation headsets is shared by a large number of VR HMD vendors,” Werner says. “This drives technology development and innovation.”

    PC Gaming

    Another successful VR demo, at HTC, that showed room scale gaming that actually worked.
    Another successful VR demo, at HTC, that showed room scale gaming that actually worked.
    For decades, we’ve used gamepads, joysticks, keyboards, mice and other peripherals to make PCs and video game consoles understand where we’re looking at. With eye tracking, your computer already knows what you’re looking at and can react accordingly.
    “When you want to interact with an object you just look at it and press a button,” says Werner. “The computer understands which object you want to interact with. You don’t need to drag the mouse or controller to the place you are already looking.”
    Whether it’s about hacking at an object, aiming at a target, designating a location for the game character to run, or simply changing the direction of the point-of-view camera, eye tracking might make it a whole lot easier for gamers to interact with the gaming environment. This can mean a big deal for games that require a high level of mouse and controller handling.
    While it might render previously challenging games too easy, it can also pave the way for games that are much more fast paced.
    Eye tracking can also create a cleaner and less intrusive user interface.
    “In games, graphical artists spend a lot of time creating beautiful environments,” Werner says, “and are in a constant battle with UI interaction designers that need to place UI elements on top, since it clutters the immersive feeling.”
    With eye tracking, Werner explains, you can hide the UI or make it transparent and only make it visible when the gamer’s gaze is directed toward it. “This creates a more immersive feeling and solves a constant battle between graphical artists and UI designers,” he says.
    For simulations and virtual worlds, eye tracking can enable gaze aware objects, where game objects or characters will react to the gaze of the player to make the simulation more realistic, Werner says. This means that you’ll have to be careful not to stare too long at a mercenary’s purse when entering a tavern in your favorite RPG.
    Tobii itself offers a line of tack-on eye-tracking devices and eye-tracking embedded laptopsand has worked with gaming companies on eye-tracking-enhanced versions of popular games such as Rise of the Tomb Raider, Deus Ex and Watch Dogs 2.
    It is unlikely that eye tracking will replace controllers any time soon, but according to Werner, thanks to the technology, “PC games will take into account one of the most powerful interaction methods we as humans have, our eyes. PC gamers will be able to utilize an additional control mechanism complementing the mouse and the controller. This will drive more natural and interaction without taking anything away.”

    Medicine and accessibility

    Medicine doctor hand working with modern computer interface as medical concept
    Beyond the consumer level, the benefits of eye tracking expand to other realms where measurements of human gaze are key to obtaining results and insights.
    “There is an increasing interest in using eye tracking to help diagnose — and potentially treat –neurological disorders,” says Bryn Farnsworth, science editor at biometric research company iMotions. “For example, infants usually like to look at images with people’s faces—scenes that have a social element.”
    Farnsworth explains that infants that go on to develop autism are much more likely to have a preference for images that feature geometric shapes, while for children with the Williams syndrome, Farnsworth says, the situation is reversed, meaning that they show a marked preference for social scenes in comparison to neurotypical children.
    This all suggests, Farnsworth says, “that the analysis of eye movements may help guide early diagnosis.”
    research paper by students at UCSD states that eye tracking technology holds promise as an objective methodology for characterizing the early features of autism, because it can be implemented with virtually any age or functioning level.
    Labs such as iMotions are helping researchers obtain those metrics through eye tracking devices in order to better understand and assess the conditions of patients.
    Eye tracking company RightEye uses the technology to help physicians in administering test and finding symptoms for illnesses ranging from simple concussions to Alzheimer’s disease and dyslexia, and to help treat children with autism.
    Eye tracking can also be a breakthrough for patients with physical disabilities, especially as affordable, consumer-level devices become available. “This is a large area for eye tracking and it is evolving,” says Tobii’s Werner. Gaze keyboards and control panels powered by eye tracking, Werner explains, give people with diseases such as cerebral palsy and spinal cord injuries a means to communicate, control their environment and develop skills through therapy.

    Advertising

    facebook-live-ads-gif
    In the current state, the best metric advertisers get from ads are impressions and click numbers. But those numbers do not precisely reflect the effectiveness of ad campaigns because a lot of what gets counted as impressions goes to waste on non-human sources. That is something that changes with eye tracking technology.
    “The advertising industry is currently in the midst of some major upheaval when it comes to universal standards for measuring ad impact,” says Porco, the Impax chief executive. “The whole concept of ‘viewability’ is now being redefined to make more sense in the age of ad blockers and bot traffic.”
    With eye tracking technology, online advertisers will be able to measure exactly how many actual human eyes actually view their ads when they appear on the page. While gaining precise metrics would be nearly impossible until such time as every computer and mobile device is embedded with eye tracking technology, using eye tracking does give insights into how users interact with ads.
    In the physical world, however, eye tracking is already showing promise.
    “Market research firms are experimenting with directly measured biometric data to precisely determine the composition of people in out-of-home media environments such as retail stores, for audience measurement purposes,” Porco says.
    Porco’s company, Impax Media, is investing heavily in eye tracking technology along with other computer vision techniques to collect attention metrics from its proprietary in-store advertising screens. “We’re big believers that the future of the ad industry is going to be grounded in attention metrics, as opposed to impressions, and eye tracking is, hands down, the best way to track attention,” Porco says.
    The data, Porco says, helps advertisers and location partners to assess audience interest in various messaging angles, and to correlate this information with parameters like location, timing and demographics. “It’s great for media buyers seeking to get the most for their budgets, and for store managers dealing with questions about everything from inventory to staff shift schedules.”
    While retailers always benefit from collecting information about customers, the area is somewhat a gray and is often subject to controversy and falls across privacy regulations. However, Porco underlines that there’s no need to collect identity parameter in order to glean useful insights, and anonymized data about gaze point, age and gender along with duration of view suffice.

    Market research

    research-kit
    It’s important for market researchers to “evaluate people’s interactions and expectations across the whole omnichannel customer journey and its key touchpoints,” says Simone Benedetto, UX researcher at TSW, an Italy-based market research lab.
    Recent advances in eye tracking, Benedetto explains, open up new possibilities in both lab and real world neuromarketing tests.
    “It is crucial for us to involve users in product and service design and evaluation,” Benedetto says. “This doesn’t mean just to ask them their opinion, but to collect objective data coming from their eyes and brain while interacting with the product or service.”
    TSW uses mobile eye tracking units along with other wearables in order to get precise user and customer metrics on a wide variety of products and services, both digital (such as online ads, mobile apps, websites, software and device control panels) and physical (such as print material, product packages, cars, home furniture and retail stores).
    Being able to weigh the user’s natural interaction with products and services enables researchers to identify real usability problems and frustration points and collect actionable information that gives insights into customer satisfaction and engagement and drive design decisions.
    “One of the great advances last year has been the introduction of object tracking in relation to the analysis of data from mobile eye trackers,” says iMotions’ Farnsworth, referring to the process by which specific visual features can be delineated from a scene, and information about how that particular feature is attended to can then be recorded.
    “This means that an individual can, for example, wear portable eye tracking glasses, interact with their environment normally, and how they attend to certain features can be automatically analyzed —how long they look at a street map when they’re out walking, if they notice an advert that they passed,” Farnsworth says. “Being able to automatically understand how specific features are attended to clearly has great ramifications for understanding humans, and using that knowledge further.”
    “From my perspective there’s a huge market behind the exploitation of eye tracking into UX-neuromarketing investigations,” Benedetto says. “Eye tracking allows the implicit measurement of user behavior, and turns that measurement into quantitative objective data. We have only relied on subjective data for years, and it’s definitely time for a change.”

    The future of eye tracking

    Close Up of blue eye with computer circuit board lines, digital composite
    Close Up of blue eye with computer circuit board lines, digital composite
    Tobii’s Werner told me he believes a new paradigm of PC usage will emerge, where eye tracking is a fifth modality that, in combination with touch screens, mouse/touchpad, voice and keyboard, will make computers much more productive and intuitive. “Gaze always precedes any kind of action that you do with mouse, keyboard and voice, so much smarter user interactions will be designed using these technologies,” he says.
    As vision is the most used sense among human beings, being able to track and measure it digitally will have a great impact on how we make our intentions known to computers, wittingly or unwittingly.