Digital Synesthesia

Want to see with your tongue? Boing Boing's David Pescovitz looks at technology that blurs the boundaries between our five senses. What if you...

Want to see with your tongue? Boing Boing's David Pescovitz looks at technology that blurs the boundaries between our five senses.

What if you could see with your skin? Or taste what you see? While those kinds of experiences might suggest a mental disorder, or an acid trip, the ability to substitute your senses by choice is on the horizon. A confluence of new technologies are leading to a kind of digital synesthesia.

Synesthesia, of course, is the fascinating neurological phenomenon whereby stimulation of one sense involuntarily triggers another sensory pathway. A synesthete might taste sounds or hear colors. But the ability to reroute the senses could dramatically help blind individuals, for example, or restore the sense of touch to amputees wearing prosthetic limbs.

At Institute for the Future, where I'm a researcher, my colleagues and I have spent the last few months exploring the notion that "everything is programmable," or will be soon. The idea is that emerging technologies-from pervasive computers to synthetic biology-are making it possible to program our bodies and our worlds to desired specifications. Increasingly, we are looking at the entire world through a computational lens. As part of that research, we've been collecting "signals"-events, developments, articles, scientific publications-that taken together, give indications of key trends. We've entered these in our public Signtific signals database and tagged them based on their subject matter. I've found many research efforts suggesting how we may reprogram our senses in the future.

For example, there are the "Flavor Tripping" parties fueled by Synsepalum dulcificum, aka "Miracle Fuit," the West African berry that temporarily reprograms your taste buds to make anything sour or bitter taste perfectly sweet. And there's the story of Daniel Kish, the blind psychologist who, by clicking his tongue, uses echolocation to "see." In the realm of digital synesthesia, numerous projects are attempting to leverage tactile feedback in the form of clothing outfitted with tiny vibrators. Instead of picking up your phone to read a text message, you might feel the words spelled on your back.

The late Paul Bach-y-Rita could be considered the father of all technology used to reprogram the human senses. In 1963, Bach-y-Rita developed a "Tactile to Visual Sensory Substitution" device. It converted images from a camera to tactile sensations that a blind person could feel on his or her back. Bach-y-Rita's research was all based on the notion of "sensory substitution." The brain, he argued, was not hardwired and that a working sense, say touch, could be used to replace a failing one, e.g. vision. His ideas around the plasticity of the human brain were very controversial at the time but widely accepted today. He continued his research on sensory substitution technology at the University of Wisconsin-Madison, and his company Wicab, until his death in 2007.

"I can connect anything to anything," Bach-y-Rita said in a profile in The Telegraph shortly before his death:

We see with our brains, not with our eyes. When a blind man uses a cane he sweeps it back and forth, and has only one point, the tip, feeding him information through the skin receptors in the hand. Yet this sweeping allows him to sort out where the doorjamb is, or the chair, or distinguish a foot when he hits it, because it will give a little. Then he uses this information to guide himself to the chair to sit down. Though his hand sensors are where he gets the information and where the cane "interfaces" with him, what he perceives is not the cane's pressure on his hand but the layout of the room: chairs, walls, feet, the three-dimensional space. The receptor surface in the hand becomes merely a relay for information, a data port.

In the latest incarnation of Bach-y-Rita's work, the data port is the tongue. The company he co-founded, Wicab, has developed a visual prosthetic for the blind that converts images from a video camera into tactile sensations on the tongue. The system, called BrainPort, pairs a head-mounted digital video camera with a postage stamp-size electrode array that sits on the tongue. A small computer translates the visual information into a pattern that is "displayed" on the tongue.

"The tactile image is created by presenting white pixels from the camera as strong stimulation, black pixels as no stimulation, and gray levels as medium levels of stimulation, with the ability to invert contrast when appropriate," reads the company's website. "Users often report the sensation as pictures that are painted on the tongue with champagne bubbles."

The BrainPort is not yet FDA approved, but clinical studies have been quite exciting. During trials, blind test subjects had their brains scanned while using the device. Interestingly, even though the device provides tactile sensation, visual regions of the brain were activated.

Seeing with your tongue may seem unusual, but arguably not as weird as "skin vision." A researcher at Tel Aviv University suggests that humans might be able to "see" with their skin. Engineering professor Leonid Yaroslavsky hopes that through biomimicry, new kinds of imaging technology might be developed that obviates traditional optics. Yaroslavsky presented his theories on the subject in a scientific book titled Advances in Information Optics and Photonics. From an American Friends of Tel Aviv University press release:

Skin vision is not uncommon in nature. Plants orient themselves to light, and some animals-such as pit vipers, who use infrared vision, and reptiles, who possess skin sensors-can "see" without the use of eyes. Skin vision in humans is likely a natural atavistic ability involving light-sensitive cells in our skin connected to neuro-machinery in the body and in the brain, explains Prof. Yaroslavsky.

While the first people to reprogram their senses are likely to be people with a sense that has failed them, the technology will likely trickle down. Eventually, the hard lines between our five senses may be blurred. And in a world where everything is programmable, five may be a choice, not a limit.

Do you see what I'm saying?

David Pescovitz is co-editor of Boing Boing, a research director at Institute for the Future, and editor-at-large of MAKE.

via Chela Horsdal / Twitter

Amazon's "The Man in the High Castle" debuted the first episode of its final season last week.

The show is loosely based on an alternative history novel by Philip K. Dick that postulates what would happen if Nazi Germany and the Empire of Japan controlled the United States after being victorious in World War II.

Keep Reading Show less
via Mike Mozart / Flickr

Chick-fil-A is the third-largest fast food chain in America, behind McDonald's and Starbucks, raking in over $10 billion a year.

But for years, the company has faced boycotts for supporting anti-LGBT charities, including the Salvation Army, the Fellowship of Christian Athletes, and the Paul Anderson Youth Home.

The Salvation Army faced criticism after a leader in the organization implied that gay people "deserve to die" and the company also came under fire after refusing to offer same-sex couples health insurance. But the organization swears it's evolving on such issues.

via Thomas Hawk / Flickr

The Fellowship of Christian Athletes explicitly announced it was anti gay marriage in a recent "Statement of Faith."

God instituted marriage between one man and one woman as the foundation of the family and the basic structure of human society. For this reason, we believe that marriage is exclusively the union of one man and one woman.

The Paul Anderson Youth Home teaches boys that homosexuality is wrong and that same-sex marriage is "rage against Jesus Christ and His values."

RELATED: The 1975's singer bravely kissed a man at a Dubai concert to protest anti-LGBT oppression

In 2012, Chick-fil-A's CEO, Dan Cathy, made anti same-sex marriage comments on a radio broadcast:

I think we are inviting God's judgment on our nation when we shake our fist at Him and say, "We know better than you as to what constitutes a marriage". I pray God's mercy on our generation that has such a prideful, arrogant attitude to think that we have the audacity to define what marriage is about.

But the chicken giant has now decided to change it's says its charitable donation strategy because it's bad for business...Not because being homophobic is wrong.

The company recently lost several bids to provide concessions in U.S. airports. A pop-up shop in England was told it would not be renewed after eight days following LGBTQ protests.

Chick-fil-A also has plans to expand to Boston, Massachusetts where its mayor, Thomas Menino, pledged to ban the restaurant from the city.

via Wikimedia Commons

"There's no question we know that, as we go into new markets, we need to be clear about who we are," Chick-fil-A President and Chief Operating Officer Tim Tassopoulos told Bisnow. "There are lots of articles and newscasts about Chick-fil-A, and we thought we needed to be clear about our message."

RELATED: Alan Turing will appear on the 50-pound note nearly 70 years after being persecuted for his sexuality

Instead, the Chick-fil-A Foundation plans to give $9 million to organizations that support education and fight homelessness. Which is commendable regardless of the company's troubled past.

"If Chick-Fil-A is serious about their pledge to stop holding hands with divisive anti-LGBTQ activists, then further transparency is needed regarding their deep ties to organizations like Focus on the Family, which exist purely to harm LGBTQ people and families," Drew Anderson, GLAAD's director of campaigns and rapid response, said in a statement.

Chick-fil-A's decision to back down from contributing to anti-LGBT charities shows the power that people have to fight back against companies by hitting them where it really hurts — the pocket book.

The question remains: If you previously avoided Chick-fil-A because it supported anti-LGBT organizations, is it now OK to eat there? Especially when Popeye's chicken sandwich is so good people will kill for it?


Oh, irony. You are having quite a day.

The Italian region of Veneto, which includes the city of Venice, is currently experiencing historic flooding. Venice Mayor Luigi Brugnaro has stated that the flooding is a direct result of climate change, with the tide measuring the highest level in 50 years. The city (which is actually a collection of 100 islands in a lagoon—hence its famous canal streets), is no stranger to regular flooding, but is currently on the brink of declaring a state of emergency as waters refuse to recede.

Keep Reading Show less
The Planet
via Gage Skidmore / Flickr and nrkbeta / flickr

The Southern Poverty Law Center (SPLC) dropped a bombshell on Tuesday, announcing it had over 900 emails that White House aide Stephen Miller sent to former Breitbart writer and editor Katie McHugh.

According to the SPLC, in the emails, Miller aggressively "promoted white nationalist literature, pushed racist immigration stories and obsessed over the loss of Confederate symbols after Dylann Roof's murderous rampage."

Keep Reading Show less
via Twitter / Bye,Bye Harley Davidson

The NRA likes to diminish the role that guns play in fatal shootings by saying, "Guns don't kill people, people kill people."

Which is the same logic as, "Hammers don't build roofs, people build roofs." No duh. But it'd be nearly impossible to build a roof without a hammer.

So, shouldn't the people who manufacture guns share some responsibility when they are used for the purpose they're made: killing people? Especially when the manufacturers market the weapon for that exact purpose?

Keep Reading Show less