On Wednesday and Thursday I visited Georgia Tech and Emory in Atlanta, and it was a trip full of happy surprises. I went to visit Steve Potter, who is best known for creating neural networks with living brain cells and giving them “bodies” to use and virtual worlds to explore.

It’s been powerfully argued by George Lakoff, Rodolfo Llinas, and others, that you can’t have cognition without having a body; brains and minds come into existence because bodies have to move about in the world and cope with its exigencies. This line of thinking says, “Forget about trying to create thinking machines by creating algorithms and rules.” Instead, create networks of sufficient complexity and turn them loose in a world where they need to do things to get by, and intelligence will emerge.

As I sat down in Steve’s red Prius (with a “Scientists for Obama” bumper sticker next to a license plate saying “FAQ WAR”), he passed me an announcement that Karl Deisseroth was speaking the next morning at Emory. My jaw dropped. Deisseroth is a leading light in optogenetics, a topic I’ve been writing about in World Wide Mind. His home base is Stanford. To see him here was beyond good luck. It was karma.

So we went to see him. Optogenetics is about using light instead of electricity to make neurons fire. At first glance that sounds counterintuitive, like jump-starting a car by beaming a flashlight at the engine. As far as the body is concerned, light doesn’t seem to do much except maybe give us a sunburn.

But everything changes when you insert new genes into the cells to make them react to light. The genes make proteins (channelrhodopsin-2 and halorhodopsin) that migrate to the cell walls. When channelrhodopsin is hit with blue light, it makes the cell fire; when halorhodopsin is hit with yellow light, the cell is inhibited from firing.

There’s two advantages to that scheme. First, you can control which cells in the brain express the new genes, by adding cell-specific promoters. So even though the new genes go into all the cells, they only get expressed in some of them, say Purkinje cells or glial cells.

That means you can decide, by designing the genes correctly, that only these cells will react to blue light and not those cells. That’s a huge leap in sophistication from electrodes, which make everything in the vicinity fire whether you want it to or not.

Second, inhibition is as important as excitation. Neurons are always getting input from other neurons and making up their little minds about whether to fire or not. The absence of an action potential is information too. Optogenetics lets you say to neurons, by shining yellow light at them, “Even though everyone is telling you to fire, don’t.” You can’t do that with electricity. Electricity only gives you an ON switch. Optogenetics gives you an ON switch and and an OFF switch.

How do you get light into a brain? For the moment, you do it the obvious way: with fiber-optic cables. People are thinking about how to get self-contained light sources into the brain, so you don’t have to have things going through the skull and skin.

Optogenetics is opening up new horizons for basic science: understanding how neurons interact. Everyone’s excited. Deisseroth got a big crowd, and he explained the basics plus new and unpublished research on how it might help understand brain diseases such as Parkinson’s.

Its implications for the design of brain-machine interfaces are significant. It’s going to be quite a while before anyone tries it in humans, not least because to get it to work, you have to infect brain tissue with viruses. The viruses have had all the bad stuff taken out, and can’t self-replicate, but still and all. It’s not going to be an easy sell to the FDA.

Then I gave my own talk to Steve’s class. I did my basic Cochlear Implant 101 talk, explaining how it works, and aired some new ideas from my second book. It went well; you know a talk’s going well when class ends and nobody gets up.

Steve and I then visited Phil Kennedy of Neural Signals. Kennedy is working on brain-machine interfaces to let locked-in people communicate.

Locked-in syndrome is rare but quite horrible. It happens when one has a brain stem stroke, which prevents the brain from sending signals to the body. The patient remains fully conscious, and fully aware of sensation, but is totally unable to move, eat, or speak. (This was the plight of Jean-Dominique Bauby, author of The Diving Bell and the Butterfly.) Kennedy’s goal is to detect and decode the brain’s activity while the patient is trying to generate phonemes, and speak them aloud.

As we were talking, the door opened and a young man was wheeled in on a very large and elaborate wheelchair. I was stunned to see it was Erik Ramsey, who has been “locked in” for ten years following a car accident when he was 16.

At first it was difficult to get a sense of his presence; his gaze tended to wander, and he couldn’t turn his head to look at a person. The only thing he could do was lift his eyes up for “yes” and down for ‘no.” His eyesight had deteriorated, his father explained, because his tear ducts were often dry – which accounted, I thought, for why he didn’t seem to see me very well. But I explained my cochlear implant to him, taking care to hold the processor in his field of view, and when I asked if he understood his eye shot upward: yes. So he was in there, paying attention.

I watch as Erik is prepped by Phil Kennedy on the left and his parents on the right. Phil is attaching the coils to his head, and his mom is injecting medications into his stomach. (Photo by Steve Potter.)

Erik Ramsey has electrodes in his brain that report on neural activity, and Kennedy attached magnetic coils – similar in principle to cochlear implant headpieces – that pick up data from them. I had hoped to see him use them to set up a communications link to Erik, because I was dying to see the poor kid get a chance to say something. No such luck. They were having him think “yes” over and over again, looking for its neural traces, apparently without much luck that day.

Kennedy sees Erik three times a week, working on the code that can suss vowels and consonants out of the vast tangle of neural activity that goes on in the brain. Consonants appear to be harder than vowels; why, I don’t know.

It was hard to keep from crying. When I had a major sensory system drop out of my head in 2001, the technology to get it back was right at hand. I have been very, very lucky. Still, I sometimes find it difficult to hear what’s going on, which forces me to paddle about in my own head for a while. Seeing a human being cut off from everything was a very hard, very emotional experience. I wondered: if he went mad, how would anyone know?

I gave a second lecture the next day, this time using some new slides on optogenetics that Ed Boyden had sent me, and that went very well too. In the Q&A afterward someone asked me whether implants could be powered by harvesting energy from the body’s own heat and motion, and I said I didn’t know.

But, I said to him, I want to talk to you afterward – because I saw he was wearing a body-mounted computer, with a tiny monitor over one eye and a one-handed keyboard in his pocket. A “gargoyle,” to use Neal Stephenson’s term. He came up afterward and introduced himself: Thad Starner, director of the Contextual Computing Group at Georgia Tech. And he fingerspelled his name. I was soon to find out why.

Talking with Thad Starner over lunch. Thad was having me repeat back phone numbers while moving stuff around on the table, to show doing complex motor activities didn’t detract from cognition. (I saw his point but wasn’t fully convinced.) You can also see two cochlear implants on the table. In fact, there are at least nine computers in this picture; can you find them all? (Photo by Steve Potter.)

Thad had been using wearable computers since his undergrad days at MIT in 1993, because, as he put it, he realized he was spending a lot of money on his his education and forgetting most of it. Not only that, much of the most interesting stuff happened out of class, where it wasn’t polite to divert one’s gaze and write on a notepad. What he wanted was to be connected to a computer all the time, where he could take notes while maintaining eye contact and look up his earlier notes.

He let me look through his monitor, and what I saw was a white computer screen hovering in space. Since the other eye was unobstructed, the screen appeared somewhat translucent. I could see both it and most of my visual field at the same time.

So he could, during a conversation, do a google search on you and take notes without breaking his gaze. The monitor was slightly below his line of sight, so you could see both his eyes; he tilted his head down just a bit and peered at you from above it, as if wearing bifocals. I found that talking with him seemed surprisingly natural, if one ignored the fact that he was a terrific clinical case of ADD.

I was fascinated, because Thad was obviously The Guy To Ask on what it might be like to live with a brain-computer interface in one’s head. I asked him about the lived experience of always being connected to a computer. Being able to take notes while staying immersed in the experience kept him focused, he said. And, he pointed out, it was fast and always available, unlike a Blackberry or iPhone. It takes 20 seconds to turn it on, find the right application, and start typing. A notepad is faster, but it’s not searchable; you can’t have your notepads from 15 years ago to hand. He does. He said that it augmented his ability to think, to remember, to recall deep background during conversations.

But, I asked him, doesn’t trying to multitask degrade your ability to converse and think in rich, sustained ways? There’s a lot of research that shows that while people believe they can multitask, in reality they’re not very good at it. Trying to talk on a cellphone while driving makes you worse at both. Trying to write while being constantly interrrupted by email makes it hard to get “in the zone.” I told him about Maggie Jackson’s book Distracted: The Erosion of Attention and the Coming Dark Age and its fears that we’re raising a generation of addicts who can’t focus on anything.

He granted my concern, but said that there’s a difference between multitasking and multiplexing. In multiplexing, tasks reinforce each other. For him, taking one-handed notes and looking up Google during conversations adds depth to his life instead of taking it away; that’s multiplexing. He also said, as we were hectically walking up the hill to his lab, that he doesn’t do email while he’s talking to people; for him, that would be multitasking.

I asked him why he was interested in sign language. He did his masters’ thesis on recognizing sign handshapes with computers, and his lab had four or five posters on ASL. Sign language – at least when initially learned – recruits the brain’s motor cortex area for much of the body, instead of just the lips and face. For that reason, it should be easier to “see” physically signed words rather than voiced ones in brainscanning. So if you taught people with ALS (amyotrophic lateral sclerosis) a number of important signs before they lost all motor control, their later efforts to make those signs might be much clearer and easier to read than imagined words. Fascinating stuff. ASL: ALS; a startling connection.

In talking with him, I quickly gave up trying to be as smart as he is. Our IQs might be similar, for all I know. But Thad has 15 years of notes at hand and Google constantly in his field of view. If he sees someone he met five years ago, he can remember the key points of the conversation. Everyone has an internal, neural representation of the richness of their lives. But Thad has an external representation of that as well, and one with a diachronic axis. He has an access to time, memory, and information that most people can’t even imagine. He isn’t playing at augmentation. He really is augmented.

It comes at a cost, to be sure. He has to carry around 8 or 10 pounds of computer hardware all the time. And he has to be willing to be conspicuously and mysteriously different, a constant spectacle.

But he is exploring the intersection of minds and bodies, in a way that is as much art as engineering. He’s a performance artist of memory, and as any true artist does, he shows the rest of us new ways to think about being human.

Minds and bodies were a constant theme of the visit. Karl Deisseroth is changing minds by changing bodies; indeed, his research shows that the old Cartesian distinction between mind and body is growing ever more useless. Steve Potter, in giving neural circuits “bodies” so that they can grow and evolve, is trying to repair that old mistake. And I met the most disconnected man in the world, and the most connected man in the world.

It was an exciting and moving visit, from the nitty-gritty of channelrhodopsin-2 and halorhodopsin to the empyrean reaches of memory. In the evenings Steve and I had long, luxurious conversations about optogenetics and whether the Internet is a hive mind. Steve has a roomy, well-stocked mind, with a home to match, and the sweetest wife in the world. The visit was a sustained conversation about the relationship of technologies to bodies, and bodies to minds, and a fascinating experience all around.

During Gallaudet’s long winter break — with the campus empty, the gym shuttered, and the cafeteria closed — I’ve taken to walking around the athletic track for exercise. It’s boring. Around and around and around and around.

A few days ago I discovered my iPhone’s earbuds in my desk and thought, Hey! What if I stuck these in my ears?

I could do it, too, because the mikes of my implants are cleverly situated in the shell of the ear. The earbuds fit in just fine, nestled right against the mikes.

But what to listen to? After some fussing around, I figured it out: I could download just about any radio show to my computer via iTunes. Syncing with my iPhone automatically transferred the files over. Then I could pull up the iPod program, which played them.

Cool.

What I find so remarkable about the iPhone is its sheer completeness. It can do just about anything a desktop computer can do. It is a calendar and a camera and a newspaper and Google Maps and a GPS locator and a notepad and a voicemail manager and a tape recorder and a video player and a contact manager and a calculator and a clock and an email client and a Web browser and a cellphone.

Because of the Web browser, it is also every single Web page on earth. It is every page of the New York Times and the Washington Post. It is every picture Cassini ever took at Saturn. It is every play Shakespeare ever wrote.

It’s also an extraordinary radio. I downloaded about a dozen episodes of NPR’s Fresh Air and WNYC’s Radiolab. I’ve been listening to them, one at a time, while orbiting the track. Around and around, but now listening to Brian Greene explaining why an infinite number of parallel universes exist, to a lawyer explaining how Bush & Co. could be tried for war crimes (yes please!), and to Oliver Sacks explaining why people get auditory hallucinations. (In listening to that show, Pop Music, I had to laugh as I heard my own voice coming out the earbuds.)

So my long quiet walks around the track have bloomed, and as if in synchrony, the campus is waking up too: students are drifting back in and the cafeteria is open again. Some signing deaf students can’t communicate very well with their families, so they come back as soon as possible, here, where their real families are. The sports teams have come back, too, to play games. Signs of laughter and life, in the dead of winter.

So here’s my second book, just published by Routledge:

It’s titled Educating Learning Technology Designers: Guiding and Inspiring Creators of Innovative Educational Tools. It’s an edited volume, with chapters by many authors.

The book is the brainchild of Chris DiGiano, an excellent fellow with whom I worked at SRI International in Menlo Park, California. It grew out of his project on educational technology. The book was a big job; it took about three years. Chris is now at Google, and our other co-editor, Shelley Goldman, is still at Stanford.

You might be wondering why I was involved in a project like this. My dissertation was on creating Web-based tools for online collaboration. Using ColdFusion, a terrific programming language, I wrote a software suite that was a bit like Blackboard (only, I like to think, much better.) It had team rooms, online voting, file libraries, chat rooms, the works. (More on that here.)

So while I’m doing very different things now, it’s nice to see that all those years I spent on my Ph.D. have had this concrete result.

And it means that World Wide Mind, which I’m writing now on contract with Simon & Schuster, is actually my third book. My fourth, if you count my dissertation. Hard to believe. Back to the word processor.

My iPhone had a software glitch yesterday, so I took it to the Apple store in Pentagon City for expert care. While I was waiting, a young and quite cute Asian woman sat down next to me. Her iPhone’s screen was shattered.

“You dropped it?” I asked her, sympathetically.

“No,” she confessed. “I threw it.”

This weekend I saw two plays, Frost/Nixon at the Kennedy Center and Fences at Gallaudet’s Elstad Auditorium.

Frost/Nixon was a one-note play. It chronicled how David Frost, a lightweight talk-show host, got Nixon to confess to wrongdoing in a TV interview three years after he resigned. (Yes, this actually happened.) The play was an obvious fantasy of Bush someday doing the same and letting the country have some kind of catharsis and closure. I’m not holding my breath.

But seeing Fences, in American Sign Language — now that was an amazing experience. It followed the travails of a black family in the late 1950s as they struggled with racism, money, and infidelity. The actors were first-rate, and in ASL, the dialogue seemed filled with lightning. It had that extraordinary combination of motion and meaning. It was dazzling to watch. Now that I’ve seen it in ASL, I can’t help feeling that the play would seem tepid and one-dimensional in the original English. To talk only with your mouth!

I’m grateful to the director, Ethan Sinnott, and the lead actors, Rian Gayle, April Jackson, and Daniel Ilaire, for bringing us that gift. It made me feel privileged to have a life where I get to see such things.

In that blessed future when civilization has thrown away its keyboards and people can just think what they want to do, I’ll say to the young ‘uns, “Oh, yes, I first did it that way back in 2008, when you had to wear funny caps and do it one letter at a time.”

Me, wearing a funny cap and “typing.”

This was at the Society for Neuroscience’s conference here in Washington DC (for those of you just getting on this blog, I’m spending the year here as a visiting professor.) I’d seen g.tec demonstrate the technology the night before, and I was eager to try it myself.

The cap has sockets for some two dozen electrodes, as you can see. “I’ve got electrodes and chips in my head already,” I said to Stefan Schaffelhofer, one of g.tec’s developers, who looked barely old enough to be out of college. “Is that going to interfere with your electrodes?”

“Oh, no, they’re passive,” Stefan cheerfully assured me. He didn’t see any problem with my wearing my processors under the cap, either.

I confess I was a little nervous when I sat down and he tugged the cap over my head. Electrical fields here, electrical fields there, who knew how they would interact? I was sure that if I called Advanced Bionics for advice, they’d say, eyeing their lawyers, “Heck no. Get out of there.”

I didn’t call Advanced Bionics.

What, I asked Stefan, was I actually supposed to do? How does one type by thinking?

He brought up a screen in which all the letters of the alphabet were arranged, in order. Look at the letter you want to type, Stefan told me. At any given instant, only one letter will appear to be illuminated. When the letter you’re looking at is illuminated, you’ll have a startle reaction that the electrodes will pick up.

“A startle reaction?” I said.

Not actually a startle, Stefan told me. My brain would generate a P300 wave, which is associated with visual recognition. Like the brain saying, “Oh! The letter I was looking at just lit up!”

So all I had to do was look at each letter I wanted until the system finally decided I was looking at it, then move on to the next. The system cycles through the letters so rapidly that “your” letter flashes several times a second.

Stefan started up the system, and letters started flashing. I looked steadily at the H. Every time it lit up, I couldn’t help thinking, “H!” And since it lit up over and over again during the next ten seconds or so, I found myself thinking, “H! H! H! H!”

And then an “H” appeared on the screen.

That was hair-raising. (Not that my hair could go up very far under the cap.)

My goodness, it worked! Quick, find the E and stare at it!

See me concentrating? “E! E! E! E!”

After a few seconds, an E silently appeared next to the H.

I moved on to the L. “L! L! L! L!”

Now the screen said, H E L.

Okay, another L…I guess I just keep looking at it, right?

Right.

By now, it was feeling less and less surprising. Something very specific and concrete happened in my mind every time “my” letter lit up. A zing, a pluck, a swish, a beep. A cotillion of neurons did their bob-and-whirl in my parietal lobe, and that created an electrical whoosh that the electrodes could easily pick up. The P300 event-related potential, as real as aggie marbles. Simple! Keep going…

After all these years, what my brain really wants to say.

“Not bad,” I said to Stefan, with a grin. “It worked. On the first try.”

I’d guess it took me about two minutes to do that. Experienced users, I’d learned the night before, could go about as fast as hunt-and-peck typers.

Was the system reading my mind? Yes and no.

Yes, in the sense that I was typing with mental activity alone.

But I think the deeper and righter answer would have to be “no.” All the system was looking for was that signature of visual recognition, the P300 event-related potential. When it saw it, it simply noted the last letter it had flashed. It had no understanding of the neurological difference beween intending an H versus an E, much less the neural activity of expressing the idea Hello out there.

I heard some scuttlebutt, however, about newer (and also noninvasive) devices that actually do decode neural activity. I’ll be looking into those.

Mindreading or not, the experience was a hoot. When I got back to Gallaudet, I proudly showed the pictures of me wearing the cap to a couple of students in the cafeteria.

“Whoa Nelly! A new era!”

I still think they think I’m interesting but a little crazy.

Today I told my first joke in ASL. It was around the dinner table, and the students were discussing Homecoming Weekend’s closing party, called the “Bash.” I didn’t go to the Bash, because I figured my students might not want Mike the Professor around while they got drunk and fooled around in the mosh pit. (I hope they fooled around in the mosh pit.) But I did go watch for a few minutes while the DJ was warming up. The party was in a concrete garage next to my dorm, and I immediately saw why: they wanted to make the most of the reverb. For deaf students, Strauss waltzes don’t do it. What does it is shockwaves coming from sound bouncing off walls and colliding with itself.

They got shockwaves. World’s Biggest Amps, Cranked Up To Eleven. I could feel my esophagus vibrating. I bet for women, it did a great job of vibrating certain exquisitely sensitive body parts. For me it was strangely soothing, like having a giant cat purring against my entire body.

So at the dinner table, I signed, “I stood and watched the music. I think it helped my sinus problems.” And I got a chuckle around the table.

So I’m feeling proud of myself tonight.

The class I’m co-teaching with Josh has been going extremely well. The students are an exceptionally smart bunch. They write such good weekly essays that Josh and I argue with each other over who gets to read them this week. It’s great to have interpreters voicing what the students are signing, because it means that for the first time in my teaching career, I have full auditory access to classroom discussions.

Students we don’t know keep showing up to visit our class – we had five or six visitors last week – and Josh and I haven’t the heart to kick them out. It dawned on me last week that I’m having the best teaching experience of my life.

Also last week, I gave a talk titled “Cochlear Implants and the Future of the Deaf World.” I started with some worrisome numbers: Gallaudet’s enrollment is dropping, and more parents and students are opting for oral modes of education. “But I’m not going where you think I’m going with this,” I said, and picked up on something that James Tucker, the superintendent of the Maryland School for the Deaf, had said at a talk the day before. He’d pointed out that even oral deaf childen using cochlear implants had been interested in being part of the deaf community. “It’s human to want a sense of belonging,” he’d said. I suggested that deaf people could use emerging technologies to create new forms of communication and community, and in so doing, become innovators and pathbreakers in such a way that hearing people would look to them for inspiration.

What technologies? Here’s where I got radical: I showed some fascinating videos of computers picking up neural signals and using them to control machines such as prosthetic limbs. Perhaps, someday, I suggested, implanted technologies might enable people to exchange thoughts in a way that can barely be imagined today. Who better to take advantage of such technologies, I suggested, than deaf people, who are already familiar with implanted technologies and already innovators in creating languages like ASL?

As Josh said later, “It was like you threw a bomb into the audience, but in a good way.”

I suspect that people here think I’m crazy but interesting. That’s okay.

Elvis seems happy enough with life in the dorm, though I think he gets bored with spending so much time in my room. I’ve been letting him visit my neighbors now and then. One of my students lives next door and she and her roommate give him bits of ham. So Elvis is always meowing to be let out to make the rounds. “You want to go visit Aunt Kelley and Aunt Stephanie?” I say to him. “Okay, go on. Just be back by nine.”

Elvis, king of the dorm.

I didn’t go to the Bash, but I did put on my good suit and go to a party the night before, called the Ball, at a local club. I’d thought it was an all-university party, teachers and students alike. Wrong. I was the only teacher at the party, and the oldest person there by 20 years. But my students seemed to get a kick out of me being there. One signed to me, using grand, theatrical movements, “Sign is totally different from English. Forget English. Throw it in the trash.” I was struggling to follow him, but in a burst of inspiration, I took off my processors. It let me focus on the visual, and that helped.

Another of my students asked me whether I was enjoying my time at Gallaudet, and I said I was, specially now that I was feeling better after having been sick. “You complain too much,” she signed to me.

“I like complaining,” I signed back to her. “Complaining is fun.”

And then I explained, in slow, careful signs that sometimes I complain just because a large percentage of the signs I know are good for complaining. I want to say something, so I use the words I have.

“You’re lying,” she signed, pointing her finger emphatically at my chest. And that was that.