Virtual reality (VR) pioneer Jeremy Bailenson discusses how VR experiences can change our perception of self, increase empathy, and lead to new forms of social interaction.
Jeremy Bailenson is the founding director of Stanford University’s Virtual Human Interaction Lab where he studies the psychology of VR, in particular how virtual experiences lead to changes in perceptions of self and others. His lab builds and studies systems that allow people to meet in virtual space, and explores the changes in the nature of social interaction. His most recent research focuses on how VR can transform education, environmental conservation, empathy, and health.
On this Episode
Find out more: futurespodcast.net
Transcript
Luke Robert Mason: You're listening to the FUTURES Podcast with me, Luke Robert Mason.
On this episode I speak to the Founding Director of Stanford University's Virtual Human Interaction Lab, Jeremy Bailenson:
"The cool thing about VR is you can do things that you can't do in the physical world. Leveraging this paradox, which is that the brain treats it as real, but you can do the impossible - you can do anything - is a really neat way to use VR." - Jeremy Bailenson, excerpt from interview
Jeremy shared his thoughts on how virtual reality experiences can change our perception of self, increase empathy, and lead to new forms of social interaction. This episode was recorded on location in London, England at the offices of publisher W.W. Norton.
Mason: So, Jeremy Bailenson, I've got to ask you: when was your first Virtual Reality experience?
Jeremy Bailenson: 1994. I was interviewing for grad school at Berkeley. I had a spare day, and I walked down to the Embarcadero in San Francisco, and there was an arcade on the Embarcadero that we had to wait in line - I think we paid 25 bucks back in '94 - and they had something called Dactyl Nightmare. It was a video game where you put on the goggles, and it was a networked VR experience, and you played some type of a video game. Imagine 1994 VR: it was running at a quarter of a second lag, and the tracking was way off, they had to reboot the system a couple of times to make it work, but it was still really special!
Mason: Was that the thing that made you think that VR was the thing that you wanted to be part of? Or was it more of the science fiction that you were reading at the time?
Bailenson: Certainly that planted the seed, but for me the icing on the cake was actually re-reading Neuromancer in graduate school. Infact, in the late 1990's my PhD was in Cognitive Science, where we were learning about how you can model thought in terms of representing how reasoning works; how categorization works; running experiments on humans; and building computer simulations on how to model that. I was kind of stagnating there, and I wasn't loving it, and at the time I read this crazy book called Neuromancer and decided, "Huh. How about instead of trying to build intelligence, I'm just going to fake it. I'm going to build these avatars and study how they work." And that's when I decided to shift.
Mason: So what was it specifically about William Gibson's Neuromancer that inspired you to work in virtual reality? Was it this notion of the consensual hallucination?
Bailenson: I mean that was part of it. The fact that you were networking, the con-sensual, the people sharing their senses. That was a lot of it. I'm teaching a class at Stanford right now - it's called Virtual People. There's 200 students in it, which is about 5% of the undergrad population at Stanford, and there's two textbooks. One is my book, Experience on Demand. The second is Neuromancer. And, and what we do in Neuromancer is we read it and we pick out every instance of a changed-human. Every instance of a virtual human where there's something going on with human identity or, or social relations that's different from what you could do in the real world. And there are so many amazing examples of how what it means to be a person is just fundamentally changed in the world of Gibson.
Mason: I wonder whether your students at Stanford...how that's slightly different from when you were reading Neuromancer. So, they've grown up in the age of Facebook and Twitter and social media where these virtual platforms are changing their notions of identity. I wonder if they're bringing an entirely different reading of Gibson to that.
Bailenson: Yeah, I suspect they are. I mean, what I like about the book is that if you read it, you'll notice he doesn't tell you anything about technology. He doesn't talk about pixels or resolution - that is left abstract. So it does travel through time a bit better than others might where they instantiate those details, but yeah. Who knows what they're thinking. I mean, what I know - what I hear from them most is that, if you've read the novel, it's a wonderful book but it's complicated and that typically takes a second read to really get it, so I tend to get them a little bit upset that I'm forcing them to read this book that they don't really understand.
Mason: And are they finding virtual reality through other means? Have they often been inside of VR before they've seen science fiction notions or narratives around the possibility or virtual reality?
Bailenson: So I arrived at Stanford in 2003, and back then we were the only VR Lab there. I mean there's some heroes from VR there: a guy named Ken Salisbury who does haptics, and there's a guy named Pat Hanrahan - that is one of the graphics heroes. There's a lot of people that do small parts of it, but I was the only VR Lab. Now of course, there are many instances of VR in terms of Labs. But what's stunning to me is there is an undergrad group called Rabbit Hole VR at Stanford, and there's 900 students I believe part of their group - affiliated with their group - which is about 20% of the undergrads at Stanford.
Mason: And what does Rabbit Hole do?
Bailenson: It's a good question, I'm not exactly sure. They do VR.
Mason: They do VR. Do they do anything else on top of the VR? I know that's against your 8 rules for how to do VR safely.
Bailenson: Don't ask, don't tell.
Mason: In that case let's talk a little more about what Stanford's Virtual Human interaction lab actually do. Firstly, why that name - Virtual Human? That seems to conjure notions of autonomous avatars rather than virtual reality experiments.
Bailenson: So when I arrived at Stanford, to get tenure at Stanford University it's very very difficult. And the way one does that is by really becoming the world's expert, or among the world's experts in a very specific area. And before tenure - my research before 2010 - most of my work was about interaction, social interaction with avatars, or agents meaning virtual humans that were controlled by people or that were autonomous. When we were naming the lab, strategically I really wanted to show the world that I wasn't just a tech guy, I wasn't a VR person. I was a person who studied the way people interact and exist inside these virtual spaces, and that was what I wanted the lab to be about.
Mason: So for you, what is virtual reality? What's the lab's definition of virtual reality? Because it feels like lots of people have so many different definitions. I know Jaron Lanier in this recent book 'Dawn of the New Everything' had 50 definitions of virtual reality, so I wonder what is your definition for that term?
Bailenson: Well, when I'm talking to people who have never tried it, I say instead of watching a movie it's as if you're inside of a movie and that's um...I think our audiences can be a bit more nuanced. So to me, VR requires tracking, rendering and display. Tracking which is measuring your body movements. Rendering is redrawing a scene based on those body movements, and then display is replacing your senses with content. So in the physical world, when you walk towards somebody they get bigger. When you turn your head towards a sound it gets louder. To make VR work, you have to track physical movements, redraw the scene from a new position, and redraw sight, sound, touch, sometimes smell, and then to replace what you're seeing using the hardware such that the display renders what is virtual as opposed to physical. And it's that confluence of those three events that define VR for me.
Mason: Are those three things are important because you're trying to represent reality as it stands out in the real world, or are they important because of the brain?
Bailenson: What makes VR special is it responds to your body. What makes VR different than other media: instead of hitting buttons, you're moving your body so you're walking towards things, you're moving your hands to grab things. It's perceptually surrounding. So with the TV when you turn your head it goes away, when you close your eyes it goes away. With VR, when you turn your head around there are things everywhere, and there are sounds that are spatialised - and so the perceptual surrounding aspect, and replying to your body. The third thing VR does and you know, this comes at a cost, you mentioned my article on safety, is that it completely blocks out the light and most of the sound from the physical world so that contributes to the immersion as well.
Mason: I mean is the aim to match the physical world or at least the cues that you expect from the physical world, or is this something interesting that can be done inside of virtual reality where you can mess with the physics of the real world and see what the brain and the body does in that sort of environment?
Bailenson: My philosophy is, you know, there are some great applications where by replicating what's in the physical world you can do things that are expensive or rare, and that's a good use for things like training. But in general, the cool thing about VR is you can do things that you can't do in the physical world and leveraging this kind of, this paradox, which is: the brain treats it as real, but you can do the impossible; you can do anything - is a really neat way to use VR.
Mason: Is there any ways to trick the brain inside of virtual reality? I know your lab is doing specific experiments with regards to say things like, empathy, that it's not so much tricks to the brain, but it's, it's using quirks that we have as human beings and, and using virtual reality to play with those, to enable individuals to have these high empathetic experiences.
Bailenson: In general, my colleague Byron Reeves likes to say, "the brain has not yet evolved to differentiate compelling VR from a real experience." So, humans have been around for a long time and you know, if you walk towards a lion and it growls at you, you're going to feel scared. And in VR, if you, if the scene responds to your movement, similar responses occur. So one of the lines of research we've been doing since the late 1990s is showing that when VR is done well, that the brain tends to treat that experience as if it were real and the body responds accordingly.
Mason: I think you say in the book it's called 'presence.'
Bailenson: Presence is the term psychologists use to define that magic experience of when VR is a really compelling illusion. Matthew Lombard, a professor at Temple likes to say it is, "the illusion of non-mediation". When VR is done well, you don't even know you're in VR - you're just having an experience.
Mason: For you it becomes reality almost.
Bailenson: It becomes reality for you in that moment. Absolutely.
Mason: To take the idea of presence one step further and look at someone like Mel Slater's work. I mean, he tries to break down what it is that makes someone present in VR. And he talks about the two illusions: the plausibility illusion and the place illusion. Do you agree with those? As is the notions around which thing that makes virtual reality so immersive are those two?
Bailenson: I had dinner with Mel Slater last night here in London and we talked about the possibility illusion, and I think he's one of the scholars that has made VR as good as it is. He's amazing and I'm a huge fan of his work. And, and I like his thinking on it. So Mel, you know, in the last decade -15 years - has really embraced this neuroscience approach and he's doing really transformational work.
Mason: Considering your background is cognitive psychology, I mean, how much do you see the collision between neuroscience and VR being the future of this medium?
Bailenson: There is no doubt that, you know, going back to William Gibson and Neuromancer, there was no hardware there. You jacked right into the brain and there's a growing movement, including a lot of Mel's colleagues, who are doing brain computer interface to use the brain. Not just as a metric to understand if VR is compelling, but also to be able to, you know, control the scenes by using as an input factor, electrical activity in the brain, and you know we're still a long way off from that, but there are certainly scholars that are...that are moving in that direction.
Mason: I mean, is that a direction you'd potentially move in with the Stanford Virtual Human Interaction Lab, or are you doing something more nuanced?
Bailenson: I'm a little bit old to start doing wet work myself. I'm 45, and learning. You know, I did neuroscience in grad school, but I'm going to continue to be a helper for those that are doing that work. But I'm gonna, you know, my strength is really in measuring behavior and nonverbal behavior and attitude and behavior change and I'm going to kind of continue in that domain.
Mason: Well, let's talk about some of those, those behavior change ways in which you've been using virtual reality because your experiments have been looking at virtual reality as an empathy machine. I think that's Chris Milk's term, and I know you mentioned Chris Milk in the book and how he's been using certain immersive 360 video to help with empathy, but what you're doing is a little bit more nuanced, a little bit more in depth. Could you explain sort of your interest around empathy in VR?
Bailenson: I just came from the Tribeca Film Festival a week ago. It's now April and I was just in New York City with Chris Milk, and he and I had a dinner and drinks together and talked about his new work, now is also using computer graphics. He's doing a lot of work in networked VR and these shared experiences. But we premiered a piece at Tribeca this year called Thousand Cut Journey and this is a, it's premiering as we speak in New York City about 11 hours a day for eight days straight. The arcade is open at Tribeca, and hundreds of people a day are going through this experience. This is a piece about racism, about black-white racism, and you put on the goggles and you look down and you've become Michael Sterling, a black male, and you go through his life over time. You start out as a seven year old and then you become a 13 year old and then you become a 30 year old. And throughout your life you're experiencing racial bias. And the general theme of the piece, my colleague Courtney Cogburn, she's a professor at Columbia University - studies black-white racism - this is based on, on her academic research and her life experiences, as well as those of her research team. This is really her vision from a narrative standpoint with me, you know, bolstering it based on what I know in storytelling and VR, and not just being the tech guy, but really helping her achieve the vision that her research is producing. So the theme of it is that you, as a black male, are punished for things that you do, where your white friends and colleagues do not get punished. And you start out as a kid where you throw a block and your friends have been throwing blocks - the teacher yells at you. Then you become a teenager and you try to cross a street, and for jaywalking you get pulled over and frisked, when your white friend does not, and then similar instances as you do job interviews - and it goes on. So that's a piece that's premiering as we speak. In general for the last 15 years, we've run dozens of studies that has you become someone else in VR. And then we compare that journey of walking a mile in someone else's shoes to control conditions. Things like role playing, or watching videos, or reading case studies. And in general, not every single time, and certainly not all the time and not in every single measure, but across all of these studies what pattern emerged is that as in general, VR tends to outperform those control conditions at producing behavior change and, and changing the way you behave towards others and your attitudes toward them.
Mason: I mean, one of the examples of that is the Virtual Mirror project. Could you explain that a little further?
Bailenson: So the mirror is a tool that causes, what Mel Slater calls, body transfer. He and his colleagues and you know, we've been using the virtual mirror since 2003, but I didn't invent it. The mirror was around before I got onto the scene. My colleague Jack Loomis had one in 1999. UC Santa Barbara, and I'm sure others have had them before then. But what the mirror does is it allows you to move physically in the world and to see your reflection in VR change. So in this Michael Sterling piece and Thousand Cut Journey, you become Michael Sterling. And the way that we do body transfer is that we have a scene between every age jump you see yourself in a virtual mirror. And the way you do body transfer is that you move your hands physically, you wave to yourself, and you spend some time having this, what they call synchronous behavior. You move your body, and the mirror image responds in time.
Mason: Can you talk a little further about synchronous behavior? Is that the thing that combines the virtual of these environments that you are creating, and the actual of your own body? Is there always this need for a certain degree of calibration between the two before someone finds it truly immersive?
Bailenson: Yes. So the synchronous term comes from the neuroscientist, actually. So there's a famous line of research called the rubber hand illusion. And I'll try to describe this to our listeners. With a rubber hand illusion, imagine that you're sitting at a table and you put your hand under the table so you can't see your hand. And then they put a rubber hand on top of the table where your physical hand should be. Now when you look down at that rubber hand, you see somebody stroking that hand with their finger. So on the rubber hand, you see a finger stroking it. Somebody under the table is touching your physical hand and feeling it stroke. So to sum that up, you're seeing the rubber hand get stroked and you're feeling your physical hand get stroked, but you can't see your physical hand. Over time, the brain tends to treat that rubber hand as if it were your real hand because you're having that synchronous movement where you're feeling the touch and seeing the movement on the rubber hand. Now, the illusion does not work when it's not synchronous. So the way, one way to measure this, and I know that sounds ridiculous, if you take a needle and you stick it in the rubber hand, you feel pain in your brain from the rubber hand getting getting stuck by a needle, even though obviously it's not your real hand. It doesn't work if the movement is not synchronous. In other words, if the finger you see stroking your rubber hand is not in time with the finger that's touching your physical hand, it doesn't work.
Mason: I want to return, just very quickly, back to this idea of empathy machines and the sorts of projects that you are describing. The worry is these tools might create empathy, but they also might serve to highlight certain differences between individuals. Is there a danger of how these platforms are being created to generate empathy? Could they also do the inverse in some cases?
Bailenson: I believe full-heartedly that we need to study this thoroughly. And there have been some studies that show that when you become someone else, that it activates stereotypes, and there's been other studies that show that nothing happens and that it doesn't go either way. And I believe that it's an extremely powerful medium if you try this body transfer, where you become someone else in the mirror. It's pretty intense, and I love the idea that we need to spend a lot of time working on this.
Mason: Does it really depend on the type of individual? Are some people more averse to virtual reality simulation environments versus others. Do we know what the quick is within each and every individual's brain that either makes them hyper-immersed or sort of not interested in those sorts of environments?
Bailenson: So in terms of individual differences in presence - we call this proclivity for presence - there's not been much work and the reason is, up until recently, to run a VR study, it's really hard. You get this really bulky hardware. It would break all the time. You could only bring people to the lab. Now that hardware costs a couple hundred bucks and basically runs off a laptop, we can start collecting data at scale. So Fernanda Herrera, who's a PhD student in my lab, she has now run over 3000 subjects through this one empathy study we have called 'Becoming Homeless', where you basically start a journey where you live in your house and you lose your job and you get evicted. You try living in your car. And we put this in museums. So we have a permanent exhibit at the San Jose Tech Museum. We've brought it to senior citizen's homes, we've put it at flea markets. That's a really large data set with a lot of variants in terms of who the participants are. So I believe this data site will give us some insights to answer your question, but there hasn't been much work to date, because there just haven't been large sample studies that have enough variance.
Mason: The one thing that is clear is that there are, what you call in the book, 'media effects'. Media can change people. Could you explain what 'media effects' are and why that's so important with regards to virtual reality?
Bailenson: In the field of communication, my lab at Stanford is in the Department of Communication. We study media - in particular, how media affects people and 'media effects' is the word we use for, you know, "what's the role of watching TV for five hours a day or of having a connected world via the internet?" And you know, with VR we don't really know how media is gonna affect people because we haven't been able to run these large scale studies. I just produced a report for...or I coauthored a report for Common Sense Media. Common Sense Media is an organization where parents can go to, to help understand media and kids. So, you know, I go there for, I have a six year old and a four year old. Is this movie appropriate for a six year old? It's hard to tell from the description sometimes. And what we wrote for common sense is a report about kids in VR. So it's for people to try to understand in general what we know about kids in VR and how media affects kids. In this instance, what we don't. I mean the short answer is that we have a ton of work to do. And you know, we've come up with some rules or guidelines for how you should think about VR and kids.
Mason: Right now we don't have any consumer devices that are specifically targeted to kids though, do we? And by kids, which sort of age group do you mean?
Bailenson: So in this report we study everything we know about anybody under the age of 17. And the reason we can do that is there so few academic studies that have looked at anyone under the age of 18. It's possible to review every single one in a fairly short report. So my PhD student, Jackie Bailey, who graduated last year, she's now Professor at the University of Texas. Her work while at Stanford looked at hundreds of kids who were three years old, four years old, five years old and six years old. And she had them put on the goggles and meet Grover from Sesame Street - this is a project sponsored by Sesame Street - and what we were looking at is "what's the difference for kids when they meet Grover in VR compared to when they meet him on TV? How does that affect things like executive function, how they're able to resist temptation and how it changes the way that they believe these characters are real?"
Mason: Nothing sounds more terrifying than the idea of Elmo in VR, or Elmo VR.
Bailenson: We have it in the lab. When you come to the Bay Area, you can dance with Elmo
Mason: From the perspective of that report, I mean, those sorts of experiences - very short term. So you put children in virtual reality for short period of time and then you give them something familiar, i.e. a character that they recognize from something like Sesame Street. And I wonder what your opinions are on VR and duration? I know you said that 20 minutes is the limited time anybody should spend on VR. But we're already seeing artists - there's an artist in the UK, Mark Farid, who wants to spend 28 days in VR using a project called Seeing Eye. I mean, what do you think and feel when you hear that some people want to push their experience of these environments and this hardware to the extreme?
Bailenson: I'm all for free speech and I believe anyone should be able to do what they want with their own media experiences. You know, Mark, go have at it, have fun, be safe. Make sure you have a buddy watching you. But that's an artist, and you know, artists do extreme art, and let's keep that going. In terms of...all I can, you know, I'm not a government agent and I can't and don't want to do regulation. For my friends and my family, when they ask me "what should I do?" 20 minutes is enough. Pull that thing off after 20 minutes, touch a wall, have a drink of water, say hello to a friend. Typically we find that there's not much worth going back in for after 20 minutes. I mean, there are some obviously artists that are doing this, you know, I think that's pretty neat. But for people in general, I think that there's very few VR experiences that are worth doing for more than 20 minutes right now. That may change as content gets better.
Mason: Does this align with some of the commercial interests of the sorts of folks who want to create commercial VR? Of course they want to keep people - at least I understand it that way - they want to keep people in as long as possible. They want it not to be a 20 minute experience, but to be a whole, a five hour gaming experience. I mean, how do you mediate the conversation with those sorts of individuals who come to your lab and go, Jeremy, tell us how we keep people in here for five hours cause you want to sell the shoot them up game. And we want people to be fully immersed for that period of time. I mean, ethically, what do you do in that scenario?
Bailenson: Oh, ethically it's really simple. I tell them just what I'm telling you. And even, you know, when, when I'm working with these companies and you know, it's a hard message to deliver. But you know, they're big men and big women and they, you know, they just choose whether they're going to listen to me. I've really clearly, you know, say that, you know, I don't want you to be doing this every day. I think 20 minutes is plenty. You know, there are certain experiences I think people should avoid in VR that are, you know, pretty nasty. And they listen to me. They don't necessarily hear me or, or they hear me, they don't necessarily listen to me. But you know, I keep harping on it.
Mason: Well, what are some of those experiences to avoid, if you don't mind me asking?
Bailenson: That depends on personal tendencies, but, but my guideline is if there was something that if you did it in the real world - whatever this thing is - when you went to bed that night, you wouldn't be able to sleep. You wouldn't be able to look yourself on a mirror. You wouldn't be able to look your spouse in the eye. You would just feel terrible at yourself had you done it in the real world. Those are the kinds of things I think we should avoid in VR. And that's a personal choice on my end. You know, you should do things you can't do - fly to the moon. You should do things that are dangerous, like jumping off a cliff. That's great. In VR, you're not going to get hurt if there's something that you know the experience would leave you sad about yourself as a person because the brain tends to respond to VR in a way that's real. You should avoid those in VR, in my opinion.
Mason: The counter argument could be, I mean, VR could be used as a safe environment in which to explore the darker sides of what it means to be human. For some people that may be a form of exorcism. It goes back to the same debate that individuals are having right now between child sex robots; whether that cures peadophilia or not. There are some dark things that can be done in these sort of safe environments whereby arguably nothing is at stake that could help certain individuals through certain proclivities. I mean, is there a blurred line here?
Bailenson: So let's take aside the instance where there's a therapist that's guiding somebody through who's in a very specific clinical population. Cause that's outside my expertise. But I will say in general in psychology, this notion of catharsis that experiences can be used to relieve tension is not one that has much experimental support. So when we talk about video game violence, there certainly is debate about whether or not these games desensitize you to violence in the real world. There are some people that can test that. As far as I know, there's not many credible psychologists that are arguing that it causes you to relieve tension and hence not be more aggressive later on. That is a notion that doesn't get much more in the literature.
Mason: There's a funny line with regards to PTSD studies whereby arguably VR is being hyper effective at helping people sort of recover from those violent experiences. And part of that is to do with degrees of exposure. They need to be re-exposed to the sorts of violence that they have been suffering. I wonder if we can talk just a little bit about the PTSD studies and why they've shown to be so effective, and where VR combines with, with the recovery of PTSD.
Bailenson: I think that's a great transition. And Skip Rizzo, who's probably the name most associated with this. He says very clearly that when you play video games like Call of Duty, he calls it a 'revenge fantasy'. And that is, they do that just to explore their dark place. What he doesn't argue that's clinically important to what he argues: what you use VR for is something called cognitive exposure therapy. So someone who's got PTSD, the way therapists typically try to help them is by bringing them back, mentally, To try to get them to imagine the time and the place of this trauma so that the therapist can then give them coping strategies to try to undo these associations. So it's not really exploring a dark place, it's more, you know, having to go back there. So the therapist can give you tools. And so what Rizzo does is he builds, you know, virtually rock. And when veterans come back, he puts them in there in the goggles and he's got good sight and sound. Sometimes you use virtual scent where he can have the burning smell. It's really intense. And then with a therapist, the therapist helps you unravel some of that damage while you're there.
Mason: I think that's the important thing that often we miss. It has to be guided VR. Either there has to be a therapist there or in the cases of your lab, you're on the periphery to catch people when they fall. I mean people often forget that there has to be an individual sort of helping another person through these VR experiences - it's not a me and the virtual environment scenario is it?
Bailenson: No, it's safety first. It's really unlike other forms of recreation. As far as I know the first death just occurred in virtual reality. A man in Moscow was playing a immersive video game and fell through a play class table and bled to death, and you know, it's, it's a tragedy. But we're going to see more of them, if we don't really take spotting and safety in terms of just not colliding into objects seriously.
Mason: I mean you recently wrote an article for Slate outlining some of the eight ways in which we need to be more safe with VR, and spotting is one of those. Could you explain what spotting is? Because again, I think that's one of the most overlooked things with regards to VR. People just assume they put their headset on and off they go. But there always has to be an individual there.
Bailenson: So the way you do VR is you wear these goggles and you're in a physical room and it's almost never the case that the virtual scene is the same size as the physical room. So simply walking into walls is an issue because you know, the VR room is very big and your physical room is typically pretty small. Over the years of giving tours in my lab - and I probably demo at least a thousand, maybe 2000 people per year - you know, we've had some people do some pretty extreme behaviors. I was reminded this morning that Lord Tony Hall, the head of the BBC was in my lab, and when he did the Superman demo where he lifted his hands over his head to fly like a hero, we have a floor in my lab that shakes. The floor shook, and Lord Hall just decided to do a backflip. When he took off, he raised his hands up and jumped and I was there to catch him. But I think the field of VR would have suffered had he continued along that trajectory.
Mason: I want to go back a little bit back to this, this idea of PTSD and how VR can be used in these environments. Because almost it feels like if VR can be used to relieve the symptoms of trauma, or of pain, then surely it also follows that there is a potential to use virtual reality to induce trauma or to induce pain. To my knowledge, I don't think we've had the first case of someone having PTSD from a VR experience, but won't that really prove out the efficacy of the medium, if that was allowance to happen?
Bailenson: Yeah, I know I'm not a clinical psychologist and so I can't speak of what causes PTSD per se, but I do believe these intense experiences do produce negative reactions. I mean the closest data point I have for you on this, and this is very anecdotal because we just premiered this Thousand Cut Journey - that's the piece at Tribeca right now, where you become a black male - when whites go through it, we're supposed to go and you know, feel really emotionally engaged and to rethink our role in how to help racial justice and how to help people of color. When people of color go through it, it's a trigger for some of them, and we've had more than one person have a fairly intense reaction. We're very careful. We talked to them beforehand and after. That's why this one is not up on steam yet for free. We want to be a little more careful with this one. But we've had some, you know, some people that are brought back to the times when this happened to them. And it's hard
Mason: Again back to - and I hate to keep coming back to the dark side of VR - but I think it's a thing that if we don't talk about it openly, it's the thing that the press take and they want to run VR as if it's like violent computer games. They want to assume the worst constantly. And, and I know that the folks from Be Another Lab, who you might be aware of, have said that VR has the potential to induce severe pain or suffering, whether physically or mentally, if it is applied with torturous intentionality. Most certainly the military will shortly be experimenting with VR as a form of torture if they have not begun already. I mean, doesn't that concern you - the idea that it can be used for these hyper insidious scenarios?
Bailenson: So first I want our listeners to take a deep breath. These are a couple of people in Boston saying that maybe the military might be doing this. That is, we have no data that that is occurring. So everybody take a deep breath. That being said, your point is well taken and this is a medium that's powerful and I can't look you in the eye and say that 'VR is so powerful, it can change the way you think about race, but not be used to make you feel bad.' And I mean, let's forget VR for a second. You can do a lot with a video, you can do a lot with the written word and, but as I'm arguing, VR is a next level thing in terms of its influence and its power. So we've got to be careful. I mean, look. The book came out January 30th and it's been reviewed by the New York Times and the Washington Post, The Times of London, The Wall Street Journal, and in general, people love the book. Where I get criticized is I'm not as attentive to the dark sides as I should've been. I think I've been lumped - or VR has been lumped a little bit with kind of the dark side of social media right now. In terms of its manipulation and abuse and privacy. And so even though chapter two of the book is all about the dark sides of VR, I've been pigeonholed as a hopeless optimist.
Mason: Let's return then to a little bit about how VR is being lumped in with social media because I felt Jaron Lanier has done a lot to help that narrative. So with the Dawn of the New Everything, he came up with two strong messages which were about the behavioral modification empires of social media, but also about virtual reality and how that could be the next way in which humans are manipulated from both a behavioral perspective or otherwise. And I think Jaron's done it slightly a disservice in so far as he's kind of frightened people into the idea that now reality's at stake, cause we already know our reality is being manipulated through virtual platforms. And we've seen Zucks testify in front of Congress, and Zucks of course is one of the individuals who both came to your lab and found VR trippy, and also has ownership of one of the at least most publicized cases of where a VR company has been purchased and wants to be used at scale. Now your interest is social VR. You're very excited about social VR, but it feels like what Zuckerberg terms as social VR is very different from what you outlined as social VR in the book. And I wonder if you could just highlight where are some of the differences and where are some of the similarities between your visions, I suppose?
Bailenson: Yeah. First let me start with Jaron Lanier, because I do think he's still really high on VR. He and I talk very often and I believe, I don't want to put words in his mouth, but if we can change the economic model such that everything's free and the only way you pay is with your data, I think he'll be really happy. So, you know, you can ask him directly. But I do think he's optimistic in VR, assuming we can solve some of these structural problems that we've had with other social media. On Mark, you know, Mark is...or let's talk about Facebook. Facebook is really into social VR. They have a huge group - including a number of my former PhD students - that are building Facebook spaces, which is their networked avatar space. There's also a dozen other companies in the Valley that are doing it, whether it's Altspace VR who got bought by Microsoft or High Fidelity, which is a new company from Philip Rosedale who founded Second Life, or Sansar, which is the new version of Second Life run by Linden Lab. So there's a lot of these companies that are out there. I like that it's a crowded space. If you look at the number of people who are using social network VR right now, the 'winner' in terms of the most people using is something called VR chat. And they had 20,000 people in there concurrently. And that wasn't on anybody's radar a year ago. So I think that, you know, we're all betting on Facebook to win just because of their amazing resources and talent in some ways. But you know, there's some, there's other players.
Mason: Phillip Rosendale and Facebook kind of share something interesting with regards to social VR, because they're moving slowly towards the avatars looking in the image and likeness of you, and it feels like Jaron's disappointment with VR has come from the fact that we haven't gone and created these crazy worlds that have formed morphogenic algorithms, and our avatars aren't bipedal and human looking, but they're a mesh of two individual human bodies that are writhing and thriving in whole new, different sorts of ways. I just wonder where you stand on that? Does social VR needs to look a little bit like reality to get people over the threshold to explore the possibilities there? Or should we just embrace full creativity when it comes to social VR and create worlds that we've never seen before, never experienced before, and embrace that kind of creative chaos that could come with VR?
Bailenson: I mean, the answer is yes - we should do both. So Jaron and I have published papers together on his theory, which he came up with, you know, decades ago, called Homuncular Flexibility, which is the ability for the brain to accommodate very weird bodies. And we've built avatars that have third arms and ones that when you move your physical hands, that moves your avatar's legs. And you know, I've run studies where you become a cow and other animals, and become a piece of coral. And I love all that. But you know, for me, the epic win with VR in terms of saving the planet: we're soon going to have 11 billion people on this planet and we can't all be driving to work both ways everyday and flying across the world to have hour long meetings, so something's got to give. And in the work context, you can't look like a monkey with three heads. You have to, you know, have some semblance of professionalism. So when we're doing things that require photographic realisms, given the context, we need to have the ability to do that. And when you want to, you know, share bodies and you know, grow a third arm, you can do that too - but just the right time, in the right place.
Mason: A lot of people worry that the VR is going to be the diaspora from the self. But in actual fact, it could be the medium that allows you to explore multiple selves, if social media has failed in that respect, i.e. it forces you to have a singular profile. And could VR be the thing that returns us back to the sort of mid-nineties world where you're not judged by your body or your avatar - you're judged by the sorts of experiences that you generate and create. Could it retrieve a lot of the good intentions of the mid 90 cyber punks?
Bailenson: Yeah, I mean, look, when you look at the main thinkers in this era, one of our heroes, her name is Sherry Turkle, and Sherry was hugely optimistic about the digital media as it's going to help us with, you know, self-expression and the selves. And of course she's turned around a lot on her thinking. I mean, you see this a lot when you look at the scholars who wrote brilliantly and extensively in the eighties and nineties about social media or about digital media has turned around a lot. Jaron - of course - being one of them. Sherry Turkle - another. And I don't love seeing that trajectory. It's kinda sad. I'm not saying that they're not wrong. It's always a bummer when the heroes that have, you know, really helped pave the way for this vision, you know, are not excited about where it's come.
Mason: Is it a case of we got what we got what we wanted, but it wasn't necessarily what they expected it was going to be? Do you think that that's where some of the disappointment with the future that we have now comes from? And do you think that in actual fact virtual reality is a possibility that this time we could get it right? It feels like VR is being given a much wider berth. The sort of work that you're doing is really interesting, and so far as it not rejects or ignores the commercial applications of VR, but it really proves the efficacy within a certain closed environment whilst all the commercial VR is preparing people to have these sorts of incredible experiences through these, you know, very representative gaming or 360 video. Let the commercial guys go do that whilst you're also doing your work, because your work will be the thing that will really help us transition from reality to another reality.
Bailenson: Well, I mean, the one thing I can say is that what's a little bit different now is that we've, we've made some pretty big mistakes with how economically and how structurally we've allowed social media to play out. And hopefully we can learn from them. But I don't have so much hubris that I think that, that, that anything I can do can make this different. And I try to be loud and to, you know, to talk to people like you and your listeners and hopefully we can avoid some of the mistakes we've made in the past.
Mason: I want to go a little bit back to talking about how VR is an important medium in terms of immersion and in terms of the senses, and in terms of agency, because I think the thing that we probably haven't spoken enough about is agency. You always have the ability to take the thing off with virtual reality. Storytelling is all about the agency of the individual, where other mediums of storytelling such as 2D video or cinema force you to go through from beginning to middle to end. How does human agency play and how is human agency so important when it comes to thinking about virtual reality?
Bailenson: When it comes to storytelling, when you watch a movie, when you read a book, it's linear. You know, unless it's a 'choose your own adventure', you're pretty much reading in a line. With film, the director - she chooses where you're gonna look by pointing the camera. VR is anarchy. You can look wherever you want, whenever you want, and you can walk to places.
I can give you an example: When I was at Tribeca Film Festival, one of the pieces, I did - a new piece, - it's called a Hero, and it's a really intense piece. You're in a Syrian village, and there is a dog that comes up to you and you play with the dog. And they have these two beautiful children that are kicking a soccer ball. And you know, this is a fully immersive backpack based system. You're walking around multiple rooms, computer graphics, high end graphics, really very well done. And you're sitting there playing with these kids and then you see a helicopter come and the helicopter hovers over and you go, "Oh man", you know, "it's coming", and the helicopter drops a bomb. And when the bomb hits about five meters away from you, unbeknownst to you, the artists have now wheeled in a fan that's blowing hot air and then they drop some paper or shrapnel or some, some sawdust. So you feel shrapnel hit your face. I mean, that bomb hitting and me feeling that blast was one of the most intense things I've ever done in VR. Obviously really horrible. Getting to the point of your question, what you're supposed to do in this instance - it's called Hero - after the bomb hits, there's people who are stuck under heavy objects and you have to go around and help some of them. One of the men who had gotten crushed, I just decided I wanted to help somebody off in the distance, I got up in the story itself. The directors didn't want me going over there. And what happened is that I, you know, ran over to help this guy who was underneath this big chunk of metal and I smashed my face, smashed into a two by four that they'd left sitting out for the fan because they couldn't anticipate that I was going to not do what they wanted me to do in terms of the narrative, and I smashed into a two by four and cut the inside of my mouth.
So it's really hard to predict what someone's going to do in VR and forget safety for a second. It just, it harms storytelling because you know, one of the classic aspects of film is something called plant and payoff. Early on in the first act you learn this really subtle thing about somebody and later on that subtle thing, that side long glance or the fact that they had a red shirt becomes how, you know, the twist. And in VR it's hard to do storytelling tricks like that because you never know where someone's looking. And it could be that you're doing this really complicated sidelong glance and handing somebody something under the table, but you know, the viewer, she's looking up at the ceiling and looking at the fan and how cool it looks with the light and she misses the entire thing. So the agency issue with storytelling is something that people are still working on.
Mason: You mentioned very quickly, the idea, especially in that scenario, where the bomb went off and you had the feeling of the wind and the feeling of the heat. I mean, how much do you feel like you have to get all of the senses into the virtual reality environment? I mean, how many can we get in, and there's the haptics - there's the smell that we can do, the touch in temperature to a degree? Do you think that the aim should be to get all the senses into these environments, or should we carefully mediate what we do bring in and what we leave out in the real world?
Bailenson: I mean, I think it depends on the simulation that you're going for. So I don't think you should overfit too much and sometimes sight's enough. Sometimes sight plus sound is enough. I mean, doing touch haptics is really expensive and the devices are hard to do. So you know, for certain things, for example, when we're training a quarterback to look around a field and to recognize a visual pattern, in that instance touch isn't as important because we just need to get the visual pattern. When we are training people on how to do surgery and to hold these complicated scalpels while they're cutting through bones and arteries - haptics is really important. So I think VR in general is hard to do and very expensive. And so we should choose our battles in terms of which senses we render in order of ones that are most relevant to the transfer of the task.
Mason: Have you had any experience with The Void? I did it about two weeks ago. I remember the first thing I did was look for my hands. Then I looked for my feet. My feet weren't there, but I found my hands. And the second thing I did was try and grab my friend's hand. It was such a visceral moment of 'can I touch him, and will I feel something?'. I wonder if that, as a form of VR as entertainment, that is closer to where we should be going, where it should live, instead of in the homes where we sort of self-medicate ourselves for our VR therapies and VR meditation experiences. It should be less about this consumer play and more about this full immersive experiential play.
Bailenson: Yeah, the location based VR - as we're starting to call it - is one, I think, that has a lot of promise. There's a company in the Bay area called Nomadic, and Nomadic does similar to what you described about that experience, but what makes them special is that half of the company, they're not just tech people, they are people that know how to do popup stores. So basically they can descend upon a shopping mall and within, you know, some short amount of time, have a four room setup with, you know, the objects in the right place and the fans, and the heat generators in the right place...and they can do this quickly and at scale. And I like the model. So we did a piece for NPR when the book came out and we taped it on location in an arcade in New York City. It's called VR World. I grew up in the 80s and going to the arcade was an intensely social experience. And what we saw at VR World - it was in the middle of the day - and we had all these school children, they're on field trips, running around playing with each other. It was special and really neat. On the other hand, you know, you still have people in goggles and they're not looking at you. So even though they're in the same physical space, it's not when you went to an arcade in the 80s, you all cluster around this machine and you could kind of look at each other. And so it's, I like location VR because it brings us out of the house, but I still think, you know, we need to think more about, you know, small doses.
Mason: Have you seen the film - I know you've read the book - but have you seen the film Ready Player One yet? Spielberg's kind of imaginary of what the future of VR may look like. Because I think what's interesting specifically about the film versus the book is Spielberg's attempt to imagine the physics of how these future VR spaces will work. The haptic suits, there's the gentleman who's set up as the evil character has this incredible chair that he sits in to experience the VR and yet the final battle at the end (and this is not spoiler alert), you just have these people running through the streets with their VR headsets on and you go "hold on - nothing seems to match with regards to the physics here. Some people are on treadmills, some people are just running through the streets, some people are sitting in these chairs." I mean do you find that's compelling, or do you just think that's good storytelling and we should just reject the aesthetics of Ready Player One?
Bailenson: So I saw Ready Player One the day it came out, with all the employees from High Fidelity and all the employees from Linden Lab in San Francisco.
Mason: I bet conversation in the bar afterwards was fascinating?
Bailenson: Sitting next to Rosedale, and so, you know - had there been an earthquake, the oasis would have been jeopardized because all the actual people building it were in that room. And actually the bar conversation, we went back to High Fidelity, both the Second Life employees and the High Fidelity employees, and we had beers and whiskey and talked about a lot of this stuff. And you know, look, one of the most powerful scenes for me from the movie was right when you start out in the movie - and this is not a spoiler - that you pan down this building and you see everybody in the windows alone, in their own room doing stuff that looks really weird from an outside observer. And it just kind of begins with this is our vision of what VR is, you know, and, and it goes back to this, you know, how do you get over this "VR is isolating. And then when you're using VR, you look like an idiot". How do you, how do you get over that? And then that to me is what stood out from that movie, that first scene when you're like, "wow, that's what we were striving for".
Mason: I mean, my listeners would hate me if I didn't ask you, but what were some of the opinions of folks like Philip Rosedale and the Second Life guys? Did they look at that and go, "Oh my God, that is exactly what we need to build"? Or was it like, "Come on, there's no way that's gonna ever render in our systems, why the hell would anybody do it that way?"
Bailenson: Well, we'd all read the book and we were all rooting for it to be great because you know, the VR industry will get a boost if the movie...so really the conversation was about the response that the movie was going to cause for the public. One of the big ideas behind the book and the movie, and again, not a spoiler alert, is it's just about games and you're playing these games. And so for those that have read the book and the movie, there's a big theme in the book about VR and training and how people actually get an education in VR. That doesn't appear in the movie at all. And so, you know, for me personally and a lot of the conversations I was in, I enjoyed the movie. It was a fun, you know, just nice experience. But I wished that in addition to the gaming, that some of these more, you know, useful applications of VR would have emerged.
Mason: There's another theme in both the book and the film. This idea of escapism, and escapism can be a positive thing. In the film it's escapism from this very kind of grubby environment in which these characters are living, but also escapism is used as a term that is seen as a negative when it comes to thinking about VR. I mean, where do you sit with this idea of VR being a form of escapism? Cause I think for some people that's a good thing. I mean, we complained about these kids spending hours and hours and hours in these massively online gaming environments. But for some of those kids, that's where they find their social connection. That's where they find their confidence. That's where they find their friends. Okay, they might not be looking after their bodies and they might be sitting with bad posture in front of these machines for an extended period of time. But that's arguably where they feel human. So then what's wrong with that?
Bailenson: For my friends and family, I always recommend moderation. And you know, I do get if you live in rural Alaska or if you, you know, you're not mobile and you can't leave. And there's one of my former grad students, his name is Nikki and Nick has done more work than just about anyone looking at the pro-social aspects of these network video games. These MMORPG's. So there's no doubt you can learn amazing leadership skills in these games and you can make friends that you're gonna know forever. All that being said, it's hard to watch Ready Player One and find anything positive from that world as depicted by Klein. It's, you know, it's pretty hard to see anything healthy about the physical world itself or about, you know, the fact that the entire planet's depending on the success of somebody playing an eighties video game. And so I, you know, I see it in your face that you see some positivity there, and maybe I'm a bit older and it was just hard to see the love there
Mason: To end this conversation. I really want to ask you about your future vision for VR. I mean, what is it for you that makes an environment, or a VR environment, really effective, and should these experiences be hyper customizable, hyper social? What sort of VR would you like people to experience? Because it feels like at the moment, so many people are having terrible VR experiences. A lot of people are using losing their VR-ginity to horrible branded executions using preexisting HTC hardware plus a tilt brush software. And assuming that that is the limitation of VR, I mean everything that's in the book and everything that you do in your lab shows that the VR is so much more. So I just wonder what is your vision for the best sorts of VR experiences that we can have as human beings?
Bailenson: Yeah, I love the question and just let me say that I actually think the HTC Vive is a pretty good system. I think you can have great experiences.
Mason: I've got nothing against the HTC Vive. Just against how these headsets are thrown on people and then any old content is given to them. It's branded up as Coca-Cola or McDonald's. And then there's the assumption that, okay, this is VR and I Tilt Brush on the side of a Happy Meal box and go, "Okay, I've had a VR experience".
Bailenson: Yeah I know, I totally agree with your big premise. I just want to be clear that even in my lab we often use the Vive and the Rift and their native tracking systems, because they're pretty good. And Tilt Brush in and of itself outside of an advertising/paint the Happy Meal context is a pretty neat thing to do. But your point is well taken and most of the times when I give talks I say "how many of you guys have done VR?" And more and more as time goes on, half or two thirds will have done it. And I said, "how many guys have used it twice?", and like the one hand will go up. And then the problem is people are using either hardware that's not maintained well, where the tracking is not perfect, or the laptop can't render at 90 frames a second, or they're just doing really bad experiences, ones that make you sick. The camera's moving really quickly and you're not moving your body are ones that have no point, One thing I'm excited about - to end optimistically - one of the roadblocks in VR is content creation. And it's hard to create content. It's hard for me to create content. Thousand Cut Journey - we spent a year and a half on this, and my other pieces Becoming Homeless and the Stanford Ocean Acidification Experience; thousands of hours. It takes a lot of time. As the tools like Unity and like Unreal and all these other engines are becoming easier to program and the cost of entry gets lower, what you're going to see is more people learning how to create good content. And you know, look, if you think about movies, we didn't get to Star Wars you know, when they invented the moving picture. It took decades, right? To get to something lofty, to like Citizen Kane. But it takes a while to get to great content and I think what I'm excited about is as the content creation gets democratized, somebody's going to nail it.
Mason: Jeremy Bailenson, thank you for your insight on VR.
Bailenson: I really appreciate your due diligence. Great questions. Thank you.
Mason: Thank you to Jeremy for sharing his thoughts on how virtual space will transform human experience.
You can find out more by purchasing Jeremy's new book, Experience on Demand: What Virtual Reality Is, How It Works, And What It Can Do, available now. If you like what you've heard, then you can subscribe for our latest episode or follow us on Twitter, Facebook, or Instagram @FUTURESPodcast.
More episodes, transcripts, and show notes can be found at futurespodcast.net. Thank you for listening to the FUTURES Podcast.
