Not Exactly AR And Not Exactly VR (with Lauren Goode & Anand Agarawala)

Hey, it's Eric Newcomer with Newcomer. We've got a great episode for you this week. I have two awesome guests to talk about. This is Apple Vision Hero in the future of augmented reality and virtual reality. We've got Lauren Good, a senior writer at Wired, co-host of Wired's Have a Nice Future in Gadget Labs, podcast, you can follow her on Wired.com and she got her hands on the device. So she's a rare human at the moment. So she walked us through her experience with the Apple Vision Pro. And then also on the podcast, a non-agre wallah, the CEO of Spatial, a company that helps people build 3D worlds. He's been along for this journey of virtual reality devices and sort of has been in the waiting game of a real hit. And so he had a lot of smart perspective about the Apple Vision Pro. So we had a really fun conversation. I think you'll enjoy the episode. Thank you both so much for coming on. I really appreciate it. Thanks, Eric. It's great to be on your show. All right. Lauren, you actually have used the Apple Vision Pro. So we want to start with you. We're going to be sort of speculating and talking about it, but you have experienced the product. So yeah, what was it like? Can you walk us through the whole experience of trying it out? So I'm actually wearing the Gen 3 version right now. You just can't see it. Oh, yeah. You've got all the features. They shrunk it down. Yeah, that much. So I think it's probably worth noting that when you first try the Vision Pro, there's a preamble. Right? And this I think is going to be the experience for people when they eventually buy it. You're most likely going to have to go to an Apple store or some other kind of pop-up and get, you know, scanned for this device. Because when I first went into the super secret building that was, I later found out was built specifically to give demos of this device. When Apple employee first scanned my face and my eyes and my ears, the ears are the weird one. Yeah, the ears, I mean, you think that they're going to peer directly into your ears. It's not quite like that. They're measuring the space between them so they can map spatial audio to you. And then an Apple-employed optometrist measured to my prescriptive lenses. I was wearing contact lenses that day, but I brought my glasses with me, anticipating that something was going to happen. And so by the time I got into my private demo room to put this thing on my face, it had been personally calibrated for optimal fit and optimal, like, just vision, I guess. So then the vision pro is waiting for me on a coffee table. There are two Apple representatives there. They are not wearing headset. Your heart is like a lottery at this moment or you're like, no, you have like a limited window as a reporter, right? Do you get 30 minutes with it? Yeah, you know, it was about 30 minutes. I didn't get to record anything. Like I couldn't record audio. I couldn't take notes. I couldn't like take a photo of myself. So I don't have actual like timestamps. And you know, when you're in the reality distortion field, you're in the sport text and you're like, what is time? But I think it was approximately 30 minutes. Yeah, and I put it on and I think the first thing that struck me was actually how hefty it felt considering that they've offloaded the battery because there is that external battery pack. I fitted it like there's the soft strap at those over the top of the head, the soft strap that goes in the back of the head. I started to know as I looked at it, how much of it seemed to borrow little design elements from other Apple products, like the soft foam cushioning that frames your face looks a lot like the AirPods Max and the strap that goes in the top of the head looks a lot like an Apple watch strap. So we talked about that a little bit and then the thing was on and then I did like an eye calibration thing that was very aptly and pretty intuitive. And then it actually even sort of concluded with this like little approval click that sounds like an Apple watch clicker, Apple pay clicker something. This wouldn't be an Apple review without something like the click was really thought through it. Yeah, yeah, the chime, it was really the best chime ever. Do you want me to keep going or like, I'm like, am I getting all started to buy, start the experience. Okay. So we have to have the on top of the clock scene is just as exciting, like, yeah, like usually at this point in a podcast, I pause because it like the host wants to jump and be like, okay, so get to the boy. So yeah, so then I put it on and I think that another thing worth noting is that this isn't exactly AR and it's not exactly VR and the company is positioning it as a spatial computing platform. Sorry. I love that. Shuttering right now. He's like, what did you say? spatial. We did they call it spatial? Are they naming it spatial? But no, it's like obviously that is a non company name. Okay. That's right. Google it. Google it. Yeah. So then your face is fully enveloped. And I think Apple made a big point to say that you can see the world around you, but it's a little bit of miss marketing because while you can see the world around you, it is a video pass through. It's not like there's any sort of transparency to the lenses of the glasses, like your face isn't it? The reason why you can see the world around you is because there are so many cameras in the device that are recording and streaming to you, like broadcasting to you. This is fascinating to me. So does it look like when I'm looking at like an iPhone camera to take a picture? And it's sort of like that or what is sort of the vibe of the real world as seen through the cameras? It feels like a slightly fuzzier version of the real world. Like you know when you put on, I'm going to assume on it, I know you have, but Eric, I'm going to assume you have used a quest too. Yeah. I know. I know. But you have used a quest. Yes. We've got to make this happen. It faced the PR is listening to this right now. They're like shipping onto you at the start. I know. You guys are. We have them here. I'll bring one over. Yeah. Okay. So when you, when you put on the quest, there's a moment before you get fully immersed in virtual reality where you map your living room, you create these barriers around your living room or wherever you're standing in your real environment so that it creates like these virtual walls so you don't bump into things once you can no longer see. And so you see in that moment, this very grainy, black and white, although newer versions have color pass through, but you see a very grainy, black and white image of your living room. And then it draws the barriers. It's like that, but like a lot better and nicer in color. So I was seeing this, you know, very norm core Scandinavian apple demo living room, but I was not seeing it like I would see through glasses. I was seeing it as I would see it as like a video broadcast to me. That's weird because I know you said it didn't feel like AR, but like, I mean, I've just done too many of those past two demos, but that's usually after a few minutes, your brain kind of just be like, yeah, that's my living room now. Like, you know, I mean, you kind of adapt pretty quickly. Like in true AR devices, you're seeing the real world, his optical pass through. Yeah. So it's like you're actually seeing like you're actually your eye, like the photons are going from the couch to your eye. Yeah. Exactly. It's like glasses. And then there's images on top of the glasses, but that are just blocking it. Because this is your looking at a screen that is showing screen thinking, right? Okay. That's right. That's right. Yeah. And I'm glad you explained that because I think when I say it's not like real AR, I think that's what I mean. It's that it's not using any kind of wave guide technology that refracts light and I put it in your eyeballs. Like, it's not holographic. It's not volumetric. But it is AR if you think about the literal definition of AR as augmenting your reality. And so once you are like running computer applications into this space in front of you, where you would typically be looking at your real world living room, but actually you're seeing apps and playing games and doing stuff like you are augmenting your reality, right? So it's like conceptually AR. And the responsiveness, part of what's supposed to make this better than the status quo is just like how quick it is, right? And also high depth, right? Can you get into like this sort of quality and how much that is or isn't a game changer in your view? Yeah, I think the updates were remarkably good. You are when you first fire up the, you know, the home screen, you're looking at just a almost like a dock, a floating dock of Apple apps. I didn't see there might have been third party apps on like the next page. I don't actually recall once again, could not take notes. So a lot of us have gone off with memory, but I mostly saw Apple apps like Safari and messages and photos. And then once I opened some of them, the imagery did actually look remarkably crisp. So the optics, yeah, I mean, they're pretty good. I think that the eye tracking and gesture control is a bit of a game changer here because typically when you're wearing a VR headset, you're using game controllers in order for your hands to be able to like navigate the world around you. There's so much compute power in this thing. And so many cameras that Apple like the device scans your hands and after that initial scan, your hands are in the frame, your real life hands. And you just use them by tapping your fingers together to manipulate and scroll through apps. It's worth saying. This is like the future. It's insane, right? You're going in optometrist. It's like, I feel like every sci-fi movie, this is like it. I'm not saying that means this is fully functional. It's going to change society. We're all going to use it. But like, this is like as living sci-fi as you can get where your hands are controlling things. We're like living the world through cameras. Like, they have like basically medical professionals examining you to make it work. I just want to, it's weird. It's amazing. I don't know if that scanning stuff is that critical. I mean, like, you know, on an Oculus, like, so one of the things are doing is IPD checking so that when you get it, it just works. But like, you know, Quest has like those three settings and those get 80% of humans or 90% of humans. So I think some of that stuff, and like the spatial audio, like if you just plopped it on to Lauren's setting, I think it would be fine. You could take that, you know, that physician part out of it. But otherwise, yes. It is pretty trippy. Yeah. Yeah. And the other element of the gesture control is that the way you select apps, that's where the eye tracking comes in because there are the external facing cameras that see what your hands are doing. And then there are the internal cameras that see what your eyes are looking at. And there's almost no latency. I mean, if there's latency, it's milliseconds. And Apple has said like the threshold should be like 12 milliseconds or fewer. And when you look at an app, like when I looked at the, and Apple was like sort of guiding me and they were also holding an iPad off to the side where they could see what I was looking at. So it was very controlled demo. They said like, look at the photos app. And I looked at the photos app and it just sort of was like emphasized in the row of apps. And then it's like, look to the next app in that app. So it's just like the face computer knows where you are looking. And then you click it with your hand to open it. And then there's an expand option in apps and use your finger to like select the expand option. You can, you know, scroll through to the photos, open them up into panoramic mode. Yeah, everything you're doing because the voice control was not set up. Yeah, it was not ready. Everything you're doing is with your eyes or your hands. I think this is one of the most powerful things and most breakthrough things that they've done is like your eyes are I think two orders of magnitude faster than your hands in terms of response rate and speed and everything. And this is the thing that I find like in the TVs of all the reviews of red MKBHD said it's magical. I think it's a hyper IO device, a hyper and put up a device that was speed at which you can kind of look at stuff and move like it feels like you're just, it's almost like mind control. And like that, it might be the greatest breakthrough of it. And that's maybe why they're leaning towards a productivity applications. I think it's going to be like, you know, when you're on your phone and you're like, I'll answer this email on my computer because I want to use my keyboard. It's going to be that same level of like, I'll answer this on my vision pro, you know, it remains to be seen what you need to do all that kind of pointing and stuff for them. And I've seen some demos too that are like, you just get used to looking at stuff and it lighting up like Lauren described, right? And when you look at icons, they light up. And then when you look at the real world, when you take it off, it's not going to do that. And you're going to be kind of like, oh, you're going to want that super reality. It isn't really interesting. But I think that like that eye tracking aside from all the other stuff, I'm going to want that on my Mac, you know, without the headset, you know, like I think that's a potential real breakthrough that's maybe hidden in here. And that fits with sort of like the launch of the iPhone where it came with this, you know, gesture control that was so perfect and human that it felt like, okay, this power is a new generation of devices. Now it gets swiped your TV, right? Like it's just used to it. And people have tried that on the desktop before you might recall many years ago, there was a sort of called leap motion that was doing really interesting stuff around gesture control on PCs. And then there's another company called Toby that does eye tracking. And I think those were really cool technologies in search of applications, like maybe they would be good in games or maybe they would be good even for like consumer surveys to figure out where people were looking on their machines and like what they were gravitating towards. But I think with Apple, like with anything with Apple, there's so much validation that comes with Apple entering a market and the technology is so good that all of a sudden you're like, oh, this could actually become mainstream in some other way. And your eyes are like the best indicator of your intent, right? Like even you're not even aware of where you're looking, you know, I mean, think about the ramification to ads and, you know, it's just fast. I mean, it is really much closer to your hands. Your intent of your mind is like two degrees away in your hands, but it's like one degree away in your eyes. You know, I mean, like it's much more automatic. Right. So then there's the other thing that I thought was pretty, I don't know if I'm called stand out but very different about this was the ability to dial back the opacity of the immersiveness. Like you could see the living room and then run, you know, a little window of an application in front of you or you could twist the dial and then just go almost full of the R. And when I saw that, I thought, oh, to me, this seems like it's something that's geared towards app developers because then an app developer has the option to say I'm building a VR app or an AR app or something in between. What did you make of that as someone on the app developer side? It's interesting. You know, the magic leap, a ship to the reality button and they had a similar intent. They didn't work out that well. Like so you click the button and the ideas you go back to reality. So much more rudimentary. And the thing is a switching time was so long that it like because they thought you'd be wearing this thing all day and you want to be popping in and out of reality. So it's an interesting through line to this switch. I think like it's cool. I think you need people to feel really grounded and be able to like someone just walked in the door and you need to say hello or you know, you're probably just jumped on your lap. Like you need that. I don't know if the dial, I mean, it will be nice. I'm sure it'll be nice and aptly as an app though like you can either dim completely your experience down to reality or you can dim parts of it. And so like for us like in spatial like the avatars like for you can example, you can imagine an experience where you dim just the environment, but the avatars are still there. You know what I mean? And so that like I think developers could potentially use it in intriguing ways. Like I'm watching, I could be in that space, you know, you're watching the Mandalorian on that crazy spaceship that they showed. And hopefully my friends are with me and I dimmed the thing because other people are in the room too. And now I can see the friends still. I still do Mandalorian. I think like if you do selective dimming, I think it could be quite intriguing. It's also like you definitely need that switch back and forth reality to be really smooth because it is on your phone. To me the big question here is like is it enough of an AR device? Is it enough of a VR device or is it this weird like half and half? And I think like the dimmer sort of gets to that. You know what's interesting? They showed almost no VR mode at all. I can't even remember a single real time 3D thing they showed except for these immersive panoramas and like even watching the movies like even the 3D dinosaur thing which everyone thought they were going to do in 3D was like a pseudo shallow depth Tyrannosaurus right? It wasn't like look behind you. There's like a thing. You know what I mean? It's not a VR device as it was positioned. I think like they didn't show a single game. I experienced a little bit of that in the demo. So when I pulled up I'm trying to remember which video clip it was. There were a few video clips that were like set up in advance. You know once again this is a very controlled demo. I watched Avatar 2 in 3D. I watched this sizzle reel that Apple put together of all of these incredible nature scenes and then I watched the John Favreau directed dinosaur clip. So at one point Apple said looked at the left and there were these pre-crafted environments that I could select with my eyes and hands and one of them was called cinematic mode. So I was watching this you know this video content in front of me in the you know the living room that I was sitting and then when I chose cinematic mode it actually made it a lot more immersive. Now there were some edges there there at least like we're kind of wavy or floaty edges. So it's not like I felt like I could look up at the sky or look behind me and just be like completely in the world. But it was like very very close it was up against the edge of full VR I would say. But you're not walking around right like it's kind of it's basically 360 video I mean not 360 360 360 360 stereoscopic video and there's 28 million pixels I think that it's speculated they don't have the processing power to actually render real time 3D in a way that you can move around. Like can you game on this device like that that is a big question. I didn't do any and there were no gaming maps in the demo that I had it was either like some facetimes and messaging although I couldn't even message back with a person some web browsing and like video watching there wasn't anything around gaming and a video like one of the things that I experienced in the air headsets that I've tested like even like snap spectacles and those are only available to developers and like the only last a half hour or so in a heat up really quickly like but they do like pure wave guide AR and they also do spatial anchoring. So like you would draw this like artistic squiggly into the world in front of you and then it would be anchored there and you could walk up to it you could scale it up you can walk around it you can manipulate it with your hands there were objects floating in space that you could just manipulate and it lists really cool way and like I did not experience that in the Apple headset. In the gaming front I mean so one of the developer labs showed apparently I didn't see me yet but one of them saying the rec room in 3d and so that was like I think immersive but it also raised the question what's the control scheme for these games right if there's just hand tracking like you know PlayStation VR and not Quest have joy pads with multiple buttons so you can run like we're just asking how are they going to do like I get how you do shoot in rec room but how do you do run how do you do jump how do you do duck I mean you could do those in real life it's going to get fatiguing real quick so you need controls you could imagine a gesture scheme where you do that. There were also like some of the games were actually in 2d mode and you know when we're chatting the guidance to developers is start with your iPad app and then you know that's kind of the 2d frame and there's even guidance on like okay well you can use your 2d iPad joystick with your fingers which is kind of an awkward way to play. So is it a productivity device like you think it's built it's like really set up to well I guess to like do zoom type things and then to watch movies are like what was sort of the one to use case that they were sort of painting for you in this. I kind of felt like they threw everything at the wall with this frankly the messaging was a little bit muddled around it even the demo video that they showed during the wwc keynote showed like every example in the book right there's the person on the flight with it there's the dad browsing memories there's the woman who's doing work and then goes down to the fridge to get her sparkling water it felt that way in the demo too it was like here's a little bit of messaging here's a little bit of FaceTime here's some entertainment also you in a colleague can collaborate on this design thing and that to me signals that they don't really have the killer app quite figured out yet and actually in one of the comparisons that I've been making in conversation with folks is that it very much reminds me of the iPad in that way I think everyone's been saying oh but what about the iPhone you know if you're down on this right now maybe you're a lot I because look people were skeptical about the iPhone and look how that changed the world I don't think that that's the app comparison I think much in the way that the iPad it's mostly a home device some people are power users and might travel with it or some people might use it as their primary PC but it is very much a consumption device it sits at home you give it to your kids sometimes occasionally you do work on it it's a little bit more of an isolated experience right like you're using it yourself for the most part sometimes it casts other content I think like this could potentially if it ends up being as successful as the iPad then that will be a success for this product because it is it is ultimately very hard I think to convince people to wear a face computer for an extended period of time I think a non and I were talking and sort of our prep call about like just like getting people to use these VR devices right they like use them a lot in the beginning and they fall off and nobody uses them anymore and it's just like a phone is much easier to grab than a VR device and so to me I don't know I I feel like my view is much more like this is an all or nothing thing either they get it so that people really want it to put on it's magical it feels like way ahead of everything else and so then people are putting it on and then people get word from their friends like no I'm like in this thing I'm doing XYZ for a long period of time or like it's early adopters you try it don't find a sustained use case and it's not even to iPad level like I don't quite see this sort of middle case personally I think I can see both I mean I think the iPad thing like you bring your iPad to their plane because it's just way better there now quit you bring this to the airplane you know that's probably the number one place right I think that the that's about first class envy there like oh my god yeah 500 dollar device in first class wearing these things like that is going to be that'll be the symbol of like the class divide in America well the morning after the event I got a text message from a CEO not a big tech CEO but more of like a lifestyle CEO who just wrote in all caps I will not wear the goggles on a plane I thought that was interesting that's the wally moment be lying flat in first class with headset like being entertained and being fed it's gonna have it's that's the future it's not the sign to you know but I do agree like my number one learning after working a VR for seven years day in day out is like the transformative moment will be like when people actually shift to this device it needs to be an all day wearer device otherwise there's a switching cost and the switching cost is always going to be way higher than your phone which is in your pocket and it's so easy we can't even switch to our watch because the phone is so good and this is like honest the whole time right like that is the key I think and that's why the thing I was most excited to hear from more you know and folks who tried it is can you use it like all day can you imagine doing that and I'm hearing not so much like joy in a certain space is quite red after her demo and they said it was the fitting or whatever but I just can't imagine I mean I can't wear ski goggles all day and they have no computer them so I think you've got to go from goggles to glasses for this to be that kind of mainstream thing and that's the path we're on and this is like a huge injection of energy and Apple you know sex appeal this is the Ferrari the Tesla Roadster of that category right and there's an arms race now between two trillion dollar companies trying to win your face and so that's great that's great for the consumer it's great for the developer Lauren how long do you think you could wear the device I know you only got like maybe this 30 minutes with it but did you feel like at the annual fatigued or you felt like no I'm comfortable I could keep going you know I thought I could keep going actually I have worn the Maddo quest 2 for I don't know maybe as long as an hour before some friends and I used to play Beat Saber together we were all in different cities and we would you know put on our headsets and toss up and play Beat Saber and like beat his little disembodied avatars floating in the metaverse and that was fun this setup was always really annoying like especially if you hadn't used them at a quest 2 in a while then you picked it up and put it on there was almost inevitably some update to download and then you'd be like wait how do I navigate this thing with this dorky hand controllers again I don't think I ever used it for more than an hour and I always say that when I take something like that off it feels like my forehead is breathing a sigh of relief right I'm sure after an hour or so in the Apple headset I would feel the same way I mean maybe I would get so engrossed in a movie that I would want to keep wearing it for the full duration of a movie but I could also picture at some point being like you know what I really just want to take this off and watch this on a big flat screen because maybe there's a little bit of depth here maybe there's a little bit of interactivity like the dinosaur was interacting with me or the butterfly was interacting with me but once you're over that novelty you know what you're reminded me of is like the first days of covid when you spend eight hours a day on zoom and how fatiguing it was and when you're done you're like oh I guess like that's kind of the same thing on 2d experience but you can last eight hours you know yeah yeah that's true I've done days where I've like spent like when we've been pitching or whatever talking to businesses like four hours five hours in a haul lens like it's exhausting for sure you do lose yourself in it when you're in there because especially if there's another human in there with you because you're just kind of like you know talking but it's tough oh my gosh I can't believe you've spent four or five hours at a haul lens did you get like a special award for that that is like my god I wrote a story for wired ones that was just entirely about how long it took me to get set up for a meeting in haul lens like it was so comically bad that required meetings upon meetings to anyway kudos to you on and I'm just curious like when you mention this inevitability of shrinking the form factor like you think that we're headed to glasses I think that would be great because they would be a lot more lightweight but I just think the physical batteries are still like there's only so much we can do around innovating lithium ion batteries as long as that's what we are using in silicon is being innovated on in such a way that it's become a lot more power efficient and that helps with battery life but like we are up against like physical and chemical limitations with batteries so as long as we have those are we ever really going to get super lightweight powerful glasses yeah I think okay the wire that is fair and I think the interesting thing about this is that they say it's all day where they actually said that and meaning and I'm sorry well but I want they mean by that right but but the quest pro you actually can't run all day if it's plugged in because it draws too much but you know like they so yeah they fix that and the thing is like this so Apple fix that yeah right yeah vision pro there's some fairness in the claim given the quest can't deliver all day even plugged in quest pro okay the thing is they've externalized the battery in this case so can you imagine a pair of glasses and real x real now is a couple years old and that's like glasses e you know you could imagine an Apple caliber device delivering something like that with external battery and we're not going to solve the battery problem for a while except for like kind of Apple watch scenarios where it's like episodically on during your day but I can imagine a world where like that's maybe not too far away maybe it's you know like where you have a x real style chunkier glasses with a wire it's going to need a wire you think people will just get used to carrying around the battery pack if the glasses make it worthwhile enough well like then what's the world in which we're carrying around phones and we have a watch on our wrist and we've got this battery pack I'm aware in glasses like what's the benefit then to the glasses that you can't just do on your phone but you know what's crazy Apple has made us all carry around headphones okay who carried around headphones before they're winning the pocket who would have thought we'd be carrying around that and this the watch you know there's so much crap we're carrying for them if it delivers value these deliver AirPods deliver a lot of value so that's the question the other thing I've learned is that the phone is so damn good and delivers so much value so quickly and it's just really hard to beat so the bar is really high to replace it really I think that I feel like optimistic about some of the technology that apple has just introduced especially considering it's the first version I still feel very skeptical about the market the category the product as a category I know you have to run around like a shaman who's seen the future so your schedule is very limited so we will let you go soon but like another writer wrote apple vision isn't the future do you agree with something that straightforward or what's your prediction here yeah that was Kate nubs is great piece for wired and I have to say I give Kate a lot of respect because I think that a lot of the tech press is like dancing around the question and maybe being a little bit nice because I think that you risk a lot less when you are optimistic and writing something positive about an apple product because it's apple and they typically deliver like you're gambling a little bit less when you take that position versus hey this is going to be a flop and this is not the future and so I give a lot of respect to Kate for writing that I think there's a real possibility this is a flop I'm not saying I'm calling that right now I think I tend to think that it's going to find a niche market and that apple will find a way because it is an operations company as much as a computer company to eke out margins on it or find that niche or find a way to make it valuable to some stuff that it's consumer device the vision pro they're going to sell so few of them that they're always going to be able to claim that they don't have any they could just sell like one a day at every store and just never have enough and never run out you know I mean it's such a obviously niche product that I think I feel like the bar it has to step over is so so low yeah but as a category people have been talking about VR headsets for like literally decades at this point and by the way I would say that the people who have been prognosticating on this are predominantly men who have these like techno utopian visions of such things and and like it hasn't come to be now like granted transit love you guys it's so singly on this podcast it's true also I mean I don't know I'm you a techno utopian Eric you're a journalist so maybe not I'm optimistic about the matter that would be the second half once you go we're going to go into full tap techno utopian and we're just going to be over here like you're already they don't know anything like what you're not here to really shit on the idea we're just going to go wild you know but I know what I will tune back in and listen to that later on for sure I want to hear it you have to say I tend to think that this is something I still think that there's been that vision of let's do face computing for a very long time and there is a reason why the majority of the population has not opted to spend a lot of their time hours of their time wearing something that cuts off some of the most personal sensory organs of your body like even with Apple owning our pocket is on unput it so well when you're wearing the AirPods that is augmented reality but you can still see and smell and all that stuff and actually there's like even pass through hearing when you're using the iPhone you're still in the real world even though you're digitally connecting with all of your family and friends like when you're using iPad you're consuming entertainment but you're in your bedroom and you're comfy bed and you're like I just think like putting something on our faces for an extended period of time is a really big ask yeah I mean just to you know if you said the Tesla wrote when you saw the Tesla road street you said this is this is not the future I think that would be quite shortsighted like of course not everyone's going to drive a Tesla roadster but EVs like it was the gateway drug to EVs right but EVs are solving such a huge societal problem which is how do we make this particular category of transportation sexy enough so that people are less reliable on internal combustible engine vehicles that are contributing in major ways to our climate problem well you have to lie to me and that I can't no but vision prosol no but I agree with that but people don't care about self I mean like just to be cynical like people don't really care about solving the problem the care about convenience and luxury and you know what I mean like they also made it wasn't a Prius it was like it was sexy right like the other thing just to respond to something said earlier it's like I get a bite my head off is it like I smiled before I bite something like a shark the like you know you mentioned like that is there any use case because it's like there's a whole bunch of stuff there's living room there's like you know how would you describe I mean it's the bull case like that there is I'm skeptical too but the you know the bull case is how would you describe how you use your iPhone there's a lot of different stuff you know like if you're describing to an alien species you know like what would you say like he's a social he's a for email he's a for browser it's the everything device it is so it is a general purpose like I think in some ways it's okay part of this is like as VR developers we never had this caliber of device before to experiment with so in some ways it's a development platform now that's what I'm really curious about are they going to see these devices like water like magically been quested to stimulate development because they're really expensive and the most devs won't be able to afford them and they're taking a really big and the economics won't necessarily work out to this something we talked about earlier is like the economics that work in VR today are on quest there are companies that have made tens of millions hundreds of millions and those have to charge twenty nine ninety nine out of the gate as a one time fee because why because you don't you know the retention is kind of low in these not none of users an engagement to do freemium and recurring payments that business models the venture capital or one ninety nine like when can you do a one ninety nine app on a headset like you need crazy volumes for that so that is a bit out that's a really great point at this point on end if you had limited resources and you had to develop for either the quest two or three or this apple vision pro which would you choose well the good news what we're doing both and the good news is for most developers it'll be relatively like if you're doing a 3d volumetric app like if you're 3d developer if you're doing 3d volumetric app like a a rec room or a spatial or something it's quite easy to like we already support AR kit you know like our app already sports AR kits so we already have both you know and so it'll be relatively easy for people to pick both and as we talked about yesterday I think these will converge a bit like I think probably quest will have a high end device and a low end and they're kind of you know like the quest pro and their quest three or whatever and apple is already rumored to be shipping every day you know five four five hundred dollar device as well so I think like it's pretty easy to support both there's nothing that like aside from the eye tracking that like you know and that's pretty I mean quest could be you know definitely support that as well so I think both the answers both because of unity it's not expensive to develop for both well I look forward to trying spatial on apple vision pro and I think you just made a really good plea here if like jaws or Tim or listening that you guys should send a free kit to a spatial over there so but I do have to run thank you so much for having this one has been really fun enjoy enjoy the metaverse we will see yeah great that was so fun I want to just like is it enough VR or enough AR is the thing I keep coming back to you like you've got me really worried that it can play video games which just feels like such a key yeah early adopter use case for a product like that and would be classic apple history to like underestimate the value of video games it is very intriguing that they really didn't show any gaming use cases on that headset and it's an entertainment device and then secondly they did announce the game port kit I don't know if you notice that but where you can easily like they're allowing developers who built PC games to much easier bring them to the Mac direct translation direct X APIs using wine kind of thing so they are getting game they're trying to improve the Mac they change some settings so that you can move into like a gaming mode for your higher performance gaming mode and stuff so there is energy in Mac gaming for Mac for the first time in a long time the funny thing is the iPhone is the biggest gaming platform in the world so it's like now yeah I mean I think that it was probably a conscious choice maybe it's a positioning thing versus meta because meta is so hardcore the games now the games is the one vertical that's proven games and then fitness you know to a much lesser degree but those are the two verticals that are really shown to take off for VR and you know part of it's I don't know if they've viewed as a VR device as much right can you you know give a little bit of your history with ARVR I feel like we sort of jumped in understandably for the conversation about like what's this device like please tell us you've seen you've been to the mountaintop like I'm down with like you know tell us but just like so if you will have context in your sort of journey sort of I feel like true believer but then also sort of exhausted by how long it takes to like oh man I've got to do like more than just be a true I don't know how to charge it out that is a fantastic question yeah so I've been working on 3D user interfaces for like 20 years first company called bump top 3d desktop user face that used a physics engine back in 2004 it was my master thesis it was like a new TED Talk on it sold it to Google in 2010 and I was doing 3d interfaces on a 2d screen fully 3d interface on a 2d screen on 2d touch screen and then spatial and so I was part of the andra actually maybe it's worth giving the context as part of android starting at in 2010 when it was under 100 people the team and version of Claire Froyo and then watching and then you know all the way to ice cream sandwich which was like expanse of growth these are all operating systems yes oh you don't know I don't want to do this stuff it's it's it's it's there letters of the alphabet so it's like E Claire F Froyo I ice cream sandwich so just to give you a little you know thing yeah now they're just numbers which is way more boring but because back then the team would get ice cream sandwiches when we finished the release stuff but anyway that epic expansion of mobile when I left Google I was like yo that's gonna like I tried the haul ends I felt like I was looking into literally a tunnel vision to the future and that there was a huge opportunity software opportunity to get in there early and that wasn't 2016 when I started spatial with my co-founder Jenna because it was like hey building 3d software is hard and there's very few people have that skill set that can combine kind of the 3d graphic stuff you need which you probably develop those that skill sets typically found in gaming but then also the UI and design expertise you need for like making stuff sensible to the you know common person so we started and I anticipated like a lot of people that the headsets would have been ramped much closer to the mobile ramp right like mobile ramped crazy quickly to get to like a billion devices I think right I think it was like five years from like one million smartphones to one billion and so you know I think a lot I mean you know VCs were investing too like Facebook had just bought an Oculus oh yeah there was a ton of excitement yeah the haul ends two was out right you know like a haul ends one was out like there was it was it was like fast times I mean that's a challenge of any sort of I mean start up you know you want to time the sort of early in this sort of wave up so the year in the right position when things but you don't want to be too early where it's like oh it didn't take off at the pace we wanted and that's okay when we started in 2016 we thought 2019 2020 for an Apple headset and it was always people make the joke it was always two to three years out and every year to kind of push out I think the problems were physics hard to get it right and you know I think Apple definitely wanted to launch glasses not goggles and you know just not really possible to the type of quality they wanted so that's the challenge you know and I think it didn't accelerate as fast as folks some people will be like I told you so it's going to take five to ten years but you know the bulls were out there for sure and thought we could have gotten further faster so that's the spatial moves to be more like you're also going to deal with flat screen regulars yeah I mean so we are one of the few platforms that so we start off on haul we basically went to bigger and bigger platforms start off on haul lens which is $3,000 so we've been there this is you know this is this price point and then COVID happened and then we support quest two and so our usage grew like 50x because the market was so much bigger for quest two and then we supported mobile and then we grew again and then we supported web and then we grew a ton like 100x basically from where we were before and so now you can join spatial and the cool thing is you are a first-class citizen on all these devices whether you're on mobile whether you're on web or whether you're on VR and basically our usage is split across these platforms and no surprise majority of our platform 9% of our usage is web and mobile and then you know sub 10% for game developers to build like experiences games for people to play around with on these different devices so anyway you're deep in you've been on this journey you've sort of like okay wow we can actually get a lot of users on web you think so like are you fatigued of being optimistic or what's your like having been through that sort of journey and live through this and like hope for the future a lot like where are you right now with Apple Vision Pro it's that's a great question I think you'll be seven years in October on the official journey of you know just an EA or VR company we gave up on waiting for the perfect headset about halfway through because every years like this is the year it's gonna come it's gonna come and like it was just like all right screw this we got to own our own destiny we got to go where the users are we can have a door we can have a foot in the future but we've got to work for the now or start up we have to live and survive right and so i'm still like i think also after i don't know how many hours i've logged in the headset at this point i mean probably thousand plus you know like over you know five six years or whatever and watching the keynote on monday was really inspiring to be honest it was like a whole team after was done was cheering like we just sent this in the mood like we almost it felt like we were launching something and there was a lot of our stuff that like you know we'd seen a lot of a stuff that we built in the lab and because we kind of grown a little detached from it not to attach but like it's like hey the collaboration use case is gonna be really tough to make work in this environment in this in this kind of state of the market but it's such a sexy device and i we all can't wait to use it you know it's funny we did an internal poll about 20 people answered who's gonna buy one yes no maybe two people said yes think like and the rest were split between no and maybe wow that's not very optimistic if you're sort of a company this sort of plays in this right well the price of accuracy so if it's four five hundred bucks then it's a different equation i mean that's like a lot of money we did the math you know like a MacBooks like let's say three grand you use it roughly let's be optimistically three years you're roughly paying if you use to five hours a day a work day you're paying about a buck and hour to use it which is not bad and you know a MacBook will solve your problem where this you're definitely gonna have a MacBook two you know you're not you're gonna need like a computer right i mean yeah i mean but i'm so energized and excited like vc's are reaching out and like all these people like press are reaching out it's like there's never been so much excitement and i think like because of how mainstream apple is like my sister's reaching out and you know like i love this idea that your sister stop talking to you and i was like oh my god apple like i have got that brother of mine he's sort of always playing around with his stuff no but you know what i mean like the person like the brista knows about it and i mean brista's like like like this is brista ready you know what i mean like it's just yeah it's intriguing how mass market apple is and how can capture the imagination and that fanboy community the fanatical apple community can help get the product through you know to the promised land i think i feel like a lot of the argument is like okay they have this device this is the pro i mean they expressly branded as pro and then it's like dot dot they'll have the air obviously it'll be cheaper but like you know my view has been that like the actual quality of the experience is like what's so key and been the barrier and then just like you know my tv is sharper than some of these VR devices or whatever like those things really matter so the question is just like will it degraded air version of this be sort of a mass market device if it can't even deliver like the pro level experience which we're not even sure the pro level experience has the graphics processing and everything to do what people really want so are you optimistic about like an air version of this device yeah definitely man i'm optimistic about any apple device they're so good at hardware i mean i'd like i'm not like okay okay guilty is charge of an apple fanboy but they've got the goods they launch the iMac the iphone the ipad the ipod they're so good at hardware at the MacBook like the AirPods each one of these is a multi billion dollar industry on its own right but i think the other thing is about this strategy of going hyper high-end as the technology curve goes you a lot of that stuff you'll be able to trickle down or you'll learn stuff about what you built on the high-end that you can actually turns out maybe you can't get it in the four five hundred it's like model three model X right i totally agree with the test the metaphor and you know apple's been doing that for a long time and that that makes total sense to me and i think people fixating on the price it's expensive it's a niche it's a niche sort of product at the moment it's a suck not everyone can afford a Ferrari man it sucks i mean it's a it's a bummer that technology has to be that way where the first gen is not accessible but unfortunately that's just how tech works but you know computers used to cost 30 thousand dollars so it will get cheaper you know right and the beauty of it is at the end of the day they want to be able to sell a device that they can make money on to everybody yeah yeah that drives the prices down and then and that's just sort of how this and i think what i like about their strategic positioning is that like we've seen the devices you can get for five hundred bucks we're not satisfied with those and we don't think the public is satisfied with those and so we need to like go to the next level and yes it's going to be incredibly expensive but we hope we can convince the public with like that device that like the the the vision pro today is maybe what the quest six will be like you know at that quest six price point you know i mean at that in that number of years and so they're trying to do that today and see okay what in use cases that enable they're trying to really accelerate that path the future i wanted to pull up like mark zuckerberg's comments about division pro like and gave you to react i mean he you know apple announced the headset you know i was really curious what they're gonna ship he hasn't played with it yet and he says this mark zuckerberg from what i've seen initially i'd say the good news is that there's no kind of magical solutions that they have to any of the constraints on laws of physics that our teams haven't already explored and thought of they went with a higher resolution display and between that and all the technology they put in there to power it it costs seven times more and requires so much energy that now you need a battery and a wire attached to it to use it they made that design trade off and it might make sense for the cases that they're going for and basically saying you know we want to make things for everybody you know we have different philosophies they sell sort of elite stuff and like we feel good about this i mean do you think this is good for a facebook in that it validates the category or is it bad for facebook in that it's a potentially better product that eats this lunch i think it's good for the market at some point of maturity would need all like all those big tech companies to get involved right like if you're just the one company doing it that means it's probably not a big enough market right and so i think that like it absolutely validates it people get it now i think especially let like now that there's two trillion dollar companies i think there's a lot less people you have to answer to in terms of like wall street and stuff it's like okay now it's a competitive thing now we can't lose i think like the positioning is interesting you know about the like lower price point thing you know i think the social stuff like i get this like hey this is not the kind of device i want to have but like there will be social on right and i sort of trust apple more to get like the human to human things and i don't know mark sucker berg's company necessarily but he can tell himself whatever he wants i mean yeah apples grade it like i don't know the human feeling like human company like this was built for a person they understood you know like i feel like they get that so i i don't quite see uh would suck saying on that point but certainly the pricing and the accessibility and sort of their strategic differences yeah i mean this is i think this is great i mean i think also i heard on was it i think this retiree podcast that like mark loves playing strategy games and he hates to lose very competitive and so this will be very intriguing this kind of will double down as resolve you know to really you know amp up you know actually wasn't it was ronnie saying that ronnie out of it's the co-founder magically because he's met both tim and and mark it's it's it's gonna be really exciting i mean it now it will value the thing though is that like on the cost the pro quest pro is very expensive and so like i don't know which is a uh thousand twelve hundred something like that so that's definitely not an every person's device right but maybe i mean it is great that they are providing something in a more accessible price point although now it's went up in price they're not subsidizing it as much it used to be two ninety nine which was incredibly cheap and you know they're done their kind of sensitivity studies were like what price do we have to get it where it's like you know it's intriguing you know there was a period i think it was during the pandemic where if i like the metaverse was going to be here tomorrow and like we had that whole discussion it can become a semantic argument and i'm guilty of sort of the semantic games where it's like you know fortnight you are a perpetual character and you can go to a concert and that feels metaverse like can we expand that out but i just wanted to take stock you know now that the metaverse hype sort of died but now we're sort of like back in this sort of like world again with this device like how do you see this the metaverse conversation they had and what's your view on to i think coming to be a little cheesy paraphrase the metaverse is already here it's just not evenly distributed right like i think we live in a 2d metaverse the average american spends 10 hours a day on a screen i think of roughly four of those on a phone four of those on a laptop and then two of those on a tv so like that is a lot of 2d time now and what i mean by that is we live in a 2d digital we spend a majority of our day in digital experiences all right and they're 2d and your future computer will be a headset it's really kind of a matter of time and so when the device is 3d by nature those experiences all become 3d by nature and so netflix will now be i look on my couch and i'm watching i've seen my friends with me and we're watching the game together you know or or the listener is going to be shouting right now you sort of did a dot dot dot on the the future is going to be a headset right i there are i see obviously that why that could seem intuitive but there to some i think there's a perfect reasonable thing that like if you believe in ar especially if you believe like oh man continue to access to the real world is like a valuable thing of this device it's like whoa let me tell you what's a better version of a are like everything else besides your screen is still the real thing you know i mean it's like in some ways like a screen is a great solution to a r because it doesn't block you off from the rest of the world and the part that you want to be a screen is there you know like okay i remember when i was a kid and like i watched like futuristic visions of the future and maybe like i remember like reading i think popular science or something like that and they show like you're gonna have your wallpaper is going to be tv screens and you're going to be like you're going to have like the menus will be you know tv screens and stuff and there's no way like this is back in the crt where the fricking tv's are massive and stuff like the big zenith boxes i'm not that old but like you know what i mean they were chunky and now like restaurants give you an iPad for a menu sometimes or like there's tv's just plastered everywhere right so the digital world is kind of like eating you know like the metaphors is eating you know like software eats everything like the you know the hardware or you know it's kind of eating the world too in that sense so i hear your pie i shouldn't have said goggles are the future like but i do think glasses like if you think about it okay here's the question if you could have a pair of glasses that can be the exact same or maybe slightly bigger than your current laptop to your eye or your phone you look down your hand you see the phone and it's just as cheap and light and it's always on would you wear that probably yeah yeah i mean i'm a believer so then it's like when does that happen when does that happen right why don't you you want your screen on demand if you can project things out like if if it's lightweight i mean the problem you know there are these physics questions as mark zuckerberg alluded to as like apples clearly run into you know their device uses cameras it is not sort of optical so there are these like intractable physics problems that have been a real barrier those are hard for glasses yep those are hard i mean i think we'll get there i don't know it feels like inevitable i mean it's just a question of when right is it like five years ten years i mean the contact lens stuff is kind of crazy too if you see the doctor people well just that there's a company i forget what it's called now but they've got like a really compelling contact lens display where you can actually get i think it's like maybe five twelve by five twelve pixels or something i mean i'm just thinking like let's listen to some podcasts they're talking about emulating from the founder of naughty dog Jason Rubin it was bosses boss the future podcast the you know the interesting cto facebook yeah so he was just he was just on early this week and he said that like he played em you he's a founder crash bandicoot naughty dog and he said he plays playstation on emulation and their resolution was like five twelve by three eighty four or something crazy so like we have four k now in your hand and so just looking at the arc of tech like right it's kind of insane you know like the stuff we had where kids like literally dot matrix printers right and you know i mean and things are exponentially getting faster in terms of innovation right so like chat gbt got to a hundred million users in months so things are exponentially and they do kind of compound so i wouldn't bet against they're not being glasses you know in ten year maybe twenty twenty for sure i think twenty is a for sure twenty i mean it's the classic thing where what we're bad at the vacations what are the vacations i know i love to see them well it's been great having you on the podcast super exciting we all yeah we're gonna have to have an experience for ourselves that's our episode thanks so much to lore and good and a non agriwala for coming on the show on american newcomer has been the newcomer podcast shout out to Tommy here in our audio editor Riley can sell out my chief of staff young chomsky for the wonderful theme music subscribe at newcomer.co i really appreciate our pain subscribers and like comment subscribe on youtube and please review us on apple podcast thanks so much and see you next week good bye good bye good bye good bye good bye good bye good bye