Take our user survey and make your voice heard.
Image: Yagi-Studio/iStock

Editing memories, spying on our bodies, normalizing weird goggles: Apple’s new Vision Pro has big ambitions

By Luke Heemsbergen

Apple Vision Pro is a mixed-reality headset – which the company hopes is a “revolutionary spatial computer that transforms how people work, collaborate, connect, relive memories, and enjoy entertainment” – that begins shipping to the public (in the United States) later this week.

Critics have doubted the appeal of the face-worn computer, which “seamlessly blends digital content with the physical world”, but Apple has pre-sold as many as 180,000 of the $3,500 gizmos.

What does Apple think people will do with these pricey peripherals? While uses will evolve, Apple is focusing attention on watching TV and movies, editing and reliving “memories”, and – perhaps most importantly for the product’s success – having its customers not look like total weirdos.

Apple hopes the new device will redefine personal computing, like the iPhone did 16 years ago, and Macintosh did 40 years ago. But if it succeeds, it will also redefine concerns about privacy, as it captures enormous amounts of data about users and their environments, creating an unprecedented kind of “biospatial surveillance”.

Spatial computing

Apple is careful about its brand and how it packages and describes its products. In an extensive set of rules for developers, the company insists the new headset is not to be referred to as a “headset”. What’s more, the Apple Vision Pro does not do “augmented reality (AR), virtual reality (VR), extended reality (XR), or mixed reality (MR)” – it is a gateway to “spatial computing”.

Spatial computing, as sketched out in the 2003 PhD thesis of U.S. software engineer Simon Greenwold, is: “human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces”. In other words, the computer can interact with things in the user’s physical surroundings in real time to provide new types of experiences.

The Vision Pro has big shoes to fill for new user experiences. The iPhone’s initial “killer apps” were clear: the internet in your pocket (including portable access to Google Maps), all your music on a touch screen, and “visual voicemail”.

Sixteen years later, all three of these seem unremarkable. Apple has sold billions of iPhones, and some 80% of humans now use a smartphone. Their success has all but killed off earlier tools like paper maps and music CDs (and the ubiquity of text, image and video messaging has largely done away with voicemail itself).

Killer apps

We don’t yet know what the killer apps of spatial computing might be – if any – but here is where Apple is pointing our attention.

The first is entertainment: the Vision Pro promises “the ultimate personal theatre”.

The second is an attempt to solve the social problem of walking around with a weird headset covering half your face. An external screen on the goggles shows a constantly updated representation of your eyes to offer important social cues about your gaze to those around you. Admittedly, this looks weird. But Apple hopes it is less weird and more useful than trying to interact with humans wearing blank aluminium ski goggles.

The third is the ability to capture and and relive “memories”: recording and playback of 3D visual and audio from real events. Reviewers have found it striking: "This was stuff from my own life, my own memories. I was playing back experiences I had already lived."

Apple has patented tools to select, store, and annotate digital “memories”. These memories are files, and potentially products, to be shared in “spatial videos” recorded on the latest iPhones.

Biospatial surveillance

There is already a large infrastructure devoted to helping tech companies track our behaviour in order to sell us things. Recent research found Facebook, for example, receives data from an average of around 2,300 companies on each individual user.

Read more: Explainer: what is surveillance capitalism and how does it shape our economy?

Spatial computing offers a step change to this tracking. In order to function, spatial computing records and uses vast amounts of intimate data about our bodies and surroundings.

One study on headset design noted no fewer than 64 different streams of biometric and physiological data, from eye tracking and pupil response to subtle changes in the body’s electromagnetic field.

Your face tomorrow

This is not “consumer” data like the brand of toothpaste you buy. It is more akin to medical data.

For instance, analysing a person’s unconscious movements can reveal their emotional state or even predict neurodegenerative disease. This is called “biometrically inferred data” as users are unaware their bodies are giving it up.

Apple suggests it won’t share this type of data with anyone, and Apple has proven better than most companies on privacy. But biospatial surveillance puts more of ourselves in use for spatial computing, in ways that are expanding.

It starts simply enough in the pre-order process, where you need to scan your facial features with your iPhone (to ensure a snug fit). But that’s not the end of it.

Apple’s patent about memories is also about how to “guide and direct a user with attention, memory, and cognition” through feedback loops that monitor “facial recognition, eye tracking, user mood detection, user emotion detection, voice detection, etc. [from a] bio-sensor for tracking biometric characteristics, such as health and activity metrics […] and other health-related information”.

Social questions

Biospatial surveillance is also the key to Apple’s attempt to solve the social problems created by wearing a headset in public. The external screen showing a simulated approximation of the user’s gaze relies on constant measurement of the user’s expression and eye movement with multiple sensors.

Your face is constantly mapped so others can see it – or rather see Apple’s vision of it. Likewise, as passersby come into range of the Apple Vision Pro’s sensors, Apple’s vision of them is automagically rendered into your experience, whether they like it or not.

Apple’s new vision of us – and those that surround us – shows how the requirements and benefits of spatial computing will pose new privacy concerns and social questions. The extensive biospatial surveillance that captures intimate biometric and environmental data redefines what personal data and social interactions are possible for exploitation.

Luke Heemsbergen is Senior Lecturer, Digital, Political, Media, Deakin University.

The Conversation is an independent and nonprofit source of news, analysis and commentary from academic experts.

© The Conversation

©2024 GPlusMedia Inc.

Login to comment

There is absolutely no way in hell that I would ever put one of these weird goggles on my face!!!! I do not need to relive any memories using this crap nor work looking like some computerized zombie with that thing on!!!!

-5 ( +1 / -6 )

Seeing that it is an Apple product they'll probably sell quite a few just because it is branded Apple.

Considering that it costs 3500 dollars I can't imagine that very many outside the Apple fanbase will buy one though.

-3 ( +2 / -5 )

Those prices are far beyond my budget and I guess most other people's budgets too. Although the technology is very impressive, it also doesn't really fit to our average human's biological and psychological body conditions. Most of us will stand it only for a maximum 30 minutes per session. I had the possibility to wear such a really professional device / headset for free at a coal mining museum and it was a great 3D and multimedia experience including 360 degrees incoming data, but although very happy for this rare opportunity I was also very relieved when taking the device off after those about 10 minutes. It makes one dizzy, stressed, later tired and slight stomach and head aches afterwards too.

1 ( +2 / -1 )

I don't think it's my thing but at $4000-5000, it's at the bottom of my wish list.

0 ( +2 / -2 )

Indeed. Me too wallace. Another useless gadget. Don't want to clutter up my 1LDK anymore, do we?

-6 ( +0 / -6 )

I do not believe the Apple Vision Pro is another useless gadget. There will be many good uses for them. Just not for me.

3 ( +5 / -2 )

For people asking themselves what are these used mostly used for - mainly for 3D porn. Yep, it's a thing, it gained huge traction thanks to increased internet speeds and CoVID imposed isolation, and there are a lot of free or very affordable sites. It basically the only industry where VR hasn't tanked - everywhere else is losing traction. There you have it.

-3 ( +0 / -3 )

No one is interested in VR. The future is AR. When we can have the technology on a pair of normal glasses, we'll wonder how we ever managed without it. This could be the start of something that makes the iPhone pale into insignificance. Get the stock while you can...

-2 ( +2 / -4 )

I will respectfully pass on this wonderful item. I won’t be in this world long and I want to enjoy reality while I can. On the other hand this item might be beneficial to people who are bedridden or from time to time used in educational setting to explore history or something like that.

3 ( +3 / -0 )

Are these also see-through glasses like they used to sell from ads in the comic books? Now, that would be a good selling point.

1 ( +1 / -0 )

To counter the negative comments here, I'd just like to point out that Apple has sold all of the preorders for this device. Sold out!

3 ( +3 / -0 )

This will be used by rich hipsters, in luxury services (fashion, real estate, holidays) and in industry. The price is too high for mainstream use.

The killer app will be 'stripping' passers-by in real time, getting the body shape/movements right. Which will kick up a bit of a stink.

-2 ( +0 / -2 )

The picture used in this article is not the apple vision pro it is the cheap knock off that you put your smart phone in and is not even close to what even the quest 2 looks like when you wear it. That being said the apple vision pro is not meant for the normal consumers like the first iPhone was not meant for consumers. It is meant for developers and prosumers who will build apps for future devices that will be cheaper then this head set. That being said VR is not going anywhere. In fact its getting bigger but i wouldn't expect anyone who isn't in the VR space to see the raise in where this tech is going (as well as AR tech)

I was definitely one of those people who when the smart phone came out was saying "I will never buy that thing it's too expensive and all i need is a cell phone to make phone calls anyway" but know i have 2 in both my pockets and wouldn't dream of going back to flip phones or any other style of cell phone. As the tech progresses it will get cheaper and more main stream any using it will be no different then using a smart phone now.

2 ( +2 / -0 )

The Vision Pro is smaller than the one in the photo.


0 ( +1 / -1 )

Login to leave a comment

Facebook users

Use your Facebook account to login or register with JapanToday. By doing so, you will also receive an email inviting you to receive our news alerts.

Facebook Connect

Login with your JapanToday account

User registration

Articles, Offers & Useful Resources

A mix of what's trending on our other sites