MR definition
Mixed reality (MR) is technology that blends the physical and digital worlds so virtual objects can be placed in, and interact with, a real environment in real time. MR devices map rooms, surfaces and hands, letting digital content sit on tables, hide behind real furniture and respond to gestures, between augmented reality and fully virtual reality.
How does mixed reality work?
A mixed reality headset continuously scans its surroundings with cameras and depth sensors, building a 3D mesh of walls, floors and furniture. It tracks the user's head, hands and often eyes. With that understanding, virtual objects can be anchored to real surfaces, hidden correctly behind real objects, lit to match the room and manipulated with natural gestures. Spatial anchors let content stay in the same physical place across sessions and between users.
Devices take two approaches. Optical see-through headsets such as Magic Leap 2 project images onto transparent lenses. Video passthrough headsets such as Meta Quest 3, Apple Vision Pro and Samsung Galaxy XR, the first Android XR headset, capture the world with cameras and display it on screens combined with virtual content. Microsoft popularized the term mixed reality with its HoloLens headsets, since discontinued, building on the reality-virtuality continuum described by researchers Paul Milgram and Fumio Kishino in 1994.
Mixed reality vs AR vs VR
Virtual reality replaces the real world completely. Augmented reality overlays digital content on the real world, often without understanding it deeply, such as a label floating in front of a phone camera. Mixed reality sits between them: digital content is aware of the physical space and interacts with it. In practice the terms overlap, and modern headsets switch between full VR and passthrough mixed reality in the same session, which is why many companies now use the umbrella term extended reality (XR).
Mixed reality use cases
- Remote assistance: experts place arrows and notes on equipment a field technician is looking at.
- Guided assembly and maintenance with step-by-step holographic instructions.
- Full-scale design reviews of cars, machinery or buildings in a real room.
- Surgical planning with 3D models built from patient scans.
- Training on real equipment enhanced with virtual hazards and guidance.
- Collaborative meetings around shared 3D models.
- Retail and interior planning at true scale in the customer's space.
Development tools for mixed reality
Most MR applications are built with Unity or Unreal Engine. The OpenXR standard provides a common interface across headsets, and toolkits such as the open-source Mixed Reality Toolkit (MRTK) offer ready-made interaction components. Meta provides SDKs for passthrough, scene understanding and hand tracking on Quest, Apple's visionOS uses SwiftUI and RealityKit for spatial apps, and Google's Android XR supports Jetpack XR, Unity and OpenXR. Each platform publishes interaction guidelines covering comfortable distances, target sizes and gesture design.
Good MR design respects the real environment: content should not block walkways or hide hazards, and users must always be able to see what matters around them. Safety reviews belong in every MR project, especially in factories, hospitals and warehouses.
Challenges and how to start
Headsets remain expensive compared with phones, battery life limits long sessions, and field of view, weight and comfort still constrain all-day use. Device management, content pipelines for 3D models and integration with business systems add effort beyond the app itself. The strongest cases are where hands-free, in-context information saves real time or prevents costly errors. Nexzem builds mixed reality pilots for training, maintenance and design review, measuring time saved and error rates against current methods before scaling.