How the LiDAR Sensor in the New iPad Can Be a Game-Changer for AR

How the LiDAR Sensor in the New iPad Can Be a Game-Changer for AR

Apple has long been leaning towards augmented reality (AR). The company started emphasizing AR with the launch of its ARKit in 2017. Since then, the Cupertino-based tech giant has been experimenting with AR and developing it with each passing year. However, this year, the company launched its latest iPad Pro equipped with a hardware component that specifically enhances the AR capabilities of mobile devices. Yes, you got it right, LiDAR sensor.

How does it work

Light detection and ranging (LiDAR) is a technique often used in automated robots and cars to create a real-world map. The sensor works with the help of a laser beam. These laser beams are emitted into the environment by the projector and the sensor waits for the reflection of these laser beams to return. As reflections touch the sensor, the system calculates the amount of time that each laser beam takes back into the sensor and based on this analysis a real-world 3D map is created

Now, decent traditional LiDAR sensors are as large as fist sizes and are expensive. Apple was able to cram in an operational LiDAR sensor inside the iPad Pro's iPhone 11-like camera module. Now, this step increases the AR capabilities of mobile devices by leaps and bounds.

How does AR work without LiDAR systems?

AR that works with the help of computer vision (such as in iPhones) relies on the speed of the camera to analyze the environment. Therefore, to perform AR work efficiently, you have to move your camera very carefully to make the system recognize flat surfaces. However, LiDAR sensors work immediately and do not require any careful movements. It starts to map the environment as soon as the camera is opened and in no time creates a 3D model of the real world.

How can AR improve on mobile devices?

Now, LiDAR sensors in mobile devices can open up a whole range of AR possibilities and even erase some limitations of traditional AR systems. First, it is faster than a computer vision AR and can map the world in real-time. Secondly, it is more accurate in capturing the details of the environment. And third, unlike computer vision-dependent AR, LiDAR sensors do not require any light to capture objects in the environment. Instead, it uses laser beams that enable the system to see things without any visible light.

These limitations are prevalent in recent mobile devices with AR capabilities, but not in the iPad Pro. As Apple was able to fit a LiDAR sensor into an iPad, it wouldn't be a shock if they also integrated one into their upcoming iPhones, as the leak suggests.