Niantic has announced the general availability of the first set of augmented reality tools from its Lightship platform, enabling developers across the world to realize their visions for augmented reality (AR) and the real world metaverse. The Niantic Lightship Augmented Reality Developer Kit (ARDK) is now available for all AR developers around the world at Lightship.dev. 

Watch the Lightship global launch keynote.

The Lightship Platform is the foundation for Niantic’s products, built on years of experience developing and running titles ranging from Ingress to Pokémon GO and Pikmin Bloom.

As part of the Lightship Platform, the ARDK is a cross-platform software development kit that runs seamlessly on billions of Android and iOS devices across the globe, according to the company. It brings together the tools and technologies that power the top three AR features—real-time mapping, understanding, and sharing—so developers can create unique AR experiences that seamlessly utilize some or all of the ARDK tools. Over time, the company plans to make more Lightship Platform features available to developers.

Real-Time Mapping: Meshing

With Lightship, any public environment in the world can become a new experience where people explore, gather, and connect, and the key to making that possible is the ability to map reality.Relying on Niantic’s advancements in using smartphone camera sensors to build depth understanding, Meshing APIs in the ARDK create real-time 3D mesh maps, understanding the topography and surfaces around the user. A mesh persists throughout the app session, expanding and improving as the user moves.

With Occlusion APIs, virtual objects appear in the right depth order relative to real-world objects, so if a user kicks their virtual ball over a hedge, they continue to see the hedge, but no longer see the ball.

Building on years of research, Niantic’s depth technology automatically adapts to any Android or iOS device’s camera, from standard RGB sensors to the newest LiDAR-powered devices. The result is a dynamic 3D mesh map that enables developers to create more visually realistic AR layers, taking into account both location and contextual understanding of the unique space surrounding any user within the AR application.

Understanding: Semantic Segmentation

Semantic Segmentation APIs ensure digital objects will interact correctly with different surfaces in the real world, and appear—or not appear—where they realistically should.

Semantic Segmentation APIs in the ARDK use computer vision-based capabilities to instantly identify different elements of a real-world environment, such as ground, sky, water, buildings, and more. These elements inform the way virtual content will react within a real space.

Semantic Segmentation allows developers to focus on the experiences they want to create, while the ARDK dynamically understands the environment, bringing creations realistically to life.

Sharing: Multiplayer Experiences

Sharing is a critical component to making all of these AR experiences feel real. ARDK’s multiplayer APIs enable developers to create AR sessions supporting up to five players concurrently. This allows for more interactive and collaborative applications that feel realistic and seamless, keeping virtual content, players, and their interactions in sync, all in real time.

To keep the shared experiences fast, the ARDK is backed by a P2P networking stack and managed server, so developers don’t need to write code to support multiplayer experiences. Additionally, ARDK provides some lightweight multiplayer gaming functions, including a player lobby system, a synchronized clock, and session-persistent storage.

Lightship ARDK Availability

The core APIs in the Lightship ARDK, including Semantic Segmentation and Meshing, are currently free to use for developers, no matter how many people use their apps. The Multiplayer API is free for apps with fewer than 50,000 monthly active users; a fee applies for apps with more than 50,000 monthly active users. Multiplayer APIs are free for the first six months of use, regardless of the number of people using the app.

These terms apply to all developers who sign up for the Lightship ARDK before May 1, 2022.