Aechelon Technology, a global defense firm focused on geospatial simulation technology and AI training systems, announced Project Orbion as an AI-enabled digital twin of the Earth.
I got a look at a demo today in San Francisco and it showed a pretty amazing 3D rendering of the Earth, rich with satellite detail and enhanced in parts with AI technology. A number of partners are helping with various applications of the project, which uses satellite imagery and AI to created highly detailed 3D renderings of the Earth.
It’s a very ambitious project to build something that is like a foundation for the metaverse, and it has a number of key partners. It’s not done yet, but the ecosystem for the cooperation is now in place, the parties said.
Project Orbion partners include Niantic Spatial (San Francisco) for its Large Geospatial Model reconstruction and visualization service; Iceye (Helsinki) for space-based imaging radar; BlackSky (Herndon, Va.) for very high resolution, high-cadence Earth observation imagery and Distance Technologies (Helsinki) for their Light Field 3D Displays.
Each additional tech fills the gap that others cannot. The resulting Skybeam technology allows things like video on the ground to be integrated into the maps in real time to give situational awareness for both human and autonomous devices for the purpose of precision training.
The company, based in South San Francisco, said Orbion is a groundbreaking new initiative that will integrate best-of-class technology solutions to create a live digital twin of the Earth. All of this is complete with accurate physics, real-time weather and more in full Synthetic Reality (SR).
The tech will be dual use, for use with military applications as well as civilian applications, said Javier Castellar, chief strategy officer at Aechelon, in a briefing in San Francisco. For many years, the company has used its tech to help pilots learn how to fly for the past 27 years, said Nacho Sanz-Pastor, CEO of Aechelon, in a press briefing. One of the pilot programs that Aechelon provides is for the U.S. Coast Guard, as those pilots fly 14,000 missions a year and rescue as many as 4,000 people a year.
Project Orbion is a way to combine all the tech that is coming in separately to companies today. Combined and fully visualized, it could have many more uses than just the defense tech today.
Castellar said tech will be necessary to detect millimeter changes on the ground, and some tech is available now for that. Sanz-Pastor noted that GPS was once purely the domain of military uses and it moved to commercial uses, and the same is happening here.
“We create the environment where the pilots fly, and it’s something we call synthetic reality,” Sanz-Pastor said. “We focus on extreme accuracy because the missions we train our pilots on have very little room for error.”
But for the longest time, much of the data on the planet that the company had was static. It could take months for the data on the planet to be updated. Lately, tech has come to the rescue. Iceye helps with satellite radar, BlackSky contributes visual satellite updates, Aechelon handles live 4D video (blending video with 3D), and Niantic Spatial supplies on-the-ground info and AI reconstruction. Distance helps visualize the tech with its displays.
Aechelon’s Project Orbion is a living synthesis of real-time satellite imagery, radar intelligence, video photogrammetry and AI all combined with the most accurate rendition of the Earth, reconstructing and distributing our world in dynamic 3D.
From monitoring wildfires to floods, from wartime troop movements to shipping routes in peacetime, from urban rescues to remote disasters – Project Orbion helps situational
awareness, understanding and safe human decision making.
Synthetic Reality data supplied from Project Orbion is designed to train leading-edge defense and corporate AI models with real-world data – not just human-generated data, but actual ground truth conditions data. It’s where cutting-edge AI is fused into a digital twin of the planet.
Project Orbion is capable of penetrating darkness, clouds and smoke to enable users to react to natural disasters and defense-oriented use cases – all with the most up-to-date 3D representation of the world. This is the Earth’s true Digital Twin – perceived by AI and protected by Human natural perception.
A goal of the collaboration is to explore integrating forthcoming versions of Niantic Spatial’s Visual Positioning System (VPS) with Aechelon’s simulation systems. In GPS-denied environments where satellite signals are unavailable or compromised, VPS will allow ground teams to achieve centimeter-level localization and navigation.
This capability is critical for complex and time-sensitive search-and-rescue operations.
All of these cutting-edge technologies are being integrated to utilize Aechelon’s military-grade Synthetic Reality visualization, AI training and simulation systems.
“Aechelon has spent decades creating the most realistic synthetic environments for mission-critical defense training,” said Nacho Sanz-Pastor, oundcofer and CTO of Aechelon Technology. “The challenge has always been keeping pace with changes in the physical world and training humans and autonomous systems with realistic worldwide information. Our Skybeam product combines several innovative geospatial platforms with our own AI-driven, worldwide multi-sensor baseline to become a powerful new solution.
The partnership with Niantic Spatial, Iceye, BlackSky and Distance will enable Project Orbion (over the next months and years) to address dual-use, mission-critical applications that until now were just in the realm of science fiction.”
“By combining our platform to scan, visualize and understand the world with Aechelon’s simulation technology, we can provide emergency and first responders the most accurate understanding of the environments in which they operate,” said Brian McClendon, Niantic Spatial’s CTO, in a statement. “Geospatial understanding unlocks a new level of situational awareness that allows teams to plan and execute missions with greater confidence and safety.”
McClendon worked at Silicon Graphics decades ago like many of the veterans on the project. In a press briefing, he sai Niantic’s games were sold to Scopely this year for $3.5 billion, but Niantic Spatial remained as a separate tech company using the tech data that was the underlying foundation for games like Pokémon Go.
“We are working on geospatial data,” McLendon said.
Keyhole and Google Earth enabled us to zoom in and see the world. Now Niantic Spatial aims to transform how humans and machines can visualize the world. It uses phones and drones to collect data on the ground and above the ground to reconstruct a 3D model of places like Pier 39 in San Francisco. The result is a high-quality model in 3D, reconstructed in a demo with Aechelon. The goal is to create consumer tools, creating a living model of the world that humans and machines can talk to.
It takes in real-world data via a fusion of scan processing, 3D reconstruction, alignment and georeferencing. Then it feeds them into the large geospatial services model.
Niantic Spatial’s tech knows where you are pointing your phone at any given position on Earth. Then it can incorporate that data on the ground into the large language model that can then improve the accuracy of the reconstruction of geo data from other sources. McClendon said the hope is to scan an area using real-time visual tech like phones and then upload it so pilots can use it on specific missions in the same area.
One of the things the ground data from phones and other sources can do is reach areas GPS can’t, like urban canyons. And it can also deal with areas where GPS jamming is taking place.
“At Iceye, we design, build and operate the world’s largest group of imaging radar satellites – that allows us to react to anything that happens on the ground, regardless of the conditions and very quickly,” said Pekka Laurila, Iceye Co-Founder and Chief Strategy Officer (CSO). “Aechelon began evaluating Iceye SAR radar imagery from orbit by fusing the data 1:1 using its Skybeam platform to update their global geospatial database. The goal was to penetrate clouds, adverse weather and smoke to ensure timely collection on the Earth for testing critical defense applications, which now can be expanded for dual use-cases on training corporate AI systems.”
Finland-based Iceye was founded in 2014 for creating imaging satellites for space. It now has more than 900 people and it has developed a network of 54 satellites and it keeps adding to that. It uses synthetic aperture imaging to see through clouds and see in the dark. It uses a massive microsatellite constellation to access a location in minutes and repeat with high frequency. Right now it can capture at five to 10 times a day, Laurila said in a press briefing.
With those satellites, it can create high-resolution imaging of objects on the ground, from tankers to military vehicles. Laurila showed an image of the bomber base that Ukraine attacked with drones in Russia. It showed intact bombers as well as those that were destroyed by Ukraine’s drones.
Aside from the military uses, insurance companies are big users of the technology, Laurila said. With the new tech from Project Orbion, more commercial uses are expected to blossom.
“BlackSky’s AI-first approach carries immense value in informing a new world of AI-enabled real-time applications – digital twins, virtual and augmented reality – that increasingly moves at machine speed,” said Brian O’Toole, BlackSky CEO, in a statement. “BlackSky’s unique ability to deliver reliable, high-cadence imagery and analytics in real time is enabled by our dedication to proprietary end-to-end AI-enabled automation across our entire architecture. Transformative opportunities like Project Orbion are changing the way humanity will experience and explore the world.” around us and can reshape how we respond to vital national security, economic or natural disaster events for the better.”
Black Sky’s said some customers want data as much as six minutes after visual capture. In a given day, it can capture 10 images in a day in the same area, Scot Currie, vice president of geospatial solutions at Black Sky, said in a press briefing. Over time, the metaverse might be doable but it requires customers to come forward and finance the infrastructure, including putting a lot more satellites into orbit.
“Distance and Aechelon have integrated next-generation 3D lightfield AR HUDs to advance aerospace simulation and defense applications. These cutting-edge systems support real-time battlefield intelligence, target analysis, and invite AR training – all powered by Aechelon’s Synthetic Reality ecosystem. This collaboration has the potential to become the operating system for human AI interaction, enabling intuitive decision making and making
autonomous AI systems safer – under control,” said Urho Konttori, CEO of Distance Technology.
Kontorri said in a press briefing that the mixed-reality tech is being used for both civilian and defense purposes. He noted wit the war in Europe, the goal is to rebuild defense capabilities. The two-year-old company has 45 people and it is based in Helsinki. It uses tech for auto to enhance things like HUD systems and create a distance 3D lightfield for a car’s vision systems.
Aechelon is a global leader in simulation and training systems, specializing in geo-specific flight simulators and physics-based sensor AI training. Its cutting-edge Synthetic Reality platform replicates real-world environments with unmatched precision, enabling both humans and machines to operate in scenarios of extraordinary detail.
This technology allows pilots and emergency responders to train for complex, high-stakes situations in immersive, ultra-realistic virtual settings. Created for the most demanding use-cases in training, simulation and analysis applications, Aechelon synthetic reality has been the tactical simulation choice of militaries throughout the free world for decades and now leverages this same trusted architecture for civilian use.
Aechelon Technology is a Sagewind Capital platform company, which invested in a majority stake in Aechelon in August 2024. Aechelon has not disclosed how much it raised, but it has been profitable for more than 27 years.
Following this investment, the existing technical and management teams continued to expand rapidly, driving both organic growth and strategic acquisitions, such as the Radar Simulation Division from Compro (Radar Toolkit).
Aechelon employs approximately 250 people and is growing steadily. Its physical offices are located in South San Francisco (HQ), Orlando, Texas, Kansas City and Madrid (Spain). The company has 11 U.S. patents, while its broader portfolio has 30 patents worldwide.
The firm works with military contractors including Lockheed Martin, Boeing, Leonardo and virtually all main aerospace contractors and commercial satellite companies. The data is used by the likes of U.S. Department of Defense (Army, Navy, Air Force, Marine Corps, SOCOM, Coast Guard), Homeland Security and Allied Nations. Aechelon’s synthetic reality tech is used in more than 1,000 high-end simulators and data centers at its customers.
Other firms like Palantir are likely to be partners or customers.
Mike Zyda, former USC computer science professor and a gen AI columnist at IEEE, said in an interview with GamesBeat, ” I like what they’ve got because they can connect to live satellites. Every 25 seconds, they can get an update on the data and change the image and send out that image to clients. I think that’s completely awesome.”
He added, “The fact that it’s connected to live satellites is the cool part. People have been building simulation in simulated environments forever, and now that they can put this and run this on a PC with an Nvidia card inside and get you to one meter or one centimeter data at various parts of the world. I think that’s pretty darn awesome.”
You could use that data to monitor progress on a construction site, or on the military side see where someone is building an underground cavern to hide missiles and weapons, he said.
“There are signs that are visible from space. And if you can just collect that data over time and compare it, this company has the ability to generate alerts to you, saying that building site is now covered in sand, and here’s what it looked like a year ago,” Zyda said.
He noted the tech probably requires a lot more satellites to capture the world in real time, and that would create a huge problem with space junk.
Rob Enderle, an analyst with Enderle Research, also said the tech was very impressive.
“As we look at creating the digital twin of the planet, which will be used for weather forecasting and for planning for the future, having this level of detail increases substantially the accuracy of any predictions that revolve around that Earth.”
As far as metaverse vibes go, he said, “It could be a foundational element for a future metaverse project. Being able to to explore the world digitally is certainly a part of the ability of these artificial twins projects. The potential for gaming, for simulation, for experimentation in a virtual setting, has increased substantially with this level of technology, and it’s a stepping stone to the future.”
Asked about whether there were customers interested in metaverse applications, Castellar said the company is focused on business-to-business applications but he expects interest to develop on the metaverse front. McClendon at Niantic Spatial said that the tech exists to sync VR experiences with AR visuals through mixed reality.
Sanz-Pastor said that it will be critical to ensure that the tech does not fall into the hands of bad actors or that it could be “poisoned” by others with ill intent.