A team of researchers at NC State University is employing advanced 3D modeling and artificial intelligence to tackle the complex task of sorting microscopic fossils that are no larger than a grain of sand. These tiny organisms, known as foraminifera or “forams,” have inhabited Earth”s oceans for more than 100 million years. When they perish, their shells accumulate on the seafloor, capturing chemical information that reveals historical patterns in ocean environments. For scientists engaged in paleoceanography, these fossils represent invaluable data.
The primary challenge stems from the enormous quantity and minute size of these specimens. Analyzing foraminifera demands sorting through countless similarly shaped objects, making the process both laborious and time-consuming. To address this issue, a research team led by Edgar Lobaton developed Forabot, an open-source robotic system designed for sorting and imaging these microfossils.
“Identifying these fossils is very challenging, which is what prompted our research,” stated Lobaton, a professor of electrical and computer engineering at NC State and co-author of a paper detailing the work. However, Forabot was still in development. The hardware required frequent and meticulous adjustments to effectively sort items smaller than a pencil tip. “We aimed to discover a more efficient method to enhance Forabot,” Lobaton noted.
Innovative 3D Imaging Techniques
The solution came in the form of Foram3D, a groundbreaking technique that transforms paleoceanographic research by creating photorealistic, three-dimensional images of foraminifera. The research team adapted an existing algorithm based on the growth patterns of these shells to generate accurate 3D representations of the fossils. This method involved integrating the intricate 3D geometries of the foraminifera, including internal shell structures, to create mathematically precise digital models.
Collaboration with a paleontologist ensured that the virtual models accurately reflected the characteristics of seven representative foraminifera species. Following the creation of these digital facsimiles, the researchers simulated Forabot”s operational process. Utilizing the newly generated 3D models, they improved the system”s sorting accuracy from 82% to 89%. The simulations facilitated adjustments to the imaging system, enhancing accuracy without necessitating hardware changes. Once the optimal settings were determined, fine-tuning the hardware became a more manageable task.
“These simulations enabled us to identify the best imaging conditions and are now steering the development of a new robotic system focused on 3D reconstruction—an essential step toward automating the identification process for these microfossils,” said Sanjana Banerjee, a co-author and Ph.D. student in electrical engineering at NC State. “Our work lays a strong foundation for studying the growth and morphology of various foraminifera species and addresses significant challenges in micropaleontology, such as limited data availability and accurate shape recovery.”
AI”s Role in Fossil Analysis
The methodology employed by Lobaton and his team in developing the Foram3D technique shows potential for enhancing any robotic system designed to identify and sort objects with intricate shapes. By utilizing the 3D geometric models of foraminifera, the research team assessed how advanced AI models can reconstruct 3D shapes from a limited set of 2D images, simulating real-world scenarios with imperfect lighting. This technology leverages Neural Radiance Fields (NeRF), a deep learning approach that can “fill in the gaps” in photographs, capturing angles or sides of an object that are not visible.
The NeRF reconstruction of synthetic foraminifera demonstrates its capabilities using Forabot”s renderings. “The technique was originally devised to enhance robotic systems sorting and identifying microscopic marine fossils for climate research, but it has the potential to serve as a model for various other applications,” Lobaton explained. “Possible uses extend to the isolation of microbes and pathogens at a microscopic level as well as sorting agricultural products on a larger scale.” To facilitate further research, Lobaton”s team has made the code base for this project publicly available.
The paper titled “Foram3D: A Pipeline for 3D Synthetic Data Generation and Rendering of Foraminifera for Image Analysis and Reconstruction” is published open access in the journal Marine Micropaleontology. This work was co-authored by Turner Richmond, a former Ph.D. student at NC State; Michael Daniele, an associate professor of electrical and computer engineering at NC State; and Thomas Marchitto, a professor of geological sciences at the University of Colorado, Boulder.
This article is based on a news release from NC State University.
