From “AI That Sees a Single Object” to “AI That Compares Multiple Objects”

As of 2026, AI technology is moving beyond the stage of understanding and describing text and images towards the stage of accurately perceiving and using real-world spaces and objects. In manufacturing settings, for example, determining whether multiple industrial parts can be assembled correctly and comparing differences in part geometries are critical processes. However, conventional vision-language models have primarily focused on understanding and describing single object-based images or point clouds. These models have difficulty handling multiple objects simultaneously and explaining geometric relationships, such as which parts can be joined or how they differ.

Against this backdrop, there is a growing demand for “AI capable of comparing multiple objects, understanding differences in shape, combinations, and variations, and explaining these differences in natural language.” This research and development project aims to meet these societal needs by creating new AI technologies that support and streamline human decision-making processes in applied fields such as design and manufacturing.

Researchers at AIST have developed a point-cloud language model that can understand and explain geometric relationships among multiple objects.

Comparing multiple objects or parts to identify differences in their shapes and their spatial relationships is a critical process in manufacturing. Although AI technology has increasingly been adopted in manufacturing in recent years, conventional vision-language models have been limited to recognizing and describing single objects. This makes it difficult for them to understand the geometric relationships among multiple objects. To address this limitation, the researchers constructed “Multi-Object in 3D (MO3D),” a dataset comprising approximately 70,000 high-quality point-cloud samples, each paired with question-and-answer annotations. MO3D serves as a new training and evaluation framework for comparing multiple objects and understanding their geometric relationships. Using this dataset, the researchers developed the “Multi 3D Large Language Model (Multi-3DLLM),” a point-cloud language model that can compare multiple objects at the part level and explain the details in natural language such as “which components are joined together” and “where their shapes differ.” In evaluation experiments, our proposed model surpassed the performance of existing vision-language models. For all description tasks (object comparison, object joining, and object changes) using MO3D, Multi-3DLLM demonstrated improved performance, with its question accuracy rate increasing by about 1.8 times compared to conventional methods. This technology realizes an AI model that can understand geometric relationships among multiple objects and explain them in words. As a result, it is expected to contribute to improved work efficiency in various fields (ex, design and manufacturing) including robot-assisted part sorting, assembly support, shape comparison, and editing support in 3D design software. Furthermore, the MO3D developed in this research is expected to advance physical AI research on understanding the geometric relationships between multiple objects.

The research results were presented at the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2026 (June 3-7, 2026, Denver, USA). In addition, the point cloud language model “Multi-3DLLM” and the “MO3D” dataset developed for this study are available on GitHub:
(https://github.com/KohsukeIde/BeyondSIngleObject).