A robot system that can handle medical ampoules was developed through collaboration between the National Institute of Advanced Industrial Science and Technology (AIST), Ookuma Electric Co., Ltd. and Applied Vision Systems Co. (AVS). The system uses 3D shape measurement and object recognition based on the advanced 3D vision system developed by AIST. The ampoules do not need to be arranged in any particular way for the system to work.
Machinery exists that can take sorted ampoules off hospital shelves, however there is no automated system for sorting the ampoules, leaving room for medical mishaps caused by human error. The development of the current system is expected to result in a reduction of medical mistakes and improvement in pharmacists’ working environment.
Currently, pharmacists dispense medicine from ampoules based on doctors' prescriptions. At big hospitals, the amount of medicine dispensed is quite large, so the burden on pharmacists is also quite large, resulting in up to 15% of malpractice cases stemming from the dispensation of incorrect medicines.
In order to prevent these kinds of accidents, it is necessary to automate the handling of medical ampoules. According to rules devised by the Japan Federation of Medical Devices Associations, ampoules must be labeled with UCC EAN 128 barcodes, and this allows machines to automatically recognize the kind of medicine contained in the ampoule. However, the task to approach ampoules to the bar code readers is not automated, and since humans are involved, the possibility of mistakes caused by human error cannot be avoided.
For these reasons, we felt it necessary to develop a system that could automatically recognize medical ampoules, use robots to handle them, dispense the medicine properly using the information contained in the barcodes, and store the medicine properly on the shelves.
One of the biggest issues facing the developers was finding a way for the system to recognize and identify the ampoules. Ampoules are usually taken out of their packaging and put on the shelves in no particular order and eventually they get disorganized. Because of this, it is not possible to use simple two-dimensional image processing. Also, the ampoules are glossy, so it is even difficult for traditional three-dimensional systems to recognize the containers.
In order to solve this problem, Ookuma Electric Co., Ltd. engaged in three-way joint research with AIST and AVS. AIST provided its advanced 3D vision system and AVS provided the application software in order to develop this system.
Two-dimensional image processing is not sufficient to recognize objects that are piled up or in various positions, so it is necessary to use three-dimensional systems. However, medical ampoules have a glossy surface that makes even three-dimensional processing difficult due to the halation caused by reflections. This new system is based on an advanced 3D vision system and improves on the stereo correlation method so that with the help of a multiple camera system, even items with glossy surfaces can be detected.