Vol.2 No.2 2009

30/98

Research paper : A secure and reliable next generation mobility (Y. Satoh et al.)−110−Synthesiology - English edition Vol.2 No.2 (2009) Stereo image processing can be achieved by a minimum of two cameras (binocular stereo) without any other measures such as mirrors. Yet, accuracy can be improved by using more cameras that will allow multiple results for the reliability assessment of the measurements. In the stereo omni-directional camera, trinocular stereo was used in consideration of the balance of accuracy and camera-head size.3.3 Image integrationFigure 4 shows an example of image integration. The stereo omni-directional camera is composed of multiple cameras, and an omni-directional image is obtained by integrating the images of individual cameras using software. In our intelligent electric wheelchair, one of the functions considered for future addition is remote transmission of the omni-directional video using a cell phone line to provide remote support. Therefore, it is necessary to integrate the images in good quality, while risk detection of the surrounding environment must be done frequently, and the computation cost must be minimized as much as possible.In general, a camera lens has greater distortion and light fall-off at the edge of the image (Fig. 5). This is not a major issue when viewing one image as in an ordinary digital camera, but gaps and differences in lightness occur at the boundaries when integrating multiple images. To solve this problem, it is necessary to: (1) correct the barrel-shaped distortion of the lens, (2) correct the peripheral light fall-off of the lens, (3) conduct geometric conversion from the coordinate system of individual cameras to the integrated coordinate system, (4) correct the color variation between the cameras, and (5) conduct the blending process for smooth connection of the boundaries between the images. Due to the limitation of space in the paper, we leave the specific computation equations to a referenced paper[11]. Since they are nonlinear conversions containing several trigonometric functions, they require over ten seconds (when 3.2 GHz CPU was used) for a single image composition. In order to improve the performance, we determined all the parameters that are dependent on the properties of the camera-head and camera unit, and performed above calculations in advance to make a transformation look-up table. With the look-up table, one finely corrected omni-directional image can be obtained from twelve raw color images with no correction at all. The time required to process an omni-directional image of 512 × 256 pixels is only 10 ms or less.3.4 Estimation of camera-head positionTo obtain accurate information on the surrounding environment of the electric wheelchair, it is necessary to know accurately at what position the camera-head of the stereo omni-directional camera is attached to the electric wheelchair. In the initial design, the pose of the camera-head was obtained by detecting the direction of the gravity using an acceleration sensor fixed to the support bar of the camera-head when the wheelchair was stationary. However, two issues emerged when testing the prototype: (1) the movement of the camera-head due to unevenness and bumps during the test run was greater than expected, and it became necessary to estimate and correct the camera-head pose parameters in real time, and (2) it was discovered that a lifter (a device to hoist up the user and move him/her to the seat of the electric wheelchair) may be needed for the rider to mount and dismount the electric wheelchair. Therefore, we employed a swinging attachment arm to prevent interference of the camera-head with the lifter, but this caused slight changes of the attachment position after every swing and fix. Therefore, a method to estimate the camera-head pose parameters in real time became necessary.Fig. 4 High quality and high speed image integration.The omni-directional image is produced in high quality and in high speed from 12 raw images that include lens distortion and peripheral light fall-off. Although it is difficult to present the omni-directional image in 2D, we present the image in Mercator projection, as in a world map.Fig. 5 Lens distortion correction.The person in the photo is holding a ruler in his hand. In the left photo, the ruler is arched since this is the image before correction. On the right is the corrected image. The distortions of the line of the ceiling and other parts are also corrected.Image integration processIntegrated omni-directional image12 raw images for12 directionsUse of look-up table→Achieved integration at 10 ms or less(1)Barrel-shaped distortion correction(2)Peripheral light fall-off correction(3)Geometric conversion(4)Color correction(5)Blending

※このページを正しく表示するにはFlashPlayer9以上が必要です