Image information processing system, image information processing method, image information processing program, and automobile

• US 20060115123A1
• Filed: 11/29/2005
• Published: 06/01/2006
• Est. Priority Date: 11/30/2004
• Status: Active Grant

• Current Assignee: Arriver Software AB (Qualcomm, Inc.)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Nagaoka, Nobuharu, Kanai, Kazunori
• Granted Patent: US 7,672,481 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/21 using cameras
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An image information processing system permits stable display of image information regarding an object on an image display device even if an image area of the object recognized through an infrared camera is unstable. In the image information processing system, a first image area recognition unit recognizes a first image area associated with an object on the basis of a second variable based on the position of the object measured through the infrared camera. Further, a second image area recognition unit estimates the first image area on the basis of a first variable and a second variable based on the running mode of an automobile, and recognizes the estimated image area as a second image area. Then, an image information control unit causes a head-up display to display one or both of the first image information associated with the first image area and the second image information associated with the second image area.

Claims

1. An image information processing system for processing an image of an object shot by an infrared camera mounted in a mobile body and then displaying image information regarding the object on an image display device mounted in the mobile body, comprising:

first variable measurement means for measuring a first variable based on a running mode of the mobile body;

second variable measurement means for measuring a second variable based on the position of the object through the infrared camera;

first image area recognition means for recognizing an image area associated with the object as a first image area on the basis of the second variable measured by the second variable measurement means;

second image area recognition means for estimating an image area recognized by the first image area recognition means on the basis of the first variable measured by the first variable measurement means and the second variable measured by the second variable measurement means and then recognizing the estimated image area as a second image area; and

image information control means for displaying, on the image display device, one or both of first image information associated with the first image area recognized by the first image area recognition means and second image information associated with the second image area recognized by the second image area recognition means.

2. The image information processing system according to claim 1, further comprising:

composite image area recognition means that combines the first and the second image areas recognized by the first and the second image area recognition means, respectively, and then recognizes the result as a composite image area, wherein the image information control means causes the image display device to display one of the first and the second image information associated with the first and the second image areas, or the composite image information associated with the composite image area of the first and the second image areas recognized by the composite image area recognition means.

3. The image information processing system according to claim 2, wherein the image information control means causes the image display device to display composite image information associated with the composite image area of the first and the second image areas if the first and the second image areas have an overlapping portion.

4. The image information processing system according to claim 1, wherein the second image area recognition means estimates the second variable, which will be measured by the second variable measurement means in the future, on the basis of the first variable measured by the first variable measurement means and the second variable measured by the second variable measurement means, and then recognizes a second image area on the basis of the estimated second variable.

5. The image information processing system according to claim 4, wherein

the second variable measurement means performs measurement, taking a part or all of the second variable as a quaternion indicating the position of the object, and

the second image area recognition means estimates the second variable by using a quaternion expressing a spatial displacement based on the first variable measured by the first variable measurement means and the second variable measured by the second variable measurement means.

6. The image information processing system according to claim 5, wherein

the first variable measurement means measures the speed of the mobile body and the angular velocity about a certain axis as first variables, and

the second image area recognition means estimates the second variable by using a quaternion that expresses a turning motion specified by the speed and the angular velocity measured by the first variable measurement means.

7. The image information processing system according to claim 1, wherein, if the second variable measured by the second variable measurement means is unstable, then the second image area recognition means recognizes the second image area on the basis of the second variable measured by the second variable measurement means before such unstableness.

8. The image information processing system according to claim 7, wherein the second image area recognition means determines that the second variable is unstable if the measurement of the second variable by the second variable measurement means is disabled, or a variation amount of the second variable measured by the second variable measurement means exceeds a threshold value.

9. The image information processing system according to claim 1, wherein, if the first image area recognized by the first image area recognition means is stable, then the image information control means causes the first image information associated with the first image area to be displayed on the image display device, whereas, if the first image area recognized by the first image area recognition means is unstable, then the image information control means causes the second image information associated with the second image area recognized by the second image area recognition means to be displayed on the image display device.

10. The image information processing system according to claim 9, wherein the image information control means determines that the first image area is unstable if the difference between or the mean value of one or a plurality of representative points of the first image area recognized by the first image area recognition means and one or a plurality of representative points of the second image area recognized by the second image area recognition means exceeds a threshold value.

13. An automobile, as the mobile body, comprising the image information processing system according to claim 1.

11. An image information processing method for processing an image of an object shot by an infrared camera mounted in a mobile body and then displaying image information regarding the object on an image display device mounted in the mobile body, comprising:

a first variable measurement step for measuring a first variable based on a running mode of the mobile body;

a second variable measurement step for measuring a second variable based on the position of the object through the infrared camera;

a first image area recognition step for recognizing an image area associated with the object as a first image area on the basis of the second variable measured in the second variable measurement step;

a second image area recognition step for estimating an image area recognized in the first image area recognition step on the basis of the first variable measured in the first variable measurement step and the second variable measured in the second variable measurement step and then recognizing the estimated image area as a second image area, and

an image information control step for causing the image display device to display one or both of first image information associated with the first image area recognized in the first image area recognition step and second image information associated with the second image area recognized in the second image area recognition step.

12. An image information processing program for imparting to a computer a function for processing an image of an object shot by an infrared camera mounted in a mobile body and then displaying image information regarding the object on an image display device mounted in the mobile body, the function comprising:

a first variable measurement function for measuring a first variable based on a running mode of the mobile body;

a second variable measurement function for measuring a second variable based on the position of the object through the infrared camera;

a first image area recognition function for recognizing an image area associated with the object as a first image area on the basis of the second variable measured by the second variable measurement function;

a second image area recognition function for estimating an image area recognized by the first image area recognition function on the basis of the first variable measured by the first variable measurement function and the second variable measured by the second variable measurement function and then recognizing the estimated image area as a second image area; and

an image information control function for causing the image display device to display one or both of first image information associated with the first image area recognized by the first image area recognition function and second image information associated with the second image area recognized by the second image area recognition function.