Image display apparatus, image display method, measurement apparatus, measurement method, information processing method, information processing apparatus, and identification method

• US 7,423,553 B2
• Filed: 07/14/2005
• Issued: 09/09/2008
• Est. Priority Date: 09/13/2002
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Satoh, Kiyohide, Uchiyama, Shinji, Yokokohji, Yasuyoshi, Eto, Daisuke, Yoshikawa, Tsuneo, Nakatsuru, Tomoyasu
• Primary Examiner(s): Nguyen; Tai T
• Application Number:
  US11/180,524
• Publication Number:
  US 20050264433A1
• Time in Patent Office: 1,153 Days
• Field of Search: 340/937, 340/901, 340/905, 340/988, 340/990, 340/991, 340/435, 340/461, 348/61, 348/104, 348/117, 348/180, 348/201, 702/104, 702/150, 702/181, 702/182
• US Class Current: 340/937
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages

Abstract

An external index detection unit (105) receives an image sensed by a camera (100), and detects an external index set in a real scene. A tachometer (110) measures the rotational velocities of two rear wheels of a vehicle. A vehicle measurement unit (120) measures the position and azimuth of the vehicle on the basis of the image coordinate position of the external index and the rotational velocities of the two rear wheels of the vehicle. A head index detection unit (135) receives an image sensed by a camera (130), and detects a head index set on the head of a passenger (132). A head measurement unit (140) measures the position of the head on the basis of the image coordinate position of the head index. An image generation unit (150) generates a display image based on the position and azimuth of the vehicle and the position of the head, so that the passenger can observe predetermined navigation information superimposed at a predetermined position of the real scene, and controls a projector (160) to project the generated image.

Claims

1. A measurement apparatus which measures a position and orientation of an image sensing device or an object sensed by the image sensing device by detecting indices on an image of the object sensed by the image sensing device, comprising:

an index selection unit adapted to select indices which are to be observed on the sensed image without being occluded, wherein the position and orientation of the image sensing device or the object sensed by the image sensing device are measured on the basis of information of the selected indices.

2. The apparatus according to claim 1, wherein said index selection unit comprises:

index information holding unit adapted to hold information associated with each index; and

occlusion determination unit adapted to determine a presence/absence of occlusion of each index on the basis of the information associated with each index held by said index information holding unit, and said index selection unit selects indices which are determined to be free from occlusion.

3. The apparatus according to claim 2, wherein said index information holding unit holds at least position/orientation information of the image sensing device or the object upon sensing a template used to detect each index, and said occlusion determination unit determines the presence/absence of occlusion by comparing the position/orientation information of the image sensing device or the object held by said index information holding unit, and estimated values of a position and orientation of the image sensing device or the object.

4. The apparatus according to claim 2, wherein said index information holding unit holds at least information associated with a direction of each index, and said occlusion determination unit determines the presence/absence of occlusion by comparing the information associated with a direction of each index held by said index information holding unit with an estimated value of a direction of the image sensing device.

5. The apparatus according to claim 2, wherein said index information holding unit holds at least a position of each index, and said occlusion determination unit determines the presence/absence of occlusion on the basis of the position of each index, estimated values of a position and orientation of the image sensing device, and shape information of a real space.

6. A measurement method which measures a position and orientation of an image sensing device or an object sensed by the image sensing device by detecting indices on an image of the object sensed by the image sensing device, comprising:

an index selection step of selecting indices which are to be observed on the sensed image without being occluded, wherein the position and orientation of the image sensing device or the object sensed by the image sensing device are measured on the basis of information of the selected indices.

7. The method according to claim 6, wherein the index selection step comprises:

an index information holding step of holding information associated with each index in a memory; and

an occlusion determination step of determining a presence/absence of occlusion of each index on the basis of the information associated with each index held by the memory, and the index selection step includes a step of selecting indices which are determined to be free from occlusion.

8. The method according to claim 7, wherein the memory holds at least information associated with a direction of each index, and the occlusion determination step includes a step of determining the presence/absence of occlusion by comparing the information associated with a direction of each index held by the memory with an estimated value of a direction of the image sensing device.