Display control device, display control method, program, and portable apparatus

• US 7,535,486 B2
• Filed: 09/26/2006
• Issued: 05/19/2009
• Est. Priority Date: 06/04/2004
• Status: Expired due to Fees

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Komobuchi, Hiroyoshi, Motomura, Hideto, Kanamori, Katsuhiro
• Primary Examiner(s): Rao; Andy S
• Application Number:
  US11/526,680
• Publication Number:
  US 20070019000A1
• Time in Patent Office: 966 Days
• Field of Search: 348142-160, 375/240.01, 37524021-24025
• US Class Current: 348/158
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

The present invention provides a display control method and a display control device capable of detecting a position and an orientation of an apparatus by a simple structure and controlling a display of a displayed image based on the detected position and orientation of the apparatus with ease. Two cameras are mounted on the apparatus. A user holds the apparatus in the hand, captures, by a first camera, a first image including the face of the user, and captures, by a second camera, a second image including a scene opposite to the user. A display control unit extracts features of the face from the first image and extracts scene feature points from the second image. Then, the display control unit compares the extracted results with reference graphics and the like, and calculates a user-to-apparatus distance ro and angles θo, φo which indicate a direction to the apparatus, and controls the display of the displayed image based on the calculated distance ro and angles θo, φo.

Claims

1. A display control method for controlling a display of a displayed image based on a position of an apparatus provided with a first camera and a second camera which are in a known geometric relationship, said display control method comprising:

capturing, by the first camera, a first image including an object with a known image feature;

capturing, by the second camera, a second image including a scene in the vicinity of the object;

extracting the image feature of the object from the first image;

calculating a distance from the object to the apparatus and a direction of the apparatus viewed from the object, by comparing the extracted image feature of the object to the known image feature of the object;

extracting an image feature of the second image from the second image;

calculating a distance from the object to the apparatus and a direction of the apparatus viewed from the object, by comparing the extracted image feature of the second image to an image feature extracted from a past image which has been captured by the second camera; and

controlling the display of the displayed image based on the distance and direction calculated based on the first image and the distance and direction calculated based on the second image.

2. The display control method according to claim 1,

wherein the object is a face of a user who uses the apparatus.

3. The display control method according to claim 1,

wherein said controlling of the display of the displayed image includes

calculating the distance from the object to the apparatus and the direction viewed from the object using, in a complementary manner, the distance and direction calculated based on the first image and the distance and direction calculated based on the second image, and

controlling the display of the displayed image based on the calculated distance and direction.

4. The display control method according to claim 3,

wherein said controlling of the display of the displayed image includes controlling the display of the displayed image based on the distance and direction calculated based on the first image in the case where the image feature of the second image can not be extracted and until the image feature of the second image can be extracted.

5. The display control method according to claim 1, further comprising

calculating an orientation of the apparatus with respect to the object using at least one of the image feature of the object extracted from the first image and the image feature of the second image extracted from the second image,

wherein controlling the display of the displayed image includes controlling the display of the displayed image based on the calculated distance from the object to the apparatus, the calculated direction of the apparatus, and the calculated orientation of the apparatus.

6. A display control device for controlling a display of a displayed image based on a position of an apparatus provided with a first camera and a second camera which are in a known geometric relationship, said display control device comprising:

a first feature extraction unit operable to extract an image feature of an object from a first image including a known image feature of the object, the first image being captured by said first camera;

a first calculation unit operable to calculate a distance from the object to said apparatus and a direction of said apparatus viewed from the object, by comparing, to the known image feature of the object, the image feature of the object extracted by said first feature extraction unit;

a second feature extraction unit operable to extract an image feature of a second image from the second image including a scene in the vicinity of the object, the second image being captured by said second camera;

a second calculation unit operable to calculate a distance from the object to said apparatus and a direction of said apparatus which is viewed from the object, by comparing the image feature of the second image extracted by said second feature extraction unit to an image feature extracted from a past image which has been captured by said second camera; and

a display control unit operable to control the display of the displayed image based on the distance and direction calculated by said first calculation unit, and the distance and direction calculated by said second calculation unit.

7. A portable apparatus provided with a first camera and a second camera which are in a known geometric relationship, said portable apparatus comprising

a display control device operable to control a display of a displayed image based on a position of said portable apparatus, said display control device including:

a first feature extraction unit operable to extract an image feature of an object from a first image including a known image feature of the object, the first image being captured by said first camera;

a first calculation unit operable to calculate a distance from the object to said apparatus and a direction of said apparatus viewed from the object, by comparing, to the known image feature of the object, the image feature of the object extracted by said first feature extraction unit;

a second feature extraction unit operable to extract an image feature of a second image from the second image including a scene in the vicinity of the object, the second image being captured by said second camera;

a second calculation unit operable to calculate a distance from the object to said apparatus and a direction of said apparatus which is viewed from the object, by comparing the image feature of the second image extracted by said second feature extraction unit to an image feature extracted from a past image which has been captured by said second camera; and

a display control unit operable to control the display of the displayed image based on the distance and direction calculated by said first calculation unit, and the distance and direction calculated by said second calculation unit,

wherein said display control device is operable to control the display of the displayed image based on a position of said portable apparatus.

8. A program for controlling a display of a displayed image based on a position of an apparatus provided with a first camera and a second camera which are in a known geometric relationship, said program being recorded on a recording medium and causing a computer to execute a method comprising:

extracting an image feature of an object from a first image including a known image feature of the object, the first image being captured by the first camera;

calculating a distance from the object to the apparatus and a direction of the apparatus which is viewed from the object, by comparing the extracted image feature of the object to the known image feature of the object;

extracting an image feature of a second image from the second image including a scene in the vicinity of the object, the second image being captured by the second camera;

calculating a distance from the object to the apparatus and a direction of the apparatus viewed from the object, by comparing the extracted image feature of the second image to an image feature extracted from a past image which has been captured by the second camera; and

controlling the display of the displayed image based on the distance and direction calculated by the first image, and the distance and direction calculated by the second image.