Image processing device having image sensors that pick up an object that carries out an instructing action

• US 7,970,204 B2
• Filed: 06/18/2007
• Issued: 06/28/2011
• Est. Priority Date: 08/30/2006
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Sawachi, Youichi
• Primary Examiner(s): DULANEY, KATHLEEN YUAN
• Application Number:
  US11/812,273
• Publication Number:
  US 20080056561A1
• Time in Patent Office: 1,471 Days
• Field of Search: 382/154, 382/106, 382/107, 245/158, 245/427, 463/31, 715/863, 702/150, 348/39
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

An image processing device comprising: a plurality of image pickup sections that pick-up, from respectively different positions, a same object of sensing which carries out an instructing action, and outputting image information thereof; a position information sensing section which, by carrying out stereo matching on the plurality of image information outputted from the plurality of image pickup sections by using, as an object, two-dimensional regions, which correspond to a search space which is a three-dimensional space set in advance as a space in which the instructing action can be carried out, senses position information expressing matching positions which are positions of the object of sensing in the three-dimensional space at points corresponding to one another in the two-dimensional regions; and a specific position sensing section that senses, among the matching positions expressed by the position information, a specific position which is a matching position which matches predetermined conditions, is provided.

Claims

1. An image processing device comprising:

a plurality of image sensors that capture, from respectively different positions, a same object of sensing which carries out an instructing action, and that output image information expressing images in which the object of sensing is included;

a position information sensing section which, by carrying out stereo matching on the plurality of image information outputted from the plurality of image sensors, using, as an object, two-dimensional regions, which are regions that are narrower than a common region included commonly in each of the plurality of images expressed by the plurality of image information, and which correspond to a search space which is a three-dimensional space set in advance as a space in which the instructing action is to be carried out in order to issue an instruction, senses position information expressing matching positions, which are positions of the object of sensing in the three-dimensional space at points corresponding to one another in the two-dimensional regions; and

a specific position sensing section that senses, among the matching positions expressed by the position information sensed by the position information sensing section, a specific position which is a matching position that matches predetermined conditions, wherein;

the specific position that is sensed by the specific position sensing section is a position within a sensing space included in the search space,

the sensing space is a space within a rectangular parallelepiped, and

the predetermined conditions are that the position information sensed at the position information sensing section exists within the rectangular parallelepiped, and that, in a case in which the matching position does not exist on one surface of two predetermined, opposing surfaces among surfaces of the rectangular parallelepiped, but exists within a predetermined distance from another surface, the matching position which is nearest to the one surface is made to be the specific position.

2. The image processing device of claim 1, further comprising:

a consumed electric power controlling section which controls an amount of electric power consumed by the image processing device,

wherein the consumed electric power controlling section cancels a low electric power consumption mode when the low electric power consumption mode is set and, after a predetermined time period has elapsed after the specific position sensing section senses the specific position, the specific position sensing section again senses the specific position.

3. The image processing device of claim 1, further comprising:

a deriving section that derives a state amount which expresses a state of movement of the specific position sensed by the specific position sensing section; and

a sensing section that senses the instructing action based on the state amount derived by the deriving section.

4. The image processing device of claim 1, further comprising a coordinate rotating section which, in accordance with a positional relationship between the specific position sensed by the specific position sensing section and an origin of the three-dimensional space, rotates the three-dimensional space thereby yielding a new three-dimensional space.

5. The image processing device of claim 1, wherein the position information sensing section compares, for each of the image information outputted from a same image sensor, the plurality of image information outputted from the plurality of image sensors and the plurality of image information outputted from the plurality of image sensors after a predetermined time period has elapsed, and executes the stereo matching in a case in which image information differs.

6. An image processing device comprising:

a plurality of image sensors that capture a same object of sensing from respectively different positions, and that output image information expressing images that include the object of sensing, wherein the object of sensing carries out an instructing action that instructs execution of an operation processing which carries out a predetermined operation;

a position information sensing section that carries out stereo matching on the plurality of image information outputted from the plurality of image sensors, using, as an object, two-dimensional regions, which are regions that are narrower than a common region included commonly in each of the plurality of images expressed by the plurality of image information, and which correspond to a search space that is a three-dimensional space set in advance as a space in which the instructing action is to be carried out in order to issue an instruction, the position information sensing section thereby sensing position information expressing matching positions, which are positions of the object of sensing in the three-dimensional space at points corresponding to one another in the two-dimensional regions;

a specific position sensing section that senses, among the matching positions expressed by the position information sensed by the position information sensing section, a specific position, wherein the specific position is a matching position that matches predetermined conditions;

a display section;

a control section that effects control such that a predetermined screen corresponding to a cross-section of the three-dimensional space is displayed in a display region of the display section;

a storing section in which the operation processing is stored in advance in association with respective partial regions obtained by dividing the predetermined screen into a plurality of regions;

a deriving section that derives, based on position information of the specific position sensed by the specific position sensing section, an amount of movement of the specific position along a direction normal to the cross-section, as a state amount that expresses a state of movement of the specific position at a time when the object of sensing moves; and

a sensing section that senses the instructing action based on the state amount derived by the deriving section, such that, when the amount of movement exceeds a predetermined amount, the sensing section senses, as the instructing action, an action instructing execution of the operation processing that is stored in association with a partial region that corresponds to a position of the specific position in the cross-section at that point in time.

7. The image processing device of claim 6, wherein

the control section further effects control such that a position-indicating image, which shows a position of the specific position, is displayed at a corresponding position of the specific position in the predetermined screen, and such that a state-indicating portion, which shows the state of movement of the specific position, is displayed at each partial region, and

the control section further effects control such that a displayed state of at least one of the position-indicating image or the state-indicating portion, displayed at the partial region at a position where the position-indicating image is displayed, changes in accordance with the amount of movement.

8. The image processing device of claim 7, wherein the control section prohibits movement of the specific position along a planar direction of the cross-section, immediately before the amount of movement reaches the predetermined amount.

9. The image processing device of claim 6, wherein the control section prohibits movement of the specific position along a planar direction of the cross-section, immediately before the amount of movement reaches the predetermined amount.

10. The image processing device of claim 6, wherein the control section further effects control such that boundary lines, which partition the plurality of partial regions, are displayed on the predetermined screen.

11. The image processing device of claim 6, further comprising a sensing space setting section which sets a position of a sensing space,

wherein the sensing space setting section moves a search space, which is substantially an equal distance from each of the image sensors and which is positioned in a vicinity of the display section, in accordance with the specific position sensed by the specific position sensing section, and sets, as a position of the sensing space, the specific position at a time when the specific position has not moved for a predetermined time period.

12. The image processing device of claim 6, wherein the control section effects control such that an image, which is expressed by image information outputted from the image sensor, and a two-dimensional image, which projects a rectangular parallelepiped expressing a sensing space, are superposed, and a mirror image, which is obtained by mirror-image-converting an image obtained by the superposing, is displayed at a display region of the display section.

13. The image processing device of claim 6, further comprising a coordinate rotating section which, in accordance with a positional relationship between the specific position sensed by the specific position sensing section and an origin of the three-dimensional space, rotates the three-dimensional space thereby yielding a new three-dimensional space.

14. The image processing device of claim 6, wherein the position information sensing section compares, for each of the image information outputted from a same image sensor, the plurality of image information outputted from the plurality of image sensors and the plurality of image information outputted from the plurality of image sensors after a predetermined time period has elapsed, and executes the stereo matching in a case in which image information differs.