Setting operation area for input according to face position

• US 9,672,413 B2
• Filed: 10/24/2013
• Issued: 06/06/2017
• Est. Priority Date: 11/06/2012
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Inada, Tetsugo, Yokokawa, Yutaka, Doucet, Nicolas, Yoshida, Takumi
• Primary Examiner(s): CERULLO, LILIANA P
• Application Number:
  US14/438,405
• Publication Number:
  US 20150278583A1
• Time in Patent Office: 1,321 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

Provided is an information processor which readily permits operation input to be made so as to point a position on a screen when an operation input is received from a user using a captured image obtained by imaging the user. The information processor acquires a captured image including a user'"'"'s face, identifies the position of the user'"'"'s face included in the acquired captured image, sets an operation area at a position on the captured image determined in accordance with the identified face position, detects a detection target within the operation area, and receives, as a user-pointed position, a position on the screen corresponding to a relative position of the detected detection target within the operation area.

Claims

1. An information processor comprising:

a captured image acquisition section configured to acquire a captured image including a face of a user;

a face position identification section configured to identify a position of the face of the user included in the captured image;

an operation area setting section configured to set an operation area at a position on the captured image adjacent the face of the user, the operation area being smaller than the captured image and being a detection area, any remaining areas of the captured image not established as detection areas being non-detection areas; and

an operation reception section configured to ignore for purposes of detection any non-detection areas and to detect a detection target from within the operation area, and receive, as a user-pointed position, a position on a display image corresponding to a relative position of the detected detection target within the operation area such that a horizontal distance of the user-pointed position from a vertical edge of the display image is proportional in a ratio to a horizontal distance of the target position from a corresponding vertical edge of the operation area and such that a vertical distance of the user-pointed position from a horizontal edge of the display image is proportional in the ratio to a vertical distance of the target position from a corresponding horizontal edge of the operation area.

2. The information processor of claim 1, wherein the operation reception section detects, from within the operation area, a hand of the user as the detection target.

3. The information processor of claim 2, wherein

the captured image acquisition section acquires two images captured by a stereo camera and generates, on the basis of the two captured images, distance distribution data representing a distance distribution from the stereo camera to a subject, and

the operation reception section detects a position of the subject closest to the stereo camera within the operation area as a position of the hand of the user.

4. The information processor of claim 1, wherein the operation reception section detects a controller held by the user in his or her hands from within the operation area as the detection target.

5. The information processor of claim 4, wherein

the controller has a light-emitting section, and

the operation reception section detects a position of a light emitted from the light-emitting section from within the operation area as a position of the controller.

6. The information processor of claim 1, wherein

the captured image includes two faces of first and second users,

the face position identification section identifies positions of the respective faces of the first and second users,

the operation area setting section sets first operation area on the captured image in accordance with the position of the face of the first user and sets second operation area on the captured image in accordance with the position of the face of the second user, and

the operation reception section receives, as a position pointed to by the first user, a position on the screen corresponding to a relative position of the detection target detected within the first operation area and receives, as a position pointed to by the second user, a position on the screen corresponding to a relative position of the detection target detected within the second operation area.

7. The information processor of claim 1 further comprising:

a display image generation section configured to generate a display image by arranging an object within the captured image and display the display image on a screen of a display device, wherein

the display image generation section determines the position where the object is arranged in accordance with the position of a controller held by the user in his or her hands detected from within the captured image.

8. A control method for controlling an information processor to perform a process that matches an operation input from a user, the control method comprising:

acquiring a captured image including a face of a user;

identifying a position of the face of the user included in the captured image;

setting an operation area at a position on the captured image adjacent the face of the user, the operation area being smaller than the captured image and being a detection area, any remaining areas of the captured image not established as detection areas being non-detection areas; and

ignoring for purposes of detection any non-detection areas and detecting a detection target from within the operation area, and receiving, as a user-pointed position, a position on a display image corresponding to a relative position of the detected detection target within the operation area such that a horizontal distance of the user-pointed position from a vertical edge of the display image is proportional in a ratio to a horizontal distance of the target position from a corresponding vertical edge of the operation area and such that a vertical distance of the user-pointed position from a horizontal edge of the display image is proportional in the ratio to a vertical distance of the target position from a corresponding horizontal edge of the operation area.

9. An apparatus comprising a computer operable to execute a computer program for causing a computer to function as:

a captured image acquisition section acquiring a captured image including a face of a user;

a face position identification section identifying a position of the face of the user included in the captured image;

an operation area setting section setting an operation area at a position on the captured image adjacent the face of the user, the operation area being smaller than the captured image and being a detection area, any remaining areas of the captured image not established as detection areas being non-detection areas; and

an operation reception section ignoring for purposes of detection any non-detection areas and detecting a detection target from within the operation area, and receiving, as a user-pointed position, a position on a display image corresponding to a relative position of the detected detection target within the operation area such that a horizontal distance of the user-pointed position from a vertical edge of the display image is proportional in a ratio to a horizontal distance of the target position from a corresponding vertical edge of the operation area and such that a vertical distance of the user-pointed position from a horizontal edge of the display image is proportional in the ratio to a vertical distance of the target position from a corresponding horizontal edge of the operation area.

10. A non-transitory, computer-readable information storage medium storing a program for a computer, the program including:

acquiring a captured image including a face of a user;

identifying a position of the face of the user included in the captured image;

setting an operation area at a position on the captured image adjacent the face of the user, the operation area being smaller than the captured image and being a detection area, any remaining areas of the captured image not established as detection areas being non-detection areas; and

ignoring for purposes of detection any non-detection areas and detecting a detection target from within the operation area, and receiving, as a user-pointed position, a position on a display image corresponding to a relative position of the detected detection target within the operation area such that a horizontal distance of the user-pointed position from a vertical edge of the display image is proportional in a ratio to a horizontal distance of the target position from a corresponding vertical edge of the operation area and such that a vertical distance of the user-pointed position from a horizontal edge of the display image is proportional in the ratio to a vertical distance of the target position from a corresponding horizontal edge of the operation area.