Information processing apparatus, information processing system, and information processing method

• US 10,462,358 B2
• Filed: 04/28/2015
• Issued: 10/29/2019
• Est. Priority Date: 06/20/2014
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Otani, Takahide
• Primary Examiner(s): Wang, Xi
• Application Number:
  US15/314,962
• Publication Number:
  US 20170201677A1
• Time in Patent Office: 1,645 Days
• Field of Search: 396 51, 348 1416, 348 1408, 348 1409
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

This information processing apparatus includes a control unit that acquires two images including a face of a user as an object, the two images being picked up from different positions in a vertical direction of the face of the user, determines whether or not each face in the two images is a face of the same person, and operates a mode in which on the basis of an image of an eye part in one of the two images, a line-of-sight correction image obtained by modifying an image of an eye part of the other of the two images is generated when each face in the two images is the face of the same person.

Claims

1. An information processing apparatus, comprising:

a central processing unit (CPU) configured to;

determine a first face in a first image and a second face in a second image are of a same user, wherein

the first image is captured by a first imaging device from a first position,

the second image is captured by a second imaging device from a second position, and

the first position is different from the second position;

determine, based on the determination that the first face and the second face are of the same user, a direction in which the first imaging device and the second imaging device are arranged is along one of a horizontal orientation of a user face or a vertical orientation of the user face; and

replace, based on the determination that the direction is along the one of the horizontal orientation of the user face or the vertical orientation of the user face, an eye part of the first image with an eye part of the second image to generate a line-of-sight correction image.

2. The information processing apparatus according to claim 1, wherein

the CPU is further configured to acquire, each of the first image that includes the first face and the second image that includes the second face, as an object, and

the first position and the second position are in a vertical direction of the user face.

3. The information processing apparatus according to claim 1, wherein the CPU is further configured to

output an alert based on the determination that the direction is along other than one of the horizontal orientation of the user face or the vertical orientation of the user face.

4. The information processing apparatus according to claim 3, wherein the CPU is further configured to output, as the alert, information for correction of the direction in which the first imaging device and the second imaging device are arranged.

5. The information processing apparatus according to claim 2, wherein

the first imaging device is an upper imaging device in the vertical direction of the user face as the object, and the second imaging device is a lower imaging device in the vertical direction of the user face as the object, and

an image quality of the first image captured by the first imaging device is higher than an image quality of the second image captured by the second imaging device.

6. The information processing apparatus according to claim 5,

wherein the CPU is further configured to:

determine, in a mode in which the line-of-sight correction image is generated, whether a vertical positional relationship between the first imaging device and the second imaging device in the vertical direction of the user face as the object is that the first imaging device is above the second imaging device, and

output an alert based on the vertical positional relationship being other than that the first imaging device is above the second imaging device.

7. The information processing apparatus according to claim 6,

wherein the CPU is further configured to output, as the alert, information for correction of the vertical positional relationship between the first imaging device and the second imaging device.

8. The information processing apparatus according to claim 2, wherein the first imaging device is in the information processing apparatus, and the second imaging device is a separate imaging device detachably mounted on the information processing apparatus.

9. The information processing apparatus according to claim 1, wherein the CPU is further configured to edit the generated line-of-sight correction image.

10. An information processing apparatus, comprising:

a first imaging device configured to capture a first image from a first position;

a second imaging device configured to capture a second image from a second position, wherein the first position is different from the second position; and

a central processing unit (CPU) configured to;

determine a first face in the first image and a second face in the second image are of a same user;

determine, based on the determination that the first face and the second face are of the same user, a direction in which the first imaging device and the second imaging device are arranged is along one of a horizontal orientation of a user face or a vertical orientation of the user face; and

replace, based on the determination that the direction is along the one of the horizontal orientation of the user face or the vertical orientation of the user face, an eye part of the first image with an eye part of the second image to generate a line-of-sight correction image.

11. An information processing method, comprising:

in an information processing apparatus;

determining a first face in a first image and a second face in a second image are of a same user, wherein

the first image is captured by a first imaging device from a first position,

the second image is captured by a second imaging device from a second position, and

the first position is different from the second position;

determining, based on the determination that the first face and the second face are of the same user, a direction in which the first imaging device and the second imaging device are arranged is along one of a horizontal orientation of a user face or a vertical orientation of the user face; and

replacing, based on the determination that the direction is along the one of the horizontal orientation of the user face or the vertical orientation of the user face, an eye part of the first image with an eye part of the second image to generate a line-of-sight correction image.

12. An information processing system, comprising:

a central processing unit (CPU) configured to;

determine a first face in a first image and a second face in a second image are of a same user, wherein

the first image is captured by a first imaging device from a first position,

the second image is captured by a second imaging device from a second position, and

the first position is different from the second position;

determine, based on the determination that the first face and the second face are of the same user, a direction in which the first imaging device and the second imaging device are arranged is along one of a horizontal orientation of a user face or a vertical orientation of the user face; and

replace, based on the determination that the direction is along the one of the horizontal orientation of the user face or the vertical orientation of the user face, an eye part of the first image with an eye part of the second image to generate a line-of-sight correction image.

13. A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a processor of an information processing apparatus, cause the information processing apparatus to execute operations, the operations comprising:

determining a first face in a first image and a second face in a second image are of a same user, wherein

the first image is captured by a first imaging device from a first position,

the second image is captured by a second imaging device from a second position, and

the first position is different from the second position;

determining, based on the determination that the first face and the second face are of the same user, a direction in which the first imaging device and the second imaging device are arranged is along one of a horizontal orientation of a user face or a vertical orientation of the user face; and

replacing, based on the determination that the direction is along the one of the horizontal orientation of the user face or the vertical orientation of the user face, an eye part of the first image with an eye part of the second image to generate a line-of-sight correction image.