Display control device, display control method, and program

• US 9,411,419 B2
• Filed: 10/29/2014
• Issued: 08/09/2016
• Est. Priority Date: 11/13/2013
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kasahara, Shunichi, Rekimoto, Junichi
• Primary Examiner(s): Buttram, Todd
• Application Number:
  US14/526,778
• Publication Number:
  US 20150130839A1
• Time in Patent Office: 650 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

There is provided a display control device including a matching section configured to match a first image or sensor data output from a first imaging device or a sensor worn on a head of a first user, to a second image output from a second imaging device worn on a part other than the head of the first user, a sight estimation section configured to estimate a region corresponding to a sight of the first user in the second image, on the basis of a result of the matching, and a display control section configured to generate an image expressing the sight of the first user using the second image on the basis of a result of the estimation of the sight, and display the image expressing the sight of the first user toward a second user that is different from the first user.

Claims

1. A display control device comprising:

a matching section configured to match a first image or sensor data output from a first imaging device or a sensor worn on a head of a first user, to a second image output from a second imaging device worn on a part other than the head of the first user;

a sight estimation section configured to estimate a region corresponding to a sight of the first user in the second image, on the basis of a result of the matching; and

a display control section configured to generate an image expressing the sight of the first user using the second image on the basis of a result of the estimation of the sight, and display the image expressing the sight of the first user toward a second user that is different from the first user.

2. The display control device according to claim 1, wherein

the matching section matches the first image output from the first imaging device, to the second image, and

the sight estimation section estimates that a region matched to the first image in the second image includes the region corresponding to the sight of the first user.

3. The display control device according to claim 1, wherein

the matching section matches the sensor data output from the sensor, to the second image, and

the sight estimation section estimates that a region corresponding to a direction of the first user, estimated on the basis of the sensor data is the region corresponding to the sight of the first user.

4. The display control device according to claim 3, wherein

the sensor includes an acceleration sensor, an angular velocity sensor, a gyro sensor, or a geomagnetic sensor.

5. The display control device according to claim 1, further comprising:

a sight line estimation section configured to estimate a sight line direction of the first user,

wherein the display control section generates an image expressing the sight line direction of the first user, in addition to the sight of the first user.

6. The display control device according to claim 1, wherein

the display control section smoothes movement of the sight of the first user, when generating the image expressing the sight of the first user.

7. The display control device according to claim 1, wherein

the display control section generates an image in which the first image is combined with the region corresponding to the sight of the first user in the second image.

8. The display control device according to claim 1, wherein

the display control section generates an image marking the region corresponding to the sight of the first user in the second image.

9. The display control device according to claim 1, wherein

the display control section generates an image in which a display position of the second image is changed in such a manner that the region corresponding to the sight of the first user is positioned frontward.

10. The display control device according to claim 1, wherein

the display control section generates an image of the region corresponding to the sight of the first user, the region being cut out from the second image.

11. A display control method comprising:

matching a first image or sensor data output from a first imaging device or a sensor worn on a head of a first user, to a second image output from a second imaging device worn on a part other than the head of first user;

estimating a region corresponding to a sight of the first user in the second image, on the basis of a result of the matching; and

generating, by a processor, an image expressing the sight of the first user using the second image on the basis of a result of the estimation of the sight, and displaying the image expressing the sight of the first user toward a second user that is different from the first user.

12. A non-transitory computer-readable medium having a set of computer executable instructions embodied thereon for causing a computer to execute a method comprising:

match a first image or sensor data output from a first imaging device or a sensor worn on a head of a first user, to a second image output from a second imaging device worn on a part other than the head of the first user;

estimate a region corresponding to a sight of the first user in the second image on the basis of a result of the matching; and

generate an image expressing the sight of the first user using the second image on the basis of a result of the estimation of the sight, and display the image expressing the sight of the first user toward a second user that is different from the first user.