Display control device, display control method, and program for displaying an annotation toward a user

• US 10,460,022 B2
• Filed: 08/13/2014
• Issued: 10/29/2019
• Est. Priority Date: 11/13/2013
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kasahara, Shunichi, Rekimoto, Junichi
• Primary Examiner(s): Nguyen, Maikhanh
• Application Number:
  US15/025,753
• Publication Number:
  US 20160239472A1
• Time in Patent Office: 1,903 Days
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

To be able to further facilitate interaction between users using AR technologies. Provided is a display control device including: a picture image acquiring unit configured to acquire a moving image corresponding to a sight of a first user in real time; a display control unit configured to display the moving image toward a second user different from the first user, and an annotation detection unit configured to detect that an annotation input by the second user with respect to the moving image is displayed in the sight of the first user or is output to be displayed. The display control unit further displays the displayed or output annotation toward the second user.

Claims

1. A display control device comprising:

circuitry configured to;

acquire a moving image corresponding to a sight of a first user in real time;

initiate display of the moving image on a display toward a second user different from the first user;

detect that an annotation input by the second user with respect to the moving image is displayed in the sight of the first user or is output to be displayed;

initiate display of the displayed or output annotation on another display in the sight of the first user;

initiate display of the displayed or output annotation on the display toward the second user;

detect that the displayed or output annotation displayed on the another display is gazed by the first user;

initiate, on the display toward the second user, display of a notification indicating that the displayed or output annotation displayed on the another display is gazed by the first user;

output the annotation at a predetermined rate to display the annotation on the another display in the sight of the first user, and queue the annotation that is input over the predetermined rate; and

set the predetermined rate on the basis of an operation performed by the first user or sensing information of the first user.

2. The display control device according to claim 1, wherein the circuitry is further configured to:

detect that the annotation is queued to be displayed or to be output; and

initiate display of the queued annotation on the display toward the second user, the queued annotation and the displayed or output annotation being displayed concurrently on the display toward the second user.

3. The display control device according to claim 2, wherein the circuitry is further configured to:

initiate display of the displayed or output annotation on the display in a first form and display of the queued annotation on the display in a second form, the first form and the second form being different from each other.

4. The display control device according to claim 2, wherein the circuitry is further configured to:

initiate, on the display toward the second user, display of the displayed or output annotation integrated into the moving image; and

initiate, on the display toward the second user, display of the queued annotation on the display toward the second user outside the moving image.

5. The display control device according to claim 1, wherein the circuitry is further configured to:

set the predetermined rate at 0 temporarily, on the basis of the sensing information of the first user.

6. The display control device according to claim 5, wherein the circuitry is further configured to:

initiate, on the display toward the second user, display of a notification indicating that the predetermined rate is 0.

7. The display control device according to claim 1, wherein the circuitry is further configured to:

set the predetermined rate at a maximum value until an amount of the queued annotation becomes less than a predetermined threshold value, on the basis of the operation performed by the first user.

8. The display control device according to claim 1, wherein the circuitry is further configured to:

delete at least a part of the queued annotation on the basis of an operation performed by the second user.

9. The display control device according to claim 1, wherein the circuitry is further configured to:

initiate, on the display toward the second user, display of the notification indicating that the displayed or output annotation is gazed by the first user, based on detecting that the displayed or output annotation is gazed by the first user.

10. The display control device according to claim 1, wherein the circuitry is further configured to:

initiate, on the display toward the second user, display of the moving image into which the displayed or output annotation is integrated.

11. The display control device according to claim 10, wherein the circuitry is further configured to:

execute a process for integrating the displayed or output annotation into the moving image.

12. The display control device according to claim 1, wherein the circuitry is further configured to:

detect that the displayed or output annotation is gazed by the first user when a sight line of the first user is identical with a direction in which the displayed or output annotation is displayed on the another display.

13. The display control device according to claim 1, wherein the circuitry is configured to:

detect that a sight line of the first user is identical with a direction in which the displayed or output annotation is displayed on the another display; and

detect that the displayed or output annotation is gazed by the first user, based on detecting that the sight line of the first user is identical with the direction in which the displayed or output annotation is displayed on the another display.

14. A display control method comprising:

acquiring a moving image corresponding to a sight of a first user in real time;

displaying the moving image on a display toward a second user different from the first user;

detecting, by a processor, that an annotation input by the second user with respect to the moving image is displayed in the sight of the first user or is output to be displayed;

displaying the displayed or output annotation on another display in the sight of the first user;

displaying the displayed or output annotation on the display toward the second user;

detecting that the displayed or output annotation displayed on the another display is gazed by the first user;

displaying, on the display toward the second user, a notification indicating that the displayed or output annotation displayed on the another display is gazed by the first user;

outputting the annotation at a predetermined rate to display the annotation on the another display in the sight of the first user, and queue the annotation that is input over the predetermined rate; and

setting the predetermined rate on the basis of an operation performed by the first user or sensing information of the first user.

15. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising:

acquiring a moving image corresponding to a sight of a first user in real time;

displaying the moving image on a display toward a second user different from the first user;

detecting that an annotation input by the second user with respect to the moving image is displayed in the sight of the first user or is output to be displayed;

displaying the displayed or output annotation on another display in the sight of the first user;

displaying the displayed or output annotation on the display toward the second user;

detecting that the displayed or output annotation displayed on the another display is gazed by the first user;

displaying, on the display toward the second user, a notification indicating that the displayed or output annotation displayed on the another display is gazed by the first user;

outputting the annotation at a predetermined rate to display the annotation on the another display in the sight of the first user, and queue the annotation that is input over the predetermined rate; and

setting the predetermined rate on the basis of an operation performed by the first user or sensing information of the first user.