Display control device, display control method, and recording medium

• US 9,886,798 B2
• Filed: 02/16/2016
• Issued: 02/06/2018
• Est. Priority Date: 03/28/2013
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tsurumi, Shingo
• Primary Examiner(s): SHENG, TOM V
• Application Number:
  US15/045,246
• Publication Number:
  US 20160163117A1
• Time in Patent Office: 721 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

There is provided a display control device including a display controller configured to place a virtual object within an augmented reality space corresponding to a real space in accordance with a recognition result of a real object shown in an image captured by an imaging part, and an operation acquisition part configured to acquire a user operation. When the user operation is a first operation, the display controller causes the virtual object to move within the augmented reality space.

Claims

1. A device comprising:

circuitry configured to

recognize a real object from an image data of a real space,

set, based on the recognized real object, a virtual object onto a determined position,

output a signal to control display of the virtual object at the determined position,

recognize a user operation,

output a signal to change a position of the virtual object in response to a first user operation recognized as the user operation, and

output a signal to control size of the virtual object in response to a second user operation recognized as the user operation,

wherein the virtual object remains fixedly displayed at the determined position until the user operation is recognized.

2. The device according to claim 1, wherein the user operation comprises an operation of two fingers.

3. The device according to claim 1, wherein the user operation comprises an operation of holding the virtual object by two fingers.

4. The device according to claim 1, wherein the user operation specifies the virtual object as a target of operation.

5. The device according to claim 4, wherein the user operation comprises an operation performed by two fingers.

6. The device according to claim 1, wherein the virtual object is selected from among a plurality of candidate virtual objects.

7. The device according to claim 1, wherein the first user operation comprises a drag operation.

8. The device according to claim 1, wherein the device is a transmissive head mounted display.

9. The device according to claim 1, wherein the circuitry is further configured to recognize an environment of the real space based on the image data of the real space.

10. The device according to claim 9, wherein the circuitry is further configured to determine a position and an attitude of the device based on the recognized environment.

11. The device according to claim 9, wherein the virtual object is set onto the determined position based on the recognized environment.

12. The device according to claim 9, wherein the recognizing of the environment includes recognizing a wall or a pillar.

13. The device according to claim 1, wherein the circuitry is further configured to recognize an environment of the real space using simultaneous localization and mapping (SLAM).

14. The device according to claim 13, wherein the circuitry is further configured to determine a position and an attitude of the device based on the recognized environment.

15. The device according to claim 13, wherein the virtual object is set onto the determined position based on the recognized environment.

16. The device according to claim 13, wherein the recognizing of the environment includes recognizing a wall or a pillar.

17. The device according to claim 1, wherein the circuitry is further configured to acquire a gravity vector indicating a gravity of the real space.

18. The device according to claim 17, wherein the circuitry is further configured to control an attitude of the virtual object based on the gravity vector.

19. The device according to claim 1, further comprising:

a 3-axis acceleration sensor.

20. The device according to claim 1, further comprising:

a camera configured to obtain the image data of the real space.

21. The device according to claim 1, wherein the circuitry is further configured to output a signal to control display of an indicator that indicates the size of the virtual object.

22. The device according to claim 1, wherein when the size of the virtual object reaches a threshold size, the size of the virtual object is restricted from further changing.

23. A method comprising:

recognizing a real object from an image data of a real space;

setting, based on the recognized real object, a virtual object onto a determined position;

displaying the virtual object at the determined position;

recognizing a user operation,

changing a position of the virtual object in response to a first user operation recognized as the user operation; and

controlling size of the virtual object in response to a second user operation recognized as the user operation,

wherein the virtual object remains fixedly displayed at the determined position until the user operation is recognized.

24. A non-transitory computer-readable recording medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising:

recognizing a real object from an image data of a real space;

setting, based on the recognized real object, a virtual object onto a determined position;

displaying the virtual object at the determined position;

recognizing a user operation,

changing a position of the virtual object in response to a first user operation recognized as the user operation; and

controlling size of the virtual object in response to a second user operation recognized as the user operation,

wherein the virtual object remains fixedly displayed at the determined position until the user operation is recognized.