INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND COMPUTER PROGRAM

• US 20180101223A1
• Filed: 02/18/2016
• Published: 04/12/2018
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: ISHIHARA, ATSUSHI, ISHIKAWA, TSUYOSHI, AGA, HIROYUKI, KAWASAKI, KOICHI, NISHIBE, MITSURU, KUSANO, YUJI
• Granted Patent: US 10,452,128 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

An information processing apparatus including circuitry configured to acquire information indicating a spatial relationship between a real object and a virtual object, and initiate generation of a user feedback based on the acquired information, the user feedback being displayed to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real world, and wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.

Claims

1. An information processing apparatus comprising:

circuitry configured to

acquire information indicating a spatial relationship between a real object and a virtual object, and

initiate generation of a user feedback based on the acquired information, the user feedback being displayed to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real world,

wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.

2. The information processing apparatus according to claim 1, wherein a size of display of the user feedback is changed as the spatial relationship between the real object and the virtual object is changed.

3. The information processing apparatus according to claim 1, wherein a size of display of the user feedback is displayed to be smaller as the real object becomes closer to the virtual object.

4. The information processing apparatus according to claim 1, wherein a shape of display of the user feedback is changed when the spatial relationship between the real object and the virtual object is changed.

5. The information processing apparatus according to claim 1, wherein a visual quality of display of the user feedback is changed as the spatial relationship between the real object and the virtual object is changed.

6. The information processing apparatus according to claim 5, wherein at least one of a color, a brightness, or transmittance of display of the user feedback is changed as the spatial relationship between the real object and the virtual object is changed.

7. The information processing apparatus according to claim 6, wherein the color of display of the user feedback is displayed to be lighter as the real object becomes closer to the virtual object.

8. The information processing apparatus according to claim 1, wherein a movement of display of the user feedback is changed as the spatial relationship between the real object and the virtual object is changed.

9. The information processing apparatus according to claim 8, wherein at least a position, a vibration, or a trajectory of display of the user feedback is changed as the spatial relationship between the real object and the virtual object is changed.

10. The information processing apparatus according to claim 1, wherein at least a portion of the virtual object is not displayed as the real object approaches the virtual object.

11. The information processing apparatus according to claim 1, wherein at least a portion of the virtual object is lighted up as the real object approaches the virtual object.

12. The information processing apparatus according to claim 1, wherein the spatial relationship includes a distance between the real object and the virtual object.

13. The information processing apparatus according to claim 12, wherein the characteristic of the user feedback is changed based on whether the real object is within a predetermined distance from the virtual object.

14. The information processing apparatus according to claim 13, wherein a characteristic of the user feedback which is changed in a case that the real object is within the predetermined distance from the virtual object is different from a characteristic of the user feedback which is changed in a case that the real object is not within the predetermined distance from the virtual object.

15. The information processing apparatus according to claim 13, wherein the circuitry is further configured to set the predetermined distance in front of and behind the virtual object as a range of operation of the virtual object.

16. The information processing apparatus according to claim 1, wherein the virtual object overlaps the real object when the real object is pushed through a location corresponding to a display position of the virtual object.

17. The information processing apparatus according to claim 1, wherein a warning feedback is given when the real object is pushed through a location corresponding to a display position of the virtual object.

18. The information processing apparatus according to claim 1, wherein display of at least a portion of the virtual object is terminated when the real object is pushed through a location corresponding to a display position of the virtual object.

19. The information processing apparatus according to claim 1, wherein the user feedback includes a display portion for an input operation performed by a user.

20. The information processing apparatus according to claim 1, wherein the spatial relationship includes at least one of a distance, a direction, or posture information between the real object and virtual object.

21. The information processing apparatus according to claim 1, wherein the user feedback includes a sound or a vibration.

22. The information processing apparatus according to claim 1, wherein the user feedback is stereoscopically displayed.

23. The information processing apparatus according to claim 1, wherein the characteristic of the user feedback is changed stepwise or in relation to a changing of the spatial relationship between the real object and the virtual object.

24. The information processing apparatus according to claim 1, wherein a magnitude of the characteristic of the user feedback is changed in relationship to a changing of the spatial relationship between the real object and the virtual object.

25. The information processing apparatus according to claim 1, wherein a location of displaying the user feedback is determined based on a location of a target whose position is indicated by a trajectory direction of the real object.

26. An information processing method comprising:

acquiring information indicating a spatial relationship between a real object and a virtual object;

generating a user feedback based on the acquired information and displaying the user feedback to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real word,

wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.

27. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising:

acquiring information indicating a spatial relationship between a real object and a virtual object;

generating a user feedback based on the acquired information; and

displaying the user feedback to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real word,

wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.