INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND PROGRAM

• US 20140300636A1
• Filed: 01/24/2013
• Published: 10/09/2014
• Est. Priority Date: 02/03/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Koga, Yasuyuki, Miyazaya, Yusuke
• Granted Patent: US 9,898,863 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 7/304 For headphones

Abstract

Provided is an information processing apparatus including a sound output control unit configured to generate localization information of a sound marker based on a virtual position, and a sound output unit configured to output a sound associated with the sound marker, based on the localization information, wherein the virtual position is determined based on a position of a real object present in a space.

Claims

1. An information processing apparatus comprising:

a sound output control unit configured to generate localization information of a sound marker based on a virtual position; and

a sound output unit configured to output a sound associated with the sound marker, based on the localization information,

wherein the virtual position is determined based on a position of a real object present in a space.

2. The information processing apparatus of claim 1, wherein the sound output unit is further configured to audibly output the sound concurrently with a displaying of a visual marker on a display unit at the virtual position.

3. The information processing apparatus of claim 1, wherein the sound output control unit is further configured to determine a sound source position of the sound marker.

4. The information processing apparatus of claim 3, wherein the sound output control unit is further configured to determine the sound source position of the sound marker based on the virtual position.

5. The information processing apparatus of claim 3, wherein the sound output control unit is further configured to determine the sound source position of the sound marker based on a distance between a position of a user and the virtual position.

6. The information processing apparatus of claim 5, wherein when the distance between the position of the user and the virtual position is greater than a threshold value, the sound output control unit is further configured to determine the sound source position of the sound marker to be at a location that is located closer to being directly in front of the user than to the virtual position.

7. The information processing apparatus of claim 6, wherein when the distance between the position of the user and the virtual position is less than a threshold value and reducing, the sound output control unit is further configured to determine the sound source position of the sound marker as correspondingly moving from a front side of the user towards the real object.

8. The information processing apparatus of claim 5, wherein when the distance between the position of the user and the virtual position is less than a threshold value, the sound output control unit is further configured to determine the sound source position of the sound marker to be located at a corresponding point along a circumference having a radius of the threshold value and centered on the position of the user.

9. The information processing apparatus of claim 8, wherein as the real object passes from a front side of the user to a hack side of the user, the sound output control unit is further configured to determine the sound source position of the sound marker as moving along the circumference from the front side of the user to the back side of the user until reaching a position located directly behind the user at which point the sound source position of the sound marker then recedes away from the user along a path of a vector.

10. The information processing apparatus of claim 5, wherein:

when the distance between the position of the user and the virtual position is greater than a threshold value, the sound output control unit is further configured to determine the sound source position of the sound marker to be on a front side of the user; and

when the user approaches a physical location corresponding to the virtual position, the sound output control unit is further configured to determine the sound source position of the sound marker as moving from the front side of the user in a direction towards the virtual position.

11. The information processing apparatus of claim 5, wherein:

as the distance between the position of the user and the virtual position decreases to approach a threshold value, the sound output control unit is further configured to determine the sound source position of the sound marker as moving closer to the position of the user along a vector that is parallel to a straight ahead direction of the user;

as the distance between the position of the user and the virtual position decreases from the threshold value to a point in time at which the virtual position is directly in front of the user, the sound output control unit is further configured to determine the sound source position of the sound marker a moving along a circumference having a radius of the threshold value and centered on the position of the user towards a position located directly in front of the user; and

as the virtual position moves from being directly in front of the user towards a back side of the user, the sound output control unit is further configured to determine the sound source position of the sound marker as moving from the position located directly in front of the user and along the circumference towards the back side of the user and out of a visible region of the user.

12. The information processing apparatus of claim 5, wherein the sound output control unit is further configured to change the sound source position of the sound marker according to a change in an orientation of a face of a user.

13. The information processing apparatus of claim 3, wherein the sound output control unit is further configured to set the sound source position of the sound marker as a same location as the virtual position, when the virtual position is within a visible area in front of a user.

14. The information processing apparatus of claim 3, wherein the sound output control unit is further configured to determine the sound source position of the sound marker such that a user perceives output sound as originating from a location of the real object.

15. The information processing apparatus of claim 1, further comprising:

a display unit configured to display a visual marker at the virtual position in a superimposing manner over a video image.

16. The information processing apparatus of claim 1, further comprising:

a head mounted display configured to display a visual marker at the virtual position.

17. The information processing apparatus of claim 1, wherein when the real object is present within a visible region of a user, the virtual position is set to be located at the position of the real object.

18. The information processing apparatus of claim 1, wherein the sound output control unit is further configured to generate the localization information by filtering information associated with real objects located in proximity to a user.

19. The information processing apparatus of claim 1, wherein the sound has a directivity based on the localization information.

20. The information processing apparatus of claim 1, wherein a movement of the real object is determined by detecting a change between a first point in time and a second point in time of a frequency of the sound as perceived by a user.

21. The information processing apparatus of claim 1, wherein the sound marker is not visible to a user.

22. An information processing method comprising:

generating localization information of a sound marker based on a virtual position; and

outputting a sound associated with the sound marker, based on the localization information,

wherein the virtual position is determined based on a position of a real object present in a space.

23. A non-transitory computer-readable medium embodied with a program, which when executed by a computer, causes the computer to perform a method comprising:

generating localization information of a sound marker based on a virtual position; and

outputting a sound associated with the sound marker, based on the localization information,

wherein the virtual position is determined based on a position of a real object present in a space.