INFORMATION PROCESSING APPARATUS AND INFORMATION PROCESSING METHOD, DISPLAY APPARATUS AND DISPLAY METHOD, AND INFORMATION PROCESSING SYSTEM

• US 20160093107A1
• Filed: 03/04/2014
• Published: 03/31/2016
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: YAMAMOTO, KAZUYUKI, HOSODA, YASUHIDE, KII, MANABU
• Granted Patent: US 10,269,180 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/217 using environment-related information, i.e. information generated otherwise than by the player, e.g. ambient temperature or humidity
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

Interaction between a virtual object and the real space is to be presented in a preferred manner.

In accordance with a user'"'"'s viewing or finding of an actual object 151, the virtual object 150 corresponding to the actual object 151 appears. The actual object 151 includes output devices that generate outputs such as vibration, electric pulse, heat or cooling air, wind, sound, light, a transfer, and a jump. The actual object 151 performs reaction to action performed on the actual object 151 by the virtual object 150. Where interaction is conducted between the virtual object 150 and the actual object 151 in synchronization, a stronger impression of reality can be given to the user.

Claims

1. An information processing apparatus comprising:

an output unit configured to apply action to an actual object; and

a control unit configured to control output from the output unit in accordance with action performed on the actual object by a virtual object, or action to be performed on the virtual object by the actual object.

2. The information processing apparatus according to claim 1, further comprising

identification information for identifying the information processing apparatus.

3. The information processing apparatus according to claim 1, further comprising

a receiving unit configured to receive a result of detection of action to be performed on the actual object by the virtual object,

wherein the control unit controls output from the output unit in accordance with the received result of detection.

4. The information processing apparatus according to claim 1, further comprising

a virtual object control unit configured to control action of the virtual object,

wherein the control unit controls output from the output unit in accordance with the action of the virtual object controlled by the virtual object control unit.

5. The information processing apparatus according to claim 1, wherein the output unit includes at least one of the output devices:

a vibrating device, a pulse generating device, a heat generating device, a cooling device, an air blowing device, an acoustic device, a light emitting device, and a transfer device,

the at least one of the output devices being incorporated into the actual object.

6. An information processing method comprising:

the step of acquiring action performed on an actual object by a virtual object, or action to be performed on the virtual object by the actual object; and

the step of applying action to the actual object in accordance with the action performed on the actual object by the virtual object, or the action to be performed on the virtual object by the actual object.

7. A display apparatus comprising:

a detecting unit configured to detect a specific actual object; and

a display unit configured to display a virtual object in response to the detection of the specific actual object.

8. The display apparatus according to claim 7, wherein

the detecting unit identifies the actual object, and

the display unit displays the virtual object corresponding to the identified actual object.

9. The display apparatus according to claim 7, wherein the detecting unit identifies the actual object based on identification information accompanying the actual object, or identifies the actual object through object recognition.

10. The display apparatus according to claim 7, wherein

the detecting unit detects the actual object from a field of view of a user, and

the display unit displays the virtual object overlapped on the actual object.

11. The display apparatus according to claim 7, further comprising

a virtual object control unit configured to control action of the virtual object.

12. The display apparatus according to claim 11, wherein the virtual object control unit controls appearance or disappearance of the virtual object in accordance with motion of a user.

13. The display apparatus according to claim 11, wherein the virtual object control unit controls an amount of information about the virtual object displayed on the display unit, in accordance with motion or a state of a user, or a time of the day.

14. The display apparatus according to claim 11, wherein the virtual object control unit controls action of the virtual object on the actual object, or controls action of the virtual object in accordance with action to be received by the virtual object from the actual object.

15. The display apparatus according to claim 11, wherein

the detecting unit detects action of the virtual object on the actual object, or action to be received by the virtual object from the actual object, and

the virtual object control unit controls action of the virtual object based on a result of the detection performed by the detecting unit.

16. The display apparatus according to claim 11, wherein the virtual object control unit controls action of the virtual object to be synchronized with action of the actual object.

17. The display apparatus according to claim 7, wherein,

when used, the display unit is mounted on a head or a facial part of a user,

the display apparatus further comprises a location/posture detecting unit configured to detect a location and a posture of the head or the facial part of the user, and

the display unit corrects display of the virtual object in the opposite direction from a change in the location or the posture of the head or the facial part of the user.

18. A display method comprising:

the detection step of detecting a specific actual object; and

the display step of displaying a virtual object in response to the detection of the specific actual object.

19. An information processing system comprising:

a control apparatus configured to control action of a virtual object;

a display apparatus configured to detect an actual object and display the virtual object corresponding to the actual object; and

an output apparatus configured to apply action to the actual object in accordance with action performed on the actual object by the virtual object, or action to be performed on the virtual object by the actual object.

20. An information processing system comprising:

a display apparatus configured to detect an actual object and display a virtual object corresponding to the actual object, and control action of the virtual object; and

an output apparatus configured to apply action to the actual object in accordance with action performed on the actual object by the virtual object, or action to be performed on the virtual object by the actual object.