Virtual reality

• US 10,559,110 B2
• Filed: 03/10/2017
• Issued: 02/11/2020
• Est. Priority Date: 03/11/2016
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Shenton, Martin, Echenique, Martin, Blackshaw, Claire, Shapiro, Elisheva
• Primary Examiner(s): Chin, Michelle
• Application Number:
  US16/082,449
• Publication Number:
  US 20190088018A1
• Time in Patent Office: 1,068 Days
• Field of Search: None
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 13/58 by computing conditions of game characters, e.g. stamina, strength, motivation or energy level
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 13/40 : of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/003 : Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 : Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06V 40/28 : Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 7/157 : defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

A virtual reality apparatus comprises an image generator to generate images representing a virtual environment. The images are to be displayed to a user via a head mountable. The virtual environment includes an avatar representation of the user within the virtual environment so that the user'"'"'s viewpoint of the virtual environment substantially corresponds to avatar'"'"'s viewpoint corresponding to that user. The image generator is responsive to operation of user controls by the user to configure the hands of the avatar representing that user. The image generator is further configured to generate respective facial configurations of the avatar for display to other users viewing the virtual environment as a mapping of the configuration of the hands of the avatar, so that the hand configuration of the avatar corresponding to a user provide an indication, to that user, of the facial configuration of the avatar corresponding to that user.

Claims

1. A virtual reality apparatus comprising:

an image generator configured to generate images representing a virtual environment, for display to a user by a head mountable display to be worn by the user;

the virtual environment including an avatar representation of the user positioned within the virtual environment so that the user'"'"'s viewpoint of the virtual environment substantially corresponds to a viewpoint of the avatar corresponding to that user; and

one or more user controls,

in which the image generator is operable to map an operation of the one or more user controls by the user on to one of a plurality of configurations of the hands of the avatar representing the user, and

wherein the image generator is configured to generate respective facial configurations of the avatar for display to other users viewing the virtual environment by mapping respective ones of the plurality of configurations of the hands of the avatar to the respective facial configurations, so that a hand configuration of an avatar corresponding to a given user provides an indication, to the given user, of the facial configuration of the avatar corresponding to the given user.

2. Apparatus according to claim 1, in which the avatar has two hands.

3. Apparatus according to claim 1, in which the user controls comprise respective controls allowing separate configuration of each of the avatar'"'"'s hands.

4. Apparatus according to claim 3, in which each avatar hand has:

a base configuration assumed by that avatar hand in an absence of operation of the respective controls; and

one or more active configurations assumed by that avatar hand in response to operation of the respective controls.

5. Apparatus according to claim 4, in which:

the image generator is configured to generate a current facial configuration in response to the current configuration of the avatar hands so that each permutation of avatar hand configurations corresponds to a respective facial configuration.

6. Apparatus according to claim 1, in which:

the head mountable display comprises a microphone;

at least some of the facial configurations comprise at least two configuration versions; and

the image generator is configured to select between versions of a facial configuration according to audio information detected by the microphone.

7. Apparatus according to claim 1, in which:

at least some of the facial configurations comprise at least two configuration versions; and

the image generator is configured to select between versions of a facial configuration according to a degree of activation of the one or more user controls.

8. Apparatus according to claim 1, in which the image generator is configured to generate an avatar body configuration in dependence upon that avatar'"'"'s user'"'"'s head orientation.

9. Apparatus according to claim 1, in which the image generator is configured to generate images representing other avatars corresponding to other users interacting with the virtual environment.

10. Apparatus according to claim 9, in which the image generator is configured to generate an image representing another avatar having a facial configuration and a hand configuration dependent upon user control operations by a corresponding user.

11. Apparatus according to claim 9, in which the image generator is configured to generate an image representing another avatar having an eye direction dependent upon a position, relative to the other avatar, of an eye-look trigger item.

12. Apparatus according to claim 11, in which the eye-look trigger item is another avatar in the virtual environment engaging in an activity.

13. Apparatus according to claim 1, further comprising the head mountable display.

14. A virtual processing system comprising two or more apparatuses according to claim 1, the image generator of each apparatus being configured to generate images for display to a respective user of that apparatus.

15. A method of generating images representing a virtual environment, for display to a user by a head mountable display to be worn by the user, in which the virtual environment includes an avatar representation of the user positioned within the virtual environment so that the user'"'"'s viewpoint of the virtual environment substantially corresponds to a viewpoint of the avatar corresponding to that user, the method comprising:

detecting operation of one or more user controls;

generating the images responsive to the detected operation of the one or more user controls by the user, in which generating the images comprises mapping the detected operation of the one or more user controls by the user on to one of a plurality of configurations of the hands of the avatar representing the user; and

generating respective facial configurations of the avatar for display to other users viewing the virtual environment by mapping respective ones of the plurality of configurations of the hands of the avatar to the respective facial configurations, so that a hand configuration of an avatar corresponding to a given user provides an indication, to the given user, of the facial configuration of the avatar corresponding to the given user.

16. A non-transitory, machine-readable storage medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the one or more processors to perform the method of claim 15.