Method for an augmented reality character to maintain and exhibit awareness of an observer

• US 10,137,374 B2
• Filed: 02/27/2018
• Issued: 11/27/2018
• Est. Priority Date: 03/30/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Miller, Thomas, Weising, George
• Primary Examiner(s): Wang, Jin-Cheng
• Application Number:
  US15/906,448
• Publication Number:
  US 20180185757A1
• Time in Patent Office: 273 Days
• Field of Search: 345633-634, 715758-764
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2340/14 : Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user

Abstract

A method comprising: detecting a position of a portable device, the portable device including a camera that captures a real-world scene, and identifying an anchor within the real-world scene; generating an augmented reality scene formed by inserting a virtual character into the real-world scene, the virtual character configured to perform an action indicating awareness of the position or activity of the portable device; displaying the augmented reality scene on a display; wherein the action indicating awareness comprises adjusting a view direction, in the augmented reality scene, of the virtual character towards the portable device, and wherein the adjustment of the view direction of the virtual character changes a relationship of the view direction to the anchor, wherein the adjustment of the view direction of the virtual character includes adjustment of a head of the virtual character so as to look in the direction of the portable device.

Claims

1. A method for causing a virtual character to demonstrate awareness of a portable device'"'"'s position or activity, the method comprising, detecting a position of a portable device, the portable device including a camera that captures a real-world scene, and identifying an anchor within the real-world scene, the anchor being tracked during movement of the portable device;

generating an augmented reality scene formed by inserting a virtual character into the real-world scene as captured by the camera, the virtual character being positioned in the augmented reality scene in relation to the identified anchor, the virtual character configured to perform an action indicating awareness of the position or activity of the portable device;

displaying the augmented reality scene on a display;

wherein the action indicating awareness comprises adjusting a view direction, in the augmented reality scene, of the virtual character;

wherein the adjustment of the view direction of the virtual character is in the direction of the position of the portable device and is defined by adjustment from a current view direction to an adjusted view direction towards the portable device, and wherein the adjustment of the view direction of the virtual character changes a relationship of the view direction to the anchor, wherein the adjustment of the view direction of the virtual character includes adjustment of a head of the virtual character so as to look in the direction of the portable device.

2. The method of claim 1, wherein the insertion of the virtual character into the real-world scene comprises placement of the virtual character on a surface of the anchor identified within the real-world scene.

3. The method of claim 1, wherein the anchor is selected from the group consisting of one or more augmented reality tags and one or more features within the real-world scene.

4. The method of claim 1, wherein the performance of an action indicating awareness occurs automatically, without requiring a command from a user.

5. The method of claim 1, further comprising,

continuously updating the adjustment of the view direction of the virtual character in the direction of the position of the portable device, as the portable device moves in three-dimensional space.

6. The method of claim 1, wherein the action indicating awareness further comprises moving the virtual character towards the portable device.

7. The method of claim 1, wherein the display is included in the portable device.

8. A method for causing a virtual character to demonstrate awareness of a portable device'"'"'s position or activity, the method comprising,

capturing a real-world scene at a camera of a portable device;

identifying and tracking one or more markers in the real-world scene;

tracking a position of the portable device;

generating a virtual reality scene positioned with reference to the one or more markers in the real-world scene, the virtual reality scene including a virtual character positioned with reference to the one or more markers in the real-world scene, the virtual character configured to perform an action indicating awareness of the position or activity of the portable device;

displaying the virtual reality scene on a display;

wherein the action indicating awareness comprises adjusting a view direction, in the virtual reality scene, of the virtual character, by adjusting a head of the virtual character;

wherein the adjustment of the view direction is in the direction of the position of the portable device, and wherein the adjustment of the view direction changes a relationship of the view direction to the one or more markers; and

continuously updating the adjustment of the view direction as the portable device moves in three-dimensional space.

9. The method of claim 8, wherein the markers are selected from the group consisting of one or more augmented reality tags and one or more features within the real-world scene.

10. The method of claim 8, wherein the performance of an action indicating awareness occurs automatically, without requiring a command from a user.

11. The method of claim 8, wherein the adjustment of the view direction of the virtual character includes adjustment of eyes of the virtual character so as to look in the direction of the portable device.

12. The method of claim 8, further comprising,

continuously updating the adjustment of the view direction of the virtual character in the direction of the position of the portable device, as the portable device moves in three-dimensional space.

13. The method of claim 8, wherein the action indicating awareness further comprises moving the virtual character towards the portable device.

14. The method of claim 8, wherein the display is included in the portable device.

15. A method for causing a plurality of virtual characters to demonstrate awareness of a portable device'"'"'s position or activity, the method comprising,

detecting a position of a portable device, the portable device including a camera that captures a real-world scene, and identifying an anchor within the real-world scene, the anchor being tracked during movement of the portable device;

generating an augmented reality scene formed by inserting a plurality of virtual characters into the real-world scene as captured by the camera, the virtual characters being positioned in the augmented reality scene in relation to the identified anchor, each of the virtual characters configured to perform an action indicating awareness of the position or activity of the portable device;

displaying the augmented reality scene on a display;

wherein the action indicating awareness comprises adjusting a view direction, in the augmented reality scene, of the virtual characters;

wherein the adjustment of the view direction of each of the virtual characters is in the direction of the position of the portable device and is defined by adjustment from a current view direction to an adjusted view direction towards the portable device, and wherein the adjustment of the view direction of each of the virtual characters changes a relationship of the view direction to the anchor, wherein the adjustment of the view direction of each of the virtual characters includes adjustment of a head of the virtual character so as to look in the direction of the portable device.

16. The method of claim 15, wherein the anchor is selected from the group consisting of one or more augmented reality tags and one or more features within the real-world scene.

17. The method of claim 15, wherein the performance of an action indicating awareness occurs automatically, without requiring a command from a user.

18. The method of claim 15, further comprising,

continuously updating the adjustment of the view direction of each of the virtual characters in the direction of the position of the portable device, as the portable device moves in three-dimensional space.

19. The method of claim 15, wherein the action indicating awareness further comprises moving the virtual characters toward the portable device.

20. The method of claim 15, wherein the display is included in the portable device.