Augmented reality and physical games

• US 10,478,717 B2
• Filed: 07/31/2017
• Issued: 11/19/2019
• Est. Priority Date: 04/05/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Robbins, Steven John, Westerinen, William J., Hanson, Lisa M., Son, Sung Ho, Wattles, Richard J.
• Primary Examiner(s): Pierce, Damon J
• Application Number:
  US15/664,653
• Publication Number:
  US 20170326446A1
• Time in Patent Office: 841 Days
• Field of Search: None
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 2009/0615 transparent
• CPC Class Codes: A63F 2009/0619 with an opaque part
• CPC Class Codes: A63F 2009/2435 using a video camera
• CPC Class Codes: A63F 2009/2442 Sensors or detectors
• CPC Class Codes: A63F 2250/307 stereoscopic
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: A63F 3/00643 Electric board games; Electric features of board games electric word or number games A63F3/0421; computer chess G06F; electric raffle games A63F3/081
• CPC Class Codes: A63F 9/24 Electric games; Games using electronic circuits not otherwise provided for video games A63F13/00; computerized gaming systems G07F17/32

Abstract

Augmented reality and physical game techniques are described. In one or more implementations, an indication is received by a computing device of a location of a physical gaming piece of a game. An augmentation is computed based on the indication by the computing device to be displayed as part of the game. The augmentation is displayed by the computing device on a display device that is at least partially transparent such that a physical portion of the game is viewable through the display device concurrently with the augmentation.

Claims

1. An apparatus comprising:

a light engine;

a light guide configured to receive light from the light engine, the light guide being at least partially transparent such that at least a portion of physical surroundings of the apparatus are viewable through the light guide; and

one or more modules implemented at least partially in hardware, the one or more modules being configured to detect a presence of a physical game in the physical surroundings, compute an augmentation based at least partially on one or more available moves for a physical game piece in a current state of play of the physical game, and cause the light engine to output the augmentation for display by the light guide.

2. The apparatus of claim 1, where the one or more modules comprises one or more of a touch sensor, a camera, a keyboard, a cursor, a gyroscope, an accelerometer, and a magnetometer.

3. The apparatus of claim 1, wherein the one or more modules are configured to detect the presence of the physical game via a manual interaction.

4. The apparatus of claim 1, wherein the one or more modules are configured to recognize a gesture made by a user and compute the augmentation based at least in part on the gesture recognized.

5. The apparatus of claim 4, wherein the one or more modules are configured to recognize a gesture comprising an eye movement.

6. The apparatus of claim 1, wherein the one or modules are configured to locate one or more markers in the physical surroundings to determine an orientation and a position of the apparatus.

7. The apparatus of claim 1, wherein the one or more modules are further configured to output one or more of an augmentation describing the current state of play of the physical game and an augmentation comprising instructions related to the current state of play of the physical game.

8. The apparatus of claim 1, wherein the apparatus comprises a tablet, a mobile phone, or a wearable device.

9. The apparatus of claim 1, wherein the one or more modules are configured to receive data communicated from another apparatus, the data usable to compute the augmentation.

10. The apparatus of claim 1, wherein the one or more modules are further configured to output the augmentation using auto-stereoscopic techniques.

11. The apparatus of claim 1, wherein the light guide comprises:

a layer to implement touch sensors across a front surface of the display;

a layer of an optically transparent material;

a diffraction grating matrix;

a layer of a material of a lower optical index than the diffraction grating matrix; and

an electro-chromic layer.

12. A method comprising:

receiving an input to a computing device of a current state of play of a physical game;

computing an augmentation based on the input and one or more available moves for a physical game piece in the current state of play of the physical game, the augmentation to be displayed as part of the physical game; and

displaying the augmentation by the computing device on a display device that is at least partially transparent such that a physical portion of the game is viewable through the display device concurrently with the augmentation.

13. The method of claim 12, wherein the input comprises one or more of a manual interaction of a user with the computing device and a gesture recognized by the computing device.

14. The method of claim 13, wherein the manual interaction comprises one or more of a touch input, a cursor movement, a keystroke, and a voice command and the gesture comprises one or more of a pupil dilation, an eye movement, and a hand motion of one or more user.

15. The method of claim 12, wherein the current state of play comprises a score in the physical game.

16. The method of claim 12, wherein the input is received by one or more of a camera, a touchscreen, and a keyboard.

17. A method for displaying an augmentation, the method performed on a computing device by one or more modules implemented at least partially in hardware, the method comprising:

detecting features of a physical surroundings of an apparatus via image data received from a camera;

computing a geometry of the physical surroundings based upon the features detected;

detecting a presence of a physical game in the physical surroundings from the geometry of the physical surroundings;

computing the augmentation based at least partially on one or more available moves for a physical game piece in a current state of play of the physical game; and

outputting the augmentation for display via a display device in a location and a position based upon one or more of the features, the geometry, and the physical game detected in the physical surroundings.

18. The method of claim 17, wherein detecting the features comprises capturing images of one the physical surroundings via the camera and computing a location of one or more of an edge, a corner and a placement of an object in the images.

19. The method of claim 17, wherein detecting the features comprises using a depth camera.

20. The method of claim 17, wherein detecting the presence of the physical game comprises detecting the presence based upon a known size and a known shape of the physical game.