SYSTEM AND METHOD FOR INTERACTING WITH VIRTUAL OBJECTS IN AUGMENTED REALITIES

• US 20130178257A1
• Filed: 01/06/2012
• Published: 07/11/2013
• Est. Priority Date: 01/06/2012
• Status: Abandoned Application

• Current Assignee: Augaroo, Incorporated
• Original Assignee: Augaroo, Incorporated
• Inventors: Langseth, Justin
• Application Number:
  US13/345,189
• Publication Number:
  US 20130178257A1
• Time in Patent Office: Days
• US Class Current: 463/4
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

The system and method described herein may be used to interact with virtual objects in augmented realities. In particular, the system and method described herein may include an augmented reality server to host software that supports interaction with virtual objects in augmented realities on a mobile device through an augmented reality application. For example, the augmented reality application may be used to create and deploy virtual objects having custom designs and embedded content that can be shared with other users to any suitable location, similarly interact with virtual objects and embedded content that other users created and deployed using the augmented reality application, participate in games that involve interacting with the virtual objects, obtain incentives and targeted advertisements associated with the virtual objects, and engage in social networking to stay in touch with friends or meet new people via interaction with the virtual objects, among other things.

Claims

1. A mobile device for interacting with virtual objects in augmented realities, comprising:

one or more location sensors configured to determine a location associated with the mobile device; and

a processor configured to;

communicate the location associated with the mobile device to an augmented reality server, wherein the augmented reality server correlates the location associated with the mobile device to one or more virtual objects;

receive, from the augmented reality server, data associated with the one or more virtual objects correlated to the location associated with the mobile device, wherein the data associated with the one or more virtual objects includes locations associated with the one or more virtual objects;

display, on the mobile device, an augmented reality that superimposes the one or more virtual objects over one or more images that represent physical surroundings associated with the mobile device, wherein the one or more virtual objects are superimposed over the one or more images at the locations associated with the one or more virtual objects; and

process one or more requests to interact with the one or more virtual objects in the augmented reality using an augmented reality application.

2. The mobile device of claim 1, wherein the one or more images that represent the physical surroundings associated with the mobile device include a map associated with a physical area that encompasses the location associated with the mobile device.

3. The mobile device of claim 1, wherein the one or more images that represent the physical surroundings associated with the mobile device include a satellite view associated with a physical area that encompasses the location associated with the mobile device.

4. The mobile device of claim 1, further comprising a camera configured to sense the physical surroundings associated with the mobile device, wherein the physical surroundings sensed with the camera represents a current viewpoint associated with the mobile device.

5. The mobile device of claim 4, wherein the processor is further configured to:

obtain a position and an orientation associated with the mobile device using the one or more location sensors; and

communicate the position and the orientation associated with the mobile device to the augmented reality server, wherein the augmented reality server correlates the position and the orientation associated with the mobile device to the location associated with the mobile device to map the current viewpoint associated with the mobile device.

6. The mobile device of claim 4, wherein the one or more images that represent the physical surroundings associated with the mobile device include a map associated with a physical area that encompasses the location associated with the mobile device and the current viewpoint associated with the mobile device.

7. The mobile device of claim 1, wherein the one or more requests cause the augmented reality application to collect the one or more virtual objects, destroy the one or more virtual objects, or move the one or more virtual objects to new locations if the locations associated therewith are substantially near to the location associated with the mobile device.

8. The mobile device of claim 1, wherein the one or more requests cause the augmented reality application to obtain content or virtual items embedded in the one or more virtual objects from the augmented reality server or upload content or virtual items to embed in the one or more virtual objects to the augmented reality server if the locations associated therewith are substantially near to the location associated with the mobile device.

9. The mobile device of claim 8, wherein the content or virtual items obtained from the augmented reality server and the content or virtual items uploaded to the augmented reality server includes text, pictures, graphics, audio, video, icons, games, software, incentive, or advertising content associated with the one or more virtual objects.

10. The mobile device of claim 1, wherein the physical surroundings associated with the mobile device include a virtual field and the one or more virtual objects superimposed over the one or more images in the augmented reality include a three-dimensional virtual ball and a three-dimensional virtual goal, and wherein the one or more requests relate to playing an interactive game in which gestures associated with the mobile device cause the three-dimensional virtual ball to move within the virtual field, goals are scored if the gestures cause the three-dimensional virtual ball to enter the three-dimensional virtual goal, and the displayed augmented reality includes a virtual scoreboard to track information relating to the interactive game.

11. The mobile device of claim 10, wherein the gestures associated with the mobile device cause the three-dimensional virtual ball to move to a new location within the virtual field based on a current direction, elevation angle, and speed associated with the gestures at a time when the location associated with the mobile device and the location associated with the three-dimensional virtual ball are within a predetermined proximity.

12. The mobile device of claim 10, wherein the virtual field comprises a virtual space that overlays a first physical space where one or more players on a first team in the interactive game are located with a second physical space where one or more players on a second team in the interactive game are located.

13. The mobile device of claim 10, wherein the processor is further configured to communicate with the augmented reality server over a persistent connection using low-latency communication technology to synchronize the one or more requests that relate to playing the interactive game in substantially real-time.

14. The mobile device of claim 1, wherein the processor is further configured to manage a treasure or scavenger hunt in which multiple users are given clues to describe the locations associated with the one or more virtual objects in a predetermined sequence and subsequent virtual objects in the predetermined sequence only become visible to the multiple users upon collecting prior prerequisite virtual objects in the predetermined sequence.

15. The mobile device of claim 1, wherein the processor is further configured to communicate with the augmented reality server over a persistent connection using low-latency communication technology to synchronize the one or more requests to interact with the one or more virtual objects with one or more other requests that one or more other mobile devices initiate to interact with the one or more virtual objects in substantially real-time.

16. The mobile device of claim 15, wherein the processor is further configured to:

receive one or more events from the augmented reality server that relate to the one or more other requests that the one or more other mobile devices initiated to interact with the one or more virtual objects over the persistent connection, wherein the one or more events indicate that the one or more other requests were initiated earlier in time than the one or more requests processed to interact with the one or more virtual objects; and

update the augmented reality displayed on the mobile device to reflect the one or more other requests that were initiated prior in time in lieu of the one or more requests processed to interact with the one or more virtual objects.

17. The mobile device of claim 1, wherein the processor is further configured to:

zoom out on the one or more images that represent the physical surroundings associated with the mobile device to represent a larger geographic area;

receive, from the augmented reality server, data associated with multiple virtual objects having locations in the larger geographic area represented in the one or more images; and

form one or more clusters to contain the multiple virtual objects in response to the locations associated therewith having a predetermined proximity to one another, wherein the one or more clusters indicate a number of the multiple virtual objects contained therein.

18. The mobile device of claim 17, wherein the processor is further configured to display, on the mobile device, a list that identifies the multiple virtual objects in the one or more clusters in response to a selection associated with the one or more clusters, wherein the one or more requests relate to one of the multiple virtual objects selected from the displayed list.

19. The mobile device of claim 1, wherein the one or more requests cause the augmented reality application to post comments relating to the one or more virtual objects on the augmented reality server.

20. The mobile device of claim 1, wherein the one or more virtual objects superimposed over the one or more images in the augmented reality have three-dimensional shapes and custom designs, images, or photographs wrapped around surfaces associated therewith, and wherein movements associated with the mobile device cause the displayed augmented reality to show the one or more virtual objects from different perspectives.

21. The mobile device of claim 1, wherein the mobile device comprises a smartphone, augmented reality glasses, augmented reality contact lenses, a head-mounted display, or augmented reality technology directly tied to a human brain.

22. The mobile device of claim 1, wherein the processor is further configured to communicate criteria to restrict or control access to the one or more virtual objects to the augmented reality server.

23. A method for interacting with virtual objects in augmented realities, comprising:

determining a location associated with a mobile device via one or more location sensors on the mobile device;

communicating, by a processor on the mobile device, the location associated with the mobile device to an augmented reality server that correlates the location associated with the mobile device to one or more virtual objects;

receiving, at the mobile device, data from the augmented reality server that relates to the one or more virtual objects correlated to the location associated with the mobile device, wherein the data that relates to the one or more virtual objects includes locations associated with the one or more virtual objects;

displaying, on the mobile device, an augmented reality that superimposes the one or more virtual objects over one or more images that represent physical surroundings associated with the mobile device, wherein the one or more virtual objects are superimposed over the one or more images at the locations associated with the one or more virtual objects; and

processing one or more requests to interact with the one or more virtual objects in the augmented reality using an augmented reality application executing on the mobile device.

24. The method of claim 23, wherein the one or more images that represent the physical surroundings associated with the mobile device include a map associated with a physical area that encompasses the location associated with the mobile device.

25. The method of claim 23, wherein the one or more images that represent the physical surroundings associated with the mobile device include a satellite view associated with a physical area that encompasses the location associated with the mobile device.

26. The method of claim 23, further comprising sensing the physical surroundings associated with the mobile device using a camera on the mobile device, wherein the sensed physical surroundings represents a current viewpoint associated with the mobile device.

27. The method of claim 26, further comprising:

obtaining a position and an orientation associated with the mobile device using the one or more location sensors; and

communicating, by the processor, the position and the orientation associated with the mobile device to the augmented reality server, wherein the augmented reality server correlates the position and the orientation associated with the mobile device to the location associated with the mobile device to map the current viewpoint associated with the mobile device.

28. The method of claim 26, wherein the one or more images that represent the physical surroundings associated with the mobile device include a map associated with a physical area that encompasses the location associated with the mobile device and the current viewpoint associated with the mobile device.

29. The method of claim 23, wherein processing the one or more requests includes using the augmented reality application to collect the one or more virtual objects, destroy the one or more virtual objects, or move the one or more virtual objects to new locations if the locations associated therewith are substantially near to the location associated with the mobile device.

30. The method of claim 23, wherein processing the one or more requests includes using the augmented reality application to obtain content or virtual items embedded in the one or more virtual objects from the augmented reality server or upload content or virtual items to embed in the one or more virtual objects to the augmented reality server if the locations associated therewith are substantially near to the location associated with the mobile device.

31. The method of claim 30, wherein the content or virtual items obtained from the augmented reality server and the content or virtual items uploaded to the augmented reality server includes text, pictures, graphics, audio, video, icons, games, software, incentive, or advertising content associated with the one or more virtual objects.

32. The method of claim 23, wherein the physical surroundings associated with the mobile device include a virtual field and the one or more virtual objects superimposed over the one or more images in the augmented reality include a three-dimensional virtual ball and a three-dimensional virtual goal, and wherein processing the one or more requests includes using the augmented reality application to play an interactive game in which gestures associated with the mobile device cause the three-dimensional virtual ball to move within the virtual field, goals are scored if the gestures cause the three-dimensional virtual ball to enter the virtual goal, and the displayed augmented reality includes a virtual scoreboard to track information relating to the interactive game.

33. The method of claim 32, wherein the gestures associated with the mobile device cause the three-dimensional virtual ball to move to a new location within the virtual field based on a current direction, elevation angle, and speed associated with the gestures at a time when the location associated with the mobile device and the location associated with the three-dimensional virtual ball are within a predetermined proximity.

34. The method of claim 32, wherein the virtual field comprises a virtual space that overlays a first physical space where one or more players on a first team in the interactive game are located with a second physical space where one or more players on a second team in the interactive game are located.

35. The method of claim 32, further comprising communicating with the augmented reality server over a persistent connection using low-latency communication technology to synchronize the one or more requests that relate to playing the interactive game in substantially real-time.

36. The method of claim 23, further comprising managing, via the augmented reality application, a treasure or scavenger hunt in which multiple users are given clues to describe the locations associated with the one or more virtual objects in a predetermined sequence and subsequent virtual objects in the predetermined sequence only become visible to the multiple users upon collecting prior prerequisite virtual objects in the predetermined sequence.

37. The method of claim 23, further comprising communicating with the augmented reality server over a persistent connection using low-latency communication technology to synchronize the one or more requests to interact with the one or more virtual objects with one or more other requests that one or more other mobile devices initiate to interact with the one or more virtual objects in substantially real-time.

38. The method of claim 37, further comprising:

receiving, at the augmented reality application executing on the mobile device, one or more events from the augmented reality server that relate to the one or more other requests that the one or more other mobile devices initiated to interact with the one or more virtual objects over the persistent connection, wherein the one or more events indicate that the one or more other requests were initiated earlier in time than the one or more requests processed to interact with the one or more virtual objects; and

updating, via the augmented reality application, the augmented reality displayed on the mobile device to reflect the one or more other requests that were initiated prior in time in lieu of the one or more requests processed to interact with the one or more virtual objects.

39. The method of claim 23, further comprising:

zooming out on the one or more images that represent the physical surroundings associated with the mobile device to represent a larger geographic area;

receiving, at the mobile device, data associated with multiple virtual objects having locations in the larger geographic area represented in the one or more images from the augmented reality server; and

forming, via the augmented reality application, one or more clusters to contain the multiple virtual objects in response to the locations associated therewith having a predetermined proximity to one another, wherein the one or more clusters indicate a number of the multiple virtual objects contained therein.

40. The method of claim 39, further comprising displaying, on the mobile device, a list that identifies the multiple virtual objects in the one or more clusters in response to a selection associated with the one or more clusters, wherein the one or more requests relate to one of the multiple virtual objects selected from the displayed list.

41. The method of claim 23, wherein processing the one or more requests includes using the augmented reality application to post comments relating to the one or more virtual objects on the augmented reality server.

42. The method of claim 23, wherein the one or more virtual objects superimposed over the one or more images in the augmented reality have three-dimensional shapes and custom designs, images, or photographs wrapped around surfaces associated therewith, and wherein movements associated with the mobile device cause the displayed augmented reality to show the one or more virtual objects from different perspectives.

43. The method of claim 23, wherein the mobile device comprises a smartphone, augmented reality glasses, augmented reality contact lenses, a head-mounted display, or augmented reality technology directly tied to a human brain.

44. The method of claim 23, further comprising communicating criteria to restrict or control access to the one or more virtual objects from the augmented reality application to the augmented reality server.

45. A computer-readable storage medium storing computer-readable instructions for interacting with virtual objects in augmented realities, wherein executing the computer-readable instructions on a mobile device causes the mobile device to:

use one or more location sensors on the mobile device to determine a location associated with the mobile device;

communicate the location associated with the mobile device to an augmented reality server, wherein the augmented reality server correlates the location associated with the mobile device to one or more virtual objects;

receive, from the augmented reality server, data associated with the one or more virtual objects correlated to the location associated with the mobile device, wherein the data associated with the one or more virtual objects includes locations associated with the one or more virtual objects;

display an augmented reality that superimposes the one or more virtual objects over one or more images that represent physical surroundings associated with the mobile device, wherein the one or more virtual objects are superimposed over the one or more images at the locations associated with the one or more virtual objects; and

process one or more requests to interact with the one or more virtual objects in the augmented reality using an augmented reality application.

46. The computer-readable storage medium of claim 45, wherein the one or more images that represent the physical surroundings associated with the mobile device include a map associated with a physical area that encompasses the location associated with the mobile device.

47. The computer-readable storage medium of claim 45, wherein the one or more images that represent the physical surroundings associated with the mobile device include a satellite view associated with a physical area that encompasses the location associated with the mobile device.

48. The computer-readable storage medium of claim 45, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to use a camera on the mobile device to sense the physical surroundings associated with the mobile device, wherein the physical surroundings sensed with the camera represents a current viewpoint associated with the mobile device.

49. The computer-readable storage medium of claim 48, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to:

use the one or more location sensors to obtain a position and an orientation associated with the mobile device; and

communicate the position and the orientation associated with the mobile device to the augmented reality server, wherein the augmented reality server correlates the position and the orientation associated with the mobile device to the location associated with the mobile device to map the current viewpoint associated with the mobile device.

50. The computer-readable storage medium of claim 48, wherein the one or more images that represent the physical surroundings associated with the mobile device include a map associated with a physical area that encompasses the location associated with the mobile device and the current viewpoint associated with the mobile device.

51. The computer-readable storage medium of claim 45, wherein the one or more requests cause the augmented reality application to collect the one or more virtual objects, destroy the one or more virtual objects, or move the one or more virtual objects to new locations if the locations associated therewith are substantially near to the location associated with the mobile device.

52. The computer-readable storage medium of claim 45, wherein the one or more requests cause the augmented reality application to obtain content or virtual items embedded in the one or more virtual objects from the augmented reality server or upload content or virtual items to embed in the one or more virtual objects to the augmented reality server if the locations associated therewith are substantially near to the location associated with the mobile device.

53. The computer-readable storage medium of claim 52, wherein the content or virtual items obtained from the augmented reality server and the content or virtual items uploaded to the augmented reality server includes text, pictures, graphics, audio, video, icons, games, software, incentive, or advertising content associated with the one or more virtual objects.

54. The computer-readable storage medium of claim 45, wherein the physical surroundings associated with the mobile device include a virtual field and the one or more virtual objects superimposed over the one or more images in the augmented reality include a three-dimensional virtual ball and a three-dimensional virtual goal, and wherein the one or more requests relate to playing an interactive game in which gestures associated with the mobile device cause the three-dimensional virtual ball to move within the virtual field, goals are scored if the gestures cause the three-dimensional virtual ball to enter the three-dimensional virtual goal, and the displayed augmented reality includes a virtual scoreboard to track information relating to the interactive game.

55. The computer-readable storage medium of claim 54, wherein the gestures associated with the mobile device cause the three-dimensional virtual ball to move to a new location within the virtual field based on a current direction, elevation angle, and speed associated with the gestures at a time when the location associated with the mobile device and the location associated with the three-dimensional virtual ball are within a predetermined proximity.

56. The computer-readable storage medium of claim 54, wherein the virtual field comprises a virtual space that overlays a first physical space where one or more players on a first team in the interactive game are located with a second physical space where one or more players on a second team in the interactive game are located.

57. The computer-readable storage medium of claim 54, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to communicate with the augmented reality server over a persistent connection using low-latency communication technology to synchronize the one or more requests that relate to playing the interactive game in substantially real-time.

58. The computer-readable storage medium of claim 45, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to manage, via the augmented reality application, a treasure or scavenger hunt in which multiple users are given clues to describe the locations associated with the one or more virtual objects in a predetermined sequence and subsequent virtual objects in the predetermined sequence only become visible to the multiple users upon collecting prior prerequisite virtual objects in the predetermined sequence.

59. The computer-readable storage medium of claim 45, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to communicate with the augmented reality server over a persistent connection using low-latency communication technology to synchronize the one or more requests to interact with the one or more virtual objects with one or more other requests that one or more other mobile devices initiate to interact with the one or more virtual objects in substantially real-time.

60. The computer-readable storage medium of claim 59, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to:

receive one or more events from the augmented reality server that relate to the one or more other requests that the one or more other mobile devices initiated to interact with the one or more virtual objects over the persistent connection, wherein the one or more events indicate that the one or more other requests were initiated earlier in time than the one or more requests processed to interact with the one or more virtual objects; and

update the displayed augmented reality to reflect the one or more other requests that were initiated prior in time in lieu of the one or more requests processed to interact with the one or more virtual objects.

61. The computer-readable storage medium of claim 45, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to:

zoom out on the one or more images that represent the physical surroundings associated with the mobile device to represent a larger geographic area;

receive, from the augmented reality server, data associated with multiple virtual objects having locations in the larger geographic area represented in the one or more images; and

form one or more clusters to contain the multiple virtual objects in response to the locations associated therewith having a predetermined proximity to one another, wherein the one or more clusters indicate a number of the multiple virtual objects contained therein.

62. The computer-readable storage medium of claim 61, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to display a list that identifies the multiple virtual objects in the one or more clusters in response to a selection associated with the one or more clusters, wherein the one or more requests relate to one of the multiple virtual objects selected from the displayed list.

63. The computer-readable storage medium of claim 45, wherein the one or more requests cause the augmented reality application to post comments relating to the one or more virtual objects on the augmented reality server.

64. The computer-readable storage medium of claim 45, wherein the one or more virtual objects superimposed over the one or more images in the augmented reality have three-dimensional shapes and custom designs, images, or photographs wrapped around surfaces associated therewith, and wherein executing the computer-readable instructions on the mobile device further causes the mobile device to update the displayed augmented reality to show the one or more virtual objects from different perspectives based on movements associated with the mobile device.

65. The computer-readable storage medium of claim 45, wherein the mobile device comprises a smartphone, augmented reality glasses, augmented reality contact lenses, a head-mounted display, or augmented reality technology directly tied to a human brain.

66. The computer-readable storage medium of claim 45, wherein executing the computer-readable instructions on the mobile device further causes the mobile device to communicate criteria to restrict or control access to the one or more virtual objects to the augmented reality server.