Systems, Methods and Apparatuses of Seamless Integration of Augmented, Alternate, Virtual, and/or Mixed Realities with Physical Realities for Enhancement of Web, Mobile and/or Other Digital Experiences

• US 20190108686A1
• Filed: 07/27/2018
• Published: 04/11/2019
• Est. Priority Date: 07/28/2017
• Status: Active Grant

• Current Assignee: Magical Technologies LLC
• Original Assignee: Magical Technologies LLC
• Inventors: Spivack, Nova, Fu, Yenyun, Hoerl, Matthew, Pena, Armando
• Granted Patent: US 10,740,804 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 20/123 Shopping for digital content
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones

Abstract

Systems, Methods and Apparatuses of Seamless Integration of Augmented, Alternate, Virtual, and/or Mixed Realities with Physical Realities for Enhancement of Web, Mobile and/or Other Digital Experiences are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, to create an alternate reality environment of a physical location. The alternate reality environment can include: a representation of a real environment associated with the physical location and/or the virtual object. Moreover, in the alternate reality environment, one or more of, perceptibility of the virtual object and perceptibility of the representation of the real environment is configurable or adjustable by the human user.

Claims

1. A method to create an alternate reality environment of a physical location, the method, comprising:

causing to be perceptible or unperceptible, to a human user, a virtual object in the alternate reality environment;

wherein, the alternate reality environment includes;

a representation of a real environment associated with the physical location;

the virtual object;

further wherein, in the alternate reality environment, one or more of, perceptibility of the virtual object and perceptibility of the representation of the real environment is configurable or adjustable by the human user.

2. The method of claim 1, further comprising:

providing access to a virtual world component of the alternate reality environment to the human user, the virtual object being associated with the virtual world component;

wherein, the human user is affiliated with the virtual world component such that the human user is able to access the virtual world component in the alternate reality environment and has permission to perform one or more of;

access the virtual object in the virtual world component;

edit the virtual object in the virtual world component;

author additional virtual objects to be associated with the virtual world component.

3. The method of claim 1, wherein:

the alternate reality environment includes a first virtual world component and a second virtual world component;

wherein, the first virtual world component is affiliated with a first entity and the second virtual world component is affiliated with a second entity;

controlling access of a first set of virtual objects associated with the first virtual world in the alternate reality environment based on a first set of policies or permissions, the first set of policies or permissions being determined at least in part by the first entity;

controlling access of a second set of virtual objects associated with the second virtual world in the alternate reality environment based on a second set of policies or permissions, the second set of policies or permissions being determined at least in part by the second entity.

4. The method of claim 1, further comprising, updating a representation of the virtual object in the alternate reality environment, to depict action or inaction of the virtual object:

or updating the representation of the virtual object in the alternate reality environment in response to detecting one or more of interaction with the virtual object with respect to the virtual object and a command issued to the virtual object.

6. The method of claim 1, further comprising, updating a representation of the virtual object in the alternate reality environment, to depict action or inaction of the virtual object;

wherein, the action of the virtual object includes, generating an output or response to a query command to obtain metadata from the virtual object;

wherein, the query command is issued by one or more of, the human user, another human user and another virtual object.

7. The method of claim 1, further comprising, updating a representation of the virtual object in the alternate reality environment, to depict action or inaction of the virtual object;

wherein, the action of the virtual object includes, one or more of;

a talk action;

wherein, the talk action of the virtual object is initiated responsive to one or more of, a text query, a speech query or command received by the virtual object;

a visble action;

wherein the visible action of the virtual object causes a visual component of the representation of the virtual object to change for different users or difference devices used by the different users in the real environment to access the alternate reality environment;

an audible action;

wherein, the audible action causes an audible component of the representation of the virtual object to change for different users or difference devices used by the different users in the real environment to access the alternate reality environment.

10. The method of claim 1, further comprising, updating a representation of the virtual object in the alternate reality environment, to depict action or inaction of the virtual object;

wherein, the action of the virtual object includes a touch reaction;

wherein, the touch reaction causes, one or more of;

a surface of the virtual object to be interacted with;

internals of the virtual object to be interacted with or exposed;

one or more behaviors of the virtual object to be initiated or triggered;

further wherein, the touch reaction is initiated in response to action by one or more of, the human user, another human user, and another virtual object;

wherein, the touch reaction includes reaction to one or more of move, hit shoot, poke actions.

12. The method of claim 1,

wherein, an action of the virtual object includes one or more of;

(I) a collection reaction;

wherein, the collection reaction causes, one or more copies of the virtual object to be collected by the human user or another human user;

further wherein, the collection reaction is initiated in response to action by one or more of, the human user, the other human user;

(ii) a put reaction;

wherein, the put reaction causes, one or more copies of the virtual object to be placed in a different physical location;

further wherein the ut reaction is initiated in response to action by one or more of, the human user, the other human user;

(iii) a give reaction;

wherein, the give reaction causes, one or more copies of the virtual object to be given to another human user;

further wherein, the give reaction is initiated in response to action by the human user.

15. The method of claim 1, wherein:

an action of the virtual object includes one or more of;

(i) an annotation reaction;

wherein, the annotation reaction is initiated in response to an annotation action by the human user;

further wherein, the annotation action includes one or more of, commenting, liking, rating, tagging, and linking;

(ii) an edit reaction;

wherein, the edit reaction causes specification or metadata of the virtual object to be modified;

further wherein, the edit reaction is initiated in response to action by the human user;

(iii) a publish action;

wherein, the publish action enables the virtual object to publish one or more of;

content a notification, an event, a command to a content stream of multiple content streams associated with the virtual object.

18. The method of claim 1, wherein:

an action of the virtual object includes a subscribe/follow action;

wherein, the subscribe/follow action enables the virtual object to subscribe to content or events from an external content stream;

wherein, the external content stream is associated with another virtual object;

wherein, the external content stream associated with the other virtual object includes a public content stream which is publicly accessible in the alternate reality environment.

20. The method of claim 1, wherein:

an action of the virtual object includes autonomous behavior in the alternate reality environment;

wherein, the autonomous behavior includes one or more of;

chasing the human user, randomly roaming around a path or region, hiding, doing things to or interacting with other virtual objects, doing things to or interacting with the human user or another human user.

22. The method of claim 1, wherein:

an action of the virtual object includes behaviors determined by if-then rules;

wherein, an action of the virtual object includes one or more of, self destruction, shoot at the human user, get shot by the human user, being selectively hidden, react to user, change over time, morph from one type of object to another, move from the physical location to another physical location, traverse a path over time.

24. The method of claim 1, wherein:

the human user is represented as a further virtual object in the alternate reality environment;

wherein, the further virtual object representing the human user includes a 3D rendered avatar depicted to other human users in the alternate reality environment;

further comprising;

depicting. in the alternate reality environment, an information halo associated with the human user;

wherein, the information halo is depicted in or around a vicinity of a representation of the human user;

wherein the information halo includes one or more halo elements.

26. The method of claim 1, further comprising:

enabling the human user to follow the virtual object in the alternate reality environment;

generating a notification to notify the human user via the alternate reality environment when the virtual object is followed by the human user, responsive to occurrence of a triggering event associated with the virtual object;

wherein, the virtual object represents, one or more of, a second human user, a non-human user character, a place, a location, and a thing.

28. The method of claim 1, wherein:

the alternate reality environment comprises a browsing environment;

wherein the browsing environment enables the human user to discover one or more relevant objects in the alternate reality environment through, one or more of;

physical exploration of the physical location and areas physically surrounding the physical location;

virtual exploration of the alternate reality environment through digital navigation via a user interface of the alternate reality environment.

29. The method of claim 1, wherein:

the alternate reality environment comprises a commerce environment;

wherein, the commerce environment enables the human user to carry out a transaction with respect to another entity in relation to the virtual object;

wherein, the transaction enables the human user to carry out one or more of;

purchase the virtual object from the other entity;

sell the virtual object to the other entity;

rent the virtual object to or from the other entity;

license the virtual object to/from the other entity;

wherein, the virtual object represents one or more of;

a physical good in or near the physical location;

services offered in or near the real environment;

a virtual item with real world value;

a virtual item with value in the alternate reality environment.

30. The method of claim 1, wherein, the alternate reality environment is rendered at least in part in a hologram;

wherein, the hologram is accessible in 3D and in 360 degrees.

33. The method of claim 1,

wherein the representation of the real environment includes a scene of the real environment;

wherein the scene of the real environment includes one or more of, a photograph of the real environment, a photorealistic production or illustration of the real environment, a recorded video of the real environment, a live video or live stream of the real environment.

34. The method of claim 1, wherein, the virtual object is made perceptible or unperceptible to the human user in the alternate reality environment depending on a rule set;

wherein, the rule set includes one or more of, temporal parameter, a social parameter, a search parameter and a filter parameter.

37. The method of claim 1,

wherein, the virtual object is made perceptible or unperceptible to the human user in the alternate reality environment depending on a rule set;

wherein, the rule set includes a proximity parameter;

wherein, the virtual object is associated with a real place in the physical location;

wherein, the proximity parameter includes an indication of a physical distance between the human user and the real place.

5. (canceled)

8. (canceled)

9. (canceled)

11. (canceled)

13. (canceled)

14. (canceled)

16. (canceled)

17. (canceled)

19. (canceled)

21. (canceled)

23. (canceled)

25. (canceled)

27. (canceled)

31. (canceled)

32. (canceled)

35. (canceled)

36. (canceled)

38. A system to generate an augmented reality environment associated with a geographical location, the system, comprising:

a processor;

memory having stored having stored thereon instructions, which when executed by a processor, cause the processor to;

cause to be perceptible, to a human user, a virtual object in the augmented reality environment;

wherein, the augmented reality environment includes;

a representation of a real environment associated with the geographical location;

the virtual object;

further wherein, the augmented reality environment is accessible by the human user and a second human user.

39. The system of claim 38, wherein:

the augmented reality environment includes a first avatar to represent the human user and a second avatar to represent the second human user;

an action between the first and second avatars includes, one or more of, messaging, conducting transactions, friending or unfriending, sharing identity or contact information, sharing, one or more of a place, a location, an event and the virtual object with one another;

an action between the first and second human users in the augmented reality enviroment include, one or more of, virtual physical interactions, commenting, tagging and liking.

41. The system of claim 38

wherein, the alternate reality augmented includes a first avatar to represent the human user and a second avatar to represent a second human user;

wherein, the human user is able to friend the second human user in the augmented reality environment through initiating an action with the second avatar in the augmented reality environment;

wherein, the human user receives a notification in the alternate reality environment responsive to detection of a triggering event relating to the second human user.

42. The system of claim 38:

wherein, an interaction between the human user and the second human user in the augmented reality environment includes sending or leaving a message for the second human user by the human user;

wherein the message is associated with a given place in the geographical location;

further wherein, the message is accessible by the second human user when location data of the second human user meets a criteria in relation to the given place in the geographical location;

wherein the augmented reality enviroment enables the human user to filter for a set of virtual objects in augmented reality environment that are associated with the second human user;

wherein, the set of virtual objects associated with the second human user include one or more of, virtual objects accessed or purchased by the second human user.

44. The system of claim 38, wherein, the processor:

generates a first rendering of the augmented reality environment as viewed from a perspective of the human user;

wherein, a second human user of the augmented reality environment is able to access the first rendering of the augmented reality environment;

generates a second rendering of the augmented reality environment as viewed from a perspective of the second human user to include a projection of the first rendering.

45. The system of claim 38, wherein, the processor:

uses a wireless signal to determine a relative location of a second human user from a perspective of the human user in the geographical location;

wherein, alternate augmented environment includes a second avatar to represent the second human user;

positions a location of the second avatar in the human user'"'"'s view of the augmented reality environment based on the relative location of the second human user from the perspective of the human user in the geographical location.

47. The system of claim 38, wherein, the processor:

creates a further virtual object to include user profile metadata of the human user;

wherein, the further virtual object is to be depicted in the augmented reality environment;

wherein, the virtual object depicted in the augmented reality environment, includes an avatar associated with a second human user;

determines that a physical location data of the second human user meets a criteria;

causes the further virtual object to be perceptible to the second human user in the augmented reality environment;

enables the human user to engage in a social action with the second human user in the augmented reality environment.

49. The system of claim 38, wherein, the processor:

tracks a relationship between the first and second human user in the augmented reality environment;

facilitates and records the social action in which the first and second human user have engaged in the augmented reality environment;

updating a social graph for the alternate reality environment based on the social action between the first and second human user.

40. (canceled)

43. (canceled)

46. (canceled)

48. (canceled)

50. (canceled)

51. A machine-readable storage medium, having stored thereon instructions, which when executed by a processor, cause the processor to implement a method to create a mixed reality environment associated with a geographical location, the method, comprising:

causing to be discernible or indiscernible, to a human user, a virtual object in the mixed reality environment;

wherein, the mixed reality environment includes;

a representation of a physical environment associated with the geographical location;

the virtual object;

wherein, the virtual object includes metadata regarding a physical place in the geographical location depicted in the representation of the physical environment associated with the physical location.

52. The method of claim 51, wherein:

the virtual object is authored by a second human user, and accessible by the human user when location data of the human user meets a criteria in relation to the physical place in the geographical location;

the metadata includes one or more of, reviews, comments, suggestions, ratings, polls, social media data, tweets, feeds, posts, photos, and videos relating to the physical place in the the geographical location.

53. (canceled)