Architecture for augmented reality environment

• US 9,007,473 B1
• Filed: 03/30/2011
• Issued: 04/14/2015
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Worley, William Spencer III, Weatherford, William Thomas, Coley, Christopher, Crump, Edward Dietz
• Primary Examiner(s): DIEP, TRUNG T
• Application Number:
  US13/075,948
• Time in Patent Office: 1,476 Days
• Field of Search: 348/42, 348/115, 348/207.11, 348/211.11, 348/211.12, 348/211.1, 348/211.3, 348/211.99, 348/744, 709/217
• US Class Current: 348/211.11
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 21/2347 involving video stream encryption
• CPC Class Codes: H04N 9/31 Projection devices for colour picture display , e.g. using electronic spatial light modulators [ESLM]
• CPC Class Codes: H04N 9/3179 Video signal processing therefor
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

An architecture has one or more systems for generating augmented reality environments configured to access cloud services over a network. A user is authenticated within the environments, and once authenticated is permitted to interact with the cloud services using the augmented reality environments as well as client resources provided within the environments. The client resources may include devices or things that are primary intended for use within the environments, and devices or things that are not typically associated with augmented reality environments. The architecture enables the client resources to function as thin client interfaces to the cloud services.

Claims

1. A system comprising:

an augmented reality functional node having at least a projector and a camera for generating an augmented reality environment;

a spatial analysis module to cause the projector to project structured light onto physical surfaces in the augmented reality environment and analyze the structured light to at least determine information about surfaces within the augmented reality environment;

an authentication module to identify and authenticate a user as a specific user in the augmented reality environment based at least in part on a facial image captured using the camera of the augmented reality functional node, the authentication module to grant the user a privilege to at least temporarily interact with the augmented reality environment in response to identification and authentication of the user;

a network interface to connect the augmented reality functional node to a remote computing system; and

one or more client resources available within the augmented reality environment that, in response to use by the user, are configured to be identified and tracked by the augmented reality functional node based at least in part on the structured light that is used to determine the information about the surfaces, the one or more client resources to facilitate interaction with the remote computing system.

2. The system of claim 1, wherein the authentication module resides locally within the augmented reality environment.

3. The system of claim 1, wherein the augmented reality functional node includes a computing device, and the network interface and the authentication module are implemented by the computing device.

4. The system of claim 1, wherein the authentication module resides within the remote computing system.

5. The system of claim 1, wherein the remote computing system comprises cloud services accessible via a network.

6. The system of claim 1, wherein at least some of the client resources are separate from the augmented reality functional node and comprise passive elements with surfaces onto which images may be projected by the projector of the augmented reality functional node.

7. The system of claim 1, wherein the client resources comprise functional devices that are independently equipped to enable access to the remote computing system.

8. The system of claim 1, wherein the client resources comprise mobile communication devices.

9. The system of claim 1, wherein the user is a first user who is granted first access rights by the authentication module, and wherein the authentication module further identifies and authenticates a second user using at least the camera of the augmented reality functional node, the second user being granted second access rights by the authentication module in response to identification and authentication of the second user and while the first user maintains the first access rights.

10. The system of claim 9, wherein the second access rights are different from the first access rights.

11. A method comprising:

generating an augmented reality environment within a scene, the augmented reality environment having one or more resources available within the scene to facilitate user interaction with remote computing services;

projecting structured light onto surfaces within the augmented reality environment, the structured light detected by an imaging device enabling identification of the surfaces within the augmented reality environment;

authenticating a user within the scene based at least in part on facial imagery or profile imagery captured using the imaging device to determine an identity of the user and to determine at least one permission associated with the user, wherein the facial imagery or the profile imagery includes a three-dimensional model created from detected structured light captured by the imaging device; and

upon authentication of the user associated with the at least one permission, enabling access by the user to the remote computing services using the one or more resources available within the scene.

12. The method of claim 11, wherein the one or more resources are configured to submit requests to the remote computing services and receive results.

13. The method of claim 11, wherein the user is granted one of multiple different levels of access based on the at least one permission associated with the user.

14. The method of claim 11, wherein at least one of the resources comprises a passive surface onto which images may be projected as part of the augmented reality environment, the passive surface being disconnected and separate from the imaging device.

15. The method of claim 11, wherein at least one of the resources comprises a non-electronic device.

16. The method of claim 11, wherein at least one of the resources comprises an electronic device having at least one input component or output component.

17. The method of claim 11, further comprising detecting entry of the user into the augmented reality environment, and wherein the authenticating the user is performed in response to the detecting the entry of the user into the augmented reality environment.

18. The method of claim 11, wherein the authenticating further comprises capturing an interactive movement of the user while the user is within the augmented reality environment.

19. The method of claim 11, wherein the authenticating further comprises receiving an audible voice command from the user while the user is within the augmented reality environment.

20. The method of claim 11, wherein the authenticating further comprises receiving a non-vocal audio command from the user while the user is within the augmented reality environment.

21. The method of claim 11, wherein the authenticating further comprises authenticating first user data and second user data which is different from the first user data.

22. The method of claim 21, wherein the first user data comprises a biometric parameter and the second user data comprises data other than a biometric parameter.

23. The method of claim 21, wherein the first user data comprises a biometric parameter and the second user data comprises confirmation of presence of a device belonging to the user.

24. A method comprising:

receiving user data about a user, the user data being captured while the user is within an augmented reality environment, the user data including at least facial imagery or profile imagery captured using an imaging device;

authenticating the user based at least in part on the user data to determine an identity of the user and to determine that the user has permission to access a remote computing service, wherein the authenticating comprises at least a first level of authentication that is conducted locally within the augmented reality environment and a second level of authentication that is conducted remotely;

receiving a request from a resource that is resident in the augmented reality environment, the request being generated as a result of the authenticated user interacting with the resource within the augmented reality environment; and

serving, via the remote computing service, responsive to the request, at least one of application service or data to the resource.

25. The method of claim 24, wherein the authenticating comprises verifying the identity of the user based on the at least facial imagery or profile imagery captured using the imaging device.

26. The method of claim 24, wherein the user data further comprises a biometric parameter obtained from the at least facial imagery or profile imagery.

27. The method of claim 24, wherein the resource comprises an augmented reality environment (ARE) resource that is enabled as part of the augmented reality environment.

28. The method of claim 24, wherein the resource comprises a non-augmented reality environment (non-ARE) resource that is present within, but separate from, the augmented reality environment.

29. The method of claim 24, further comprising projecting structured light onto surfaces within the augmented reality environment, the structured light detected by an imaging device enabling identification of the surfaces within the augmented reality environment;

and wherein the facial imagery or the profile imagery includes a three-dimensional model created from detected structured light captured by the imaging device.

30. The method of claim 24, further comprising assessing a level of access based on the authenticating and granting access to certain application services or data based on the access level.

31. One or more non-transitory computer-readable media storing compute-executable instructions that, when executed by one or more processors, cause the one or more processors to:

generate an augmented reality environment within a scene that includes a first user and a second user, the augmented reality environment having one or more resources available within the scene to facilitate user interaction of multiple users with remote computing services;

authenticate the first user within the scene based at least in part on facial imagery of the first user or profile imagery of the first user captured using an imaging device to determine an identity of the first user and to determine at least one permission associated with the first user; and

authenticate the second user within the scene based at least in part on facial imagery of the second user or profile imagery of the second user captured using the imaging device to determine an identity of the second user and to determine at least one permission associated with the second user.

32. The one or more non-transitory computer-readable media of claim 31, wherein the at least one permission associated with the first user is different than the at least one permission associated with the second user.

33. The one or more non-transitory computer-readable media of claim 31, wherein at least one of the resources comprises a passive surface onto which images may be projected as part of the augmented reality environment, the passive surface being disconnected and separate from the imaging device.

34. The one or more non-transitory computer-readable media of claim 31, wherein at least one of the resources comprises an electronic device having at least one input component or output component.

35. The one or more non-transitory computer-readable media of claim 31, wherein the authenticating further comprises capturing an interactive movement of the first user while the first user is within the augmented reality environment.

36. The one or more non-transitory computer-readable media of claim 31, wherein the authenticating further comprises receiving at least one of an audible voice command from the first user or a non-vocal audio command caused by the first user while the first user is within the augmented reality environment.

37. The one or more non-transitory computer-readable media of claim 31, wherein the authenticating further comprises authenticating biometric data and user data which is different from the biometric data.