Virtual reality applications

• US 9,690,457 B2
• Filed: 08/24/2012
• Issued: 06/27/2017
• Est. Priority Date: 08/24/2012
• Status: Expired due to Fees

• Current Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Levien, Roy, Malamud, Mark
• Primary Examiner(s): BARRETT, RYAN S
• Application Number:
  US13/821,560
• Publication Number:
  US 20140059458A1
• Time in Patent Office: 1,768 Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 8/61 Installation
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

Augmented reality technology is described. The technology can detect objects in a scene, identifying one or more installed or available applications based on the detected objects, and place icons representing the identified applications proximate to the detected objects in a display of the scene, e.g., so that a user can start or install the identified applications. The technology can also facilitate interaction with an identified object, e.g., to remotely control a recognized object.

Claims

1. A method performed by a computing system having a processor, the method comprising:

digitizing a vision of a scene received from an image capture device to provide a digitized scene;

detecting multiple objects in the digitized scene, wherein detecting the multiple objects comprises recognizing digitized objects in the digitized scene;

receiving contextual information associated with a user, wherein the contextual information comprises first contextual information and second contextual information;

identifying installed applications based on the detected objects, wherein the identifying includes identifying the installed applications based on one of the first contextual information or the second contextual information, further wherein the identifying includes searching a list of attributes for an attribute associated with one or more of the detected objects, the list of attributes provided by the user and associated with at least one of the installed applications;

identifying an object from the detected objects based on another one of the first contextual information or the second contextual information to provide the identified object;

placing an icon that represents at least one of the identified installed applications proximate to the identified object in a display of the digitized scene, wherein the at least one of the identified installed applications is selected based on the identified object;

receiving a user selection of the icon;

responsive to the reception of the user selection of the icon, selecting the identified object;

determining that the identified object can be interacted with, wherein the determination comprises querying a database about the identified object;

obtaining at least one specification regarding an interaction with the identified object, wherein the at least one specification includes information regarding commands to interact with the identified object and information regarding user interface elements; and

providing a user interface to interact with the identified object based on the at least one specification in accordance with the user interface elements.

2. The method of claim 1, wherein the digitized scene is displayed on an output device.

3. The method of claim 1, wherein at least one of the first contextual information or the second contextual information is received via an input from the user.

4. The method of claim 1, wherein at least one of the first contextual information or the second contextual information is based on detected positional information.

5. The method of claim 1, wherein detecting the multiple objects includes employing one or more image recognition methods.

6. The method of claim 1, further comprising causing an installed application that corresponds to the icon to launch.

7. The method of claim 1, wherein the list of attributes is received via a keyboard input.

8. The method of claim 1, further comprising:

receiving an input to associate an installed application not identified with an object detected in the digitized scene, and associating the installed application with the detected object;

receiving an input to disassociate an application identified in correspondence with an object detected in the digitized scene, and disassociating this application from the detected object;

or a combination thereof.

9. The method of claim 1, wherein at least one of the first contextual information or the second contextual information comprises time of day information, and wherein identifying the object from the multiple objects is based on the time of day information.

10. The method of claim 1, wherein at least one of the first contextual information or the second contextual information includes time of day information, and wherein the time of day information is received via an input from the user.

11. The method of claim 1, further comprising:

identifying applications not stored locally on the computing system based on the detected objects, wherein identifying the applications includes identifying applications not stored locally based on one of the first contextual information or the second contextual information, wherein identifying the applications includes searching a list of attributes for an attribute associated with one or more of the detected objects, and wherein the list of attributes provided by the user and associated with at least one of the applications is not stored locally; and

placing an icon that, when selected by the user, installs a corresponding application on the computing device, wherein the icon is placed proximate to the identified object in a display of the digitized scene.

12. The method of claim 1, wherein providing the user interface to interact with selected object based on the at least one specification comprises:

providing, from the at least one specification, instructions to provide aspects of the user interface; and

providing the aspects of the user interface to control the selected object.

13. The method of claim 1, wherein providing the user interface to interact with selected object based on the at least one specification comprises:

providing, from the specification, instructions regarding the user interface; and

providing the user interface to control the selected object.

14. A computer-readable storage device having stored thereon instructions that, if executed by a computing device, cause the computing device to perform operations comprising:

digitize a vision of a scene to provide a digitized scene;

detect multiple objects in the digitized scene, wherein the detection of the multiple objects comprises recognition of digitized objects in the digitized scene;

receive contextual information associated with a user, wherein the contextual information comprises first contextual information and second contextual information;

identify installed applications based on the detected objects, wherein the identification includes identification of the installed applications based on one of the first contextual information or the second contextual information, further wherein the identification includes searching a list of attributes for an attribute associated with one or more of the detected objects, the list of attributes provided by the user and associated with at least one of the installed applications;

identify an object from the detected objects based on another one of the first contextual information or the second contextual information;

place an icon, that when selected by the user, launches at least one of the identified installed applications proximate to the identified object in a display of the digitized scene, wherein the at least one of the identified installed applications is selected based on the identified object;

receive a user selection of the icon;

responsive to the reception of the user selection of the icon, select the object;

determine that the object can be interacted with, wherein the determination comprises querying a database about the object,

obtain a specification regarding an interaction with the object, wherein the specification includes commands to interact with the object and user interface elements; and

provide a user interface to interact with the object based on the specification in accordance with the user interface elements.

15. The computer-readable storage device of claim 14, wherein the digitized scene is displayed on an output device.

16. The computer-readable storage device of claim 14, wherein at least one of the first contextual information or the second contextual information is received via an input from the user.

17. The computer-readable storage device of claim 14, wherein at least one of the first contextual information or the second contextual information is based on detected positional information.

18. The computer-readable storage device of claim 14, wherein the detection of the multiple objects includes employment of one or more image recognition methods.

19. The computer-readable storage device of claim 14, wherein the list of attributes is received via a keyboard input.

20. The computer-readable storage device of claim 14, wherein the operations further comprise:

receive an input to associate an installed application not identified with an object detected in the digitized scene, and associate the installed application with the detected object;

receive an input to disassociate an application identified in correspondence with an object detected in the digitized scene, and disassociate this application from the detected object;

or a combination thereof.

21. The computer-readable storage device of claim 14, wherein at least one of the first contextual information or the second contextual information comprises time of day information, and wherein the identification of the object from the multiple objects is based on the time of day information.

22. The computer-readable storage device of claim 14, wherein at least one of the first contextual information or the second contextual information includes time of day information, and wherein the time of day information is received via an input from the user.

23. The computer-readable storage device of claim 14, wherein the operations further comprise:

identify applications not installed based on the detected objects, wherein the identification of the applications includes identification of applications not installed based on one of the first contextual information or the second contextual information; and

place an icon that, when selected by the user, installs a corresponding application, the icon placed proximate to the identified object in a display of the digitized scene.

24. A system, comprising:

a processor; and

a memory comprising instructions that, in response to execution by the processor, cause the system to perform operations comprising;

detect multiple objects in a digitized scene, wherein the detection of the multiple objects comprises recognition of digitized objects in the digitized scene;

receive contextual information associated with a user, wherein the contextual information comprises first contextual information and second contextual information;

identify installed applications based on the detected objects and on one of the first contextual information or the second contextual information, wherein the identification of the installed applications includes search a list of attributes for an attribute associated with one or more of the detected objects, the list of attributes provided by the user and associated with at least one of the installed applications;

identify an object from the detected objects based on another one of the first contextual information or the second contextual information;

place an icon, that when selected by the user launches at least one of the identified installed applications proximate to the identified object in a display of the digitized scene, wherein the at least one of the identified installed applications is selected based on the identified object;

receive a user selection of the icon, wherein the user selection launches at least one of the identified installed applications;

responsive to the reception of the user selection of the icon, select the identified object;

determine that the identified object can be interacted with, wherein the determination comprises a query of a database about the object;

obtain at least one specification regarding an interaction with the identified object, wherein the at least one specification includes information regarding commands to interact with the identified object and information regarding user interface elements; and

provide a user interface to interact with the object based on the at least one specification.

25. The system of claim 24, wherein the system further comprises a digital camera, and wherein the memory further comprises instructions to receive the digitized vision of the digitized scene.

26. The system of claim 24, wherein the digitized scene is displayed on an output device.

27. The system of claim 24, wherein at least one of the first contextual information or the second contextual information is received via an input from the user.

28. The system of claim 24, wherein at least one of the first contextual information or the second contextual information is based on detected positional information.

29. The system of claim 24, wherein the instructions to detect include instructions to use one or more image recognition methods.

30. The system of claim 24, wherein the list of attributes is received via a keyboard input.

31. The system of claim 24, further comprising instructions to:

receive an input to associate an installed application not identified with an object detected in the digitized scene, and associate the installed application with the detected object;

receive an input to disassociate an application identified in correspondence with an object detected in the digitized scene, and disassociate this application from the detected object;

or a combination thereof.

32. The system of claim 24, wherein at least one of the first contextual information or the second contextual information comprises time of day information, and wherein the identification of the object from the multiple objects is based on the time of day information.

33. The system of claim 24, wherein at least one of the first contextual information or the second contextual information includes time of day information and wherein the time of day information is received via an input from the user.