ASSIGNING A VIRTUAL USER INTERFACE TO A PHYSICAL OBJECT

• US 20150185825A1
• Filed: 12/30/2013
• Published: 07/02/2015
• Est. Priority Date: 12/30/2013
• Status: Abandoned Application

• Current Assignee: RPX Corporation
• Original Assignee: DAQRI, LLC
• Inventors: Mullins, Brian
• Application Number:
  US14/144,370
• Publication Number:
  US 20150185825A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A system and method for assigning a virtual user interface to a physical object is described. A virtual user interface for a physical object is created at a machine. The machine is trained to associate the virtual user interface with identifiers of the physical object and tracking data related to the physical object. The virtual user interface is displayed in relation to the image of the physical object.

Claims

1. A machine comprising:

a processor comprising an augmented reality application, the augmented reality application having;

a virtual user interface creation module configured to create a virtual user interface; and

a virtual user interface assigning module configured to associate the virtual user interface with identifiers of a physical object and tracking data related to the physical object, the virtual user interface displayed in relation to an image of the physical object.

2. The machine of claim 1, wherein the virtual user interface comprises a menu of information identifying the physical object.

3. The machine of claim 1, wherein the virtual user interface comprises interactive virtual functions associated with functions of the physical object.

4. The machine of claim 1, wherein the virtual user interface creation module is configured to create the virtual user interface from a selection of templates of virtual interfaces of similar physical objects.

5. The machine of claim 1, wherein the virtual user interface creation module is configured to create a custom virtual user interface.

6. The machine of claim 1, wherein the machine is a viewing device comprising:

a physical object identifier module configured to generate identifiers of the physical object comprising feature points of the physical object,

wherein the tracking data comprise a location of the physical object, a location of the viewing device viewing the physical object, an orientation of the viewing device, and a distance between the location of the viewing device and the location of the physical object.

7. The machine of claim 6, wherein the viewing device comprises:

an optical device configured to capture the image of the physical object;

a virtual user interface displaying module configured to;

determine the identifiers of the physical object and tracking data using the image of the physical object,

identify the physical object based on the identifiers of the physical object and tracking data, and

retrieve the virtual user interface associated with the physical object; and

a display configured to display the retrieved virtual user interface in relation to the image of the physical object.

8. The machine of claim 7, wherein the viewing device comprises:

a display in a mobile communication device hand held by a user or a transparent display mounted to a head of the user;

a plurality of sensors configured to determine a location and an orientation of the viewing device relative to the physical object in a physical environment local to the viewing device,

wherein the location includes a geographic location determined based on wireless data generated by the viewing device or triangulated from predefined references of the physical environment,

wherein the orientation is determined based on gyroscope data from the viewing device or is externally determined using a three-dimensional camera sensor.

9. The machine of claim 1, wherein the machine is a server comprising:

a storage device configured to store a database of virtual user interfaces and corresponding identifiers of physical objects and tracking data related to physical objects; and

a physical object detector module configured to;

receive the identifiers of the physical object and tracking data from a viewing device,

identify the physical object based on the identifiers of the physical object and tracking data and the database of identifiers of physical objects and tracking data related to the physical objects, and

retrieve the virtual user interface associated with the identified physical object.

10. The machine of claim 1, wherein the machine is a server comprising:

a storage device configured to store a database of virtual user interfaces and corresponding identifiers of physical objects and tracking data related to physical objects; and

a physical object detector module configured to;

receive the image of the physical object and tracking data from a viewing device,

generate feature points in the image of the physical object and tracking data from the viewing device;

identify the physical object based on the generated feature points of the image of the physical object and tracking data and the database of identifiers of physical objects and tracking data related to physical objects, and

retrieve the virtual user interface associated with the identified physical object.

11. A method comprising:

creating a virtual user interface for a physical object with a viewing device; and

training the viewing device to associate the virtual user interface with identifiers of the physical object and tracking data related to the physical object, the virtual user interface displayed in relation to an image of the physical object.

12. The method of claim 11, wherein the virtual user interface comprises a menu of information identifying the physical object.

13. The method of claim 11, wherein the virtual user interface comprises interactive virtual functions associated with functions of the physical object.

14. The method of claim 11, further comprising:

creating the virtual user interface from a selection of templates of virtual interfaces of similar physical objects.

15. The method of claim 11, further comprising:

creating a custom virtual user interface.

16. The method of claim 11, further comprising:

generating identifiers of the physical object comprising feature points of the physical object,

wherein the tracking data comprise a location of the physical object, a location of the viewing device viewing the physical object, an orientation of the viewing device, and a distance between the location of the viewing device and the location of the physical object.

17. The method of claim 16, further comprising:

capturing the image of the physical object;

determining a location and an orientation of the viewing device relative to the physical object in a physical environment local to the viewing device;

determining the identifiers of the physical object and tracking data using the image of the physical object;

identifying the physical object based on the identifiers of the physical object and tracking data;

retrieving the virtual user interface associated with the physical object; and

generating, in a display of the viewing device, the retrieved virtual user interface in relation to the image of the physical object,

wherein the location includes a geographic location determined based on wireless data generated by the viewing device or triangulated from predefined references of the physical environment,

wherein the orientation is determined based on gyroscope data from the viewing device or is externally determined using a three-dimensional camera sensor,

wherein the display comprises a transparent display mounted to a head of a user.

18. The method of claim 11, further comprising:

storing a database of virtual user interfaces and corresponding identifiers of physical objects and tracking data related to physical objects;

receiving the identifiers of the physical object and tracking data from the viewing device at a server;

identifying, at the server, the physical object based on the identifiers of the physical object and tracking data and the database of identifiers of physical objects and tracking data related to the physical objects; and

retrieving, from the server, the virtual user interface associated with the identified physical object.

19. The method of claim 11, further comprising:

storing a database of virtual user interfaces and corresponding identifiers of physical objects and tracking data related to physical objects;

receiving the image of the physical object and tracking data from the viewing device;

generating feature points in the image of the physical object and tracking data from the viewing device at a server;

identifying, at the server, the physical object based on the identifiers of the physical object and tracking data and the database of identifiers of physical objects and tracking data related to the physical objects; and

retrieving, from the server, the virtual user interface associated with the identified physical object.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

creating a virtual user interface for a physical object at a viewing device; and

training the viewing device to associate the virtual user interface with identifiers of the physical object and tracking data related to the physical object, the virtual user interface displayed in the viewing device in relation to an image of the physical object.