SYSTEMS AND METHODS FOR AUGMENTED REALITY

• US 20110141254A1
• Filed: 11/17/2010
• Published: 06/16/2011
• Est. Priority Date: 11/17/2009
• Status: Active Grant

• Current Assignee: The Active Network Incorporated (Cvent Incorporated)
• Original Assignee: The Active Network Incorporated (Cvent Incorporated)
• Inventors: ROEBKE, Mark J., DAVIDSON, Alistair Neil
• Granted Patent: US 8,730,312 B2
• Time in Patent Office: Days
• US Class Current: 348/61
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method for generating an augmented reality display on a display device according to embodiments of the present invention includes overlaying augmented reality data onto an actual image using three dimensional positioning data and three dimensional orientation data of the display device. The augmented reality data may include at least one line feature and/or at least one video feature, and may take into account both elevation of the device and the three-dimensional direction in which the imaging device is pointed.

Claims

1. A method comprising:

receiving image data from a device, wherein the image data represents visual data within a viewing range of the device;

receiving geographic position data about the device from a positioning system, the geographic position data including at least latitude, longitude, and elevation of the device;

receiving orientation data about the device from an orientation system, wherein the orientation data is three-dimensional orientation data;

querying a spatial database, the spatial database comprising one or more feature data sets, each of the one or more feature data sets including at least a three-dimensional location identification and a feature identifier;

receiving an indication of a feature range distance, wherein a field of view includes the viewing range of the device within the feature range distance from the device;

generating a displayed feature data set by identifying one or more of the one or more feature data sets whose three-dimensional location identification intersects the field of view; and

merging the displayed feature data with the image data to create an augmented reality display.

2. The method of claim 1, displaying the augmented reality display.

3. The method of claim 2, further comprising displaying the augmented reality display on a screen, wherein the displayed feature data comprises a virtual feature, and wherein a position of the virtual feature on the screen corresponds to an actual feature represented by the virtual feature.

4. The method of claim 2, wherein displaying the augmented reality display comprises overlaying the displayed feature data set onto the image data.

5. The method of claim 1, wherein at least two of the one or more feature data sets include a common feature identifier, and wherein the at least two of the one or more feature data sets are part of a line feature.

6. The method of claim 5, wherein the line feature includes a start point and an end point, wherein the start point is within the displayed feature data set, wherein the end point is not within the displayed feature data set, wherein the device is within a selected distance of the start point, and wherein the augmented reality display includes a representation of the start point but not the end point.

7. The method of claim 5, wherein the line feature includes a start point and an end point, wherein the start point is not within the displayed feature data set, wherein the end point is within the displayed feature data set, wherein the device is within a selected distance of the end point, and wherein the augmented reality display includes a representation of the end point but not the start point.

8. The method of claim 5, wherein the line feature includes a start point and an end point, wherein the device is outside of a selected distance from the start point and the end point and within the selected distance from the line feature, and wherein the augmented reality display includes a representation of a lowest end point or a highest start point of the line feature that intersects the field of view.

9. The method of claim 8, wherein the start point is the highest start point.

10. The method of claim 5, wherein the line feature includes a start point and an end point, wherein the device is outside of a selected distance from the start point and the end point and the line feature, and wherein the augmented reality display includes a representation of a direction of the line feature.

11. The method of claim 2, further comprising displaying additional information about a feature data set in response to a user request.

12. The method of claim 11, wherein the additional information comprises a distance from the device to a location of the feature data set.

13. The method of claim 2, wherein displaying the augmented reality display comprises:

identifying one or more of the one or more feature data sets in which the at least one three-dimensional location identification is within a selected distance from the device; and

displaying the one or more of the one or more feature data sets at the elevation of the device.

14. The method of claim 1, wherein the augmented reality display includes at least one video feature.

15. A method for generating an augmented reality display on a display device, the method comprising overlaying augmented reality data onto an actual image using three dimensional positioning data and three dimensional orientation data of the display device.

16. The method of claim 15, wherein the augmented reality data comprises at least one line feature.

17. The method of claim 15, wherein the augmented reality data comprises at least one video point feature.

18. A computer readable medium including instructions that, when executed by a processor, cause the processor to:

receive image data from a device, wherein the image data represents visual data within a viewing range of the device;

receive geographic position data about the device from a positioning system, the geographic position data including at least latitude, longitude, and elevation of the device;

receive orientation data about the device from an orientation system, wherein the orientation data is three-dimensional orientation data;

query a spatial database, the spatial database comprising one or more feature data sets, each of the one or more feature data sets including at least a three-dimensional location identification and a feature identifier;

receive an indication of a feature range distance, wherein a field of view includes the viewing range of the camera within the feature range distance from the device;

generate a displayed feature data set by identifying one or more of the one or more feature data sets whose three-dimensional location identification intersects the field of view; and

merge the displayed feature data with the image data to create an augmented reality display.

19. A system comprising:

a camera configured to generate image data, the camera having a viewing range;

a positioning system configured to generate geographic position data, the geographic position data including at least latitude, longitude, and elevation of the camera;

an orientation system configured to generate three-dimensional orientation data;

a display device; and

a computing device configured to;

receive the image data, the geographic position data, and the three-dimensional orientation data;

receive an indication of a feature range distance, wherein a field of view includes the viewing range of the camera within the feature range distance from the device;

query a spatial database, the spatial database comprising one or more feature data sets, each of the one or more feature data sets including at least a three-dimensional location identification and a feature identifier;

generate a displayed feature data set by identifying one or more of the one or more feature data sets whose three-dimensional location identification intersects the field of view;

merge the displayed feature data with the image data to create an augmented reality display; and

display the augmented reality display on the display device.

20. The system of claim 19, wherein the positioning system is a global positioning system.

21. The system of claim 19, wherein the orientation system comprises a compass.

22. The system of claim 19, wherein the orientation system comprises an accelerometer.

23. The system of claim 19, wherein the orientation system comprises a tilt angle sensor.

24. The system of claim 21, wherein the orientation system further comprises an accelerometer and a tilt sensor.