Mobile augmented reality system
First Claim
Patent Images
1. An apparatus comprising:
- a processor; and
a memory including instructions that, when executed by the processor, cause the processor to;
detect a surface of a structure included in a live view captured by an image sensor, the structure to include a point-of-interest (POI),determine a position and an orientation of the image sensor relative to the POI based, at least in part, on the detected surface of the structure included in the live view and data from a second sensor,extract, from the live view, a visual feature of the structure based on the position and the orientation, the visual feature associated with the POI,track the position and the orientation of the image sensor while the image sensor captures the live view, andgenerate display data including image data associated with the visual feature to augment the live view, the image data being three-dimensionally consistent with the position and the orientation of the image sensor when displayed in the live view at the POI on the detected surface of the structure.
2 Assignments
0 Petitions
Accused Products
Abstract
Embodiments of the invention relate to systems, apparatuses and methods to provide image data, augmented with related data, to be displayed on a mobile computing device. Embodiments of the invention display a live view augmented with information identifying an object amongst other objects. Embodiments of the invention may utilize other related data, such as 3D point cloud data, image data and location data related to the object, to obtain a specific location of an object within the live view. Embodiments of the invention may further display a live view with augmented data three-dimensionally consistent with the position and orientation of the image sensor of the mobile computing device.
-
Citations
20 Claims
-
1. An apparatus comprising:
-
a processor; and a memory including instructions that, when executed by the processor, cause the processor to; detect a surface of a structure included in a live view captured by an image sensor, the structure to include a point-of-interest (POI), determine a position and an orientation of the image sensor relative to the POI based, at least in part, on the detected surface of the structure included in the live view and data from a second sensor, extract, from the live view, a visual feature of the structure based on the position and the orientation, the visual feature associated with the POI, track the position and the orientation of the image sensor while the image sensor captures the live view, and generate display data including image data associated with the visual feature to augment the live view, the image data being three-dimensionally consistent with the position and the orientation of the image sensor when displayed in the live view at the POI on the detected surface of the structure. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A memory comprising instructions that, when executed by at least one machine, cause the at least one machine to at least:
-
detect a surface of a structure included in a live view captured by an image sensor, the structure to include a point-of-interest (POI); determine a position and an orientation of the image sensor relative to the POI based, at least in part, on the detected surface of the structure included in the live view and data from a second sensor; extract, from the live view, a visual feature of the structure based on the position and the orientation, the visual feature associated with the POI; track the position and the orientation of the image sensor while the image sensor captures the live view; and generate display data including image data associated with the visual feature to augmenting the live view, the image data being three-dimensionally consistent with the position and the orientation of the image sensor when displayed in the live view at the POI on the detected surface of the structure. - View Dependent Claims (13, 14, 15)
-
-
16. A method comprising:
-
accessing data identifying a point-of-interest (POI); detecting, by executing an instruction with at least one processor, a surface of a structure included in a live view captured by an image sensor, the structure to include the POI; determining, by executing an instruction with the at least one processor, a position and an orientation of the image sensor relative to the POI based, at least in part, on the detected surface of the structure included in the live view and data from a second sensor; extracting, from the live view, a visual feature of the structure using the position and the orientation, the visual feature associated with the POI; tracking the position and the orientation of the image sensor while the image sensor captures the live view; and generating, by executing an instruction with the at least one processor, display data including image data associated with the visual feature for augmenting the live view, wherein the image data is three-dimensionally consistent with the position and the orientation of the image sensor when displayed in the live view at the POI on the detected surface of the structure. - View Dependent Claims (17, 18, 19, 20)
-
Specification