Object-based localization
First Claim
1. An object-based localization system comprising:
- a frame data receiving module configured to receive, from a remote client device via a data communication network, image frame data for a plurality of frames captured by an imaging camera included in the remote client device, the plurality of frames including a first frame;
an object detector configured to receive the image frame data, automatically detect a plurality of real-world objects captured in the image frame data, and automatically classify the plurality of real-world objects as being associated with respective object classes;
an object class and instance dependent keypoint selector configured to automatically identify a plurality of object class and instance dependent keypoints for the plurality of real-world objects based on the object classes associated with the plurality of real-world objects;
a location selector module configured to select a current location record from a database based on at least a determination that the object classes associated with the plurality of real-world objects correlate with object classes associated with object instances identified by the selected current location record; and
a localization module configured to estimate a first pose of the remote client device for the first frame based on at least the identified plurality of object class and instance dependent keypoints and location data indicated by the selected current location record, and to transmit the estimated first pose to the remote client device.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques for localizing a device based on images captured by the device. The techniques include receiving, from a device via a data communication network, image frame data for frames captured by an imaging camera included in the device, the frames including a first frame, automatically detecting real-world objects captured in the image frame data, automatically classifying the detected real-world objects as being associated with respective object classes, automatically identifying object class and instance dependent keypoints for the real-world objects based on the object classes associated with the real-world objects, and estimating a pose of the device for the first frame based on at least the identified object class and instance dependent keypoints.
-
Citations
18 Claims
-
1. An object-based localization system comprising:
-
a frame data receiving module configured to receive, from a remote client device via a data communication network, image frame data for a plurality of frames captured by an imaging camera included in the remote client device, the plurality of frames including a first frame; an object detector configured to receive the image frame data, automatically detect a plurality of real-world objects captured in the image frame data, and automatically classify the plurality of real-world objects as being associated with respective object classes; an object class and instance dependent keypoint selector configured to automatically identify a plurality of object class and instance dependent keypoints for the plurality of real-world objects based on the object classes associated with the plurality of real-world objects; a location selector module configured to select a current location record from a database based on at least a determination that the object classes associated with the plurality of real-world objects correlate with object classes associated with object instances identified by the selected current location record; and a localization module configured to estimate a first pose of the remote client device for the first frame based on at least the identified plurality of object class and instance dependent keypoints and location data indicated by the selected current location record, and to transmit the estimated first pose to the remote client device. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method of localizing a device, the method comprising:
-
receiving, from a remote client device via a data communication network, image frame data for a plurality of frames captured by an imaging camera included in the remote client device, the plurality of frames including a first frame; automatically detecting a plurality of real-world objects captured in the image frame data; automatically classifying the plurality of real-world objects as being associated with respective object classes; automatically identifying a plurality of object class and instance dependent keypoints for the plurality of real-world objects based on the object classes associated with the plurality of real-world objects; selecting a current location record from a database based on at least a determination that the object classes associated with the plurality of real-world objects correlate with object classes associated with object instances identified by the selected current location record; estimating a first pose of the remote client device for the first frame based on at least the identified plurality of object class and instance dependent keypoints and location data indicated by the selected current location record; and transmitting the estimated first pose to the remote client device. - View Dependent Claims (8, 9, 10, 11, 12, 13)
-
-
14. A method of localizing a device, the method comprising:
-
capturing, with an imaging camera included in the device, image frame data for a plurality of frames at a rate based at least in part on whether the imaging camera is capturing image frame data associated with a region of interest, the plurality of frames including a first frame captured at a first time; transmitting, via a data communication network, the image frame data from the device to a remote object-based localization system; receiving, at the device at a second time, a first pose estimate generated by the remote object-based localization system for the device for the first time; and determining, at the device, a second pose estimate of the device for a third time after the second time based on at least the received first pose estimate, wherein the second pose estimate is different than the first pose estimate. - View Dependent Claims (15, 16, 17, 18)
-
Specification