Method and apparatus for object identification and location correlation based on received images

• US 10,346,683 B2
• Filed: 10/26/2017
• Issued: 07/09/2019
• Est. Priority Date: 12/31/2014
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Anastassov, Anton, Giurgiu, Gavril Adrian, Jang, Dongwook
• Primary Examiner(s): Yentrapati, Avinash
• Application Number:
  US15/794,601
• Publication Number:
  US 20180060655A1
• Time in Patent Office: 621 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps

Abstract

A method, apparatus and computer program product are provided for object identification and location correlation based on received images. A method is provided including receiving identity data associated with an object derived from one or more of a plurality of images determining an identity of an object in one or more of the plurality of images, receiving location information associated with a camera which captured the plurality of images, and correlating the identified object with a location for each of the respective images of the plurality of images.

Claims

1. A method comprising:

receiving images from two or more perspectives of an object in an environment;

determining identity data associated with the object within the received images, wherein the object is identified by the identity data;

receiving an indication of a location at which each of the two or more images were captured;

determining an angle of observation for the object identified in each of the two or more images relative to the location at which the image was captured; and

establishing a location of the object based, at least in part, on a position of the object within each of the received images, the associated angle of observation of the object for each of the received images, and a location at which each of the two or more images were captured,

wherein establishing a location of the object is further based on an offset distance from a location at which a respective image was captured, wherein the offset is of a predetermined distance based on a position of the object within the respective image.

2. The method of claim 1, wherein determining identity data comprises performing pattern recognition on the image to determine identity data associated with the object.

3. The method of claim 1, wherein the indication of the location at which at least one of the two or more images was captured is established based on interpolation.

4. The method of claim 3, wherein establishing the indication of the location based on interpolation comprises interpolating between two locations using a weighted average based on:

a first location of an image capture device that captured the at least one of the two or more images before the image was captured and an associated first time stamp;

a second location of the image capture device that captured the at least one of the two or more images after the image was captured and an associated second time stamp; and

a third time stamp indicating a time at which the at least one of the two or more images was captured, wherein a weight afforded to each of the first location and the second location is established based on the relative proximity of the third time stamp to the respective first time stamp and second time stamp.

5. The method of claim 1, wherein establishing the location of the object based, at least in part on a position of the object within each of the received images, the associated angle of observation of the object for each of the received images, and the location at which each of the two or more images were captured is performed based on triangulation of known locations and angles to establish the location of the object.

6. The method of claim 1, wherein the predetermined distance is a first predetermined distance in response to the object appearing on a left side of the respective captured image, the predetermined distance is a second predetermined distance in response to the object appearing on a right side of the respective captured image, and the predetermined distance is a third predetermined distance in response to the object appearing at a top of the respective captured image.

7. The method of claim 1, further comprising:

updating stored map data based on the established location of the object in response to a probability of the established location of the object satisfying a threshold.

8. The method of claim 7, wherein updating stored map data comprises plotting the object on the location on a map of the stored map data.

9. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the processor, cause the apparatus to at least:

receive images from two or more perspectives of an object in an environment;

determine identity data associated with the object within the received images, wherein the object is identified by the identity data;

receive an indication of a location at which each of the two or more images were captured;

determine an angle of observation for the object identified in each of the two or more images relative to the location at which the image was captured; and

establish a location of the object based, at least in part, on a position of the object within each of the received images, the associated angle of observation of the object for each of the received images, and a location at which each of the two or more images were captured,

wherein causing the apparatus to establish a location of the object is further based on an offset distance from a location at which a respective image was captured, wherein the offset is of a predetermined distance based on a position of the object within the respective image.

10. The apparatus of claim 9, wherein causing the apparatus to determine identity data comprises causing the apparatus to perform pattern recognition on the image to determine identity data associated with the object.

11. The apparatus of claim 9, wherein the indication of the location at which at least one of the two or more images was captured is established based on interpolation.

12. The apparatus of claim 11, wherein causing the apparatus to establish the indication of the location based on interpolation comprises causing the apparatus to interpolate between two locations using a weighted average based on:

a first location of an image capture device that captured the at least one of the two or more images before the image was captured and an associated first time stamp;

a second location of the image capture device that captured the at least one of the two or more images after the image was captured and an associated second time stamp; and

a third time stamp indicating a time at which the at least one of the two or more images was captured, wherein a weight afforded to each of the first location and the second location is established based on the relative proximity of the third time stamp to the respective first time stamp and second time stamp.

13. The apparatus of claim 9, wherein causing the apparatus to establish the location of the object based, at least in part on a position of the object within each of the received images, the associated angle of observation of the object for each of the received images, and the location at which each of the two or more images were captured is performed based on triangulation of known locations and angles to establish the location of the object.

14. The apparatus of claim 9, wherein the predetermined distance is a first predetermined distance in response to the object appearing on a left side of the respective captured image, the predetermined distance is a second predetermined distance in response to the object appearing on a right side of the respective captured image, and the predetermined distance is a third predetermined distance in response to the object appearing at a top of the respective captured image.

15. The apparatus of claim 9, wherein the apparatus is further caused to:

update stored map data based on the established location of the object in response to a probability of the established location of the object satisfying a threshold.

16. The apparatus of claim 15, wherein causing the apparatus to update stored map data comprises causing the apparatus to plot the object on the location on a map of the stored map data.

17. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code portions stored therein, the computer-executable program code portions comprising program code instructions configured to:

receive images from two or more perspectives of an object in an environment;

determine identity data associated with the object within the received images, wherein the object is identified by the identity data;

receive an indication of a location at which each of the two or more images were captured;

determine an angle of observation for the object identified in each of the two or more images relative to the location at which the image was captured; and

establish a location of the object based, at least in part, on a position of the object within each of the received images, the associated angle of observation of the object for each of the received images, and a location at which each of the two or more images were captured,

wherein establishment of a location of the object is further based on an offset distance from a location at which a respective image was captured, wherein the offset is of a predetermined distance based on a position of the object within the respective image.

18. The computer program product of claim 17, wherein the program code instructions configured to establish the location of the object based, at least in part on a position of the object within each of the received images, the associated angle of observation of the object for each of the received images, and the location at which each of the two or more images were captured is performed based on computer program code instructions configured to triangulate known locations and angles to establish the location of the object.