Methods for assessing image change and devices thereof

• US 9,633,050 B2
• Filed: 04/04/2014
• Issued: 04/25/2017
• Est. Priority Date: 02/21/2014
• Status: Active Grant

• Current Assignee: Wipro Limited
• Original Assignee: Wipro Limited
• Inventors: Lenka, Anil Kumar, Hosabettu, Raghavendra, Rugminibai, Abhijith Vijaya Kumar
• Primary Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US14/245,486
• Publication Number:
  US 20150242686A1
• Time in Patent Office: 1,117 Days
• Field of Search: None
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures

Abstract

A method, non-transitory computer readable medium, and an image analysis computing device that retrieves, based on a captured version of an object in a received image, training images which display related versions of the object and items of data related to the related versions of the object of the training images. Keypoints which are invariant to changes in scale and rotation in the captured version of the object in the received image and in the related versions of the object in the training images are determined. Changes to the object in the received image based on any of the determined keypoints in the related version of the object which do not match the determined keypoints in the captured version of the object are identified. The identified changes in the captured version of the object in the received image are provided.

Claims

1. A method for assessing image change, the method comprising:

retrieving, by an image analysis computing device, based on a captured version of an object in a received image, one or more training images which display one or more related versions of the object and one or more items of data related to the one or more related versions of the object of the one or more training images;

determining, by the image analysis computing device, one or more keypoints which are invariant to changes in scale and rotation in the captured version of the object in the received image and in the related versions of the object in the one or more training images;

identifying, by the image analysis computing device, one or more changes to the object in the received image based on any of the determined one or more keypoints in the related version of the object which do not match the determined one or more keypoints in the captured version of the object;

determining, by the image analysis computing device, one or more contours in the related versions of the object in the one or more training images by thresholding the one or more training images;

applying, by the image analysis computing device, a weightage to the one or more determined contours;

identifying, by the image analysis computing device, one or more parts of the object in the related versions of the object based on the determined one or more contours;

storing, by the image analysis computing device, the one or more identified parts of the related versions of the object, the applied weightage, and one or more items of data related to the one or more identified parts; and

providing, by the image analysis computing device, the identified one or more changes in the captured version of the object in the received image.

2. The method as set forth in claim 1 further comprising:

adjusting, by the image analysis computing device, the captured version of the object of the received image based on the related versions of the object of the one or more identified training images prior to the determining the one or more keypoints, wherein the adjusting comprises one or more of resizing and reorienting the captured version of the object of the received image based on the related version of the object of the one or more training images, adjusting image contrast, normalizing brightness, removing background, removing noise, or segmenting the received image.

3. The method as set forth in claim 1, wherein the one or more items of data related to the one or more identified parts comprises a material the one or more identified parts are made from.

4. The method as set forth in claim 1, wherein the identifying the one or more changes further comprises:

classifying, by the image analysis computing device, each of the determined one or more keypoints in the captured version of the object which do not match the determined one or more keypoints in the related versions of the object as one or more parts of the captured version of the object in the received image;

comparing, by the image analysis computing device, the classified one or more parts of the captured version of the object in the received image with the stored one or more parts in the related versions of the object in the one or more training images; and

identifying, by the image analysis computing device, the one or more changes based on the comparison of the classified one or more parts and the stored one or more parts.

5. The method as set forth in claim 4 wherein the classifying each of the determined one or more keypoints in the captured version of the object comprises:

determining, by the image analysis computing device, a cluster associated with each of the of the determined one or more keypoints in the captured version of the object in the received image which do not match the determined one or more keypoints in the related versions of the object in the one or more training images, wherein each cluster comprises a set of descriptors related to the determined one or more keypoints; and

identifying, by the image analysis computing device, the one or more parts of the captured version of the object in the received image associated with the determined unmatched keypoints based on the identified cluster.

6. The method as set forth in claim 4 further comprising:

determining, by the image analysis processing device, one or more segments on the classified one or more parts of the captured version of the object in the received image;

determining, by the image analysis processing device, a type of change in the identified one or more changes in the captured version of the object in the received image by comparing the one or more determined segments in the captured version and one or more segments in the stored one or more identified parts of the related versions of the object in the one or more training images along with one or more items of data related to the one or more identified parts to identify a type of change.

7. The method as set forth in claim 1 further comprising:

identifying, by the image analysis computing device, a descriptor for the one or more keypoints in the captured version of the object in the received image and in the related versions of the object in the one or more training images; and

determining, by the image analysis computing device, when one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object.

8. The method as set forth in claim 7 wherein the determining when one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object further comprises:

determining, by the image analysis computing device, a value for a match between the one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object;

determining, by the image analysis computing device, when the determined value for the match between the one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object satisfies a threshold value; and

determining, by the image analysis computing device, based on the determined value satisfying the threshold value, that the captured version of the object in the received image matches the related versions of the object in the one or more training images.

9. An image analysis computing device, comprising:

a processor coupled to a memory and configured to execute programmed instructions stored in the memory, comprising;

retrieving, based on a captured version of an object in a received image, one or more training images which display one or more related versions of the object and one or more items of data related to the one or more related versions of the object of the one or more training images;

determining one or more keypoints which are invariant to changes in scale and rotation in the captured version of the object in the received image and in the related versions of the object in the one or more training images;

identifying one or more changes to the object in the received image based on any of the determined one or more keypoints in the related version of the object which do not match the determined one or more keypoints in the captured version of the object;

determining one or more contours in the related versions of the object in the one or more training images by thresholding the one or more training images;

applying a weightage to the one or more determined contours;

identifying one or more parts of the object in the related versions of the object based on the determined one or more contours;

storing the one or more identified parts of the related versions of the object, the applied weightage, and one or more items of data related to the one or more identified parts; and

providing the identified one or more changes in the captured version of the object in the received image.

10. The device as set forth in claim 9 wherein the processor is further configured to execute programmed instructions stored in the memory further comprising:

adjusting the captured version of the object of the received image based on the related versions of the object of the one or more identified training images prior to the determining the one or more keypoints, wherein the adjusting comprises one or more of resizing and reorienting the captured version of the object of the received image based on the related version of the object of the one or more training images, adjusting image contrast, normalizing brightness, removing background, removing noise, or segmenting the received image.

11. The device as set forth in claim 9, wherein the one or more items of data related to the one or more identified parts comprises a material the one or more identified parts are made from.

12. The device as set forth in claim 9, wherein the identifying the one or more changes further comprises:

classifying each of the determined one or more keypoints in the captured version of the object which do not match the determined one or more keypoints in the related versions of the object as one or more parts of the captured version of the object in the received image;

comparing the classified one or more parts of the captured version of the object in the received image with the stored one or more parts in the related versions of the object in the one or more training images; and

identifying the one or more changes based on the comparison of the classified one or more parts and the stored one or more parts.

13. The device as set forth in claim 12 wherein the classifying each of the determined one or more keypoints in the captured version of the object comprises:

determining, by the image analysis computing device, a cluster associated with each of the of the determined one or more keypoints in the captured version of the object in the received image which do not match the determined one or more keypoints in the related versions of the object in the one or more training images, wherein each cluster comprises a set of descriptors related to the determined one or more keypoints; and

identifying, by the image analysis computing device, the one or more parts of the captured version of the object in the received image associated with the determined unmatched keypoints based on the identified cluster.

14. The device as set forth in claim 12 wherein the processor is further configured to execute programmed instructions stored in the memory further comprising:

determining one or more segments on the classified one or more parts of the captured version of the object in the received image;

determining a type of change in the identified one or more changes in the captured version of the object in the received image by comparing the one or more determined segments in the captured version and one or more segments in the stored one or more identified parts of the related versions of the object in the one or more training images along with one or more items of data related to the one or more identified parts to identify a type of change.

15. The device as set forth in claim 9 wherein the processor is further configured to execute programmed instructions stored in the memory further comprising:

identifying a descriptor for the one or more keypoints in the captured version of the object in the received image and in the related versions of the object in the one or more training images; and

determining when one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object.

16. The device as set forth in claim 15 wherein the determining when one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object further comprises:

determining a value for a match between the one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object;

determining when the determined value for the match between the one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object satisfies a threshold value; and

determining based on the determined value satisfying the threshold value, that the captured version of the object in the received image matches the related versions of the object in the one or more training images.

17. A non-transitory computer readable medium having stored thereon instructions for assessing image change comprising machine executable code which when executed by a processor, causes the processor to perform steps comprising:

retrieving, based on a captured version of an object in a received image, one or more training images which display one or more related versions of the object and one or more items of data related to the one or more related versions of the object of the one or more training images;

determining one or more keypoints which are invariant to changes in scale and rotation in the captured version of the object in the received image and in the related versions of the object in the one or more training images;

identifying one or more changes to the object in the received image based on any of the determined one or more keypoints in the related version of the object which do not match the determined one or more keypoints in the captured version of the object;

determining one or more contours in the related versions of the object in the one or more training images by thresholding the one or more training images;

applying a weightage to the one or more determined contours;

identifying one or more parts of the object in the related versions of the object based on the determined one or more contours;

storing the one or more identified parts of the related versions of the object, the applied weightage, and one or more items of data related to the one or more identified parts; and

providing the identified one or more changes in the captured version of the object in the received image.

18. The medium as set forth in claim 17 wherein the medium further comprises machine executable code which, when executed by the processor, causes the processor to perform steps further comprising:

adjusting the captured version of the object of the received image based on the related versions of the object of the one or more identified training images prior to the determining the one or more keypoints, wherein the adjusting comprises one or more of resizing and reorienting the captured version of the object of the received image based on the related version of the object of the one or more training images, adjusting image contrast, normalizing brightness, removing background, removing noise, or segmenting the received image.

19. The medium as set forth in claim 17, wherein the one or more items of data related to the one or more identified parts comprises a material the one or more identified parts are made from.

20. The medium as set forth in claim 17, wherein the identifying the one or more changes further comprises:

classifying each of the determined one or more keypoints in the captured version of the object which do not match the determined one or more keypoints in the related versions of the object as one or more parts of the captured version of the object in the received image;

comparing the classified one or more parts of the captured version of the object in the received image with the stored one or more parts in the related versions of the object in the one or more training images; and

identifying the one or more changes based on the comparison of the classified one or more parts and the stored one or more parts.

21. The medium as set forth in claim 20 wherein the classifying each of the determined one or more keypoints in the captured version of the object comprises:

determining, by the image analysis computing device, a cluster associated with each of the of the determined one or more keypoints in the captured version of the object in the received image which do not match the determined one or more keypoints in the related versions of the object in the one or more training images, wherein each cluster comprises a set of descriptors related to the determined one or more keypoints; and

identifying, by the image analysis computing device, the one or more parts of the captured version of the object in the received image associated with the determined unmatched keypoints based on the identified cluster;

adjusting the captured version of the object of the received image based on the related version of the object of the one or more identified training images prior to determining the one or more keypoints, wherein the adjusting comprises one or more of resizing and reorienting the captured version of the object of the received image based on the related version of the object of the one or more training images, adjusting image contrast, normalizing brightness, removing background, removing noise, or segmenting the received image.

22. The medium as set forth in claim 20 wherein the medium further comprises machine executable code which, when executed by the processor, causes the processor to perform steps further comprising:

determining one or more segments on the classified one or more parts of the captured version of the object in the received image;

determining a type of change in the identified one or more changes in the captured version of the object in the received image by comparing the one or more determined segments in the captured version and one or more segments in the stored one or more identified parts of the related versions of the object in the one or more training images along with one or more items of data related to the one or more identified parts to identify a type of change.

23. The medium as set forth in claim 17 wherein the medium further comprises machine executable code which, when executed by the processor, causes the processor to perform steps further comprising:

identifying a descriptor for the one or more keypoints in the captured version of the object in the received image and in the related versions of the object in the one or more training images; and

determining when one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object.

24. The medium as set forth in claim 23 wherein the determining when one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object further comprises:

determining a value for a match between the one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object;

determining when the determined value for the match between the one or more identified descriptors in the captured version of the object match one or more identified descriptors in the related versions of the object satisfies a threshold value; and

determining based on the determined value satisfying the threshold value, that the captured version of the object in the received image matches the related versions of the object in the one or more training images.