ROBUST FEATURE IDENTIFICATION FOR IMAGE-BASED OBJECT RECOGNITION

• US 20190318195A1
• Filed: 06/24/2019
• Published: 10/17/2019
• Est. Priority Date: 04/24/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Song, Bing, Lin, Liwen, Jaber, Mustafa
• Granted Patent: US 10,719,731 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries

Abstract

Techniques are provided that include identifying robust features within a training image. Training features are generated by applying a feature detection algorithm to the training image, each training feature having a training feature location within the training image. At least a portion of the training image is transformed into a transformed image in accordance with a predefined image transformation. Transform features are generated by applying the feature detection algorithm to the transformed image, each transform feature having a transform feature location within the transformed image. The training feature locations of the training features are mapped to corresponding training feature transformed locations within the transformed image in accordance with the predefined image transformation, and a robust feature set is compiled by selecting robust features, wherein each robust feature represents a training feature having a training feature transformed location proximal to a transform feature location of one of the transform features.

Claims

1-31. -31. (canceled)

32. A product image feature detection device comprising:

a tangible, non-transitory, computer-readable memory configured to store robust feature detection software instructions including at least one implementation of a feature detection algorithm, and at least one product training image; and

at least one processor coupled with the memory that, upon execution of the robust feature detection software instructions;

generates training features from the at least one product training image according to the at least one implementation of the feature detection algorithm, wherein each training feature has a corresponding training feature location within the at least one product training image;

transforms at least a portion of the at least one product training image according to an image transformation, thereby forming a transformed image portion;

generates transform features from the transformed image portion according to the at least one implementation of the feature detection algorithm, wherein each transform feature has a corresponding transform feature location within the transformed image portion; and

stores in the memory a set of robust features, wherein each robust feature in the set represents a training feature, based on the image transform, having a training feature transformed location proximal to a transform feature location.

33. The device of claim 32, wherein the image transformation is user-selectable.

34. The device of claim 32, wherein the image transformation comprises an image perturbation.

35. The device of claim 32, wherein the image transformation comprises an image processing transform.

36. The device of claim 35, wherein the image processing transform includes at least one the following:

a Gaussian filter, a color transform, an edge enhancing transform, and a lossy compression.

37. The device of claim 32, wherein the image transformation comprises a geometric transform.

38. The device of claim 37, wherein the geometric transform comprises at least one of the following:

a skewing, a rotation, and a shearing.

39. The device of claim 32, wherein the image transformation includes a scale transform.

40. The device of claim 39, wherein the scale transform comprises an up-scaling.

41. The device of claim 39, wherein the scale transform comprises a down-scaling.

42. The device of claim 39, wherein the scale transform comprises a linear scaling.

43. The device of claim 39, wherein the scale transform comprises a scaling factor of at least two.

44. The device of claim 39, wherein the scale transform comprises a scaling factor based on subject matter of the at least one product training image.

45. The device of claim 32, wherein the at least one product training image comprises a near-sequence image.

46. The device of claim 32, wherein the at least the portion of the at least one product training image comprises a patch.

47. The device of claim 32, wherein the at least one product training image includes a consumer product.

48. The device of claim 32, wherein the at least one product training image includes a vehicle.

49. The device of claim 32, wherein the at least one product training image includes clothing.

50. The device of claim 32, wherein the at least one product training image includes a game.

51. The device of claim 32, wherein the at least one product training image includes a package.

52. The device of claim 32, wherein the at least one product training image includes a device.

53. The device of claim 32, wherein the at least one product training image comprises at least one video frame.

54. The device of claim 32, wherein each robust feature comprises a robustness measure with respect to the image transformation.

55. The device of claim 54, wherein the robustness measure reflects subject matter characteristics of the at least one product training image.

56. The device of claim 32, wherein the at least one product training image is a member of a test library of product images.

57. The device of claim 32, wherein the at least one implementation of the feature detection algorithm includes an implementation of one of the following:

a scale-invariant feature transform (SIFT), Fast Retina Keypoint (FREAK), Histograms of Oriented Gradient (HOG), Speeded Up Robust Features (SURF), DAISY, Binary Robust Invariant Scalable Keypoints (BRISK), FAST, Binary Robust Independent Elementary Features (BRIEF), Harris Corners, Edges, Gradient Location and Orientation Histogram (GLOH), Energy of image Gradient (EOG), an edge detection algorithm, and Transform Invariant Low-rank Textures (TILT) feature detection algorithm.

58. The device of claim 32, wherein the device is a mobile device.

59. A product image feature detection method comprising:

generating training features from at least one product training image according to at least one implementation of a feature detection algorithm, wherein each training feature has a corresponding training feature location within the at least one product training image;

transforming at least a portion of the at least one product training image according to an image transformation, thereby forming a transformed image portion;

generating transform features from the transformed image portion according to the at least one implementation of the feature detection algorithm, wherein each transform feature has a corresponding transform feature location within the transformed image portion; and

storing in memory a set of robust features, wherein each robust feature in the set represents a training feature, based on the image transform, having a training feature transformed location proximal to a transform feature location.

60. A computer program product embedded in a non-transitory computer readable medium comprising product image feature detection instructions executable by a computer processor to perform operations comprising:

generating training features from at least one product training image according to at least one implementation of a feature detection algorithm, wherein each training feature has a corresponding training feature location within the at least one product training image;

transforming at least a portion of the at least one product training image according to an image transformation, thereby forming a transformed image portion;

generating transform features from the transformed image portion according to the at least one implementation of the feature detection algorithm, wherein each transform feature has a corresponding transform feature location within the transformed image portion; and

storing in memory a set of robust features, wherein each robust feature in the set represents a training feature, based on the image transform, having a training feature transformed location proximal to a transform feature location.