Image processing

• US 9,269,105 B2
• Filed: 02/25/2013
• Issued: 02/23/2016
• Est. Priority Date: 10/31/2005
• Status: Active Grant

• Current Assignee: Sony Europe (Amsterdam) BV (Sony Group Corp.)
• Original Assignee: Sony Europe Limited (Sony Group Corp.)
• Inventors: Lang, Mikael Carl, Porter, Robert Mark Stefan, Thorpe, Jonathan Richard, Prayle, Paul Edward, Beresford, Ratna, Williams, Michael John
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US13/776,061
• Publication Number:
  US 20130173420A1
• Time in Patent Office: 1,093 Days
• Field of Search: 382117-118, 382/305, 382/165, 705 261- 272
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06Q 30/0621 Item configuration or customization
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/45 using co-occurrence matrix computation
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An image processing method includes partitioning an image under test to form a plurality of contiguous image segments having similar image properties, deriving feature data from a subset including one or more of the image segments, and comparing the feature data from the subset of image segments with feature data derived from respective image segments of one or more other images so as to detect a similarity between the image under test and the one or more other images.

Claims

1. An image processing method by a client device, comprising the steps of:

partitioning an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

deriving feature data from a subset comprising one or more of the image segments;

transmitting at least the feature data to a server device;

receiving, from the server device, data indicative of a purchasable article or service associated with the reference feature data detected to be most similar to the feature data of one or more of the image segments; and

displaying purchase details of the article or service.

2. The method according to claim 1, in which:

the reference feature data is indicative of features of respective reference image segments;

the reference image segments are respective images segments of one or more other images; and

the comparing step is operable to detect a degree of similarity between the image under test and the one or more other images.

3. The method according to claim 1, in which at least some of the segments are contiguous.

4. The method according to claim 1, further comprising the steps of:

accessing reference feature data indicative of a plurality of reference image segments, each reference image segment having an associated data item;

detecting user selection of an image segment derived from the image under test;

accessing the data item associated with the reference image data detected to be most similar to the image segment selected.

5. The method according to claim 4, in which:

the associated data items are hyperlinks; and

the step of accessing the associated data item comprises accessing the hyperlink target associated with that hyperlink.

6. The method according to claim 4, in which the step of accessing the associated data item comprises providing to the user data indicative of a purchase price of an article or service associated with that reference image segment.

7. The method according to claim 6, further comprising the step of:

detecting user operation of a purchase control indicating that the user wishes to purchase the article or service.

8. The method according to claim 1, in which the feature data is indicative of at least colour properties.

9. The method according to claim 1, in which the feature data is indicative of at least image texture properties.

10. An image processing method by a server device, comprising the steps of:

receiving data from a client device indicative of feature data derived from a subset comprising one or more image segments of an image, each segment representing a set of pixels having similar image properties;

comparing the feature data from the subset of image segments with reference feature data so as to detect a degree of similarity between the feature data for the image segments under test and the reference feature data, in which the reference feature data or corresponding reference images have associated data items; and

transmitting, to the client device, data indicative a purchasable article or service associated with that reference feature data detected to be most similar to the feature data of one or more of the image segments.

11. An image processing method comprising the steps of:

by a client device;

partitioning an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

deriving feature data from a subset comprising one or more of the image segments; and

transmitting at least the feature data to a server device, and

by the server device;

comparing the feature data from the subset of image segments with reference feature data so as to detect a degree of similarity between the feature data for the image segments under test and the reference feature data, in which the reference feature data or corresponding reference images have associated data items;

accessing the associated data item associated with the reference feature data detected to be most similar to the feature data of an image segment; and

providing to the client device data indicative of a purchase price of an article or service associated with that reference feature data detected to be most similar to the feature data of one or more of the image segments.

12. An image processing method comprising the steps of:

by a client device and a server device connected via an interface cooperating to;

partition an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

derive feature data from a subset comprising one or more of the image segments;

compare the feature data from the subset of image segments with reference feature data so as to detect a degree of similarity between the feature data for the image segments under test and the reference feature data, in which the reference feature data or corresponding reference images have associated data items;

access the associated data item associated with the reference feature data detected to be most similar to the feature data of an image segment; and

display purchase details of an article or service associated with that reference feature data detected to be most similar to the feature data of one or more of the image segments.

13. An image processing apparatus comprising a processor configured to:

partition an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

derive feature data from a subset comprising one or more of the image segments;

transmit at least the feature data to a server device;

receive, from the server device, data indicative of a purchasable article or service associated with the reference feature data detected to be most similar to the feature data of one or more of the image segments; and

display purchase details of the article or service.

14. The apparatus according to claim 13, further comprising a camera for capturing an image under test.

15. An image processing apparatus comprising:

a processor configured to;

receive data from a client device indicative of feature data derived from a subset comprising one or more image segments of an image, each segment representing a set of pixels having similar image properties;

compare the feature data from the subset of image segments with reference feature data so as to detect a degree of similarity between the feature data for the image segments under test and the reference feature data, in which the reference feature data or corresponding reference images have associated data items; and

transmit, to the client device, data indicative a purchasable article or service associated with that reference feature data detected to be most similar to the feature data of one or more of the image segments.

16. An image processing system comprising a client device and a server device,

the client device having a processor configured to:

partition an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

derive feature data from a subset comprising one or more of the image segments; and

transmit at least the feature data to the server device, and

the server device having a processor configured to;

compare the feature data from the subset of image segments with reference feature data so as to detect a degree of similarity between the feature data for the image segments under test and the reference feature data, in which the reference feature data or corresponding reference images have associated data items;

access the associated data item associated with the reference feature data detected to be most similar to the feature data of an image segment; and

provide to the client device data indicative of a purchase price of an article or service associated with that reference feature data detected to be most similar to the feature data of one or more of the image segments.

17. An image processing system comprising a client device and a server device, each having a respective processor, the processors of the client device and the server device being configured to cooperate to:

partition an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

derive feature data from a subset comprising one or more of the image segments;

compare the feature data from the subset of image segments with reference feature data so as to detect a degree of similarity between the feature data for the image segments under test and the reference feature data, in which the reference feature data or corresponding reference images have associated data items;

access the associated data item associated with the reference feature data detected to be most similar to the feature data of an image segment; and

display purchase details of an article or service associated with that reference feature data detected to be most similar to the feature data of one or more of the image segments.

18. A non-transitory machine readable storage medium storing program code which, when executed by a client computer, causes the computer to:

partition an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

derive feature data from a subset comprising one or more of the image segments;

transmit at least the feature data to a server device;

receive, from the server device, data indicative of a purchasable article or service associated with the reference feature data detected to be most similar to the feature data of one or more of the image segments; and

display purchase details of the article or service.

19. A non-transitory machine readable storage medium storing program code which, when executed by a client and a server computer connected by an interface, causes the client computer and the server computer to cooperate to:

partition an image under test to form a plurality of image segments, each segment representing a set of pixels having similar image properties;

derive feature data from a subset comprising one or more of the image segments;

compare the feature data from the subset of image segments with reference feature data so as to detect a degree of similarity between the feature data for the image segments under test and the reference feature data, in which the reference feature data or corresponding reference images have associated data items;

access the associated data item associated with the reference feature data detected to be most similar to the feature data of an image segment; and

display purchase details of an article or service associated with that reference feature data detected to be most similar to the feature data of one or more of the image segments.