Image processing

• US 8,385,646 B2
• Filed: 09/29/2006
• Issued: 02/26/2013
• Est. Priority Date: 10/31/2005
• Status: Active Grant

• Current Assignee: Sony Europe (Amsterdam) BV (Sony Group Corp.)
• Original Assignee: Sony United Kingdom Limited (Sony Group Corp.)
• Inventors: Lang, Mikael Carl, Porter, Robert Mark Stefan, Prayle, Paul Edward, Beresford, Ratna, Thorpe, Jonathan Richard, Williams, Michael John
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US12/092,090
• Publication Number:
  US 20090310861A1
• Time in Patent Office: 2,342 Days
• Field of Search: 382/173, 382162-166, 382/108, 382/190, 382/305, 382224-228, 382260-266, 705 261- 272
• US Class Current: 382/173
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06Q 30/0621 Item configuration or customization
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/45 using co-occurrence matrix computation
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An image processing method includes partitioning an image under test to form a plurality of contiguous image segments having similar image properties, deriving feature data from a subset including one or more of the image segments, and comparing the feature data from the subset of image segments with feature data derived from respective image segments of one or more other images so as to detect a similarity between the image under test and the one or more other images.

Claims

1. An image processing method, comprising the steps of:

partitioning an image under test to form a plurality of image segments, each image segment representing a set of pixels having similar image properties, at least some of the image segments being contiguous, and the partitioning step including;

(i) deriving image property data from pixels of the image under test, and

(ii) grouping pixels into image segments in accordance with their respective image property data;

deriving feature data from a subset comprising one or more of the image segments;

comparing the feature data from the subset of image segments with feature data indicative of respective reference image segments so as to detect a degree of similarity between the image segments under test and the reference image segments;

detecting whether the image under test has a dark border; and

when the dark border is detected in the detecting, removing the dark border before the partitioning step, and reapplying the dark border after the partitioning step.

2. The method according to claim 1, wherein:

the reference image segments are respective images segments of one or more other images; and

the comparing step is operable to detect a degree of similarity between the image under test and the one or more other images.

3. The method according to claim 1, in which the partitioning step further comprise:

(iii) detecting a separation of image segments in terms of image property data differences; and

(iv) merging image segments which have less than a threshold separation.

4. The method according to claim 3, in which step (iii) is carried out recursively.

5. The method according to claim 1, further comprising the step of:

(v) spatially filtering the image segments, the spatially filtering tending to reassign pixels from small image segments into larger image segments.

6. The method according to claim 1, in which the feature data is indicative of at least colour properties.

7. The method according to claim 1, in which the feature data is indicative of at least image texture properties.

8. The method according to claim 1, further comprising the steps of:

deriving respective feature data from each image segment; and

where the subset of image segments, in respect of the image under test, comprises more than one image segment, performing the comparing step using a combination of the feature data from all of the image segments in the subset.

9. The method according to claim 1, in which each segment is contiguous.

10. The method according to claim 1, further comprising the steps of:

generating feature data indicative of a plurality of reference image segments, each reference image segment having an associated data item;

detecting a user selection of an image segment derived from the image under test; and

accessing the data item associated with the reference image segment detected to be most similar to the image segment selected by the user.

11. The method according to claim 10, in which:

the associated data items are hyperlinks; and

the step of accessing the associated data item comprises accessing the hyperlink target associated with that hyperlink.

12. The method according to claim 10, in which the step of accessing the associated data item comprises providing to the user data indicative of a purchase price of an article or service associated with that reference image segment.

13. The method according to claim 12, further comprising the step of:

detecting user operation of a purchase control indicating that the user wishes to purchase the article or service.

14. The method according to claim 10, in which the step of detecting user selection comprises:

displaying at least a part of the image under test; and

detecting user operation of a pointing device directed at that image segment.

15. A non-transitory computer-readable storage medium un which is stored computer software comprising program code which, when executed on a computer, executes a method according to claim 1.

16. An image processing apparatus, comprising:

a processor configured to partition an image under test to form a plurality of image segments, each image segment representing a set of pixels having similar image properties, at least some of the image segments being contiguous, by (i) deriving image property data from pixels of the image under test, and (ii) grouping pixels into image segments in accordance with their respective image property data, and further configured to derive feature data from a subset comprising one or more of the image segments; and

a comparator configured to compare the feature data from the subset of image segments with feature data indicative of respective reference image segments so as to detect a degree of similarity between the image segments under test and the reference image segments, wherein

the processor is configured to detect whether the image under test has a dark border, and

when the dark border is detected, the processor is configured to remove the dark border before partitioning the image, and reapply the dark border after partitioning the image.

17. An image capture apparatus comprising the image processing apparatus according to claim 16.

18. An image processing method, comprising the steps of:

partitioning an image under test to form a plurality of image segments, each image segment representing a set of pixels having similar image properties, at least some of the image segments being contiguous;

deriving feature data from a subset comprising one or more of the image segments;

comparing the feature data from the subset of image segments with feature data indicative of respective reference image segments so as to detect a degree of similarity between the image segments under test and the reference image segments;

generating feature data indicative of a plurality of reference image segments, each reference image segment having an associated data item;

detecting a user control to select an image segment derived from the image under test; and

providing access to the data item associated with the reference image segment detected to be most similar to the image segment selected by the user, by providing data indicative of a purchase price of an article or service associated with the reference image segment detected to be most similar to the image segment selected by the user.

19. An image processing apparatus comprising a processor configured to:

partition an image under test to form a plurality of image segments, each image segment representing a set of pixels having similar image properties, at least some of the image segments being contiguous;

derive feature data from a subset comprising one or more of the image segments;

compare the feature data from the subset of image segments with feature data indicative of respective reference image segments so as to detect a degree of similarity between the image segments under test and the reference image segments;

generate feature data indicative of a plurality of reference image segments, each reference image segment having an associated data item;

detect a user control to select an image segment derived from the image under test; and

provide access to the data item associated with the reference image segment detected to be most similar to the image segment selected by the user, by providing data indicative of a purchase price of an article or service associated with the reference image segment detected to be most similar to the image segment selected by the user.