Methods and systems for content processing

• US 8,520,979 B2
• Filed: 11/14/2008
• Issued: 08/27/2013
• Est. Priority Date: 08/19/2008
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Conwell, William Y.
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/271,692
• Publication Number:
  US 20100046842A1
• Time in Patent Office: 1,747 Days
• Field of Search: None
• US Class Current: 382/305
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9536 Search customisation based on social or collaborative filtering
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2215/00 Indexing scheme for image rendering
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/3278 Transmission
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

Cell phones and other portable devices are equipped with a variety of technologies by which existing functionality can be improved, and new functionality can be provided. Some relate to visual search capabilities, and determining appropriate actions responsive to different image inputs. Others relate to processing of image data. Still others concern metadata generation, processing, and representation. Yet others relate to coping with fixed focus limitations of cell phone cameras, e.g., in reading digital watermark data. Still others concern user interface improvements. A great number of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

(a) receiving an image captured with a user'"'"'s portable device;

(b) identifying a first set of plural images that are related to the captured image, said images each having associated metadata, including content descriptive metadata;

(c) identifying a subset of the metadata;

(d) identifying a second set of plural images based on metadata of images in the first set; and

(e) submitting image data, together with metadata, to an image-responsive service, to take an action based thereon, wherein at least one of said submitted image data or said submitted metadata is identified by acts (b)-(d), and wherein said submitting is performed by a processor-controlled device configured to perform such act.

2. The method of claim 1 in which (b) includes identifying plural images that are related by appearance.

3. The method of claim 1 in which (b) includes identifying plural images that are related by location.

4. The method of claim 1 in which (e) includes submitting data corresponding to the captured image to the image-responsive service.

5. The method of claim 1 in which (e) includes submitting data corresponding to one or more of the plural images in the first set to the image-responsive service.

19. The method of claim 1 that further includes capturing the image with the user'"'"'s portable device.

20. The method of claim 1 wherein the image-responsive service is a service that searches for information related to a subject depicted in the submitted image data.

21. The method of claim 1 wherein said submitting includes submitting data corresponding to one or more of the images in the second set.

22. The method of claim 1 wherein said submitting includes submitting image data, together with the identified subset of metadata, to the image responsive service.

23. The method of claim 1 wherein said submitting includes submitting image data corresponding to one or more of the images in the second set, together with the identified subset of metadata, to the image responsive service.

6. A method comprising the acts:

(a) receiving an image captured using a user'"'"'s portable device, the captured image having location data associated therewith;

(b) identifying a first set of plural images that are related by location to the captured image, said images each having associated content descriptive metadata;

(c) identifying a subset of the metadata;

(d) identifying a second image having metadata related to the identified subset of metadata, but not having associated location data; and

(e) associating location data with the second image, wherein said associating is performed by a processor-controlled device configured to perform such act.

7. A method comprising the acts:

receiving an image captured using a user'"'"'s portable device, the captured image having a location associated therewith;

identifying a first group of plural images related by in location to the captured image, each of said plural images being associated with content-descriptive metadata;

by reference to the metadata associated with the plural images, identifying;

i. a first sub-group of the images that likely have a first type of subject; and

ii. a second sub-group of the images that likely have a second type of subject; and

assessing a subject type of the captured image as likely being of the first type and/or the second type, by reference to similarity between the captured image and images of the first and second sub-groups, wherein said assessing is performed by a processor-controlled device configured to perform such act.

8. The method of claim 7 wherein the first type of subject is one of person-centric, place-centric, or thing-centric.

9. The method of claim 8 wherein the second type of subject is a different one of person-centric, place-centric, or thing-centric.

10. The method of claim 7 that includes, by reference to the metadata associated with the plural images, identifying:

i. a first sub-group of the images that likely are person-centric;

ii. a second sub-group of the images that likely are place-centric; and

iii. a third sub-group of the images that are likely thing-centric; and

assessing a subject type of the captured image as likely person-centric, place-centric, and/or thing-centric, by reference to similarity between the captured image and images of one or more the sub-groups.

11. The method of claim 7 that further includes determining candidate metadata for the captured image by reference to metadata associated with one or more of the sub-groups of images with which the captured image is most similar.

12. The method of claim 11 that further includes:

identifying a second group of plural images by reference to the location associated with the captured image, and the candidate metadata for the captured image, each of the images in the second group having content-descriptive metadata associated therewith; and

determining a refined estimate of metadata for the captured image, the determining including assessing a similarity between the captured image and each of the images in the second group, and weighting metadata associated with said images accordingly.

13. A computing system including a processor and a memory, the memory storing instructions that, when executed by the processor, cause the system to perform operations including:

(a) receiving an image captured with a user'"'"'s portable device;

(b) identifying a first set of plural images that are related to the captured image, said images each having associated metadata, including content descriptive metadata;

(c) identifying a subset of the metadata;

(d) identifying a second set of plural images based on metadata of images in the first set; and

(e) submitting image data, together with metadata, to an image-responsive service, to take an action based thereon, wherein at least one of said submitted image data or said submitted metadata is identified by acts (b)-(d).

14. A non-transitory computer-readable medium having instructions stored thereon that, if executed by a computing system, cause the system to perform operations including:

(a) receiving an image captured with a user'"'"'s portable device;

(b) identifying a first set of plural images that are related to the captured image, said images each having associated metadata, including content descriptive metadata;

(c) identifying a subset of the metadata;

(d) identifying a second set of plural images that are based in metadata of images in the first set; and

(e) submitting image data, together with metadata, to an image-responsive service, to take an action based thereon, wherein at least one of said submitted image data or said submitted metadata is identified by acts (b)-(d).

15. A computing system including a processor and a memory, the memory storing instructions that, when executed by the processor, cause the system to perform operations including:

(a) receiving an image captured using a user'"'"'s portable device, the captured image having location data associated therewith;

(b) identifying a first set of plural images that are related by location to the captured image, said images each having associated content descriptive metadata;

(c) identifying a subset of the metadata;

(d) identifying a second image having metadata related to the identified subset of metadata, but not having associated location data; and

(e) associating location data with the second image.

16. A non-transitory computer-readable medium having instructions stored thereon that, if executed by a computing system, cause the system to perform operations including:

(a) receiving an image captured using a user'"'"'s portable device, the captured image having location data associated therewith;

(b) identifying a first set of plural images that are related by location to the captured image, said images each having associated content descriptive metadata;

(c) identifying a subset of the metadata;

(d) identifying a second image having metadata related to the identified subset of metadata, but not having associated location data; and

(e) associating location data with the second image.

17. A computing system including a processor and a memory, the memory storing instructions that, when executed by the processor, cause the system to perform operations including:

receiving an image captured using a user'"'"'s portable device, the captured image having a location associated therewith;

identifying a first group of plural images related by location to the captured image, each of said plural images being associated with content-descriptive metadata;

by reference to the metadata associated with the plural images, identifying;

i. a first sub-group of the images that likely have a first type of subject; and

ii. a second sub-group of the images that likely have a second type of subject; and

assessing a subject type of the captured image as likely being of the first type and/or the second type, by reference to similarity between the captured image and images of the first and second sub-groups.

18. A non-transitory computer-readable medium having instructions stored thereon that, if executed by a computing system, cause the system to perform operations including:

receiving an image captured using a user'"'"'s portabld device, the captured image having a location associated therewith;

identifying a first group of plural images related by location to the captured image, each of said plural images being associated with content-descriptive metadata;

by reference to the metadata associated with the plural images, identifying;

i. a first sub-group of the images that likely have a first type of subject; and

ii. a second sub-group of the images that likely have a second type of subject; and

assessing a subject type of the captured image as likely being of the first type and/or the second type, by reference to similarity between the captured image and images of the first and second sub-groups.