Computerized method and device for annotating at least one feature of an image of a view

• US 9,959,644 B2
• Filed: 11/17/2011
• Issued: 05/01/2018
• Est. Priority Date: 12/21/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Monney, Mathieu, Ayer, Serge, Vetterli, Martin
• Primary Examiner(s): Nguyen, Chau T
• Application Number:
  US13/995,578
• Publication Number:
  US 20130311868A1
• Time in Patent Office: 2,357 Days
• Field of Search: 715230
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A computerized method for annotating at least one feature of an image of a view, includes the steps of obtaining the image with an image sensor of a portable device, and retrieving at least one condition. Based on the at least one condition, the method automatically selects a feature identification method among a plurality of features identification methods. It then applies the feature identification method for identifying the at least one feature, and annotates some of the identified features.

Claims

1. A computerized method for annotating at least one feature of an image of a view, comprising:

capturing the image with an image sensor of a portable device;

selecting, with a processor of the portable device, a feature identification method from a plurality of available feature identification methods based on a focusing distance, wherein the focusing distance indicates a foreground focus or a background focus on an object in the image, and wherein said plurality of feature identification methods comprises;

a first feature identification method matching features of said image with one or more of a plurality of 2D reference images that is selected when said focusing distance indicates background focus on the object in the image,

a second feature identification method matching features of said image with projections of one or more of a plurality of 3D models that is selected when said focusing distance indicates foreground focus on the object in the image, or

a combination thereof;

applying, with the processor of the portable device, said feature identification method for identifying a plurality of features of said image, and deciding which of said identified features of said image are to be annotated;

annotating, with the processor of the portable device, at least some of the identified features, resulting in an annotated image with superimposed annotations; and

displaying the annotated image with the superimposed annotations on a display of the portable device.

2. The method of claim 1, wherein the selection of the feature identification method depends on a result of an image analysis of said image.

3. The method of claim 1, further comprising

selecting the feature identification method based at least in part on a device parameter, wherein the device parameter comprises a geographical location of said portable device.

4. The method of claim 1, further comprising selecting the feature identification method based at least in part on a device parameter, wherein the device parameter comprises at least one setting parameter of a lens of said image sensor.

5. The method of claim 1, further comprising validating said selected feature identification method based on input from a user.

6. The method of claim 1, further comprising replacing the selected feature identification method with another feature identification method based on a manual selection.

7. The method of claim 1, comprising

based on said focusing distance, or a device parameter to capture said image, automatically selecting feature identification data and a set of corresponding annotations candidates on a remote server,

downloading said feature identification data and said set of corresponding annotation candidates on said portable device,

selecting a feature identification method to apply,

verifying existence of at least one correspondence between said selected and downloaded feature identification data and identified features of said image,

if a correspondence is found, annotating at least some of the identified features,

if a correspondence is not found, changing and/or widening the set of annotations candidates until a correspondence is found.

8. The method of claim 1, further comprising

based on said focusing distance, or a device parameter to capture said image, and/or the quality of a communication link between a remote server and said portable device and/or computation power of said portable device, automatically selecting a set of feature identification data on a remote server,

downloading said selected set of feature identification data on said portable device,

annotating at least some of the identified features by using said selected and downloaded feature identification data.

9. The method of claim 1, wherein:

features identification data and corresponding annotations are added with said portable device;

those added features identification data and corresponding annotations are sent to a remote server and made available to other portable devices for annotating corresponding features when identified in an image.

10. A computer-readable non-transitory storage medium encoded with instructions for causing a programmable processor to perform a method, comprising:

capturing an image with an image sensor of a portable device;

selecting a feature identification method from a plurality of available feature identification methods based on a focusing distance, wherein the focusing distance indicates a foreground focus or a background focus on an object in the image, and wherein said plurality of feature identification methods comprises;

a first feature identification method matching features of said image with one or more of a plurality of 2D reference images that is selected when said focusing distance indicates background focus on the object in the image,

a second feature identification method matching features of said image with projections of one or more of a plurality of 3D models that is selected when said focusing distance indicates foreground focus on the object in the image, or

a combination thereof;

applying the selected feature identification method for identifying a plurality of features of said image, and deciding which of said identified features of said image are to be annotated;

annotating at least some of the identified features, resulting in an annotated image with superimposed annotations; and

displaying the annotated image with the superimposed annotation on a display of the portable device.