MOBILE DEVICE IMAGE ACQUISITION USING OBJECTS OF INTEREST RECOGNITION

• US 20160094774A1
• Filed: 09/29/2014
• Published: 03/31/2016
• Est. Priority Date: 09/29/2014
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Li, Jia, Jin, Haojian
• Granted Patent: US 9,554,030 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions

Abstract

An approach is provided for acquiring images with camera-enabled mobile devices using objects of interest recognition. A mobile device is configured to acquire an image represented by image data and process the image data to identify a plurality of candidate objects of interest in the image. The plurality of candidate objects of interest may be identified based upon a plurality of low level features or “cues” in the image data. Example cues include, without limitation, color contrast, edge density and superpixel straddling. A particular candidate object of interest is selected from the plurality of candidate objects of interest and a graphical symbol is displayed on a screen of the mobile device to identify the particular candidate object of interest. The particular candidate object of interest may be located anywhere on the image. Passive auto focusing is performed at the location of the particular candidate object of interest.

Claims

1. A mobile device comprising:

one or more processors;

one or more memories communicatively coupled to the one or more processors;

a camera;

a display for displaying images to a user and for receiving user input from the user; and

a photo application executing on the mobile device and being configured to;

cause the camera to acquire an image represented by image data,

process the image data to identify a plurality of candidate objects of interest in the image,

select a particular candidate object of interest from the plurality of candidate objects of interest, wherein the particular candidate object of interest has a location other than the center of the display, and

cause passive auto focusing to be performed at the location other than the center of the display of the particular candidate object of interest.

2. The mobile device of claim 1, wherein the photo application executing on the mobile device is further configured to cause the camera to acquire a focused image after the passive auto focusing is completed.

3. The mobile device of claim 1, wherein the photo application executing on the mobile device is further configured to cause a candidate object of interest symbol to be displayed on the display at the location other than the center of the display for the particular candidate object of interest.

4. The mobile device of claim 1, wherein the photo application executing on the mobile device is further configured to, in response to a user selection of a second candidate object of interest from the plurality of candidate objects of interest, wherein the second candidate object of interest is different than the particular candidate object of interest:

cause a candidate object of interest symbol to be moved on the display from the location other than the center of the display for the particular candidate object of interest to a location of the second candidate object of interest, and

cause passive auto focusing to be performed at the location of the second candidate object of interest.

5. The mobile device of claim 1, wherein the photo application executing on the mobile device is further configured to, in response to one or more of user input that indicates a user interest in a particular location on the image that is different that the location for the particular candidate object of interest or an expiration of time:

cause the camera to acquire a second image represented by second image data, wherein the second image is different than the image and the second image data is different than the image data,

process the second image data to identify a second plurality of candidate objects of interest in the second image,

select a second particular candidate object of interest from the second plurality of candidate objects of interest, wherein the second particular candidate object of interest is different than the particular candidate object of interest and wherein the second particular candidate object of interest has a location that is both other than the center of the display and different than the location of the particular candidate object of interest, and

cause passive auto focusing to be performed at the location of the second particular candidate object of interest that is both other than the center of the display and different than the location of the particular candidate object of interest.

6. The mobile device of claim 1, wherein the photo application executing on the mobile device is further configured to process the image data to identify a plurality of candidate objects of interest in the image by:

identifying a plurality of prominent regions within the image,

assigning a plurality of bounding boxes to the plurality of prominent regions within the image,

for each bounding box from the plurality of bounding boxes, determining one or more of a color contrast score, an edge density score, or a superpixel straddling score, and

select a particular candidate object of interest from the plurality of candidate objects of interest based upon the scores for the plurality of bounding boxes.

7. The mobile device of claim 6, wherein the photo application executing on the mobile device is further configured to identify a plurality of prominent regions within the image by generating a saliency map based upon the spectral residual of an FFT on the image data.

8. One or more non-transitory computer-readable media storing instructions which, when processed by one or more processors, cause:

a photo application executing on a mobile device to;

cause a camera of the mobile device to acquire an image represented by image data,

process the image data to identify a plurality of candidate objects of interest in the image,

select a particular candidate object of interest from the plurality of candidate objects of interest, wherein the particular candidate object of interest has a location other than the center of the display, and

cause passive auto focusing to be performed at the location other than the center of the display of the particular candidate object of interest.

9. The one or more non-transitory computer-readable media of claim 8, wherein the photo application executing on the mobile device further causes the camera to acquire a focused image after the passive auto focusing is completed.

10. The one or more non-transitory computer-readable media of claim 8, wherein the photo application executing on the mobile device further causes a candidate object of interest symbol to be displayed on the display at the location other than the center of the display for the particular candidate object of interest.

11. The one or more non-transitory computer-readable media of claim 8, wherein the photo application executing on the mobile device further causes, in response to a user selection of a second candidate object of interest from the plurality of candidate objects of interest, wherein the second candidate object of interest is different than the particular candidate object of interest:

cause a candidate object of interest symbol to be moved on the display from the location other than the center of the display for the particular candidate object of interest to a location of the second candidate object of interest, and

cause passive auto focusing to be performed at the location of the second candidate object of interest.

12. The one or more non-transitory computer-readable media of claim 8, wherein the photo application executing on the mobile device, in response to one or more of user input that indicates a user interest in a particular location on the image that is different that the location for the particular candidate object of interest or an expiration of time:

causes the camera to acquire a second image represented by second image data, wherein the second image is different than the image and the second image data is different than the image data,

process the second image data to identify a second plurality of candidate objects of interest in the second image,

select a second particular candidate object of interest from the second plurality of candidate objects of interest, wherein the second particular candidate object of interest is different than the particular candidate object of interest and wherein the second particular candidate object of interest has a location that is both other than the center of the display and different than the location of the particular candidate object of interest, and

causes passive auto focusing to be performed at the location of the second particular candidate object of interest that is both other than the center of the display and different than the location of the particular candidate object of interest.

13. The one or more non-transitory computer-readable media of claim 8, wherein the photo application executing on the mobile device processes the image data to identify a plurality of candidate objects of interest in the image by:

identifying a plurality of prominent regions within the image,

assigning a plurality of bounding boxes to the plurality of prominent regions within the image,

for each bounding box from the plurality of bounding boxes, determining one or more of a color contrast score, an edge density score, or a superpixel straddling score, and

select a particular candidate object of interest from the plurality of candidate objects of interest based upon the scores for the plurality of bounding boxes.

14. The one or more non-transitory computer-readable media of claim 13, wherein the photo application executing on the mobile device is further identifies a plurality of prominent regions within the image by generating a saliency map based upon the spectral residual of an FFT on the image data.

15. A computer-implemented method comprising:

a photo application executing on a mobile device to;

cause a camera of the mobile device to acquire an image represented by image data,

process the image data to identify a plurality of candidate objects of interest in the image,

select a particular candidate object of interest from the plurality of candidate objects of interest, wherein the particular candidate object of interest has a location other than the center of the display, and

cause passive auto focusing to be performed at the location other than the center of the display of the particular candidate object of interest.

16. The computer-implemented method of claim 15, wherein the photo application executing on the mobile device further causes the camera to acquire a focused image after the passive auto focusing is completed.

17. The computer-implemented method of claim 15, wherein the photo application executing on the mobile device further causes a candidate object of interest symbol to be displayed on the display at the location other than the center of the display for the particular candidate object of interest.

18. The computer-implemented method of claim 15, wherein the photo application executing on the mobile device further causes, in response to a user selection of a second candidate object of interest from the plurality of candidate objects of interest, wherein the second candidate object of interest is different than the particular candidate object of interest:

cause a candidate object of interest symbol to be moved on the display from the location other than the center of the display for the particular candidate object of interest to a location of the second candidate object of interest, and

cause passive auto focusing to be performed at the location of the second candidate object of interest.

19. The computer-implemented method of claim 15, wherein the photo application executing on the mobile device, in response to one or more of user input that indicates a user interest in a particular location on the image that is different that the location for the particular candidate object of interest or an expiration of time:

causes the camera to acquire a second image represented by second image data, wherein the second image is different than the image and the second image data is different than the image data,

process the second image data to identify a second plurality of candidate objects of interest in the second image,

select a second particular candidate object of interest from the second plurality of candidate objects of interest, wherein the second particular candidate object of interest is different than the particular candidate object of interest and wherein the second particular candidate object of interest has a location that is both other than the center of the display and different than the location of the particular candidate object of interest, and

causes passive auto focusing to be performed at the location of the second particular candidate object of interest that is both other than the center of the display and different than the location of the particular candidate object of interest.

20. The computer-implemented method of claim 15, wherein the photo application executing on the mobile device processes the image data to identify a plurality of candidate objects of interest in the image by:

identifying a plurality of prominent regions within the image,

assigning a plurality of bounding boxes to the plurality of prominent regions within the image,

for each bounding box from the plurality of bounding boxes, determining one or more of a color contrast score, an edge density score, or a superpixel straddling score, and

select a particular candidate object of interest from the plurality of candidate objects of interest based upon the scores for the plurality of bounding boxes.

21. The computer-implemented method of claim 20, wherein the photo application executing on the mobile device is further identifies a plurality of prominent regions within the image by generating a saliency map based upon the spectral residual of an FFT on the image data.