DEEP-LEARNING-BASED SYSTEM TO ASSIST CAMERA AUTOFOCUS

• US 20200137298A1
• Filed: 10/30/2018
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Eslami, Ramin
• Granted Patent: US 10,666,858 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions

Abstract

To overcome several issues of autofocusing, a deep-learning-based autofocus system utilizes a subject'"'"'s body to autofocus on the subject'"'"'s face. The subject, including the subject'"'"'s body and face/head, are determined utilizing image processing methods, and based on the detection, the subject'"'"'s face/head is automatically focused on.

Claims

1. A method programmed in a non-transitory memory of a device comprising:

processing image content to generate a separated image content using a neural network-based algorithm including separating the image content into body areas and head areas, wherein processing the image content includes utilizing a heat map and a thresholded heat map to distinguish the head areas by bisecting the head areas;

receiving selection information; and

autofocusing on a face or head of a subject based on the separated image content and the selection information.

2. The method of claim 1 wherein the neural network-based algorithm comprises a fully convolutional network algorithm.

3. The method of claim 1 wherein receiving the selection information includes receiving a touchscreen input from a user.

4. The method of claim 1 wherein receiving the selection information includes an automatic selection by the device.

5. The method of claim 1 wherein when the selection information includes a body area of the subject, the head associated with the body area is autofocused on.

6. The method of claim 5 wherein determining the head associated with the body area includes computing a distance between one or more head areas and the body area and finding a head area with a minimum distance from the body area.

8. The method of claim 1 wherein processing the image content includes applying a head mask to isolate the head areas.

7. (canceled)

9. An apparatus comprising:

a non-transitory memory for storing an application, the application for;

processing image content to generate a separated image content using a neural network-based algorithm including separating the image content into body areas and head areas, wherein processing the image content includes utilizing a heat map and a thresholded heat map to distinguish the head areas by bisecting the head areas;

receiving selection information; and

autofocusing on a face or head of a subject based on the separated image content and the selection information; and

a processor coupled to the memory, the processor configured for processing the application.

10. The apparatus of claim 9 wherein the neural network-based algorithm comprises a fully convolutional network algorithm.

11. The apparatus of claim 9 wherein receiving the selection information includes receiving a touchscreen input from a user.

12. The apparatus of claim 9 wherein receiving the selection information includes an automatic selection by the device.

13. The apparatus of claim 9 wherein when the selection information includes a body area of the subject, the head associated with the body area is autofocused on.

14. The apparatus of claim 13 wherein determining the head associated with the body area includes computing a distance between one or more head areas and the body area and finding a head area with a minimum distance from the body area.

16. The apparatus of claim 9 wherein processing the image content includes applying a head mask to isolate the head areas.

15. (canceled)

17. A camera device comprising:

a lens;

a non-transitory memory for storing an application, the application for;

processing image content to generate a separated image content using a neural network-based algorithm including separating the image content into body areas and head areas;

receiving selection information, wherein processing the image content includes utilizing a heat map and a thresholded heat map to distinguish the head areas by bisecting the head areas; and

autofocusing, by moving the lens, on a face or head of a subject based on the separated image content and the selection information; and

a processor coupled to the memory, the processor configured for processing the application.

18. The camera device of claim 17 wherein the neural network-based algorithm comprises a fully convolutional network algorithm.

19. The camera device of claim 17 wherein receiving the selection information includes receiving a touchscreen input from a user.

20. The camera device of claim 17 wherein receiving the selection information includes an automatic selection by the device.

21. The camera device of claim 17 wherein when the selection information includes a body area of the subject, the head associated with the body area is autofocused on.

22. The camera device of claim 21 wherein determining the head associated with the body area includes computing a distance between one or more head areas and the body area and finding a head area with a minimum distance from the body area.

24. The camera device of claim 17 wherein processing the image content includes applying a head mask to isolate the head areas.

23. (canceled)

25. A method programmed in a non-transitory memory of a device comprising:

processing image content to generate a separated image content using a neural network-based algorithm including separating the image content into body areas and head areas;

receiving selection information; and

autofocusing on a face or head of a subject based on the separated image content and the selection information, wherein when the selection information includes a body area of the subject, the head associated with the body area is autofocused on, wherein determining the head associated with the body area includes computing a distance between one or more head areas and the body area and finding a head area with a minimum distance from the body area.

26. The method of claim 25 wherein the neural network-based algorithm comprises a fully convolutional network algorithm.

27. The method of claim 25 wherein receiving the selection information includes receiving a touchscreen input from a user.

28. The method of claim 25 wherein receiving the selection information includes an automatic selection by the device.

29. The method of claim 25 wherein processing the image content includes applying a head mask to isolate the head areas.