Method and Device for Controlling Auto Focusing of a Video Camera by Tracking a Region-of-Interest

• US 20100045800A1
• Filed: 12/30/2005
• Published: 02/25/2010
• Est. Priority Date: 12/30/2005
• Status: Active Grant

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Nokia Corporation
• Inventors: Miled, Mohamed, Islam, Asad, Chebil, Fehmi
• Granted Patent: US 8,089,515 B2
• Time in Patent Office: Days
• US Class Current: 348/169
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method

Abstract

The invention concerns an electronic device equipped with a video imaging process capability, which device includes a camera unit arranged to produce image frames from an imaging view which includes a region-of-interest ROI, an adjustable optics arranged in connection with the camera unit in order to focus the ROI on the camera unit, an identifier unit in order to identify a ROI from the image frame, a tracking unit in order to track the ROI from the image frames during the video imaging process and an auto-focus unit arranged to analyze the ROI on the basis of the tracking results provided by the tracking unit in order to adjust the optics. The device is arranged to determine the spatial position of the ROI in the produced image frame without any estimation measures.

Claims

1. An electronic device equipped with a video imaging process capability, which device includes

a camera unit arranged to produce image frames from an imaging view which includes a region-of-interest ROI,

an adjustable optics arranged in connection with the camera unit in order to focus the ROI on the camera unit,

an identifier unit in order to identify a ROI from the image frame,

a tracking unit in order to track the ROI from the image frames during the video imaging process,

an auto-focus unit arranged to analyze the ROI on the basis of the tracking results provided by the tracking unit in order to adjust the optics and the device being arranged to

determine the spatial position of the ROI along the Z-axis in order to detect the movement of the ROI in depth of the imaging view,

determine the spatial position of the ROI from the produced image frames,

determine the Z-axis position of the ROI from a change in the size of the ROI between produced consecutive image frames, and

adjust the optics on the basis of the determination in an established manner.

2. An electronic device according to claim 1, wherein the spatial position is a plane position of the ROI in the image frame.

3. An electronic device according to claim 1, wherein the tracking unit is arranged to perform the ROI tracking on macroblock basis in which ROI tracking is arranged to be decided whether the pixels of the current macroblock of the current image frame belong to the ROI region or to the background region and which decision is arranged to be based on the color content of the previous image frame in which the ROI region is already known.

4. An electronic device according to claim 3, wherein the tracking unit is arranged to project each macroblock of the tracking window of the current image frame into the previous image frame in which for each of the macroblocks a search area is arranged to be defined in order to determine the color content of the ROI region and background region.

5. An electronic device according to claim 4, wherein search area is arranged to be constructed by enlarging the projected macroblock in the previous image frame in each direction in order to ensure that the best match for the current macroblock is inside the search area defined for it.

6. An electronic device according to claim 5, wherein the search area is arranged to be constructed by enlarging the projected macroblock in the previous image frame in each direction by a distance equal to the estimated motion range.

7. An electronic device according to claim 3, wherein the tracking unit is arranged

to define a ROI region and a background region in the search area of the previous image frame which definitions are arranged to be based on the ROI mask of the previous image frame,

to form color histograms of the ROI region and the background region of the search area,

to analyze the said color histograms of the ROI region and background region and on the basis of the results of the analysis,

to determine the status of the pixels of the current macroblock of the current image frame whether they belong to the ROI region or to the background region, and

to update the current ROI mask based on this determination.

8. An electronic device according to claim 7, wherein the analysis of the color histograms of the ROI region and background region is arranged to be performed on the basis of the probabilities, which state if the pixel of the current macroblock is more a ROI pixel or a background pixel.

9. An electronic device according to claim 7, wherein if the ROI region inside the search area of the previous image frame is discovered to share some color content with the background region of the search area of the previous image frame, the tracking unit is arranged to perform a shape matching procedure in order to find the best location for the current macroblock in the search area in the previous ROI mask.

10. An electronic device according to claim 9, wherein the tracking unit is arranged to apply SAD method (Sum of Absolute Difference) in the shape matching procedure.

11. An electronic device according to claim 10, wherein the SAD method is arranged to be performed on a probability domain in which the best match is arranged to be determined for the current macroblock in the search area defined for that.

12. An electronic device according to claim 1, wherein the identifier unit is arranged to generate the ROI mask on the basis of the statistics of the color-content in the middle and around of the defined area including the ROI.

13. An electronic device according to claim 12, wherein identifier unit is arranged to generate search areas inside and around the defined area and to analyze the local color-content between these areas in order to decide whether the pixels of the defined area belong to target or not.

14. An electronic device according to claim 12, wherein identifier unit is arranged to perform histogram-based matching for each pixel within the defined area and a binary dilation process in order to unify the neighborhood of the pixel in the ROI mask.

15. A method comprising:

producing image frames from an imaging view which image frames include a region-of-interest ROI,

identifying a ROI from the image frame in order to perform ROI tracking process during the video imaging process,

tracking the ROI from the image frames during the video imaging process,

determining the spatial position of the ROI from the produced image frames along the Z-axis in order to detect the movement of the ROI in depth of the imaging view,

determining the Z-axis position of the ROI from a change in the size of the ROI between produced consecutive image frames,

providing the tracking results of the ROI for an auto-focus unit in order to adjust an optics, and

adjusting the optics arranged in connection with the camera unit in order to focus the ROI on the camera unit.

16. A method according to claim 15, wherein the spatial position is a plane position of the ROI in the image frame.

17. A method according to claim 15, wherein in the ROI tracking stage the ROI tracking is performed on macroblock basis in which is decided whether the pixels of the current macroblock of the current image frame belong to the ROI region or to the background region and which decision is based on the color content of the previous image frame in which the ROI region is already known.

18. A method according to claim 17, wherein in the ROI tracking stage each macroblock of the tracking window of the current image frame is projected into the previous image frame in which for each of the macroblocks a search area is defined in order to determine the color content of the ROI region and background region.

19. A method according to claim 18, wherein search area is constructed by enlarging the projected macroblock in the previous image frame in each direction in order to ensure that the best match for the current macroblock is inside the search area defined for it.

20. A method according to claim 19, wherein the search area is constructed by enlarging the projected macroblock in the previous image frame in each direction by a distance equal to the estimated motion range.

21. A method according to claim 17, wherein in the ROI tracking stage

defining a ROI region and a background region in a search area of the previous image frame which definitions are based on the ROI mask of the previous image frame,

forming color histograms of the ROI region and the background region of the search area,

analyzing color histograms of the ROI region and background region,

determining, on the basis of the results of the analysis, the status of the pixels of the current macroblock of the current image frame whether they belong to the ROI region or to the background region, and

updating the current ROI mask based on this determination.

22. A method according to claim 21, wherein the analysis of the color histograms of the ROI region and background region is performed on the basis of the probabilities, which state if the pixel of the current macroblock is more a ROI pixel or a background pixel.

23. A method according to claim 21, wherein if the ROI region inside the search area of the previous image frame shares some color content with the background region of the search area of the previous image frame then a shape matching procedure is performed in order to find the best location for the current macroblock in the search area in the previous ROI mask.

24. A method according to claim 23, wherein in the ROI tracking stage SAD method (Sum of Absolute Difference) is applied in the shape matching procedure.

25. A method according to claim 24, wherein the SAD method is performed on a probability domain in which the best match is determined for the current macroblock in the search area defined for that.

26. A method according to claim 15, wherein in the ROI identifying stage the ROI mask is generated based on the statistics of the color-content in the middle and around of the defined area including the ROI.

27. A method according to claim 26, wherein in the ROI identifying stage search areas are generated inside and around the defined area and the local color-content between these areas is analyzed in order to decide whether the pixels of the defined area belong to target or not.

28. A method according to claim 26, wherein in the ROI identifying stage a histogram-based matching is performed for each pixel within the defined area followed by a binary dilation process in order to unify the neighborhood of the pixel in the ROI mask.

29. A computer program product, stored on a computer-readable memory medium and executable by a processor, for video imaging process in order to provide ROI tracking results for an auto-focus unit which is arranged to analyze the ROI on the basis of the tracking results in order to adjust the optics arranged in connection with the camera unit in order to focus the ROI on the camera unit, the computer program product including:

code means configured to identify the ROI from the image frame produced by a camera unit,

code means configured to perform tracking of the ROI from the image frames during the video imaging process,

code means configured to determine the spatial position of the ROI along the Z-axis in order to detect the movement of the ROI in depth of the imaging view,

code means configured to determine the spatial position of the ROI from the produced image frame,

code means configured to determine the Z-axis position of the ROI from a change in the size of the ROI between the produced image frames, and

code means for adjusting the optics on the basis of the determination in an established manner.

30. The computer program product according to claim 29, wherein the program code includes code means configured to determine a plane position of the ROI in the image frame as the said spatial position.

31. The computer program product according to claim 29, wherein the program code includes code means configured to perform the ROI tracking on macroblock basis in which ROI tracking is arranged to be decided whether the pixels of the current macroblock of the current image frame belong to the ROI region or to the background region and which decision is arranged to be based on the color content of the previous image frame in which the ROI region is already known.

32. The computer program product according to claim 31, wherein the program code includes code means configured to project each macroblock of the tracking window of the current image frame into the previous image frame in which for each of the macroblocks a search area is arranged to be defined in order to determine the color content of the ROI region and background region.

33. The computer program product according to claim 32, wherein the program code includes code means configured to construct the search area by enlarging the projected macroblock in the previous image frame in each direction in order to ensure that the best match for the current macroblock is inside the search area defined for it.

34. The computer program product according to claim 33, wherein the program code includes code means configured to construct the search area by enlarging the projected macroblock in the previous image frame in each direction by a distance equal to the estimated motion range.

35. The computer program product according to claim 31, wherein the program code includes code means configured

to define a ROI region and a background region in the search area of the previous image frame which definitions are arranged to be based on the ROI mask of the previous image frame,

to form color histograms of the ROI region and the background region of the search area,

to analyze the said color histograms of the ROI region and background region and on the basis of the results of the analysis,

to determine the status of the pixels of the current macroblock of the current image frame whether they belong to the ROI region or to the background region and

to update the current ROI mask based on this determination.

36. The computer program product according to claim 35, wherein the program code includes code means configured to perform the analysis of the said color histograms of the ROI region and background region on the basis of the probabilities, which state if the pixel of the current macroblock is more a ROI pixel or a background pixel.

37. The computer program product according to claim 36, wherein the program code includes code means configured to discover if the ROI region inside the search area of the previous image frame shares some color content with the background region of the search area of the previous image frame, then to perform a shape matching procedure in order to find the best location for the current macroblock in the search area in the previous ROI mask.

38. The computer program product according to claim 37, wherein the program code includes code means configured to apply SAD method (Sum of Absolute Difference) in the shape matching procedure.

39. The computer program product according to claim 38, wherein the program code includes code means configured to perform the SAD method on a probability domain in which the best match is arranged to be determined for the current macroblock in the search area defined for that.

40. The computer program product according to claim 29, wherein the program code includes code means configured to generate the ROI mask on the basis of the statistics of the color-content in the middle and around of the defined area including the ROI.

41. The computer program product according to claim 40, wherein the program code includes code means configured to generate search areas inside and around the defined area and to analyze the local color-content between these areas in order to decide whether the pixels of the defined area belong to target or not.

42. The computer program product according to claim 40, wherein the program code includes code means configured to perform histogram-based matching for each pixel within the defined area and a binary dilation process in order to unify the neighborhood of the pixel in the ROI mask.