METHOD AND DEVICE TO DETERMINE LANDMARK FROM REGION OF INTEREST OF IMAGE

• US 20160133002A1
• Filed: 10/14/2015
• Published: 05/12/2016
• Est. Priority Date: 11/07/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: RHEE, Seonmin, KIM, Jungbae, HWANG, Youngkyoo, HAN, Jaejoon
• Granted Patent: US 10,540,540 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

At least some example embodiments disclose a device and a method for determining a landmark of an image. The device may compare, to a key landmark set as a reference landmark, a first candidate landmark detected from a region of interest (ROI) of an input image and a second candidate landmark tracked from a previous frame, and determine a landmark similar to the key landmark to be a final landmark.

Claims

1. A device for determining a landmark of an image, the device comprising:

a landmark detector configured to detect a first candidate landmark from a region of interest (ROI) of a current frame of an input image;

a landmark tracker configured to track a second candidate landmark of the current frame from final landmark information of a previous frame of the input image; and

a landmark corrector configured to determine final landmark information of the current frame based on the detected first candidate landmark and the tracked second candidate landmark.

2. The device of claim 1, further comprising:

a key landmark determiner configured to determine a key landmark of the current frame based on the first candidate landmark and the second candidate landmark.

3. The device of claim 2, wherein the key landmark determiner is configured to determine one of the first candidate landmark and the second candidate landmark to be the key landmark if a difference between a position of the first candidate landmark and a position of the second candidate landmark is less than a key threshold, and

the landmark corrector is configured to determine information associated with the key landmark to be the final landmark information of the current frame.

4. The device of claim 1, wherein the landmark corrector is configured to determine the final landmark information of the current frame based on a result of comparing at least one of the first candidate landmark and the second candidate landmark to a stored key landmark prior to the current frame.

5. The device of claim 4, wherein the landmark corrector is configured to determine the final landmark information of the current frame based on a level of similarity between the first candidate landmark and the stored key landmark and a level of similarity between the second candidate landmark and the stored key landmark.

6. The device of claim 1, wherein the landmark corrector is configured to determine the final landmark information of the current frame based on at least one of a first component including the first candidate landmark and a second component including the second candidate landmark.

7. The device of claim 6, wherein the landmark corrector is configured to determine the final landmark information of the current frame based on the first component, the second component and a corresponding reference component.

8. The device of claim 1, wherein the device further comprises:

an ROI predictor configured to predict the ROI of the current frame from the final landmark information of the previous frame.

9. The device of claim 1, wherein

the landmark tracker is configured to verify whether the tracking of the second candidate landmark is successful, and

the landmark corrector is configured to determine information associated with the first candidate landmark to be the final landmark information of the current frame if the tracking of the second candidate landmark is verified to be a failure.

10. The device of claim 1, further comprising:

an ROI detector configured to detect the ROI of the current frame.

11. A method of determining a landmark of an image, the method comprising:

detecting a first candidate landmark from a region of interest (ROI) of a current frame of an input image;

tracking a second candidate landmark of the current frame from final landmark information of a previous frame of the input image; and

determining final landmark information of the current frame based on the detected first candidate landmark and the tracked second candidate landmark.

12. The method of claim 11, further comprising:

determining a key landmark of the current frame based the first candidate landmark and the second candidate landmark.

13. The method of claim 12, wherein the determining of the key landmark comprises:

determining one of the first candidate landmark and the second candidate landmark to be the key landmark if a difference between a position of the first candidate landmark and a position of the second candidate landmark is less than a key threshold, and

the determining of the final landmark information of the current frame includes,

determining information associated with the key landmark to be the final landmark information of the current frame.

14. The method of claim 11, wherein the determining of the final landmark information of the current frame comprises:

determining the final landmark information of the current frame based on a result of comparing at least one of the first candidate landmark and the second candidate landmark to a stored key landmark prior to the current frame.

15. The method of claim 14, wherein the determining of the final landmark information of the current frame further comprises:

determining the final landmark information of the current frame based on a level of similarity between the first candidate landmark and the stored key landmark and a level of similarity between the second candidate landmark and the stored key landmark.

16. The method of claim 11, wherein the determining of the final landmark information of the current frame comprises:

determining the final landmark information of the current frame based on at least one of a first component including the first candidate landmark and a second component including the second candidate landmark.

17. The method of claim 16, wherein the determining of the final landmark information of the current frame further comprises:

determining the final landmark information of the current frame based on the first component, the second component and a corresponding reference component.

18. The method of claim 11, wherein the detecting of the first candidate landmark comprises:

predicting the ROI of the current frame from the final landmark information of the previous frame.

19. The method of claim 11, wherein

the tracking of the second candidate landmark includes,

verifying whether the tracking is successful, and

the determining of the final landmark information of the current frame includes,

determining information associated with the first candidate landmark to be the final landmark information of the current frame if the tracking of the second candidate landmark is verified to be a failure.

20. The method of claim 11, further comprising:

detecting the ROI of the current frame.