Method and device to determine landmark from region of interest of image

• US 10,540,540 B2
• Filed: 10/14/2015
• Issued: 01/21/2020
• Est. Priority Date: 11/07/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Rhee, Seonmin, Kim, Jungbae, Hwang, Youngkyoo, Han, Jaejoon
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US14/882,656
• Publication Number:
  US 20160133002A1
• Time in Patent Office: 1,560 Days
• Field of Search: None
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

At least some example embodiments disclose a device and a method for determining a landmark of an image. The device may compare, to a key landmark set as a reference landmark, a first candidate landmark detected from a region of interest (ROI) of an input image and a second candidate landmark tracked from a previous frame, and determine a landmark similar to the key landmark to be a final landmark.

Claims

1. A device for determining a landmark of an image, the device comprising:

at least one processor configured to execute computer-readable instructions to,

detect a first candidate landmark from a region of interest (ROI) of a current frame of an input image;

track a second candidate landmark of the current frame from final landmark information of a previous frame of the input image;

determine a key landmark based on the first candidate landmark and the second candidate landmark; and

determine final landmark information of the current frame based on a level of similarity between a first histogram of a first local patch adjacent to the detected first candidate landmark and a second histogram of a second local patch adjacent to the key landmark and a level of similarity between a third histogram of a third local patch adjacent to the tracked second candidate landmark and the second histogram of the second local patch adjacent to the key landmark.

2. The device of claim 1, wherein the at least one processor is configured to execute computer-readable instructions to determine the final landmark information of the current frame based on at least one of a first component including the first candidate landmark and a second component including the second candidate landmark.

3. The device of claim 2, wherein the at least one processor is configured to execute computer-readable instructions to determine the final landmark information of the current frame based on the first component, the second component and a corresponding reference component.

4. The device of claim 1, wherein the at least one processor is configured to execute computer-readable instructions to predict the ROI of the current frame from the final landmark information of the previous frame.

5. The device of claim 1, wherein

the at least one processor is configured to execute computer-readable instructions to verify whether the tracking of the second candidate landmark is successful.

6. The device of claim 1, the at least one processor is configured to execute computer-readable instructions to detect the ROI of the current frame.

13. The device of claim 1, wherein the current frame includes a face of a user.

7. A method of determining a landmark of an image, the method comprising:

detecting a first candidate landmark from a region of interest (ROI) of a current frame of an input image;

tracking a second candidate landmark of the current frame from final landmark information of a previous frame of the input image;

determining a key landmark based the first candidate landmark and the second candidate landmark; and

determining final landmark information of the current frame based on a level of similarity between a first histogram of a first local patch adjacent to the first candidate landmark and a second histogram of a second local patch adjacent to the key landmark and a level of similarity between a third histogram of a third local patch adjacent to the second candidate landmark and the second histogram of the second local patch adjacent to the key landmark.

8. The method of claim 7, wherein the determining of the final landmark information of the current frame comprises:

determining the final landmark information of the current frame based on at least one of a first component including the first candidate landmark and a second component including the second candidate landmark.

9. The method of claim 8, wherein the determining of the final landmark information of the current frame further comprises:

determining the final landmark information of the current frame based on the first component, the second component and a corresponding reference component.

10. The method of claim 7, wherein the detecting of the first candidate landmark comprises:

predicting the ROI of the current frame from the final landmark information of the previous frame.

11. The method of claim 7, wherein

the tracking of the second candidate landmark includes,

verifying whether the tracking is successful.

12. The method of claim 7, further comprising:

detecting the ROI of the current frame.

14. The method of claim 7, wherein the current frame includes a face of a user.