Object-Tracking systems and methods

• US 9,405,971 B2
• Filed: 08/14/2014
• Issued: 08/02/2016
• Est. Priority Date: 09/29/2008
• Status: Active Grant

• Current Assignee: Venus Concept, Inc.
• Original Assignee: Restoration Robotics, Inc. (Venus Concept, Inc.)
• Inventors: Qureshi, Shehrzad A., Breton, Kyle R., Tenney, John A.
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/459,968
• Publication Number:
  US 20140355834A1
• Time in Patent Office: 719 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 2017/00752 Hair removal or transplantation
• CPC Class Codes: A61B 2017/00969 used for transplantation
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 90/17 for soft tissue, e.g. breast-holding devices
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/246 : using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/40 : Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/74 : involving reference images or patches
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

A system and method for tracking, identifying, and labeling objects or features of interest, such as follicular units is provided. In some embodiments, tracking is accomplished using unique signature of the follicular unit and image stabilization techniques. According to some aspects pixel data of a region of interest in a first image is compared to pixel data of the regions of interest in a second image, and based on a result of the comparison of pixel data in the region of interest in the first and second images and the signature of the follicular unit, locating the follicular unit in the second image. In some embodiments the follicular unit is searched for in the direction of a motion vector.

Claims

1. A method for tracking a follicular unit on a body, the method comprising:

determining a signature of a follicular unit in a first image of a body surface, wherein the signature comprises a plurality of characteristics of the follicular unit;

determining pixel intensity data of a region of interest in the first image;

determining pixel intensity data of the region of interest in a second image of the body surface taken at a different point in time relative to the first image;

comparing the pixel intensity data of the region of interest in the first image with the pixel intensity data of the region of interest in the second image,

splitting the first image into foreground and background sections, assigning each of the foreground and background sections a separate weight value, and utilizing the assigned weight values to predict the change in position of the follicular unit, and

based on the comparison of the pixel intensity data of the region of interest in the first and the second images, the signature of the follicular unit, and the predicted change in position of the follicular unit, locating the follicular unit in the second image.

2. The method of claim 1, wherein determining the signature of the follicular unit comprises identifying two or more characteristics from a following list:

a length, type, caliber, emergence angle, area, shape, or color of the follicular unit.

3. The method of claim 1, the method comprising:

splitting the first image into multiple first images;

splitting the second image into multiple second images; and

wherein comparing the pixel intensity data of the region of interest in the first image and the second image comprises comparison of the each of the corresponding multiple of the first and second images to produce multiple predictions of changes in position of the follicular unit.

4. The method of claim 1, the method comprising removing from a donor area or transplanting into a recipient area the follicular unit that has been tracked.

5. The method of claim 1, wherein the method comprises an optical flow algorithmic approach.

6. The method of claim 1, the method comprising moving an image-capture device to keep the follicular unit in a field of view of the image-capture device.

7. A system for tracking a follicular unit on a body, the system comprising:

an interface configured to receive a plurality of images of a body surface from an image capture device, at least a first image comprising a follicular unit;

a signature identification component, wherein the signature identification component identifies signature information about the follicular unit in the first image; and

a motion prediction component, wherein the motion prediction component compares pixel intensity data of a region of interest in the first image with pixel intensity data for the region of interest in a second image taken at a different point in time relative to the first image, splits the first image into foreground and background sections, and assigns each of the foreground and background sections a separate weight value, and based on the comparison of pixel intensity data, the signature of the follicular unit and the assigned weight values predicts a change in position of the follicular unit from the first image to the second image to locate the follicular unit in the second image.

8. The system of claim 7, wherein the motion prediction component comprises an algorithm that implements one or more approaches from a following list:

a gray-coded bit plane approach, optical flow approach, block search image stabilization approach, or a combination of the above.

9. The system of claim 7, wherein the motion prediction component splits the first image into multiple first images and splits the second image into multiple second images, and wherein comparison of the pixel intensity data from the region of interest in the first image and the second image comprises comparison of the each of the corresponding multiple of first and second images to produce multiple predictions of changes in position of the follicular unit.

10. The system of claim 7, wherein the signature identification component and the motion prediction component are part of a single software or hardware product.

11. The system of claim 7, wherein one or both of the signature identification component and the motion prediction component comprises a processor connected to a memory and/or the image capture device.

12. The system of claim 7, wherein the system is a robotic system further comprising a robotic arm.

13. The system of claim 7, wherein the system comprises a vectoring component.

14. The system of claim 13, wherein the vectoring component is configured to calculate one or more motion vectors associated with movement of the body surface depicted in the first and the second images.

15. The system of claim 7, wherein identifying the signature information comprises identifying at least one characteristic from a following list:

a length, type, caliber, emergence angle, area, shape, or color of the follicular unit.

16. The system of claim 14, wherein the one or more motion vectors are calculated using a point cloud approach.

17. A system for tracking a follicular unit on a body surface, the system comprising:

an imaging device, the imaging device configured for capturing images; and

an image processor comprising one or more modules for executing operations on the captured images, the one or more modules comprising instructions for;

determining a signature of a follicular unit in a first image of a body surface,

wherein the signature comprises one or more characteristics of the follicular unit;

determining pixel intensity data of a region of interest in the first image;

determining pixel intensity data of the region of interest in a second image of the body surface taken at a different point in time relative to the first image;

splitting the first image into multiple first images;

splitting the second image into multiple second images;

comparing the pixel intensity data of the region of interest in the first image with the pixel intensity data of the region of interest in the second image, wherein comparing the pixel intensity data of the region of interest in the first image and the second image comprises comparison of the each of the corresponding multiple of the first and second images to produce multiple predictions of changes in position of the follicular unit, and based on the comparison of the pixel intensity data of the region of interest in the first and the second images, and the signature of the follicular unit, locating the follicular unit in the second image.

18. A method for tracking a follicular unit on a body, the method comprising:

determining a signature of a follicular unit in a first image of a body surface, wherein the signature comprises a plurality of characteristics of the follicular unit;

determining pixel intensity data of a region of interest in the first image;

determining pixel intensity data of the region of interest in a second image of the body surface taken at a different point in time relative to the first image;

comparing the pixel intensity data of the region of interest in the first image with the pixel intensity data of the region of interest in the second image, and

based on the comparison of the pixel intensity data of the region of interest in the first and the second images and the signature of the follicular unit, locating the follicular unit in the second image;

the method further comprising

splitting the first image into multiple first images;

splitting the second image into multiple second images; and

wherein comparing the pixel intensity data of the region of interest in the first image and the second image comprises comparison of the each of the corresponding multiple of the first and second images to produce multiple predictions of changes in position of the follicular unit.

22. The method of claim 18, wherein the method comprises a Gray-Coded Bit-Plane technique, block search image stabilization techniques, or a combination of the above techniques.

23. The method of claim 18, the method comprises determining a motion vector.

24. The method of claim 23, the method comprising searching for the follicular unit in the second image in a search region using the determined signature of the follicular unit, the search region defined in a direction and within a predetermined distance from the motion vector.

25. The method of claim 24, wherein the search region is defined within a predetermined angle on either side of a direction of the motion vector.

26. The method of claim 18, wherein the predictions comprise both translational and rotational components of the follicular unit.

19. A system for tracking a follicular unit on a body, the system comprising:

an interface configured to receive a plurality of images of a body surface from an image capture device, at least a first image comprising a follicular unit;

a signature identification component, wherein the signature identification component identifies signature information about the follicular unit in the first image; and

a motion prediction component, wherein the motion prediction component compares pixel intensity data of a region of interest in the first image with pixel intensity data for the region of interest in a second image taken at a different point in time relative to the first image, and based on the comparison of pixel intensity data predicts a change in position of the follicular unit from the first image to the second image to locate the follicular unit in the second image;

wherein the motion prediction component splits the first image into multiple first images and splits the second image into multiple second images, and wherein comparison of the pixel intensity data from the region of interest in the first image and the second image comprises comparison of the each of the corresponding multiple of first and second images to produce multiple predictions of changes in position of the follicular unit.

20. A system for tracking a follicular unit on a body surface, the system comprising:

an imaging device, the imaging device configured for capturing images; and

an image processor comprising one or more modules for executing operations on the captured images, the one or more modules comprising instructions for;

determining a signature of a follicular unit in a first image of a body surface, wherein the signature comprises one or more characteristics of the follicular unit;

determining pixel intensity data of a region of interest in the first image;

determining pixel intensity data of the region of interest in a second image of the body surface taken at a different point in time relative to the first image;

comparing the pixel intensity data of the region of interest in the first image with the pixel intensity data of the region of interest in the second image;

splitting the first image into foreground and background sections, assigning each of the foreground and background sections a separate weight value, and utilizing the

assigned weight values to predict the change in position of the follicular unit; and

based on the comparison of the pixel intensity data of the region of interest in the first and the second images and the signature of the follicular unit, locating the follicular unit in the second image.

21. The system of claim 20, wherein the assigned weight values are utilized to determine a motion vector.