Object-tracking systems and methods

• US 9,589,368 B2
• Filed: 06/27/2016
• Issued: 03/07/2017
• Est. Priority Date: 09/29/2008
• Status: Active Grant

• Current Assignee: Venus Concept, Inc.
• Original Assignee: Restoration Robotics, Inc. (Venus Concept, Inc.)
• Inventors: Qureshi, Shehrzad A., Breton, Kyle R., Tenney, John A.
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US15/193,594
• Publication Number:
  US 20160307333A1
• Time in Patent Office: 253 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 2017/00752 Hair removal or transplantation
• CPC Class Codes: A61B 2017/00969 used for transplantation
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 90/17 for soft tissue, e.g. breast-holding devices
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/246 : using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/40 : Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/74 : involving reference images or patches
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

A system and method for tracking, identifying, and labeling objects or features of interest, such as follicular units is provided. In some embodiments, tracking is accomplished using unique signature of the follicular unit and image stabilization techniques. According to some aspects pixel data of a region of interest in a first image is compared to pixel data of the regions of interest in a second image, and based on a result of the comparison of pixel data in the region of interest in the first and second images and the signature of the follicular unit, locating the follicular unit in the second image. In some embodiments the follicular unit is searched for in the direction of a motion vector.

Claims

1. A method for tracking a feature of interest on a body, the method comprising:

determining a signature of a feature of interest in a first image of a body surface;

determining pixel intensity data of a region of interest in the first image;

determining pixel intensity data of the region of interest in a second image of the body surface taken at a different point in time relative to the first image;

splitting the first image into multiple first images;

splitting the second image into multiple second images;

comparing the pixel intensity data of the region of interest in the first image with the pixel intensity data of the region of interest in the second image, wherein comparing comprises comparison of the each of the corresponding multiple of the first and second images to produce multiple predictions of changes in position of the feature of interest; and

based on the comparison of the pixel intensity data of the region of interest in the first and the second images and the signature of the feature of interest, locating the feature of interest in the second image.

2. The method of claim 1, wherein determining the signature comprises identifying one or more characteristics from a following list:

color, length, type, shape, and emergence angle.

3. The method of claim 1, wherein the method comprises an optical flow algorithmic approach, a Gray-Coded Bit-Plane technique, block search image stabilization techniques, or a combination of the above.

4. The method of claim 1, the method comprising using an image-acquisition device to capture images and moving the image-acquisition device to keep the feature of interest in a field of view of the image-acquisition device.

5. The method of claim 1, further comprising determining whether at least one marker in the first image is in the second image, wherein the at least one marker comprises a follicular unit, a mole, texturing, blood, a scar, a freckle, a wrinkle, a bump, or a depression of a body surface.

6. The method of claim 1, wherein the feature of interest is a bald spot and the method further comprises defining one or more hair implantation sites.

7. The method of claim 1, further comprising determining a motion vector.

8. A system for tracking a feature of interest on a body, the system comprising:

an interface configured to receive a plurality of images of a body surface from an image acquisition device, at least a first image comprising a feature of interest;

a signature identification component, wherein the signature identification component identifies signature information about the feature of interest in the first image; and

a prediction component, wherein the prediction component compares pixel intensity data of a region of interest in the first image with pixel intensity data for the region of interest in a second image taken at a different point in time relative to the first image, and based on the comparison of pixel intensity data predicts a change in position of the feature of interest from the first image to the second image to locate the feature of interest in the second image;

wherein the prediction component splits the first image into multiple first images and splits the second image into multiple second images, and wherein comparison of the pixel intensity data from the region of interest in the first image and the second image comprises comparison of the each of the corresponding multiple of first and second images to produce multiple predictions of changes in position of the feature of interest.

9. The system of claim 8, wherein one or both of the signature identification component and the prediction component comprises a processor connected to a memory and/or the image capture device.

10. The system of claim 8, further comprising a vectoring component, wherein the vectoring component is configured to calculate one or more motion vectors associated with movement of the body surface depicted in the first and the second images.

11. The system of claim 10, wherein the one or more motion vectors are calculated using a point cloud approach.

12. A method for tracking a feature of interest on a body, the method comprising:

determining a signature of a feature of interest in a first image of a body surface;

determining pixel intensity data of a region of interest in the first image;

determining pixel intensity data of the region of interest in a second image taken at a different point in time relative to the first image;

comparing the pixel intensity data of the region of interest in the first image with the pixel intensity data for the region of interest in the second image,

splitting the first image into foreground and background sections, assigning each of the foreground and background sections a separate weight value, and utilizing the assigned weight values to predict the change in position of the features of interest, and

based on the comparison of pixel data in the region of interest in the first and the second images, the signature of the feature of interest, and the predicted change in position of the feature of interest, locating the feature of interest in the second image.

13. The method of claim 12, further comprising:

splitting the first image into multiple first images;

splitting the second image into multiple second images; and

wherein comparing the pixel intensity data of the region of interest in the first image and the second image comprises comparison of the each of the corresponding multiple of the first and second images to produce multiple predictions of changes in position of the feature of interest.

14. The method of claim 12, further comprising:

determining whether at least one marker in the first image is in the second image;

calculating a motion vector corresponding to a change in position of the at least one marker from the first image to the second image; and

utilizing the motion vector corresponding to the change in position of the at least one marker from the first image to the second image to predict the change in position of the feature of interest.

15. The method of claim 14, wherein identifying the marker comprises analyzing at least one of a length, an area, a shape, a type or a color of the marker.

16. The method of claim 12, wherein predicting the change in position comprises predicting both translational and rotational components of the feature of interest.

17. The method of claim 12, wherein the signature of the feature of interest is determined by transforming the first still image to extract one or more characteristics that constitutes the signature of the feature of interest.

18. The method of claim 12, wherein determining the signature comprises identifying one or more characteristics from a following list:

color, length, type, shape, and emergence angle.

19. The method of claim 12, wherein the feature of interest is a bald spot and the method further comprises defining one or more hair implantation sites.

20. A system for tracking a feature of interest on a body, the system comprising:

an interface configured to receive from an image capture device a plurality of images of a body surface, at least a first image comprising a feature of interest;

a signature identification component, wherein the signature identification component identifies signature information about the feature of interest in the first image; and

a prediction component, wherein the prediction component compares pixel intensity data of a region of interest in the first image with pixel intensity data for the region of interest in a second image taken at a different point in time relative to the first image, splits the first image into foreground and background sections, assigns each of the foreground and background sections a separate weight value, and based on the comparison of pixel intensity data, the signature of the feature of interest and the assigned weight values predicts a change in position of the feature of interest from the first image to the second image to locate the feature of interest in the second image.

21. The system of claim 20, wherein the prediction component comprises an algorithm that implements one or more approaches from a following list:

a gray-coded bit plane approach, optical flow approach, block search image stabilization approach, or a combination of the above.

22. The system of claim 20, wherein the signature identification component and the prediction component are part of a single software or hardware product.

23. The system of claim 20, wherein one or both of the signature identification component and the prediction component comprises a processor connected to a memory and/or the image capture device.

24. The system of claim 23, wherein the processor associates an electronic identifier with at least one marker and stores the electronic identifier in the memory.

25. The system of claim 20, wherein the system is a robotic system comprising a robotic arm and at least one camera operatively connected to the robotic arm.