Object-tracking systems and methods

• US 8,848,974 B2
• Filed: 09/29/2008
• Issued: 09/30/2014
• Est. Priority Date: 09/29/2008
• Status: Active Grant

• Current Assignee: Venus Concept, Inc.
• Original Assignee: Restoration Robotics, Inc. (Venus Concept, Inc.)
• Inventors: Qureshi, Shehrzad A., Breton, Kyle R., Tenney, John A.
• Primary Examiner(s): Akhavannik, Hadi
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US12/240,724
• Publication Number:
  US 20100080415A1
• Time in Patent Office: 2,192 Days
• Field of Search: 382/100, 382103-104, 382/107, 382128-132, 382/285, 382/276, 382/218, 382/171, 345/169, 345/630, 345/156, 345/420, 345/474, 600/424, 600/426, 600/427, 600/429, 600/431, 128/905, 128/920, 128/925, 606/130, 606/187
• US Class Current: 382/103
• CPC Class Codes: A61B 2017/00752 Hair removal or transplantation
• CPC Class Codes: A61B 2017/00969 used for transplantation
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 90/17 for soft tissue, e.g. breast-holding devices
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/246 : using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/40 : Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/74 : involving reference images or patches
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

A system and method for tracking, identifying, and labeling objects or features of interest is provided. In some embodiments, tracking is accomplished using unique signature of the feature of interest and image stabilization techniques. According to some aspects a frame of reference using predetermined markers is defined and updated based on a change in location of the markers and/or specific signature information. Individual objects or features within the frame may also be tracked and identified. Objects may be tracked by comparing two still images, determining a change in position of an object between the still images, calculating a movement vector of the object, and using the movement vector to update the location of an image device.

Claims

1. A method for tracking a follicular unit, the method comprising:

identifying a follicular unit in a first image of a body surface containing follicular units;

computing a signature of the follicular unit in the first image, wherein the signature is unique to the follicular unit and comprises a plurality of characteristics of the follicular unit;

computing a motion vector associated with movement of the body surface by comparing the first image and a second image of the same body surface, the second image being taken at a different point in time relative to the first image; and

defining a search region according to and in a direction of the motion vector, wherein the search region comprises an area within a predetermined distance from the motion vector; and

searching for the signature of the follicular unit in the search region to label the follicular unit in the second image.

2. The method of claim 1, further comprising identifying at least one marker, wherein there are at least three markers in the first image.

3. The method of claim 1, further comprising identifying at least one marker, wherein the at least one marker comprises one or more of another follicular unit, a mole, a scar, a freckle, a wrinkle, a bump, or a depression of the body surface.

4. The method of claim 1, comprising identifying at least one marker in a first image using a point cloud approach.

5. The method of claim 4, further comprising calculating a centroid of a point cloud.

6. The method of claim 5, wherein the motion vector is computed by identifying a change in the centroid of the point cloud.

7. The method of claim 1, wherein computing the signature of the follicular unit in the first image comprises identifying two or more of a length, type, caliber, emergence angle, area, shape, or color of the follicular unit.

8. The method of claim 1, further comprising identifying at least one marker in the first image and the second image, labeling the at least one marker in the first image, and labeling the at least one marker in the second image with the same label as in the first image by searching for the at least one marker in the second image in the direction pointed to by the motion vector.

9. The method of claim 1, further comprising computing a second motion vector from the first image and the second image, and using the second motion vector to label the follicular unit in the second image.

10. The method of claim 9, wherein the second vector is computed using one or more of the Gray-Coded Bit-Plane, optical flow, and block search image stabilization techniques.

11. The method of claim 1, further comprising moving an image-capture device according to the motion vector to keep the follicular unit in a field of view of the image-capture device.

12. The method of claim 1, further comprising identifying at least one marker, wherein identifying the at least one marker comprises associating electronic identifiers with the at least one marker.

13. The method of claim 1, further comprising identifying at least one marker in the first image and the second image, and discarding from consideration any marker in the second image inconsistent with its location in the first imaae according to the motion vector.

14. The method of claim 1, wherein the search region is defined within a predetermined angle from the direction of the motion vector.

15. The method of claim 14, wherein defining the search region comprises specifying an angle that defines the search region on either side of the motion vector.

16. The method of claim 1, further comprising dividing the first and second images into a plurality of first and second sub-images, and computing a plurality of motion vectors from the first and second sub-images.

17. The method of claim 1, wherein the method further comprises harvesting the follicular unit that has been tracked, and further implanting the harvested follicular unit in a recipient area.

18. A system for tracking a follicular unit on a body surface, the system comprising:

an imaging device for capturing at least two images;

a signature identification component, wherein the signature identification component identifies signature information about the follicular unit, wherein the signature information comprises a plurality of characteristics of the follicular unit and forms a unique signature of the follicular unit;

a vectoring component, wherein the vectoring component calculates a motion vector associated with movement of the body surface depicted in a first image of the two images and a second image of the two images, the second image being taken at a different point in time relative to the first image; and

a tracking system for a) receiving data corresponding to the motion vector, b) defining a search region according to and in a direction of the motion vector and comprising an area within a predetermined distance from the motion vector and c) labeling the follicular unit in the second of the two images by searching for the follicular unit signature information in the search region.

19. The system of claim 18, wherein the tracking system is further programmed to move the imaging device based on the motion vector.

20. The system of claim 18, wherein the signature identification component, vectoring component, and tracking system are part of a single software or hardware product.

21. The system of claim 18, further comprising a marker identification component, and wherein the marker identification component is a part of the signature identification component.

22. The system of claim 18, wherein one or more of the signature identification component, the vectoring component, or the tracking component comprises a processor connected to at least one of memory and the imaging device.

23. The system of claim 18, wherein the system is a robotic system further comprising a robotic arm.

24. The system of claim 18, wherein the motion vector is calculated using point cloud approach.

25. The system of claim 24, wherein a point cloud comprises a plurality of follicular units and wherein the image processor determines a centroid of the plurality of follicular units and/or a centroid of one or more individual follicular units within the plurality of follicular units.

26. The system of claim 24, wherein the motion vector is calculated by identifying a change in a centroid of a point cloud.

27. The system of claim 18, wherein the search region is defined within a predetermined angle from the direction of the motion vector.

28. The system of claim 18, wherein the signature information comprises two or more characteristics from the following list:

caliber, length, type, emergence angle, area, shape or color of the follicular unit.

29. A system for labeling follicular units, comprising:

an interface adapted to receive an image data containing follicular units; and

an image processor comprising one or more modules for executing operations on the image data, the one or more modules including instructions for;

identifying a follicular unit in a first image;

computing a signature of the follicular unit in the first image, wherein the signature is unique to the follicular unit and comprises a plurality of characteristics of the follicular unit;

computing a motion vector associated with movement of the body surface by comparing the first image and a second image of the same body surface, the second image being taken at a different point in time relative to the first image; and

defining a search region according to and in a direction of the motion vector, wherein the search region comprises an area within a predetermined distance from the motion vector; and

searching for the signature of the follicular unit in the search region to label the follicular unit in the second image.

30. The system of claim 29, wherein the signature of the follicular unit comprises two or more characteristics from the following list:

caliber, length, type, emergence angle, area, shape or color of the follicular unit.

31. The system of claim 29, wherein the area of the search region comprises a region within a predetermined angle from the direction of the motion vector.

32. The system of claim 29, wherein the motion vector is calculated using a point cloud approach, the point cloud approach involves calculating a centroid of one or a plurality of follicular units.

33. The system of claim 32, wherein the motion vector is calculated by identifying a change in the centroid of a point cloud.

34. The system of claim 29, further including instructions for identifying and analyzing a plurality of markers in the first image to determine whether the plurality of markers exist in the second image, and using only those markers that exist in both the first and the second image to generate a point cloud.

35. The system of claim 29, wherein computing the motion vector comprises computing a translational component of the motion vector.