Object-Tracking Systems and Methods

• US 20100080417A1
• Filed: 03/27/2009
• Published: 04/01/2010
• Est. Priority Date: 09/29/2008
• Status: Active Grant

• Current Assignee: Venus Concept, Inc.
• Original Assignee: Restoration Robotics, Inc. (Venus Concept, Inc.)
• Inventors: Breton, Kyle R., Tenney, John A., Qureshi, Shehrzad A.
• Granted Patent: US 8,811,660 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: A61B 2017/00752 Hair removal or transplantation
• CPC Class Codes: A61B 2017/00969 used for transplantation
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 90/17 for soft tissue, e.g. breast-holding devices
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/246 : using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/40 : Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/74 : involving reference images or patches
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

A system and method for tracking, identifying, and labeling objects or features of interest is provided. In some embodiments, tracking is accomplished using unique signature of the feature of interest and image stabilization techniques. According to some aspects a frame of reference using predetermined markers is defined and updated based on a change in location of the markers and/or specific signature information. Individual objects or features within the frame may also be tracked and identified. Objects may be tracked by comparing two still images, determining a change in position of an object between the still images, calculating a movement vector of the object, and using the movement vector to update the location of an image device.

Claims

1. A system for tracking a feature of interest on a body surface, the system comprising:

an interface adapted to receive a plurality of images from an image capture device, at least a first image comprising a feature of interest;

a signature identification component, wherein the signature identification component identifies signature information about the feature of interest and detects at least one marker in the first image;

a marker referencing system, wherein the referencing system defines a frame of reference corresponding to the at least one marker, determines whether the at least one marker is in a second image, and adjusts the frame of reference corresponding to a change in position of the at least one marker from the first image to the second image to locate the feature of interest.

2. The system of claim 1, wherein the signature identification component and the marker referencing system are part of a single computer program.

3. The system of claim 1, wherein the signature identification component comprises a processor connected to memory, and wherein the processor associates an electronic identifier with the at least one marker and stores the electronic identifier in the memory.

4. The system of claim 1, wherein the marker referencing system comprises a processor connected to memory, and wherein the processor compares a location of each of the at least one marker in the first image to a location of each of the at least one marker in a second image, calculates a corresponding change in a frame of reference, and stores a result of the calculation in the memory.

5. The system of claim 1, further comprising an image capture device and wherein the image-capture device is a camera.

6. The system of claim 5, wherein the system is a robotic system further comprising a robotic arm and wherein the image capture device is operatively connected to the robotic arm.

7. The system of claim 1, wherein the feature of interest is a hair follicle and wherein the at least one marker is one or more of the anatomical landmarks on the body surface and an external fiducial.

8. The system of claim 1, wherein the at least one marker is a follicular unit and detecting the at least one marker further comprises identifying at least one of the emergence angle of the follicular unit from the body surface and the caliber of the hair.

9. The system of claim 1, wherein the marker referencing system adjusts the frame of reference by using a point cloud approach.

10. The system of claim 9, further comprising an image capture device, and wherein the image capture device is moved according to an average shift of the point cloud.

12. The system of claim 1, wherein the processor is operatively associated with the memory.

11. A system for tracking a feature of interest on a body surface, the system comprising:

an imaging device, the imaging device configured for capturing still images; and

an image processor comprising one or more modules for executing operations on the captured images, the one or more modules comprising instructions for;

identifying at least one marker in a first still image, the first still image containing a feature of interest;

determining a signature of the feature of interest in the first image;

defining a frame of reference corresponding to the at least one marker;

locating the at least one marker in a second still image; and

detecting a position of the feature of interest in the second still image using the determined signature and updating a translational component of the frame of reference based on a position of the at least one marker in the second still image.

13. A method for tracking a feature of interest on a body surface, the method comprising:

identifying at least one marker in a first still image taken by an imaging device, the first still image containing a feature of interest;

determining a signature of the feature of interest in the first still image;

defining a frame of reference corresponding to the at least one marker;

locating the at least one marker in a second still image taken by the imaging device; and

detecting a position of the feature of interest in the second still image using the determined signature and updating a translational component of the frame of reference based on a position of the at least one marker in the second still image.

14. The method of claim 13, wherein the feature of interest is a hair follicle and determining the signature of the follicular unit in the first still image comprises identifying one or more of a length, type, caliber, emergence angle, area, shape, and color of the hair follicle.

15. The method of claim 13, wherein identifying the at least one marker in a first still image comprises identifying at least three markers, and

wherein detecting the position of the feature of interest in the second still image comprises updating both translational and rotational components of the frame of reference.

16. The method of claim 13, wherein identifying the at least one marker in the first still image comprises analyzing at least one of a length, an area, a shape, a type, or a color of the at least one marker.

17. The method of claim 16, wherein the at least one marker is a follicular unit and identifying the at least one marker further comprises identifying at least one of the emergence angle of the follicular unit from the body surface and the caliber of the hair.

18. The method of claim 13, wherein the signature of the feature of interest is determined by transforming the first still image to extract one or more characteristics that constitutes the signature of the feature of interest.

19. The method of claim 13, further comprising updating the frame of reference by using a point cloud approach.

20. The method of claim 19, further comprising moving an imaging device according to an average shift of the point cloud.

21. The method of claim 13, wherein the feature of interest is a bald spot and the method further comprises defining one or more hair implantation sites.

22. A method for tracking a feature of interest on a body surface, the method comprising:

identifying at least three markers in a first still image taken by an imaging device, the first still image displaying a feature of interest;

determining a signature of the feature of interest in the first still image;

tracking at least three first objects in the first still image;

defining a frame of reference corresponding to the three markers;

determining whether the three markers are in a second still image taken by an imaging device; and

in response to at least one of the three markers not being in the second still image, detecting a position of the feature of interest in the second still image using the determined signature of the feature of interest and updating the frame of reference based on a position of any of the at least three markers in the second still image and any of the first objects in the second image,

wherein a sum of a combination of markers and first objects used to detect the position of the feature of interest is at least three.

23. The method of claim 22, wherein the signature of the feature of interest in the first still image is determined by transforming the first still image to extract one or more characteristics that constitute the signature of the feature of interest.

24. The method of claim 22, wherein at least one of the at least tree first objects comprises a follicular unit, a mole, a scar, a freckle, a wrinkle, a bump, or a depression of the body surface.

25. The method of claim 22, wherein detecting the position of the feature of interest further comprises computing a motion vector from the first still image and the second still image and using the motion vector to update the frame of reference and to locate at least one of the at least three markers.

26. The method of claim 25, wherein computing the motion vector comprises computing using one or more of a point cloud approach, Gray-Coded Bit-Plane, optical flow, and block search image stabilization approach.

27. The method of claim 25, further comprising identifying at least one first object in the first still image and locating the identified first object in the second still image, and wherein computing the motion vector comprises identifying a change in position of the identified first object from the first image to the second image.