Implementation of multi-camera tracking applications using rich color transition curve target sequences

• US 9,536,322 B1
• Filed: 06/29/2015
• Issued: 01/03/2017
• Est. Priority Date: 05/15/2007
• Status: Active Grant

• Current Assignee: Peter Harmon Smith
• Original Assignee: Peter Harmon Smith
• Inventors: Pratt, Thomas K., Smith, Peter Harmon
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US14/754,126
• Time in Patent Office: 554 Days
• Field of Search: 382/100
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

A method and apparatus for tracking a location of an object or target in a 3D space. At least one target including unique sub-targets is attached to a vision object in the 3D space. A camera disposed in a 3D space, and coupled to a computing device captures a 2D image of a target. The camera computing device identifies the sub-targets of the target and transfers the sub-target data to a computing device associated with the camera which accesses a data base of target data to identify target an, using sub-target separation distances in the database to determine 3D location and orientation of the target. Another computing device transforms the 3D locations of the detected target in the camera coordinate system into a single 3D coordinate system for the 3D space. The 3D location of targets detected by multiple cameras of different computing devices maybe daisy chained together to convert the target location in the 3D coordinate systems of each computing device into a single 3D coordinate system.

Claims

1. A method of tracking an object position in a 3D space comprising:

providing a target on an object in a 3D space, the target including a plurality of sub targets arranged in at least one unique rich color transition sequence;

determining a 3D coordinate position of a camera in the 3D space;

capturing, by the camera, a camera image frame containing the target in the 3D space;

determining a 2D coordinate position of the target in the camera image frame taken by camera;

identifying, by the camera, the identity of each sub-target of the target in the camera image frame;

communicating the sub-target identification for the target by the camera to a first computer associated with and communication coupled to the camera;

storing target identification, sub-target sequences and distances between centers of sub-targets, and a plurality of target IDs;

accessing, by the first computer, the stored target ID, sub-target sequences and center distances of the sub-target as the target image data of the target in the camera image frame; and

determining, by the first computer, the 3D location and orientation of the target in 3D space based on distances between the centers of the sub-targets in the target with distances based on camera data and the target image data.

2. The method of claim 1 wherein providing the target comprises:

arranging the sub targets with centers substantially along a best fit central line through all of the sub targets in one sequence on the target.

3. The method of claim 1 wherein providing the target comprises:

providing the target with a pair of linear sub-target sequences orthogonally arranged with respect to each other.

4. The method of claim 3 wherein providing the pair comprises:

arranging the pair of linear sub-targets sequences co-linear and co-planar to each other.

5. The method of claim 3 wherein providing the pair comprises:

forming the pair of linear sub-targets sequences as at least one of an iron cross and a carpenter square.

6. The method of claim 1, wherein a processor is associated with the camera, the method further comprising:

accessing, by the processor, a database containing the IDs of sub-target sequences used in the 3D space to determine the ID of the target in the camera image frame.

7. The method of claim 6 further comprising:

transforming, by the processor, the 2D coordinate position of the target in the camera image frame into a 3D coordinate system of the camera.

8. The method of claim 7 further comprising:

communication coupling the camera to a room computer, the room computer transforming the 3D coordinate system of the camera and the 2D coordinate position of the target in the camera image frame into a single 3D coordinate system for the 3D space.

9. The method of claim 8 wherein:

the room computer is communication coupled to a second computer;

transforming the 3D location of the target in the single 3D coordinate system to a 3D coordinate system of the second computer.

10. The method of claim 8 further comprising:

tracking, by the room computer, movement of the object through a plurality of camera image frames in the 3D space in the single 3D coordinate system of the room computer.

11. The method of claim 1 wherein:

providing the target on the object in the 3D space includes providing at least one target of a plurality of targets on a different one of a plurality of objects in the 3D space.

12. The method of claim 11, wherein a processor is associated with the camera, the method further comprising:

accessing, by the processor, database containing the IDs of sub-target sequences used in the 3D space to determine the ID of the target in the camera image frame.

13. The method of claim 12 further comprising:

transforming, by the processor, the 2D coordinate position of the target in the camera image frame into a 3D coordinate system of the camera.

14. The method of claim 11 further comprising:

communication coupling the first computer to a room computer associated with the 3D space, the room computer transforming a 3D coordinate system of the camera and the 2D coordinate position of the target in the camera image frame into a single 3D coordinate system for the 3D space.

15. An object tracking apparatus for tracking the position of an object in a 3D space, the apparatus comprising:

a target provided on the object in a 3D space, the target including a plurality of sub targets arranged in a unique rich color transition sequence;

a camera disposed in the 3D space;

a first computer associated and communication coupled to the camera;

the camera capturing a 2D image frame, the image frame containing the target;

the camera identifying an identity of each sub-target of the target in the image frame;

the camera communicating the identity of each sub-target of the target to the first computer;

the first computer accessing stored target ID, sub-target sequences and center distances of the sub-targets to identify the target in the image frame; and

the first computer determining a 3D location and orientation of the object in the 3D space based on transformation of distances between centers of the sub-targets and the target identified by the first computer with measured distances based on camera focal length and target image data.

16. The apparatus of claim 15 wherein:

the sub targets have centers substantially arranged along a best fit central line through all of the sub targets in one sequence.

17. The apparatus of claim 15 wherein:

the target includes a pair of sub-targets orthogonally arranged with respect to each other.

18. The apparatus of claim 17 wherein:

the pair of sub-targets are co-linear and co-planar to each other.

19. The apparatus of claim 18 wherein:

the pair of sub-targets are at least one of an iron cross or a carpenter square.

20. The apparatus of claim 15 comprising:

the first computer accessing a database containing the IDs of a plurality of targets disposed in the 3D space to determine the ID of the target in the image frame; and

the first computer transforming a 2D coordinate position of the target in the image frame into a 3D coordinate system of the camera.

21. The apparatus of claim 15 wherein:

the camera is communication coupled to a processor associated with the 3D space, the processor transforming a 3D coordinate system of the camera and a 2D coordinate position of the target in the image frame into a single 3D coordinate system for the 3D space.

22. The apparatus of claim 15 wherein:

the target on the object in the 3D space includes at least one unique target of a plurality of targets on a different one of a plurality of objects in the 3D space;

the processor accessing a database containing the IDs of a plurality of targets used in the 3D space to determine the ID of the at least one unique target; and

the processor transforming a respective 2D coordinate position of the at least one unique target captured by the camera into a 3D coordinate system for the 3D space.

23. A method of tracking an object position in a 3D space comprising:

providing of a plurality of targets on one object in a 3D space, each target including a plurality of sub targets arranged in at least a unique sequence;

determining a 3D coordinate position of each camera of a plurality of asynchronously operable cameras in the 3D space;

capturing, by each camera, a camera image frame containing at least one target of the plurality of targets in the 3D space;

determining a 2D position of the at least one target in the camera image frame taken by each camera;

providing a database for each of the plurality of targets including at least one of sub-target distance between center points, sub-target relative and sub-target colors;

identifying, by each camera, an identity of each sub-target of the at least one target in the image frame of each camera;

communicating the identity of each sub-target to a respective first computer associated with and communication coupled to each camera of the plurality of cameras;

storing target identification, sub-target sequences and distances between centers of sub-targets for a plurality of target IDs in an accessible memory;

accessing, by the first computer, the stored target ID, sub-target sequences and center distances of the sub-target to identify the at least one target; and

determining by the first computer, a 3D location and orientation of the at least one target in the 3D space based on distances between the centers of the sub-targets in the at least one target with distances based on camera data and target image data.

24. The method of claim 23 comprising:

coupling each of the first computers associated with each of the plurality of cameras to a second computer; and

transforming, by the second computer, 3D coordinate positions of each target identified by each of the first computers into a 3D coordinate position of each of the plurality of targets in the 3D space.

25. The method of claim 23 wherein:

each at least one unique sequence of sub-targets is a unique rich color transition sequence.

26. The method of claim 23 further comprising:

communication coupling each camera of the plurality of cameras to at least one other camera of the plurality of cameras; and

transforming, by a first camera of the plurality of cameras, a 3D coordinate position of one target identified by first camera and communicated to the first camera, into a 3D coordinate position of the at least one other camera coupled to the first camera.

27. The method of claim 23 further comprising:

communication coupling each camera of the plurality of the cameras to each other in a serial daisy chain arrangement so that all of the targets identified by the plurality of cameras has its coordinate position transformed by one of the plurality of cameras into a 3D coordinate position in the 3D space.