Object positioning method and device based on object detection results of plural stereo cameras

• US 9,615,080 B2
• Filed: 01/27/2014
• Issued: 04/04/2017
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Wang, Xin, Fan, Shengyin
• Primary Examiner(s): Shibru, Helen
• Application Number:
  US14/164,739
• Publication Number:
  US 20140218483A1
• Time in Patent Office: 1,163 Days
• Field of Search: 348 47, 348 141, 382103
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals

Abstract

Disclosed are an object positioning method and an object positioning device based on object detection results of plural stereo cameras. The method comprises a step of obtaining, when each of the plural stereo cameras continuously carries out tracking and detection with respect to each of objects, positional information of the corresponding object; a step of generating, based on the positional information of the corresponding object, a trajectory of the corresponding object; and a step of carrying out a merging process with respect to the trajectories generated corresponding to the plural stereo cameras so as to determine at least one object position.

Claims

1. An object positioning method based on object detection results of plural stereo cameras, comprising:

obtaining, when each of the plural stereo cameras continuously carries out tracking and detection with respect to each of objects, positional information of the corresponding object;

generating, based on the positional information of the corresponding object, a trajectory of the corresponding object; and

carrying out a merging process with respect to the trajectories generated corresponding to the plural stereo cameras so as to determine at least one object position, wherein the step of carrying out the merging process includes

(a) creating, for each of the plural stereo cameras, a trajectory list, and storing the respective trajectories generated corresponding to this stereo camera in the corresponding trajectory list;

(b) selecting a trajectory from the corresponding trajectory list, and comparing the selected trajectory with other trajectories stored in other trajectory lists so as to find at least one trajectory from the other trajectory lists which matches the selected trajectory;

(c) removing, when the at least one trajectory which matches the selected trajectory is found, the selected trajectory and the at least one trajectory which matches the selected trajectory from the corresponding trajectory list and the other trajectory lists, and carrying out a combination process with respect to newest positional information of the selected trajectory and the at least one trajectory so as to determine a position of the corresponding object;

(d) removing, when the at least one trajectory which matches the selected trajectory is not found, the selected trajectory from the corresponding trajectory list, and determining a position of the corresponding object based on newest positional information of the selected trajectory; and

(e) carrying out the steps (a) to (d) repeatedly until the respective trajectory lists are empty.

2. The object positioning method according to claim 1, wherein:

the positional information of the corresponding object includes timestamps, an object index, and positional coordinates corresponding to the timestamps.

3. The object positioning method according to claim 2, wherein, the step of generating, based on the positional information of the corresponding object, a trajectory of the corresponding object includes:

predetermining timestamps for resampling;

calculating, by carrying out interpolation with respect to the obtained positional information, positional information at the predetermined timestamps of the corresponding object; and

creating, based on the predetermined timestamps and the positional information corresponding to the predetermined timestamps, the trajectory of the corresponding object.

4. The object positioning method according to claim 2, wherein, the step of generating, based on the positional information of the corresponding object, a trajectory of the corresponding object includes:

predetermining a time period corresponding to the trajectory to be generated;

cutting out, from the obtained positional information, positional information of the corresponding object in the time period; and

creating, by using the positional information cut out from the obtained positional information, the trajectory of the corresponding object.

5. The object positioning method according to claim 4, wherein:

a start time point and an end time point of the time period are newest timestamps, respectively.

6. The object positioning method according to claim 5, wherein the step of carrying out the merging process with respect to the trajectories generated corresponding to the plural stereo cameras so as to determine object positions includes:

carrying out a matching process with respect to the generated trajectories corresponding to the plural stereo cameras so as to determine, for each object, at least one matched trajectory of the corresponding object;

carrying out, when the corresponding object has the at least one matched trajectory, a combination process with respect to newest positional information of the selected trajectory and the at least one matched trajectory of the corresponding object so as to determine a position of the corresponding object; and

determining, when the corresponding object does not have the at least one matched trajectory, the position of the corresponding object based on newest position information of the corresponding object.

7. The object positioning method according to claim 6, wherein:

whether or not two trajectories are matched is determined based on correlation of the two trajectories.

8. The object positioning method according to claim 5, wherein:

whether or not two trajectories are matched is determined based on correlation of the two trajectories.

9. An object positioning device based on object detection results of plural stereo cameras, comprising:

processing circuitry configured to

obtain, when each of the plural stereo cameras continuously carries out tracking and detection with respect to each of objects, positional information of the corresponding object;

generate, based on the positional information of the corresponding object, a trajectory of the corresponding object; and

carry out a merging process with respect to the trajectories generated corresponding to the plural stereo cameras so as to determine at least one object position,

wherein the processing circuitry is further configured to carry out the merging process by

(a) creating, for each of the plural stereo cameras, a trajectory list, and storing the respective trajectories generated corresponding to this stereo camera in the corresponding trajectory list;

(b) selecting a trajectory from the corresponding trajectory list, and comparing the selected trajectory with other trajectories stored in other trajectory lists so as to find at least one trajectory from the other trajectory lists which matches the selected trajectory;

(c) removing, when the at least one trajectory which matches the selected trajectory is found, the selected trajectory and the at least one trajectory which matches the selected trajectory from the corresponding trajectory list and the other trajectory lists, and carrying out a combination process with respect to newest positional information of the selected trajectory and the at least one trajectory so as to determine a position of the corresponding object;

(d) removing, when the at least one trajectory which matches the selected trajectory is not found, the selected trajectory from the corresponding trajectory list, and determining a position of the corresponding object based on newest positional information of the selected trajectory; and

(e) carrying out the steps (a) to (d) repeatedly until the respective trajectory lists are empty.

10. The object positioning device according to claim 9, wherein:

the positional information of the corresponding object includes timestamps, an object index, and positional coordinates corresponding to the timestamps.

11. The object positioning device according to claim 10, wherein the processing circuitry is further configured to:

predetermine timestamps for resampling;

calculate, by carrying out interpolation with respect to the obtained positional information, positional information at the predetermined timestamps of the corresponding object; and

create, based on the predetermined timestamps and the positional information corresponding to the predetermined timestamps, the trajectory of the corresponding object.

12. The object positioning device according to claim 10, wherein the processing circuitry is further configured to:

predetermine a time period corresponding to the trajectory to be generated;

cut out, from the obtained positional information, positional information of the corresponding object in the time period; and

create, by using the positional information cut out from the obtained positional information, the trajectory of the corresponding object.

13. The object positioning device according to claim 12, wherein:

a start time point and an end time point of the time period are newest timestamps, respectively.

14. The object positioning device according to claim 13, wherein the processing circuitry is further configured to:

carry out a matching process with respect to the generated trajectories corresponding to the plural stereo cameras so as to determine, for each object, at least one matched trajectory of the corresponding object;

carry out, when the corresponding object has the at least one matched trajectory, a combination process with respect to newest positional information of the selected trajectory and the at least one matched trajectory of the corresponding object so as to determine a position of the corresponding object; and

determine, when the corresponding object does not have the at least one matched trajectory, the position of the corresponding object based on newest position information of the corresponding object.

15. The object positioning device according to claim 14, wherein:

whether or not two trajectories are matched is determined based on correlation of the two trajectories.

16. The object positioning device according to claim 13, wherein:

whether or not two trajectories are matched is determined based on correlation of the two trajectories.