Object detecting device, object detecting method and object detecting computer program

• US 7,852,217 B2
• Filed: 12/18/2006
• Issued: 12/14/2010
• Est. Priority Date: 12/28/2005
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Kondo, Kenji, Naruoka, Tomonobu
• Primary Examiner(s): Mehmood; Jennifer
• Application Number:
  US11/919,452
• Publication Number:
  US 20090066513A1
• Time in Patent Office: 1,457 Days
• Field of Search: 340 101- 106, 340 58- 583, 3405721-5729, 382/103
• US Class Current: 340/572.1
• CPC Class Codes: G01S 13/04 Systems determining presence of a target based on relative movement of target G01S13/56
• CPC Class Codes: G01S 13/78 discriminating between different kinds of targets, e.g. IFF-radar, i.e. identification of friend or foe G01S13/75, G01S13/79 take precedence
• CPC Class Codes: G01V 15/00 Tags attached to, or associated with, an object, in order to enable detection of the object record carriers for use with machines having a detectable tag or marker G06K19/00

Abstract

When object detection means, for detecting ID information and a position of an object from outputs of a wireless tag reader, a human detection sensor, and a camera, determines that data relating to first object ID information and data relating to second object ID information, corresponding respectively to first time and second time on which human detection data indicating a presence of a human is obtained, are different from each other, the object detection means calculates a difference between first image data and second image data corresponding to the respective times to thereby detect the object position.

Claims

1. An object detecting device, comprising:

at least one wireless tag reader for detecting object ID data relating to object ID information of an object to which a wireless tag is given in a space where the object is searched;

at least one human position sensor for measuring a position of a human handling the object in the space;

at least one image-pickup device for picking up an image of a part or a whole of the space;

a storing means for storing the object ID data, human position data measured by the human position sensor, and image data picked up by the image-pickup device, while associating a time with the object ID data, the human position data and the image data, wherein the time indicates when the object ID data, the human position data and the image data were obtained; and

an object detection means for detecting object ID information and a position of the object based on the object ID data, the human position data, and the image data, stored in the storing means, wherein

when the object detection means determines that first object ID data corresponding to a first time that first human position data is obtained and relating to first object ID information and second object ID data corresponding to a second time that second human position data is obtained and relating to second object ID information, among a set of object ID data in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader, are different from each other, the object detection means calculates a difference between first image data corresponding to the first time and second image data corresponding to the second time, and in the calculation, the object detection means calculates a difference between the first image data and the second image data for an area determined based on the first human position data and the second human position data, and then, detects a position of the difference area in image data or a position obtained by converting the position to a position in the space where the object is searched, as a position of the object.

6. The object detecting device according to claim 1, wherein when calculating the difference between the first image data and the second image data, the object detection means specifies object ID information having a difference in the data relating to the first object ID information and the data relating to the second object ID information, and calculates the difference by using at least one piece of information among color, size, and shape information of the object corresponding to the object ID information having the difference.

2. An object detecting device, comprising:

at least one wireless tag reader for detecting object ID data relating to object ID information of an object to which a wireless tag is given in a space where the object is searched;

at least one human position sensor for measuring a position of a human handling the object in the space;

at least two image-pickup devices each for picking up an image of a part or a whole of the space;

a storing means for storing the object ID data human position data measured by the human position sensor, and image data picked up by the image-pickup device, while associating a time with the object ID data, the human position data and the image data, wherein the time indicates when the object ID data, the human position data and the image data were obtained; and

an object detection means for detecting object ID information and a position of the object based on the object ID data, the human position data, and the image data, stored in the storing means, wherein

when the object detection means determines that first object ID data corresponding to a first time that first human position data is obtained and relating to first object ID information and second object ID data corresponding to a second time that second human position data is obtained and relating to second object ID information, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader, are different from each other, the object detection means specifies the image-pickup device having an area determined based on the human position data from the first time to the second time as an image-picking up field of view, or the image-pickup device having a viewpoint from which the area is not hidden by any other object including a human in each of first image data corresponding to the first time and second image data corresponding to the second time, from the at least two image-pickup devices, and the object detection means calculates a difference between the first image data and the second image data corresponding to the first time and the second time respectively, picked up by the specified image-pickup device, and detects a position of an area of the difference in the image data or a position obtained by converting the position to a position in the space where the object is searched, as a position of the object.

3. An object detecting device, comprising:

at least one wireless tag reader for detecting object ID data relating to object ID information of an object to which a wireless tag is given in a space where an object is searched;

at least one human detection sensor for detecting whether a human is present in a detection range of the wireless tag reader in the space;

at least one image-pickup device for picking up an image of a part or a whole of the space;

a storing means for storing the object ID data, human detection data detected by the human detection sensor, and image data picked up by the image-pickup device, while associating a time with the object ID data, the human position data and the image data, wherein the time indicates when the object ID data, the human position data and the image data were obtained; and

an object detection means for detecting object ID information and a position of the object based on the object ID data, the human detection data and the image data, stored in the storing means, wherein

when the object detection means determines that first object ID data corresponding to a first time that first human position data is obtained and relating to first object ID data and second object ID data corresponding to a second time that second human position data is obtained and relating to second object ID information, are different from each other, the object detection means calculates a difference between first image data corresponding to the first time and second image data corresponding to the second time, and detects a position of a difference area in the image data or a position obtained by converting the position of the difference area to a position in the space where the object is searched, as a position of the object.

4. The object detecting device according to claim 3, wherein the image-pickup device is a camera, and when calculating the difference between the first image data and the second image data, the object detection means excludes an area obtained by converting the detection range of the human detection sensor to a camera coordinate system from a difference subject area for calculating the difference between the first image data and the second image data, and calculates the difference for an area from which the area has been excluded.

5. The object detecting device according to claim 3, wherein the object detection means includes:

an image data selecting unit for selecting first image data and third image data of which obtained times are different from each other, based on the first time, and selecting second image data and fourth image data of which obtained times are different from each other, based on the second time; and

an object detection unit for determining a first mask area by calculating a difference between the first image data and the third image data selected by the image data selecting unit, and determining a second mask area by calculating a difference between the second image data and the fourth image data selected by the image data selecting unit, and in calculating a difference between the first image data and the second image data, calculating the difference between the first image data and the second image data after excluding the first mask area and the second mask area from a difference subject area.

7. An object detecting method of detecting ID information and a position of an object by using object ID data relating to object ID information detected by at least one wireless tag reader for detecting the object ID data relating to the object ID information of an object to which a wireless tag is given, human detection data detected by at least one human detection sensor for detecting whether a human is present in a detection range of the wireless tag reader, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space where the object is searched, which are accumulated in association with obtained times of the respective pieces of object ID data, the method comprising:

comparing first object ID data corresponding to a first time at which the human detection data is obtained and relating to first object ID information with second object ID data corresponding to a second time at which the human detection data is obtained and relating to second object ID information;

selecting first image data and second image data respectively corresponding to the first time and the second time;

calculating a difference between the first image data and the second image data, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in the space where the object is searched, as a position of the object; and

as a result of the comparison in the object ID data comparison, when the pieces of data relating to the object ID information do not coincide with each other, carrying out the image data selection and the object detection.

8. An object detecting method of detecting ID information and a position of an object by using object ID data relating to object ID information detected by at least one wireless tag reader for detecting the object ID data relating to the object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of object ID data, the method comprising:

comparing first object ID data corresponding to a first time at which the human detection data is obtained and relating to first object ID information with second object ID data corresponding to a second time at which the human detection data is obtained and relating to second object ID information, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

selecting first image data and second image data respectively corresponding to the first time and the second time;

calculating a difference between the first image data and the second image data selected in the image data selection, detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched as a position of the object, and in calculating the difference, calculating a difference for an area determined based on the human position data from the first time to the second time; and

as a result of the comparison in the object ID comparison, when the pieces of data relating to the object ID information do not coincide with each other, carrying out the image data selection and the object detection.

9. An object detecting method of detecting ID information and a position of an object by using object ID data relating to object ID information detected by at least one wireless tag reader for detecting the data relating to the object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least two image-pickup devices each for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of object ID data, the method comprising:

comparing first object ID data corresponding to a first time when human position data was obtained and relating to first object ID information with second object ID data corresponding to a second time when human position data was obtained and relating to second object ID information, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

specifying an image-pickup device having an area determined based on the human position data from the first time to the second time as an image-picking up field of view, or an image-pickup device having a viewpoint from which the area is not hidden by other object including a human in each of first image data and second image data corresponding to the first time and the second time respectively, from the at least two image-pickup devices, and selecting the first image data and the second image data corresponding to the first time and the second time respectively, picked up by the specified image-pickup device;

calculating a difference between the first image data and the second image data selected in the image-pickup device and image data selection, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object; and

as a result of the comparison in the object ID comparison, when the pieces of data relating to the object ID information do not coincide with each other, carrying out the image-pickup device and image data selection and the object detection.

10. A computer program stored in a non-transitory computer readable medium for detecting an object, for detecting ID information and a position of an object by using a computer, by using object ID data detected by at least one wireless tag reader for detecting data relating to the object ID information of an object to which a wireless tag is given, human detection data detected by at least one human detection sensor for detecting whether a human is present in a detection range of the wireless tag reader, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space where the object is searched, which are accumulated in association with obtained times of the respective pieces of object ID data, the program comprising:

an object ID comparing means for comparing first object ID data corresponding to a first time the human detection data is obtained and relating to first object ID information with second object ID data corresponding to a first time the human detection data is obtained and relating to second object ID data;

an image data selecting means for selecting first image data and second image data respectively corresponding to the first time and the second time when the first object ID information and second object ID information do not coincide with each other as a result of the comparison by the object ID comparing means; and

an object detection means for calculating a difference between the first image data and the second image data selected by the image data selecting means, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object.

11. A computer program stored in a non-transitory computer readable medium for detecting an object, for detecting ID information and a position of an object by using a computer, by using data relating to object ID information detected by at least one wireless tag reader for detecting object ID data relating to object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of object ID data, the program causing the computer to execute:

an object ID comparing means for comparing first object ID data corresponding to a first time the human position data is obtained and relating to first object ID information with second object ID data corresponding to a second time the human relation data is obtained and relating to second object ID information, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

an image data selecting means for selecting first image data corresponding to the first time and second image data corresponding to the second time when the pieces of data relating to the object ID information do not coincide with each other as a result of the comparison by the object ID comparing means; and

an object detection means for calculating a difference between the first image data and the second image data selected by the image data selecting means, detecting a position of an area of the difference between the first image data and the second image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object, and when calculating the difference, calculating a difference for an area determined based on the human position data from the first time to the second time.

12. A computer program stored in a non-transitory computer readable medium for detecting an object, for detecting ID information and a position of an object by using a computer, by using object ID data relating to object ID information detected by at least one wireless tag reader for detecting data relating to object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least two image-pickup devices each for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of object ID data, wherein when the program causes the computer to execute:

an object ID comparing means for comparing first object ID data corresponding to a first time the human position data is obtained and relating to first object ID information with second object ID data corresponding to a second time the human position data is obtained and relating to second object ID information, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

an image-pickup device and image data selecting means for specifying an image-pickup device having an area determined based on the human position data from the first time to the second time as an image-pickup field of view, or an image-pickup device having a viewpoint from which the area is not hidden by other object including a human in each of first image data and second image data respectively corresponding to the first time and the second time, among the at least two image-pickup devices, and selecting the first image data and the second image data respectively corresponding to the first time and the second time picked up by the specified image-pickup device; and

an object detection means for calculating a difference between the first image data and the second image data selected by the image-pickup device and image data selecting means, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object,

the program causes the computer to execute

the image-pickup device and image data selecting means and the object detection means when the first object ID data does not coincide with the second object ID data as a result of the comparison by the object ID comparing means.