Object detecting device, object detecting method and object detecting computer program

• US 20090066513A1
• Filed: 12/18/2006
• Published: 03/12/2009
• Est. Priority Date: 12/28/2005
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Kondo, Kenji, Naruoka, Tomonobu
• Granted Patent: US 7,852,217 B2
• Time in Patent Office: Days
• US Class Current: 340/572.100
• CPC Class Codes: G01S 13/04 Systems determining presence of a target based on relative movement of target G01S13/56
• CPC Class Codes: G01S 13/78 discriminating between different kinds of targets, e.g. IFF-radar, i.e. identification of friend or foe G01S13/75, G01S13/79 take precedence
• CPC Class Codes: G01V 15/00 Tags attached to, or associated with, an object, in order to enable detection of the object record carriers for use with machines having a detectable tag or marker G06K19/00

Abstract

When object detection means, for detecting ID information and a position of an object from outputs of a wireless tag reader, a human detection sensor, and a camera, determines that data relating to first object ID information and data relating to second object ID information, corresponding respectively to first time and second time on which human detection data indicating a presence of a human is obtained, are different from each other, the object detection means calculates a difference between first image data and second image data corresponding to the respective times to thereby detect the object position.

Claims

1. An object detecting device, comprising:

at least one wireless tag reader for detecting data relating to object ID information of an object to which a wireless tag is given in a space where the object is searched;

at least one human position sensor for measuring a position of a human handling the object in the space;

at least one image-pickup device for picking up an image of a part or a whole of the space;

a storing means for storing the data relating to the object ID information detected by the wireless tag reader, human position data measured by the human position sensor, and image data picked up by the image-pickup device, while associating each piece of the data with obtained time thereof; and

an object detection means for detecting ID information and a position of the object based on the data relating to the object ID information, the human position data, and the image data, stored in the storing means, wherein

when the object detection means determines that data relating to first object ID information and data relating to second object ID information corresponding to first time and second time on which the human position data is obtained respectively, among the data relating to object ID information in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader, are different from each other, the object detection means calculates a difference between first image data and second image data corresponding to the first time and the second time respectively, and in the calculation, the object detection means calculates a difference for an area determined based on the human position data from the first time to the second time, and then, detects a position of the difference area in image data or a position obtained by converting the position to a position in the space where the object is searched, as a position of the object.

6. The object detecting device according to claim 1, wherein when calculating the difference between the first image data and the second image data, the object detection means specifies object ID information having a difference in the data relating to the first object ID information and the data relating to the second object ID information, and calculates the difference by using at least one piece of information among color, size, and shape information of the object corresponded to the object ID information having the difference.

2. An object detecting device, comprising:

at least one wireless tag reader for detecting data relating to object ID information of an object to which a wireless tag is given in a space where the object is searched;

at least one human position sensor for measuring a position of a human handling the object in the space;

at least two image-pickup devices each for picking up an image of a part or a whole of the space;

a storing means for storing the data relating to the object ID information detected by the wireless tag reader, human position data measured by the human position sensor, and image data picked up by the image-pickup device, while associating each piece of the data with obtained time thereof; and

an object detection means for detecting ID information and a position of the object based on the data relating to the object ID information, the human position data, and the image data, stored in the storing means, wherein

when the object detection means determines that data relating to first object ID information and data relating to second object ID information corresponding to first time and second time on which the human position data is obtained respectively, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader, are different from each other, the object detection means specifies the image-pickup device having an area determined based on the human position data from the first time to the second time as an image-picking up field of view, or the image-pickup device having a viewpoint from which the area is not hidden by other object including a human in each of first image data and second image data corresponding to the first time and the second time respectively, from the at least two image-pickup devices, and the object detection means calculates a difference between the first image data and the second image data corresponding to the first time and the second time respectively, picked up by the specified image-pickup device, and detects a position of an area of the difference in the image data or a position obtained by converting the position to a position in the space where the object is searched, as a position of the object.

3. An object detecting device, comprising:

at least one wireless tag reader for detecting data relating to object ID information of an object to which a wireless tag is given in a space where an object is searched;

at least one human detection sensor for detecting whether a human is present in a detection range of the wireless tag reader in the space;

at least one image-pickup device for picking up an image of a part or a whole of the space;

a storing means for storing the data relating to the object ID information detected by the wireless tag reader, human detection data detected by the human detection sensor, and image data picked up by the image-pickup device, while associating each piece of the data with obtained time thereof; and

an object detection means for detecting ID information and a position of the object based on the data relating to the object ID information, the human detection data and the image data, stored in the storing means, wherein

when the object detection means determines that data relating to first object ID data and data relating to second object ID information, respectively corresponding to first time and second time on which the human detection data indicating the human is present is obtained, are different from each other, the object detection means calculates a difference between first image data and second image data respectively corresponding to the first time and the second time, and detects a position of a difference area in the image data or a position obtained by converting the position of the difference area to a position in the space where the object is searched, as a position of the object.

4. The object detecting device according to claim 3, wherein the image-pickup device is a camera, and when calculating the difference between the first image data and the second image data, the object detection means excludes an area obtained by converting the detection range of the human detection sensor to a camera coordinate system from a difference subject area for calculating the difference between the first image data and the second image data, and calculates the difference for an area from which the area has been excluded.

5. The object detecting device according to claim 3, wherein the object detection means includes:

an image data selecting unit for selecting first image data and third image data of which obtained times are different from each other, based on the first time, and selecting second image data and fourth image data of which obtained times are different from each other, based on the second time; and

an object detection unit for determining a first mask area by calculating a difference between the first image data and the third image data selected by the image data selecting unit, and determining a second mask area by calculating a difference between the second image data and the fourth image data selected by the image data selecting unit, and in calculating a difference between the first image data and the second image data, calculating the difference between the first image data and the second image data after excluding the first mask area and the second mask area from a difference subject area.

7. An object detecting method of detecting ID information and a position of an object by using data relating to object ID information detected by at least one wireless tag reader for detecting the data relating to the object ID information of an object to which a wireless tag is given, human detection data detected by at least one human detection sensor for detecting whether a human is present in a detection range of the wireless tag reader, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space where the object is searched, which are accumulated in association with obtained times of the respective pieces of data, the method comprising:

comparing data relating to first object ID information with data relating to second object ID information, respectively corresponding to first time and second time on which the human detection data is obtained;

selecting first image data and second image data respectively corresponding to the first time and the second time;

calculating a difference between the first image data and the second image data selected in the image data selection, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in the space where the object is searched, as a position of the object; and

as a result of the comparison in the object ID data comparison, when the pieces of data relating to the object ID information do not coincide with each other, carrying out the image data selection and the object detection.

8. An object detecting method of detecting ID information and a position of an object by using data relating to object ID information detected by at least one wireless tag reader for detecting the data relating to the object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of data, the method comprising:

comparing data relating to first object ID information with data relating to second object ID information, respectively corresponding to first time and second time on which the human position data is obtained, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

selecting first image data and second image data respectively corresponding to the first time and the second time;

calculating a difference between the first image data and the second image data selected in the image data selection, detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched as a position of the object, and in calculating the difference, calculating a difference for an area determined based on the human position data from the first time to the second time; and

as a result of the comparison in the object ID comparison, when the pieces of data relating to the object ID information do not coincide with each other, carrying out the image data selection and the object detection.

9. An object detecting method of detecting ID information and a position of an object by using data relating to object ID information detected by at least one wireless tag reader for detecting the data relating to the object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least two image-pickup devices each for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of data, the method comprising:

comparing data relating to first object ID information with data relating to second object ID information, respectively corresponding to first time and second time on which the human position data is obtained, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

specifying an image-pickup device having an area determined based on the human position data from the first time to the second time as an image-picking up field of view, or an image-pickup device having a viewpoint from which the area is not hidden by other object including a human in each of first image data and second image data corresponding to the first time and the second time respectively, from the at least two image-pickup devices, and selecting the first image data and the second image data corresponding to the first time and the second time respectively, picked up by the specified image-pickup device;

calculating a difference between the first image data and the second image data selected in the image-pickup device and image data selection, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object; and

as a result of the comparison in the object ID comparison, when the pieces of data relating to the object ID information do not coincide with each other, carrying out the image-pickup device and image data selection and the object detection.

10. A computer program for detecting an object, for detecting ID information and a position of an object by using a computer, by using object ID data detected by at least one wireless tag reader for detecting data relating to the object ID information of an object to which a wireless tag is given, human detection data detected by at least one human detection sensor for detecting whether a human is present in a detection range of the wireless tag reader, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space where the object is searched, which are accumulated in association with obtained times of the respective pieces of data, the program comprising:

an object ID comparing means for comparing data relating to first object ID information with data of second object ID information respectively corresponding to first time and second time on which the human detection data is obtained;

an image data selecting means for selecting first image data and second image data respectively corresponding to the first time and the second time when the pieces of data relating to the object ID information do not coincide with each other as a result of the comparison by the object ID comparing means; and

an object detection means for calculating a difference between the first image data and the second image data selected by the image data selecting means, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object.

11. A computer program for detecting an object, for detecting ID information and a position of an object by using a computer, by using data relating to object ID information detected by at least one wireless tag reader for detecting data relating to object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least one image-pickup device for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of data, the program causing the computer to execute:

an object ID comparing means for comparing data relating to first object ID information with data relating to second object ID information respectively corresponding to first time and second time on which the human position data is obtained, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

an image data selecting means for selecting first image data and second image data respectively corresponding to the first time and the second time when the pieces of data relating to the object ID information do not coincide with each other as a result of the comparison by the object ID comparing means; and

an object detection means for calculating a difference between the first image data and the second image data selected by the image data selecting means, detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object, and when calculating the difference, calculating a difference for an area determined based on the human position data from the first time to the second time.

12. A computer program for detecting an object, for detecting ID information and a position of an object by using a computer, by using data relating to object ID information detected by at least one wireless tag reader for detecting data relating to object ID information of an object to which a wireless tag is given, human position data measured by at least one human position sensor for measuring a position of a human handling the object, and image data picked up by at least two image-pickup devices each for picking up an image of a part or a whole of a space, which are accumulated in association with obtained times of the respective pieces of data, wherein when the program causes the computer to execute:

an object ID comparing means for comparing data relating to first object ID information with data relating to second object ID information respectively corresponding to first time and second time on which the human position data is obtained, in which a human position measured by the human position sensor is included in a detection range of the wireless tag reader;

an image-pickup device and image data selecting means for specifying an image-pickup device having an area determined based on the human position data from the first time to the second time as an image-pickup field of view, or an image-pickup device having a viewpoint from which the area is not hidden by other object including a human in each of first image data and second image data respectively corresponding to the first time and the second time, among the at least two image-pickup devices, and selecting the first image data and the second image data respectively corresponding to the first time and the second time picked up by the specified image-pickup device; and

an object detection means for calculating a difference between the first image data and the second image data selected by the image-pickup device and image data selecting means, and detecting a position of an area of the difference in image data or a position obtained by converting the position to a position in a space where the object is searched, as a position of the object,

the program causes the computer to execute

the image-pickup device and image data selecting means and the object detection means when the pieces of data relating to the object ID information do not coincide with each other as a result of the comparison by the object ID comparing means.