Node management system and node managing program using sensing system

• US 7,710,455 B2
• Filed: 07/24/2006
• Issued: 05/04/2010
• Est. Priority Date: 10/31/2005
• Status: Expired due to Fees

• Current Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Original Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Inventors: Hashiguchi, Naoki, Onami, Yuichi, Aramaki, Koji, Nakabayashi, Sumie, Aritsuka, Toshiyuki
• Primary Examiner(s): Tran; Nhan T
• Assistant Examiner(s): DIEP, TRUNG T
• Application Number:
  US11/491,142
• Publication Number:
  US 20070101382A1
• Time in Patent Office: 1,380 Days
• Field of Search: 348/207.1, 348/211.1, 348/211.2, 348/211.3, 348/211.9, 348/143, 348/155, 348/211.4, 725/105
• US Class Current: 348/207.1
• CPC Class Codes: G08B 13/19695 Arrangements wherein non-video detectors start video recording or forwarding but do not generate an alarm themselves
• CPC Class Codes: G08B 21/22 responsive to presence or absence of persons
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices

Abstract

On the basis of state change information acquired by one of sensor nodes forming a sensor network, an instruction indicative of emitting a signal is issued to another node to acquire a photographed image from a camera. An image processing device identifies the ID and actual position of the node by issuing the signal issuance instruction to the node and detecting a signal obtained from the node from the photographed image. On the basis of personal information stored in a database, the image processing device detects a not-allowed action.

Claims

1. A node management system for managing sensor nodes and mobile nodes, in which each of said mobile nodes is associated with a respective predetermined light series pattern signal that identifies a respective holder of the mobile node, the node management system comprising:

a camera installed in a room in which said sensor nodes are distributed, the camera taking pictures of each of the respective holders of said mobile nodes upon the respective holder leaving or entering the room;

a sensor network base station communicating with said mobile nodes wirelessly;

a server including a node managing section and a sequence control section, said node managing section monitoring said sensor nodes and changes to a total number of mobile nodes in communication with said sensor network base station, the sequence control section controlling operation of both said node managing section and said imaging processing section according to a set of predetermined processing rules, each predetermined processing rule specifying a set of processing operations to take upon satisfaction of a corresponding set of conditions detected by said sensor nodes and recognized by said node managing section;

an image processing section processing each picture of each of the respective holders of said mobile nodes taken by said camera;

an application client displaying a passage map on a display device, the passage map including a corresponding position and a corresponding image of each of the respective holders who leave or enter the room, the corresponding image of each respective holder being extracted from the pictures of the respective holder taken by said camera upon the respective holder leaving or entering the room; and

a network interconnecting said sensor nodes, said mobile nodes, said server, said image processing section, and said application client,

wherein, upon recognition by said node managing section that information detected by one or more of said sensor nodes satisfies a first set of conditions corresponding to the respective holder of one of the mobile nodes leaving or entering the room, said sequence control section issues a first instruction to the image processing section to begin attempting to recognize the respective predetermined time series light pattern signal that identifies the respective holder leaving or entering the room in the pictures taken by said camera upon the respective holder leaving or entering the room and issues a second instruction to the sensor network base station to direct each of the mobile nodes in communication with the sensor network base station to begin emitting the respective predetermined time series light pattern signal that is associated with the mobile node, and, upon recognition of the respective predetermined time series light pattern signal that identifies the respective holder leaving or entering the room by said image processing section, said image processing section attempts to determine an identification (ID) of the mobile node that is associated with the respective predetermined time series light pattern signal that identifies the respective holder leaving or entering the room and, upon successfully determining the ID, outputs the ID and the image corresponding to the respective holder leaving or entering the room.

2. A node management system according to claim 1, wherein:

said image processing section is separate from said sensor nodes and said mobile nodes,

each of said mobile nodes is attached to a respective person,

said respective predetermined time series light pattern signal that is associated with each of said mobile nodes is unique to said mobile node or dynamically defined by said server; and

the first and second instructions are issued by said sequence control station at approximately simultaneously.

3. A node management system according to claim 2, wherein said sequence control section controls said camera to rotate an imaging direction of the camera according to the set of predetermined processing rules and conditions detected by said sensor nodes.

4. A node management system according to claim 3, wherein said camera is installed on a ceiling or on an upper wall in the room, and

the first set of conditions indicates detection of an opened door by a first sensor of the sensor nodes and the imaging direction of said camera is rotated by said sequence control section toward the door according to a first predetermined processing rule and the first set of conditions.

5. A node management system according to claim 4, further comprising an authority determiner for determining authority based on personal information including individual room entering authority, wherein said authority determiner determines whether or not a person associated with the ID has authority based on conditions detected by said sensor nodes, and upon determining that the person has no authority, outputs an alarm indicating a suspicious person onto a display screen.

6. A node management system according to claim 5, wherein said image processing section, upon failing to determine the ID, extracts a face image from said image corresponding to the respective holder leaving or entering the room, determines whether the respective holder is entering or leaving the room, and, upon determining that the respective holder is entering the room, outputs an indication of a presence of the respective holder onto the display screen.

7. A node management system according to claim 4, wherein said second instruction further directs said sensor network base station to allocate the respective predetermined time series light pattern signal associated with each mobile node in communication with the sensor network base station to said mobile node and said sensor node which belongs to only one base station corresponding to the door, and wherein said image processing section attempts to recognize the respective predetermined time series light pattern signal that identifies the respective holder leaving or entering the room by comparing a time series light pattern signal determined from the pictures taken by said camera upon the respective holder leaving or entering the room with said allocated respective time series light pattern signals.

8. A node management system according to claim 4, wherein:

at least one of said sensor nodes is connected to a chair in said room and transmits a detected fact of someone leaving the chart or sitting in the chair to said application client via said server;

each of said mobile nodes periodically attempts transmit a respective ID of the mobile node to the sensor network base station for transmitting to said server;

said server, upon receiving each respective ID, records an ID notification record that includes a current time, the received ID, and an identification of the sensor network base station in a historical list; and

said server periodically compares the detected fact with a previous state and transmits a detected change therebetween to said application client for updating said passage map.

9. A node management system according to claim 4, wherein said sensor network base station, upon receiving the second instruction, directs each of the mobile nodes in communication with the sensor network base station to begin emitting the respective predetermined time series light pattern signal that is associated with the mobile node except for any mobile nodes detected as having entered another room but not detected as having left the other room.

10. A computer-readable storage medium having program code embodied thereon, the program code executable by a processor to implement a method of node management, the method comprising:

acquiring images from an imaging device that is connected to a camera;

monitoring changes to a total number of mobile nodes in communication with a sensor network base station;

receiving sensed data detected by a first sensor node;

determining whether or not the received sensed data satisfies predetermined conditions;

searching for a processing sequence associated with the conditions if the sensed data satisfies the predetermined conditions;

executing the processing sequence if the sensed data satisfies the predetermined conditions, the processing sequence including;

issuing a first instruction to each mobile node in communication with said sensor network base station to begin emitting a respective predetermined time series light pattern signal that is associated with the mobile node and identifies a respective holder of the mobile node;

recognizing a first predetermined time series light pattern signal in the acquired images that is associated with a first mobile node in communication with the sensor network base station;

attempting to determine an identification (ID) of the first mobile node based on the first predetermined time series light pattern signal;

outputting the ID if the ID is determined and a first image of the images acquired from the imaging device corresponding to the respective holder of the first mobile node to use in outputting an alert on the basis of the first predetermined time series light pattern signal; and

displaying a passage map on a display device that shows a position and a face image of the respective holder of the first mobile device, the face image being extracted from the first image.

11. A node management program according to claim 10, wherein said processing sequence further includes issuing a second instruction to set an imaging direction of the camera based on the sensed data prior to acquisition of the first image from the imaging device.

12. A node management program according to claim 10, wherein said processing sequence further includes analyzing the first image from said imaging device and extracting the face image therefrom.

13. An entering/leaving management system comprising:

a camera installed in a room, the camera taking pictures of persons leaving or entering a room;

a plurality of name card type nodes each being carried by a respective person and each including a solar battery, a radio communication circuit and an LED emitting a predetermined on-off time-series light pattern signal that is uniquely associated with the name card type node and identifies the respective person carrying the name card type node, wherein each time interval during which the LED of each name card node is maintained in an on state or an off state according to the predetermined on-off time-series light pattern signal that is uniquely associated with the name card type node is not longer than a time interval between pictures taken by the camera;

at least one sensor node detecting information related to persons entering or leaving the room;

a sensor network base station communicating with said plurality of name card type nodes and the at least one sensor node wirelessly;

an image processing section processing each picture taken by said camera;

a server including a node managing section and a sequence control section, said node managing section monitoring said at least one sensor node and changes to a total number of name card type nodes in communication with said sensor network base station, the sequence control section controlling operation of both said node managing section and said imaging processing section according to a set of predetermined processing rules, each predetermined processing rule specifying a set of processing operations to take upon satisfaction of a corresponding set of conditions detected by said at least one sensor node and recognized by said node managing section;

an application client displaying a passage map on a display device, the passage map including a corresponding position and a corresponding image of each person who leaves or enters the room, the corresponding image of each person who leaves or enters the room being extracted from the pictures taken by said camera; and

a network interconnecting said at least one sensor node, said plurality of name card type nodes, said camera, said server, said image processing section and said application client;

wherein, upon recognition by said node managing section that information detected by said at least one sensor node satisfies a first set of conditions, said sequence control section issues a first instruction to the sensor network base station to direct each of the plurality of name card type nodes in communication with the sensor network base station to begin emitting the predetermined on-off time-series light pattern signal that is uniquely associated with the name card type node, and then said image processing section acquires the pictures taken by the camera, recognizes the predetermined on-off time-series light pattern signal that is uniquely associated with a first name card type node being emitted by the first name card type node in the pictures acquired from the camera, attempts to determine an identification (ID) of the first name card type node according to the predetermined on-off time-series light pattern signal that is uniquely associated with a first name card type node, and outputs the ID if the image processing section determines the ID and a position of the first name card type node.

14. An entering/leaving management system according to claim 13, wherein said image processing section, upon failing to determine the ID, extracts a face image in said pictures acquired from the camera, determines whether someone is entering or leaving the room, and, upon determining that someone is entering the room, outputs an indication of a presence of the respective holder onto a display screen.