SUBJECT SENSING IN AN ENVIRONMENT

• US 20140226855A1
• Filed: 07/02/2012
• Published: 08/14/2014
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: Yale University
• Original Assignee: Yale University
• Inventors: Savvides, Andreas, Teixeira, Thiago
• Granted Patent: US 9,922,256 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/21 for social networking applications

Abstract

A multimodal sensor network is designed to extract a plurality of fundamental properties associated with subject sensing. In one aspect, such network can combine cameras distributed in an environment with inertial sensors available in subjects'"'"' wearable devices. The network can permit anonymous detection, counting, and localization of one or more subjects utilizing the cameras. In one aspect, by fusing such information with positional data from the inertial sensors contained or coupled to wearable devices associated with the one or more subjects, the network can track and can identify each subject carrying a wearable device functionally coupled to inertial sensor(s). In one aspect, the problem of subject sensing can be divided into two parts: (1) a detection, counting, and localization (DCL) layer and (2) a tracking and identification (TI) layer, wherein such layers can be implemented via simulations and a real sensor network deployment.

Claims

1. A method for subject sensing, comprising:

generating a first layer of information associated with sensing of a subject, the first layer of information comprising presence information, counting information, and location information for a plurality of subjects in an environment, wherein the subject is contained in the plurality of subjects;

generating a second layer of information associated with sensing of the subject, the second layer of information comprising one or more of tracking information or identification information for at least one of the plurality of subjects; and

providing location information and identity information for the subject based at least on matching a portion of the first layer of information with a portion of the second layer of information.

2. The method of claim 1, wherein generating the first layer of information comprises producing a Boolean image having a value indicative of the likelihood the subject is detected.

3. The method of claim 2, wherein generating the first layer of information further comprises classifying a plurality of pixels in the Boolean image into a set of categories having a cardinality indicative of a number of subjects in the plurality of subjects.

4. The method of claim 3, wherein generating the first layer of information further comprises computing a centroid for each category of the set of categories, each centroid is indicative of a location of each subject in the plurality of subjects.

5. The method of claim 1, wherein generating the second layer of information comprises generating a 2-tuple indicate of location of the subject and an inertial measurement of the location of subject, the 2-tuple maximizes the likelihood that a portion of the location information in the first layer of information corresponds to a portion of the tracking information in the second layer of information.

6. A method for subject sensing, comprising:

detecting a plurality of subjects moving in an environment by a first sensor;

extracting first location information of a subject of the plurality of subjects based at least on the detecting action;

receiving second location information of the subject from a second sensor;

matching second location information of the subject with the first location information, thereby tracking the subject; and

, in response to the matching,

assigning a subject location to the subject based on the first location information, and

assigning a subject identity (ID) to the subject based on identifying information associated with the second sensor.

7. The method of claim 6, further comprising supplying information indicative of a location-to-ID match comprising the subject location and the subject ID.

8. The method of claim 6, wherein extracting the first location information comprises extracting anonymous coordinates of the person.

9. The method of claim 6, wherein matching the second location information of the subject with the first location information is performed in real-time.

10. The method of claim 6, wherein the first sensor is a camera and the second sensor is a wearable sensor, wherein the camera is one of a conventional camera or a feature-detecting camera.

11. The method of claim 6, wherein the first sensor is a camera and the second sensor is an inertial sensor.

12. The method of claim 6, wherein the second sensor is an radiofrequency identification (RFID) device.

13. A method for subject sensing, comprising:

receiving first location information of a subject of a plurality of subjects, the first location information is generated by first sensor;

generating second location information of the subject;

matching second location information of the subject with the first location information, thereby tracking the subject; and

, in response to the matching,

assigning a location to the subject based on the first location information, and

assigning an identity to the subject based on identifying information associated with a second sensor.

14. The method of claim 13, wherein the first sensor is a camera and the second sensor is a wearable sensor, wherein the camera is one of a conventional camera or a feature-detecting camera.

15. The method of claim 13, wherein the first sensor is a camera and the second sensor is an inertial sensor.

16. The method of claim 13, wherein the second sensor is an radiofrequency identification (RFID) device.

17. A system for subject sensing, comprising:

a plurality of sensors; and

a computing device for information analysis, the computing device functionally coupled to the plurality of sensors and configured

to transmit first location information of a plurality of subjects to a wearable device, the first location information is generated by at least one of the plurality of sensors,

to receive, from the wearable device, second location information associated with a subject of the plurality of subjects,

to match the second location information with the first location information and, in response,

to assign a subject location to the subject based on the first location information, and

to assign a subject identity (ID) to the subject based on identifying information associated with the wearable device.

18. The system of claim 17, wherein the plurality of sensors comprises a plurality of cameras.

19. The system of claim 17, wherein the wearable device comprising at least one inertial sensor.

20. The system of claim 17, wherein the wearable device is configured to generate information indicative of a location-to-ID match comprising the subject location and the subject ID.

21. The system of claim 20, wherein the wearable device is configured to supply the information indicative of a location-to-ID match comprising the subject location and the subject ID.