Apparatus and method for privacy protection of data collection in pervasive environments

• US 20060064384A1
• Filed: 09/14/2005
• Published: 03/23/2006
• Est. Priority Date: 09/15/2004
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Mehrotra, Sharad, Venkatasubramanian, Nalini, Wickramasuriya, Jehan, Alhazzazi, Mohanned, Datt, Mahesh
• Granted Patent: US 7,940,302 B2
• Time in Patent Office: Days
• US Class Current: 705/57
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G07C 9/28 the pass enabling tracking or indicating presence
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/19686 Interfaces masking personal details for privacy, e.g. blurring faces, vehicle license plates
• CPC Class Codes: G08B 13/19695 Arrangements wherein non-video detectors start video recording or forwarding but do not generate an alarm themselves
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A security system which is nonintrusive of personal privacy of a person in a space comprises at least a first localization sensor subsystem, if any, in the possession of the person; a video surveillance subsystem arranged and configured to collect visual data related to the person in the space; and a computer subsystem coupled to the localization sensor subsystem and video surveillance subsystem to associate a predetermined privacy level with the localization sensor subsystem, and to provide an access control privilege with the localization sensor subsystem, the computer subsystem determining how to present, store and/or retrieve the visual data while meeting predetermined the privacy level associated with the person.

Claims

1. A security system which is nonintrusive of personal privacy of a person in a space comprising:

at least a first localization sensor subsystem, if any, in the possession of the person;

a video surveillance subsystem arranged and configured to collect visual data related to the person in the space; and

a computer subsystem coupled to the localization sensor subsystem and video surveillance subsystem to associate a predetermined privacy level with the localization sensor subsystem, and to provide an access control privilege with the localization sensor subsystem, the computer subsystem determining how to present, store and/or retrieve the visual data while meeting predetermined the privacy level associated with the person.

2. The security system of claim 1 where the computer subsystem monitors the person only when unauthorized access is made to the space.

3. The security system of claim 1 where the localization sensor subsystem comprises an RFID tag and RFID tag reader.

4. The security system of claim 1 where the localization sensor subsystem comprises a motion detector.

5. The security system of claim 1 where the localization sensor subsystem comprises an acoustic detector.

6. The security system of claim 1 where the computer subsystem masks the visual data according to the privacy level.

7. The security system of claim 6 where the computer subsystem masks the visual data by selectively providing a degree of information loss from the visual data according to the privacy level.

8. The security system of claim 1 where the computer subsystem provides access control privilege according to association within a space of two or more specifically identified persons.

9. The security system of claim 1 further comprising a second localization sensor subsystem corresponding to an object and where the computer subsystem provides access control privilege according to association within a space of a specifically identified person with the object.

10. The security system of claim 1 where the computer subsystem comprises a sensing module which receives information from the localization sensor subsystem.

11. The security system of claim 1 where the computer subsystem comprises a sensing module which receives video data from the video surveillance subsystem.

12. The security system of claim 1 where the computer subsystem comprises a data management module communicated with the sensing module to provide access control.

13. The security system of claim 12 where the data management module comprises a policy engine to generate access control decisions and a database subsystem communicated to the policy engine to provide parameters form which the policy engine generates access control decisions.

14. The security system of claim 13 where the policy engine comprises a policy decision point submodule and a policy enforcement point submodule communicated with each other through a request/response protocol, the policy decision point submodule receiving authorization requests from the localization sensor subsystem and generating authorization decisions to the video surveillance subsystem, the database subsystem comprising a profile access database and a policy database, the policy enforcement point submodule communicated to the profile access database to read attributes therefrom and communicated to the policy database to read policy rules therefrom.

15. The security system of claim 14 where the request/response protocol through which the policy decision point submodule and policy enforcement point submodule communicate is XACML.

16. The security system of claim 1 where the computer subsystem comprises an auxiliary services module communicated with the data management module for processing incoming video data.

17. The security system of claim 16 where the auxiliary services module comprises obfuscation, motion detection and object tracking submodules.

18. The security system of claim 1 where the computer subsystem comprises an output module communicated with the auxiliary services module for customized reporting, logging or video rendering.

19. The security system of claim 3 further comprising a field generator for receiving signals from the RFID tag and relaying the received signals to the RFID reader.

20. The security system of claim 19 where the localization sensor subsystem comprises a motion detector coupled with and triggered by the field generator to read a corresponding space for an RFID tag when motion is detected by the motion detector, and if unauthorized entry into the space is determined by the computer subsystem, the video surveillance subsystem is activated to generate a processed video record according a predetermined privacy level associated with the read RFID tag, if any.

21. The security system of claim 1 where video surveillance subsystem comprises a camera for providing a video pixel stream, a foreground module communicated to the camera for providing foreground pixel detection of the space, a background module for providing a background model communicated to the foreground module, a blob module communicated to the foreground module for clustering pixels into connected image domains, an object tracking module communicated to the blob module for localizing the person in a video frame correlated to localization parameters associated with the person moving in the space, a image processing module communicated to the object tracking module for modifying video data correlated to the object according to the selectively determined privacy level, and a video rendering module communicated to the image processing module for rendering a display of the modified video data correlated to the person.

22. The security system of claim 1 where the camera is triggered by motion detection of the object in the space as generated by the localization sensor subsystem.

23. The security system of claim 21 where the image processing module comprises a pass through channel, a noise/blur filter, a pixel-coloring filter, and a bounding-box imager, one of which is activated according to the privacy level.

24. The security system of claim 13 where policy engine processes a rule set comprised of a plurality 5-tuples of the form <

r, p, oa, ts, res>

, where r designates a space, p designates a privacy level, oa designates an object in an object association list, ts designates a temporal constraint corresponding to the validity of the rule, and res designates a result corresponding to ‘

allow’

or ‘

deny’

respectively.

35. The method of claim 21 where determining how to present, store and/or retrieve the visual data comprises performing according to the privacy level one of the steps of the group comprised of passing through the visual data, filtering the visual data with a noise/blur filter, filtering the visual data with a pixel-coloring filter, or modifying the visual data with a bounding-box.

25. A method of operating a security system which is nonintrusive of personal privacy of a person in a space comprising:

localizing the person to an identified space;

collecting visual data related to the person in the space;

associating a selected privacy level with the localized person;

providing an access control privilege with the localized person; and

determining how to present, store and/or retrieve the visual data while meeting predetermined a privacy level associated with the person.

26. The method of claim 25 where collecting visual data related to the person in the space is performed only when unauthorized access is made to the space.

27. The method of claim 25 where localizing the person to an identified space comprises localizing the person by means of an RFID tag and RFID tag reader.

28. The method of claim 25 where localizing the person to an identified space comprises localizing the person by means of a motion detector.

29. The method of claim 25 where localizing the person to an identified space comprises localizing the person by means of an acoustic detector.

30. The method of claim 25 where determining how to present, store and/or retrieve the visual data comprises masking the visual data according to the privacy level.

31. The method of claim 30 where masking the visual data comprises masking the visual data by selectively providing a degree of information loss from the visual data according to the privacy level.

32. The method of claim 25 where providing an access control privilege with the localized person comprises providing the access control privilege according to association within a space of two or more specifically identified persons.

33. The method of claim 25 further localizing an object and where providing an access control privilege with the localized person comprises providing the access control privilege according to association within a space of a specifically identified person with the object.

34. The method of claim 25 where determining how to present, store and/or retrieve the visual data comprises providing a video pixel stream, providing foreground pixel detection of the space, providing a background model, clustering pixels into connected image domains, localizing the person in a video frame correlated to localization parameters associated with the person moving in the space, modifying video data correlated to the object according to the selectively determined privacy level, and rendering a display of the modified video data correlated to the person.

36. The method of claim 25 where determining how to present, store and/or retrieve the visual data comprises processing a rule set comprised of a plurality 5-tuples of the form <

r, p, oa, ts, res>

, where r designates a space, p designates a privacy level, oa designates an object in an object association list, ts designates a temporal constraint corresponding to the validity of the rule, and res designates a result corresponding to ‘

allow’

or ‘

deny’

respectively.