POINT OF RECORDATION TERMINAL APPARATUS FOR SENDING HIGH COMPLEXITY IMAGE SEQUENCES OVER AN UNRELIABLE LOW-BANDWIDTH CHANNEL

• US 20090219392A1
• Filed: 02/27/2009
• Published: 09/03/2009
• Est. Priority Date: 02/27/2009
• Status: Active Grant

• Current Assignee: Third Iron LLC, VIAAS, Inc.
• Original Assignee: Third Iron LLC
• Inventors: ROSKOWSKI, STEVEN GODDARD
• Granted Patent: US 8,558,888 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19665 Details related to the storage of video surveillance data television signal recording H04N5/76
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04L 1/1685 the supervisory signal being transmitted in response to a specific request, e.g. to a polling signal
• CPC Class Codes: H04L 2001/125 Arrangements for preventing errors in the return channel
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system for providing video intelligence as a service to a plurality of small and medium sized enterprises to distributed video recordation equipment, comprising an apparatus and computer-implemented method. An apparatus comprises a point of recordation terminal apparatus coupled to a public network. A method for operating a system for video intelligence in a public network in a secure manner.

Claims

1. An apparatus comprising

a point of recordation terminal comprising a network interface, the network interface coupled to a network,

wherein said point of recordation terminal further comprises an asset &

event capture circuit,

a reference selection and meta-tagger circuit, and

an archive store,

the asset &

event capture circuit coupled to the reference selection and meta-tagger circuit, the network interface coupled to the reference selection &

meta tagger circuit and coupled to the archive store.

2. The apparatus of claim 1 wherein a network is a private network.

3. The apparatus of claim 1 wherein a network is a cellular network.

4. The apparatus of claim 1 wherein a network is a public network and the point of recordation terminal further comprises an encrypt circuit

wherein the encrypt circuit couples the reference selection and meta-tagger circuit and the archive store to the network interface of the point of recordation terminal, whereby references are transported through the public network and stored in encrypted form and only decrypted upon request of a user.

5. A method for operating the apparatus of claim 1 comprising the processes of:

analyzing video imagery for motion matching certain behaviors,

recording imagery around an event consisting of minimal reference information and larger richer assets, storing richer assets locally if it is determined that it is inefficient to attempt transmission immediately and trickling assets when bandwidth is less constrained.

6. A system for providing security as a service to a plurality of locations by network distributed video surveillance equipment,

comprising an apparatus and computer-implemented method for operating the apparatus, wherein said apparatus comprises

a point of recordation terminal (PORT) apparatus, the PORT apparatus coupled to a local area network and having an archive store;

and at least one circuit for encrypting data; and

wherein said computer-implemented method for operating the apparatus comprises the following processes;

encrypting a reference and an asset with a private key prior to transmission via the local area and public networks;

wherein an asset comprises a plurality of video frames and a reference comprises a selected video frame selected from an asset, time, date, and identity of the PORT apparatus on which it was recorded.

7. A method of operating an apparatus for providing video surveillance as a network hosted service to a plurality of small and medium enterprises, the apparatus comprising

at least one point of recordation terminal (PORT) apparatus,

each PORT apparatus coupled to a network,

the method comprising the following processes:

capturing a plurality of video frames,

storing a plurality of video frames in an archive store;

determining an event occurrence;

selecting at least one video frame, time, date, and identification as at least one reference to a video sequence whereby each reference is indicative of where the video frame occurs in the video sequence; and

transferring the reference through a network.

8. A point of recordation terminal (PORT) apparatus for to capture, select, and transfer video frames and related data on a network without clogging local network traffic,

the PORT apparatus comprising

a processor;

a digital camera;

a video encoding &

compression circuit;

an image encoding &

compression circuit;

an event determination circuit;

a video buffer;

a reference selection circuit;

an archive store;

a private key encryption circuit;

a bandwidth controller circuit and

a network interface.

9. An apparatus for documenting at least one occurrence of an event of interest comprising a digital camera coupled to a network interface, the camera, and the network interface coupled to the following:

a means for determining when an event of interest occurs, and

a means for selecting an extent of data associated with the event of interest to accurately represent the event;

a means for efficiently recording the selected extent of data;

a means for storing the recorded events; and

a means for deriving more compact representations of the event which can assist in determining if the event is of further interest.

10. The apparatus of claim 9 wherein the means for determining when an event of interest occurs comprises a processor controlled by software to execute the following computer-implemented steps:

to determine if multiple events of interest occur in close proximity, to cause a single extent of data to be recorded indicative of multiple event representations, each of provide indication of where in the extent the event occurred

11. The apparatus of claim 9 further comprising a policy store, a connection manager, a formatting circuit, and an upload manager wherein the policy store is coupled to the connection manager, the formatting circuit, the upload manager, the bandwidth controller, and the asset and event capture circuit, wherein the formatting circuit is further coupled to the video encoding circuit, to the archive store, to the bandwidth controller, wherein the upload manager circuit is further coupled to the archive store, to the bandwidth controller, and wherein the connection manager is further coupled to the bandwidth controller and to the network interface whereby policies which determine actions upon certain conditions are received from the network by the connection manager and stored to the policy store whereby the upload manager circuit determines which and how quickly assets are transmitted via the bandwidth controller and the network interface, whereby the formatting circuit determines how to convert raw video to streamable video and how to determine the preroll and post roll parameters, whereby the connection manager changes the operating mode upon certain conditions specified in a policy stored in the policy store.

12. The apparatus of claim 9 wherein the means for storing the recorded events comprises a circuit comprising a processor controlled by software to execute the following computer-implemented steps:

to store them locally on the device and to transmit them slowly based on a policy which indicates an acceptable amount of bandwidth to utilize.

13. An apparatus for transmitting compact representations of an event of interest over an unreliable network comprising:

a means for connection comprising at least one of an IP network, a cellular network, or an IP network over cellular network;

the means for connection coupled to a first network interface circuit and to a second network interface circuit, the first network interface circuit coupled to a means for transmission of compact representations,

wherein the means for transmission of compact representations comprises a circuit comprising a processor controlled by software to execute the following computer-implemented steps;

to determine if the representation cannot be immediately transmitted,

to store the representations locally and to retry transmission at a later time; and

a means for reception coupled to the second network interface,

the means for reception comprising a circuit comprising a processor controlled by software to execute the following computer-implemented steps;

receive an https request, open an https server session, connect between an https client and an https server.

14. A point of recordation terminal comprising a circuit comprising a processor controlled by software to execute at least one of the following computer-implemented steps:

to change configuration of other circuits in the terminal,

to transmit immediately when directed by means for analysis,

to store events if immediate transmission fails, and

to specify all data should be recorded and transmitted immediately for a limited period

15. A method for operating an apparatus for to reliably maintain high complexity continuous data over a low bandwidth and unreliable connection, the apparatus comprising:

a point of recordation terminal (PORT) coupled to a connection,

the method comprising capturing, and transmitting, an event of interest,

wherein capturing an event of interest comprises the following processes;

determining when an event of interest occurs,

selecting an extent of data associated with the event of interest,

efficiently recording the selected extent of data,

deriving a compact representation of the event of interest, and

storing the recorded events;

wherein transmitting an event of interest comprises the following processes;

transmitting immediately when directed and storing if immediate transmission fails,

opening an https client session,

opening an https server session,

connecting between an https client and server,

transmitting with link level encryption,

storing recorded events of interest locally and transmitting when an acceptable amount of bandwidth becomes available.

16. A point of recordation terminal comprising:

a high resolution digital camera,

a first storage device,

a first network interface,

a circuit controlled by software to signal a motion threshold,

a circuit controlled by software to measure motion activity,

a circuit controlled by software to track motion objects with analysis artifacts,

a circuit controlled by software to encode h(dot)264 format files,

a circuit controlled by software to encode JPEG files,

a circuit to write to non-volatile removable memories,

a circuit to generate thumbs, timestamps, and motion meta-data,

a circuit controlled by software to determine if multiple events of interest occur in close proximity,

a circuit controlled by software to cause a single extent of data to be recorded indicative of multiple event representations and where they occur,

a circuit controlled by software to store assets locally on the first storage device and to retry transmission at a later time wherein

a circuit comprises a processor controlled by software instructions and the processor is coupled to the first network interface, the processor is coupled to the first storage device, and the processor is coupled to the high resolution digital camera.

17. An apparatus for generating and storing an asset comprising

means for reading and encoding a PORT identification,

means for generating a PORT unique identification for the asset,

means for generating multiple representations of the event, and

means for indicating the timing relationship between multiple representations,

18. The apparatus of claim 17 wherein the asset is an encrypted digital file.

19. The apparatus of claim 17 further comprising means for means for determining and encoding a type of event.

20. The apparatus of claim 17 further comprising means for computing and storing a digital signature.

21. The apparatus of claim 17 further comprising means for determining and storing a preroll before the start of the event.

22. The apparatus of claim 17 further comprising means for determining and storing a postroll after the end of the event.

23. The apparatus of claim 17 further comprising means for deriving a metadata value for the number of moving objects.

24. An apparatus for generating, and transmitting a reference comprising

means for reading and encoding a camera identification,

means for reading and encoding a time of day of an asset,

means for determining and storing a type of event, and

means for reading and encoding an offset of the event relative to the time of day of the asset.

25. The apparatus of claim 24 wherein a reference is an encrypted digital file.

26. The apparatus of claim 24 further comprising means for determining and storing a digital signature.

27. The apparatus of claim 24 further comprising means for deriving a low resolution, scaled still image.

28. The apparatus of claim 24 further comprising means for reading and storing a size of the asset.

29. The apparatus of claim 24 further comprising means for deriving a meta-data value for speed of moving object.

30. The apparatus of claim 24 wherein said means comprises a circuit comprising a processor coupled to computer-readable media encoded with instructions to compute meta-data values, to determine the size of an asset, to determine an event, to select a high resolution digital photograph from an image sequence, to convert an image sequence into a medium resolution video image sequence, to derive a compressed, scaled, low resolution representation from a selected high resolution digital photograph, to read camera identification and to compute a digital signature, wherein a reference comprises a plurality of digital files encoded by strong encryption.

31. A method for operating an apparatus for to reliably maintain high complexity continuous data over a low bandwidth and unreliable connection, the apparatus comprising:

a point of recordation terminal (PORT) coupled to a connection,

the method comprising capturing an event of interest,

wherein capturing an event of interest comprises the following processes

determining a start of an event of interest;

recording data associated with the event of interest;

deriving a compact representation of the event of interest;

recording the compact representation of the event of interest;

determining an end of the event of interest; and

stopping recording data.

32. The method of claim 31 wherein recording data comprises the step of encrypting the data and writing the data to non-volatile memory.

33. The method of claim 31 wherein the connection comprises an IP network.

34. The method of claim 31 wherein the connection comprises a cellular network.

35. The method of claim 31 wherein the connection comprises an IP over cellular network.

36. The PORT of claim 31 comprising

an asset &

event capture circuit,

a reference select &

meta-tagger circuit,

a formatting circuit,

an archive store,

a network interface,

a bandwidth control circuit,

an upload management circuit,

an encryption circuit, and

a connection management circuit;

the reference select &

meta-tagger circuit coupled to the asset and event capture circuit and coupled to the formatting circuit,

the archive store coupled to the formatting circuit and to the upload management circuit,

the encrypt circuit coupled to the formatting circuit and to the archive store,

the network interface coupled to the connection management circuit and to the bandwidth control circuit.

37. The PORT of claim 36 further comprising a policy store,

the policy store coupled to the connection management circuit,

the policy store coupled to the bandwidth control circuit,

the policy store coupled to the asset &

event capture circuit,

the policy store coupled to the reference select &

meta-tagger circuit, and

the policy store coupled to the upload management circuit.

38. The method of claim 31 wherein capturing an event of interest further comprises the step of reading a policy from a policy store, determining the triggers which cause certain actions and if triggered:

setting thresholds for event determination,

selecting data for association with an event of interest to be recorded,

setting parameters for deriving a compact representation, and

setting resolutions for deriving compact representation.

39. The method of claim 31 further comprising the process of transmitting an event of interest wherein transmitting an event of interest comprises the following processes:

opening a client session to a server,

transmitting the recorded compact representation of the event immediately, and

storing recorded events of interest locally for later transmission.

40. The method of claim 39 wherein transmitting the recorded compact representation comprises the steps following:

storing the compact representation in the archive store of the PORT for later transmission if immediate transmission fails and detecting that immediate transmission has failed.

41. The method of claim 40 wherein transmitting further comprises the steps of reading a policy from a policy store, determining the triggers which cause certain actions and if triggered:

setting a rate for immediate transmission,

setting a rate for later transmission,

initiating a upload of a specific recorded event, and

overwriting stored recorded events with new recorded events or a discarding new recorded event.

42. The method of claim 41 further comprising the following steps:

transmitting accumulated compact representations from the archive store of the PORT and the step of detecting when connectivity has been restored.

43. The method of claim 42 wherein transmitting accumulated compact representations comprises transmitting accumulated recorded compact representations at a limited rate determined by reading a policy store.

44. The method of claim 39 wherein later transmission comprises responding immediately to a request from a server to transmit a specific recorded event of interest.

45. An apparatus comprising a digital camera coupled to a formatting circuit coupled to an encryption circuit coupled to an archive store, wherein the encryption circuit comprises an input for reading a unique camera identification key, an input for reading a video stream from the formatting circuit, a processor for encoding the video stream with time, date, and the unique camera identification key, and an output for writing the resultant encoded video stream to the archive store.

46. An apparatus comprising a digital camera coupled to a reference select &

meta-tagger circuit coupled to a formatting circuit coupled to a connection manager circuit coupled to a network interface, wherein the connection manager circuit comprises a processor controlled by software to perform the following operations;

reading a destination IP address hardcoded onto the connection manager circuit board, receiving a compact representation of an event of interest from the reference select &

meta-tagger circuit, preparing packets with the destination IP address containing the compact representation, opening a client session with the destination IP address, and transmitting the packet as a client to a server at the destination IP address.

47. A point of recordation terminal apparatus comprising:

a network interface, the network interface coupled to a network;

an asset &

event capture circuit;

a reference selection &

meta-tagger circuit;

a bandwidth controller circuit; and

an encryption circuit, whereby captured assets and references are encrypted prior to transmission.

48. The apparatus of claim 47 further comprising an archive store coupled to the encryption circuit whereby captured assets and references are stored in encrypted form into the archive store.

49. The apparatus of claim 47 wherein the encryption circuit is uniquely associated with the specific PORT by cryptographic operation.

50. The apparatus of claim 49 wherein the encryption circuit indicates the time and date of the event of interest by cryptographic operation on the assets.

51. A method of operating the apparatus of claim 50 comprising transmitting a reference immediately while storing an asset into the archive store.

52. The method of claim 51 further comprising temporarily storing the transmitted reference and storing it to the archive store in case the transmission fails.

53. The method of claim 52 wherein storing comprises the steps of detecting when the transmission is likely to be possible again and retransmitting the reference.

54. A point of recordation terminal apparatus comprising:

a network interface, the network interface coupled to a network;

an asset &

event capture circuit;

a reference selection &

meta-tagger circuit;

wherein the network interface comprises a configuration detection circuit whereby it automatically detects and configures its network interface settings.

55. A method of operating the configuration detection circuit of claim 54 comprising sequentially trying the following processes until a working configuration is established:

DHCP, static configuration and auto-detection.

56. The method of claim 55 wherein auto-detection comprises determining the local addressing scheme;

selecting a host address not detected in the local network, probing the selected address to determine if used, and reselecting if collision is detected; and

sending a prospective transaction on at least one port to each identified hosts on the local network to determine if any act as a gateway, and selecting a host as a gateway if successful.

57. The method of claim 56 wherein determining the local addressing scheme comprises passively listening to network traffic to determine the local addressing scheme and hosts on the networks.

58. The method of claim 56 wherein determining the local addressing scheme comprises actively probing the network to determine the local addressing and hosts on the local network.

59. A point of recordation terminal apparatus comprising

a network interface, the network interface coupled to a network;

an asset &

event capture circuit;

a reference selection &

meta-tagger circuit; and

a connection manager circuit, whereby the connection manager and the network interface establish client sessions to a server at a known location.

60. A method for operating the apparatus of claim 59 comprising:

establishing client sessions to a server at a known location.

61. The method of claim 60 further comprising:

receiving commands issued by a server responding to a client.

62. The method of claim 60 further comprising:

periodically reestablishing its client connection to a server.

63. The method of claim 60 further comprising:

processing a command to quickly reestablish a client connection.

64. The method of claim 60 further comprising:

providing status indication for commands currently running in a client connection and for commands recently completed in a client connection.

65. The method of claim 60 connecting through an HTTP protocol client session.

66. The method of claim 60 connecting through an HTTPS protocol client session