Method and/or apparatus for analyzing the content of a surveillance image

• US 7,817,716 B2
• Filed: 05/29/2003
• Issued: 10/19/2010
• Est. Priority Date: 05/29/2003
• Status: Expired due to Fees

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: LSI Corporation (Broadcom, Inc.)
• Inventors: Wells, Aaron G.
• Primary Examiner(s): CZEKAJ, DAVID J
• Application Number:
  US10/447,739
• Publication Number:
  US 20040240546A1
• Time in Patent Office: 2,700 Days
• Field of Search: 37524001-24006, 37524012-24016, 375/240.26, 348143-169
• US Class Current: 375/240.01
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An apparatus comprising an input circuit, a content analyzer, a storage circuit and an output circuit. The input circuit may be configured to generate a first intermediate signal from a plurality of input video signals. The content analyzer circuit may be configured to present one or more flags in response to the intermediate signal. The storage circuit may be configured to (i) store and organize the first intermediate signal into a plurality of sequences each related to one of the input video signals and (ii) generate a second intermediate signal from the sequences. The output circuit may be configured to generate an output video signal in response to the second intermediate signal. The output circuit may be configured to embed tracking information into the output video signal in response to the one or more flags.

Claims

1. An apparatus comprising:

an input circuit configured to generate a first intermediate signal from a plurality of input video signals, wherein said first intermediate signal comprises an encoded digital video signal;

a content analyzer circuit configured to present one or more flags in response to said first intermediate signal;

a storage circuit configured to (i) store and organize said first intermediate signal into a plurality of sequences, each of said plurality of sequences corresponding to one of said input video signals and (ii) generate a second intermediate signal from said plurality of sequences; and

an output circuit configured to generate an output encoded bitstream in response to said second intermediate signal, wherein said output circuit is further configured to embed intra picture information into said output encoded bitstream in response to said one or more flags, wherein (i) said intra picture information allows said content analyzer circuit to generate said flags in response to temporal and spatial areas of interest in said encoded digital video signal, (ii) in a first mode, said content analyzer circuit (a) determines regions of greater interest and regions of lesser interest within a given channel, and (b) applies a first type of processing to said regions of greater interest comprising two or more frames and a second type of processing to said regions of lesser interest comprising two or more frames, wherein said two or more frames of said regions of greater interest and said two or more frames of said regions of lesser interest are permanently stored by said storage circuit after said first type of processing is applied to said regions of greater interest and said second type of processing is applied to said regions of lesser interest, and (iii) in a second mode, said content analyzer circuit applies said second type of processing to all frames within said given channel and said storage circuit permanently stores all of said frames of said given channel after said second type of processing is applied.

2. The apparatus according to claim 1, wherein said content analyzer circuit (i) discriminates between sequences of greater interest and sequences of lesser interest and (ii) presents the result of said discrimination to said output circuit in said one or more flags.

3. The apparatus according to claim 2, wherein said content analyzer independently changes a recording frame rate for reach of said plurality of sequences according to the result of the discrimination.

4. The apparatus according to claim 2, wherein said content analyzer enforces a minimum frame rate for storing each of said sequences.

5. The apparatus according to claim 1, wherein said plurality of sequences comprise channels of greater interest and channels of lesser interest, wherein said channels of greater interest are stored using a higher bit rate than said channels of lesser interest.

6. The apparatus according to claim 1, wherein said output circuit generates specific audio, video, control, or recording actions based on said one or more flags.

7. The apparatus according to claim 6, wherein one of said actions comprises an audio alert.

8. The apparatus according to claim 6, wherein one of said actions comprises generating a text message.

9. The apparatus according to claim 6, wherein one of said actions comprises generating a highlighted stream.

10. The apparatus according to claim 6, wherein one of said actions comprises bringing a stream of interest to the front of said output encoded bitstream.

11. The apparatus according to claim 6, wherein one of said actions comprises displaying a full resolution image sequence.

12. The apparatus according to claim 6, wherein one of said actions comprises generating a data message to present to another processing or monitoring system.

13. The apparatus according to claim 6, wherein one of said actions comprises controlling a start, stop, pause or resume of said storing.

14. The apparatus according to claim 1, wherein said content analyzer determines one or more locations of interesting events within the sequences, wherein said locations are stored in a log.

15. The apparatus according to claim 14, wherein said one or more locations comprise a time stamp in a tape storage device.

16. The apparatus according to claim 14, wherein said one or more locations comprise a file offset in an optical, magnetic, or tape storage device.

17. The apparatus according to claim 14, wherein said one or more locations are generated in response to analysis of the content of said first intermediate signal.

18. The apparatus according to claim 14, wherein said one or more locations are generated in response to other inputs to the system.

19. The apparatus according to claim 14, wherein said log is transmitted along with the output encoded bitstream.

20. The apparatus according to claim 14, wherein the storage circuit is augmented by the log to facilitate retrieval.

21. The apparatus according to claim 1, wherein said content analyzer circuit performs digital image enhancement on said regions of greater interest.

22. The apparatus according to claim 1, wherein said content analyzer circuit further tracks motion of said regions of greater interest.

23. The apparatus according to claim 1, wherein said content analyzer circuit issues information or control signals to a controller configured to steer a camera toward said regions of greater interest.

24. The apparatus according to claim 23, wherein said control signals are configured to control (i) zooming in or out, (ii) changes to brightness, (iii) changes to contrast, and (iv) edge enhancement settings.

25. The apparatus according to claim 23, wherein said control signals are configured to acquire the regions of greater interest with better temporal and/or spatial resolution.

26. The apparatus according to claim 1, wherein said flags are configured to control one or more channels within the system (i) independently when in a first mode and (ii) simultaneously when in a second mode.

27. The apparatus according to claim 1, wherein said input video signals are processed in uncompressed form.

28. The apparatus according to claim 1, wherein said output circuit is configured to generate multiple output encoded bitstreams.

29. The apparatus according to claim 1, wherein said output circuit is configured to adapt the composition and contents of one or more of the output encoded bitstreams in response to said one or more flags.

30. The apparatus according to claim 29, wherein said output circuit is configured to send an embedded alarm in one or more of the output encoded bitstreams.

31. The apparatus according to claim 1, wherein said output circuit is configured to generate one or more second output encoded bitstreams only in response to said one or more flags.

32. The apparatus according to claim 31, wherein one or more of said output encoded bitstreams is augmented by additional external data.

33. The apparatus according to claim 1, wherein said output encoded bitstream comprises a first one or more of output encoded bitstreams that are compressed and a second one or more encoded bitstreams that are uncompressed.

34. The apparatus according to claim 1, wherein said content analyzer circuit includes a video encoder configured to respond to (i) spatial and temporal filtering, (ii) spatial and temporal analysis, (iii) motion estimation, and (iv) motion compensation.

35. The apparatus according to claim 1, wherein said apparatus (i) presents index, information for viewing by a monitoring station and (ii) stores said output encoded bitstream in a data storage system.

36. The apparatus according to claim 1, wherein said flags are stored separately from said plurality of sequences.

37. The apparatus according to claim 1, wherein (i) a first set of said plurality of sequences is stored locally and (ii) a second set of said plurality of sequences is stored remotely.

38. The apparatus according to claim 1, wherein said storage circuit is further configured to (i) allocate a plurality of buffers such that each of said plurality of buffers corresponds to a respective one of said plurality of sequences and (ii) set a storage capacity of each buffer based on one or more parameters associated with the video input signal corresponding to the respective sequence.

39. An apparatus comprising:

a content analyzer circuit configured to present one or more flags in response to a plurality of input video signals;

a storage circuit configured to store said plurality of input video signals; and

an output circuit configured to generate a plurality of output encoded bitstreams in response to said stored input video signals, wherein said output circuit is further configured to embed intra picture information into said output encoded bitstreams in response to said one or more flags, wherein (i) said intra picture information allows said content analyzer circuit to generate said one or more flags in response to temporal and spatial areas of interest in said plurality of input video signals, (ii) in a first mode, said content analyzer circuit (a) determines regions of greater interest and regions of lesser interest within a given channel, (b) applies a first type of processing to said regions of greater interest comprising two or more frames and a second type of processing to said regions of lesser interest comprising two or more frames, wherein said two or more frames of said regions of greater interest and said two or more frames of said regions of lesser interest are permanently stored by said storage circuit after said first type of processing is applied to said regions of greater interest and said second type of processing is applied to said regions of lesser interest, and (iii) in a second mode, said content analyzer circuit applies said second type of processing to all frames within said given channel and said storage circuit permanently stores all of said frames of said given channel after said second type of processing is applied.

40. An apparatus for analyzing the content of a surveillance image comprising the steps of:

(A) generating a first intermediate signal from a plurality of input video signals;

(B) presenting one or more flags in response to an analysis of said intermediate signal;

(C) storing said first intermediate signal into a plurality of sequences each of said plurality of sequences corresponding to one of said input video signals;

(D) generating a second intermediate signal from said plurality of sequences; and

(E) generating an output encoded bitstream in response to said second intermediate signal, wherein intra picture information is embedded into said output encoded bitstream in response to said one or more flags, wherein (i) said intra picture information allows a content analyzer circuit to generate said one or more flags in response to temporal and spatial areas of interest in said intermediate signal, (ii) in a first mode, said content analyzer circuit (a) determines regions of greater interest and regions of lesser interest within a given channel, (b) applies a first type of processing to said regions of greater interest comprising two or more frames and a second type of processing to said regions of lesser interest comprising two or more frames, wherein said two or more frames of said regions of greater interest and said two or more frames of said regions of lesser interest are permanently stored by said storage circuit after said first type of processing is applied to said regions of greater interest and said second type of processing is applied to said regions of lesser interest, and (iii) in a second mode, said content analyzer circuit applies said second type of processing to all frames within said given channel and said storage circuit permanently stores all of said frames of said given channel after said second type of processing is applied.