Visual Image sensor system

• US 4,679,077 A
• Filed: 11/01/1985
• Issued: 07/07/1987
• Est. Priority Date: 11/10/1984
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Asahi Matsushita Electric Works Ltd. (Panasonic Holdings Corporation)
• Inventors: Yasuda, Akira, Yuasa, Hiroyoshi, Omura, Koichi
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): Kostak, Victor R.
• Application Number:
  US06/794,228
• Time in Patent Office: 613 Days
• Field of Search: 358/108, 358/105, 358/109, 340/518, 340/555, 340/565, 340/521, 340/541, 340/937
• US Class Current: 348/154
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/19658 Telephone systems used to communicate with a camera, e.g. PSTN, GSM, POTS
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: Y10S 706/902 Application using ai with detail of the ai system

Abstract

A visual image sensor system for detecting an abnormal change in a monitored scene is disclosed, wherein normal changes which may occur in the monitored scene are previously memorized to be selectively compared with changes in the monitored scene, whereby the normal changes are discriminated from the actual abnormal change to be detected. An alarm is issued when the monitored scene includes a change different from any of the memorized normal changes selected according to one or more rational selection standards, whereby false or redundant alarms due to the normal changes are prevented. For further precise detection, abnormal changes which may occur in the monitored scene are also previously memorized to be compared with the changes in the monitored scene.

Claims

1. A visual image sensor system comprising:

means for monitoring a given region and generating monitored picture data,

means for memorizing a plurality of standard picture data including normal changes of the monitored picture data,

means for memorizing one or more selection standards for selecting standard picture data to be compared with said monitored picture data,

means for memorizing one or more judgment standards to judge existence of a picture change by said comparison, and

means for comparing the monitored picture data with all the standard picture data that are selected on said selection standards and for issuing an alarm when a changed amount detected by said comparison is greater than said judgment standards.

2. The visual image sensor system as set forth in claim 1, wherein said selection standards comprise time ranges and predetermined light intensity ranges in the monitored region, said standard picture data for comparison being selected such that current time and current light intensity of the monitored region are respectively within the time range and the light intensity range of said selected standard picture data.

3. The visual image sensor system as set forth in claim 1, wherein said standard picture data are renewed when a difference between the standard picture data and the monitored picture data at a constant interval is below a predetermined level.

4. The visual image sensor system as set forth in claim 3, wherein said standard picture data to be renewed are replaced by average data of a plurality of previous picture data.

5. The visual image sensor system as set forth in claim 1, including:

line drawing generating means for generating line drawing data as standard picture data for the monitored picture data on the basis of contours and edges included in the normal standard picture data,

line designating means for designating important lines among said line drawing data as a standard data for change detection judgment of the line drawing data, and

judging means forjudging a change to be existent when said important lines are concealed to a predetermined extent.

6. The visual image sensor system as set forth in claim 1, including a flamy/non-flamy abnormality discriminator means comprising, in combination:

means for illuminating an object within the monitored region,

means for memorizing in advance normal standard picture data with and without said illumination by switching said illumination on and off when there is no abnormality,

means for taking monitored picture data with and without said illumination by switching said illumination on and off, and

means for generating a non-flamy abnormality detection signal when a change is detected only in said monitored picture data with said illumination and generating a flamy abnormality detection signal when changes are detected in both of said monitored picture data with and without said illumination.

7. The visual image sensor system as set forth in claim 1, including:

means for generating a menacing sound and/or light when a change occurs within a predetermined vigilance region of the monitored picture, and

means for judging alarm levels by the positions of the changed object after said menacing sound and/or light.

8. The visual image sensor system as set forth in claim 7, wherein said alarm level is set at a minimum when the changed object disappears after said menacing sound and/or light, in a middle range when the changed object does not move, and at a maximum when the changed object moves into a region of a higher vigilance level.

9. The visual image sensor system as set forth in claim 1, including means for transmitting the monitored picture data at the time of change detection through a narrow-band channel after information compression.

10. The visual image sensor system as set forth in claim 9, wherein only a changed region of a monitored picture is transmitted.

11. The visual image sensor system as set forth in claim 10, wherein said changed region is determined by a change detector means comprising, in combination:

means for calculating absolute values of differences between picture elements of the currently monitored picture data and of the standard picture data,

means for calculating projected values upon X axis and Y axis by summing up said absolute values on lines perpendicular to the respective axes,

means for detecting maximum values and minimum values of X coordinate data and Y coordinate data in which said projected values are greater than a standard value for projection intensity judgment, and

means for judging a change to be existent when the size of a rectangular region determined by said minimum and maximum values is greater than a standard value for projection size judgment.

12. The visual image sensor system as set forth in claim 10, wherein aid changed region is determined by a change detector means comprising, in combination:

means for calculating absolute values of differences between picture elements of the currently monitored picture data and of the standard picture data,

means for examining change magnitude of the respective picture elements which makes said absolute values zero when said absolute values are smaller than a standard value for change detection judgment of picture element wherein a picture element change can be judged to be valid,

means for calculating projected values upon X axis and Y axis by summing up said examined absolute values in lines perpendicular to the respective axes,

means for detecting maximum values and minimum values of X coordinate data and Y coordinate data in which said projected values are greater than a standard value projection intensity judgment, and

means for judging a change to be existent when the size of a rectangular region determined by said minimum and maximum values is greater than a standard value for a projection size judgment.

13. The visual image sensor system as set forth in claim 9 or 10, wherein said picture information is transmitted through exchange communication networks by automatically dialing a telephone communication means.

14. The visual image sensor system as set forth in claim 10, wherein transmitted picture information of a changed region is restored and converted into a constant size normalized picture frame.

15. The visual image sensor system as set forth in claim 10, wherein a normalized picture frame of the changed region is transmitted.

16. The visual image sensor system as set forth in claim 9, wherein line drawing data comprising contours and edges of the monitored region are transmitted before transmitting the picture data.

17. The visual image sensor system as set forth in claim 16, wherein line drawing data of a new change are transmitted instead of a previous picture data when said new change occurs while said previous picture data are being transmitted.

18. The visual image sensor system as set forth in claim 9 or 10, wherein said picture data is transmitted by a variable sampling rate coding system of line-to-line prediction residuals.

19. The visual image sensor system as set forth in claim 9 or 10, wherein said picture data is transmitted by a DPCM system of line-to-line prediction residuals.

20. The visual image sensor system as set forth in claim 18, wherein a sampling rate is reduced in changeless portions in accordance with a predetermined relationship between quantization values and sampling intervals of samples of a prediction residual signal, said quantization values of different sampling intervals being overlapped, whereby said predetermined relationship corresponds to a combination of said quantization values and said sampling intervals.

21. The visual image sensor system as set forth in claim 20, wherein a minimum quantum of samples is multiplied by a predetermined value at an input of a predictor, a minimum quantum of quantization characteristic being greater than the original minimum quantum of samples and smaller than a multiplied value of an original permissible amplitude error of said sample.

22. The visual image sensor system as set forth in claim 10, including:

means for calculating frame-to-frame residuals which are errors between current picture frame data and predicted values based on previous picture frame data already transmitted,

means for discriminating changed picture elements which are greater than a changed picture element judgment standard value smaller than said standard value for change detection judgment of picture elements from zero picture elements which are not greater than said changed picture element judgment standard value,

means for setting values of said zero picture elements at zero, and

means for transmitting said changed picture elements.

23. The visual image sensor system as set forth in claim 22, including:

means for eliminating solitary zero picture elements among changed picture elements by converting a few zero picture elements surrounded by changed picture elements into changed picture elements, and

means for eliminating solitary changed picture elements among zero picture elements by converting a few changed picture elements surrounded by zero picture elements into zero picture elements.

24. The visual image sensor system as set forth in claim 22, wherein said transmitted changed picture elements are the frame-to-frame residuals.

25. The visual image sensor system as set forth in claim 22, wherein said transmitted changed picture elements are the picture elements of the current picture frame.

26. The visual image sensor system as set forth in claim 22, wherein said zero picture elements are transmitted in a coded form showing the number of the zero picture elements and the number of zero lines in which all the picture elements are the zero picture elements, said changed picture elements being transmitted in an original form showing picture element values, the current picture frame values being replaced by the predicted picture frame values of said changed picture elements.

27. The visual image sensor system as set forth in claim 22, wherein said changed picture element is transmitted as an initial value of prediction in transition from the zero picture element to the changed picture element, said zero picture element being transmitted as a terminate value of prediction in transition from the changed picture element to the zero picture element.

28. The visual image sensor system as set forth in claim 22, wherein the number of zero codes meaning that the changed picture elements are zero and the number of zero lines meaning that all the picture elements are zero codes are transmitted in a coded form.

29. The visual image sensor system as set forth in claim 22, wherein a prediction coefficient of a frame-to-frame predictor circuit is selectable.

30. The visual image sensor system as set forth in claim 22, wherein said frame-to-frame residuals are compressed by a quantization characteristic whose compression rate is such that a general outline of a changed portion of the picture is recognizable, a writing operation into a current picture frame buffer for renewal of the current picture being inhibited to fix picture data within the current picture frame buffer as the current picture frame by instructions from a receiver, the frame-to-frame residuals between said fixed current picture frame and predicted values based on the previous picture frame already transmitted being transmitted repeatedly after information compression by quantization characteristic of a lower compression rate, whereby the predicted value approaches the actual value and resolution of the transmitted pictures is improved gradually.

31. The visual image sensor system as set forth in claim 22, wherein the change amount of the frame-to-frame residuals is, when it is large, reduced by making a spatial filter for averaging filtering said frame-to-frame residuals before transmission.

32. The visual image sensor system as set forth in claim 31, wherein said spatial filter is selectable and repeatedly operable.

33. The visual image sensor system as set forth in claim 31, wherein low frequency components of the frame-to-frame residuals are firstly transmitted by setting a previous picture frame data already transmitted at zero and by making a first spatial filter having a higher averaging ability act upon the current picture frame before transmission, high frequency components of the frame-to-frame residuals being secondly transmitted by making a second spatial filter having a lower averaging ability act upon the succeeding current picture frames, whereby the predicted value approaches the actual value and resolution of the transmitted pictures is improved gradually.

34. The visual image sensor system as set forth in claim 9, including:

means for generating rough picture elements which are roughly thinned out at a constant space,

means for calculating frame-to-frame residuals which are errors between current picture frame data and predicted values based on a previous picture frame already transmitted,

means for discriminating changed rough picture elements which are greater than a changed rough picture element judgment standard value smaller than a standard value for change detection judgment of rough picture elements from zero rough picture elements which are not greater than said changed rough picture element judgment standard value,

means for setting values of said zero rough picture elements at zero,

means for transmitting said changed rough picture elements, and

means for interpolating said changed rough picture elements,

said interpolation being carried out, only when roughly thinned out adjoining picture element lattices are changed picture elements, within a section including said adjoining lattices of changed picture elements.

35. The visual image sensor system as set forth in claim 34, wherein said interpolation is further carried out within a section including said changed rough picture elements and zero rough picture elements of said rough lattices adjoining thereto.

36. The visual image sensor system as set forth in claim 34, wherein said changed rough picture element is transmitted as said frame-to-frame residuals, the values of the predicted picture elements within the corresponding section of said changed rough picture elements being interpolated by the picture element values of said predicted picture at the positions of said changed rough picture elements, and being added to said interpolatd values of the frame-to-frame residuals, whereby a predicted picture is interpolated and restored.

37. The visual image sensor system as set forth in claim 9, wherein picture information of the monitored picture data is rapidly transmitted by picture information data of a rough picture whose lattice is rougher than a sampling lattice and such that a general outline of a changed portion of the picture is recognizable, said rough picture being interpolated in the receiver to indicate the same and wherein a fine current picture frame is transmitted after information compression by instructions from the receiver to make pictures fine, whereby resolution of the transmitted pictures are gradually improved.

38. The visual image sensor system as set forth in claim 9, wherein said transmission of the monitored picture data is, on receiving skip instructions from a receiver, interrupted and replaced by a gradual distinction improvement transmission of a new current picture frame.

39. The visual image sensor system as set forth in claim 9, wherein said transmission of the monitored picture data is, on receiving snap instructions from a receiver, interrupted and replaced by a gradual distinction improvement transmission of a new current picture frame during a constant period.

40. The visual image sensor system as set forth in claim 9, including a speaker in a transmitter, wherein said transmission of the monitored picture data is, on receiving instructions for calling by the speaker from a receiver, interrupted and replaced by a gradual distinction improvement transmission of a new current picture frame corresponding to said speaker calling during a constant period.

41. The visual image sensor system as set forth in claim 9, including a microphone in a transmitter, wherein said transmission of the monitored picture data is, on receiving instructions for listening by the microphone from a receiver, interrupted and replaced by a gradual distinction improvement transmission of a new current picture frame corresponding to said microphone listening during a constant period.

42. A visual image sensor system comprising:

means for monitoring a given region and generating monitored picture data,

means for memorizing a plurality of picture data including abnormal situations as standard picture data for the monitored picture data,

means for comparing the monitored picture data with said memorized picture data to identify a changed object,

means for memorizing judgment standard values for judging sizes and positions of changed objects for the respective picture data memorized,

means for comparing said identified picture data with the monitored picture data in accordance with said judgment standard values to issue an alarm when a change is judged to be abnormal as a result of said comparison.

43. The visual image sensor system as set forth in claim 42, including means for transmitting the monitored picture data at the time of abnormal change detection through a narrow-band channel after information compression.

44. The visual image sensor system as set forth in claim 42, including:

a knowledge base memorizing registered pattterns that include normal standard picture data and a variety of patterns of abnormal situations in line drawings changed form said picture data and that are expressed in classed list-formed models including numerical data which are based on a comprehensive relationship of features of essential and fundamental elements and on analyzed structure expressed by a relationship based on physical rules,

means for identifying the changed object by inferring from and searching said knowledge base,

means for generating current picture pattern data describing monitored scenes by judging a position and a size of the change object in accordance with restriction conditions attendant on said models in the knowledge base,

means for detecting the motion of and the moving direction of the change object by identifying the changed object on a succession of said generated pattern data, and

means for discriminating abnormal conditions from normal conditions in accordance with judgment standards comprising shapes, sizes, positions, and moving directions of the changed object.

45. The visual image sensor system as set forth in claim 44, wherein said knowledge base memorizes rules for interpreting meanings of the monitored scenes from judged results of said shapes, sizes, positions, and moving directions of the changed object,

said rules for interpreting meanings of the monitored scenes being used for judging whether or not the respective elements comprising said shapes, positions, sizes, moving directions of the changed object in the registered picture data belong to the same category with that to which the respective elements of the changed object in the monitored picture belong.