Digital video system using networked cameras

• US 6,891,566 B2
• Filed: 03/14/2001
• Issued: 05/10/2005
• Est. Priority Date: 03/14/2000
• Status: Expired due to Term

• Current Assignee: JDS Technologies, Inc.
• Original Assignee: Joseph Robert Marchese
• Inventors: Marchese, Joseph Robert
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): Henn, Timothy J.
• Application Number:
  US09/808,543
• Publication Number:
  US 20020003575A1
• Time in Patent Office: 1,518 Days
• Field of Search: 348/211.99, 348/211.14, 3482111-2113, 348/211.12, 348152-155, 348/699, 348700-702, 707/104.1, 709/231, 709/217, 709/218, 709/219, 725/116, 345/738, 345/740, 396/153
• US Class Current: 348/211.3
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04L 41/0654 using network fault recovery ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437
• CPC Class Codes: H04L 41/0677 Localisation of faults
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/6175 involving transmission via Internet broadcast-related systems characterised by the transmission system being the Internet H04H60/82
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/695 : Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/77 : between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/9205 : the additional signal being at least another television signal
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

A digital video system including a computer connected via a network to a number of video servers and cameras. The computer includes a program that provides a grid of display windows, each of which displays an image received from the camera associated with that window. The program sequentially polls each camera, accessing and displaying an image from the camera in its associated window. The program can access the cameras at different frame rates. The program stores image streams in a single file, concatenating each successive image onto the end of the file. The file is then indexed using SOI and EOI markers to permit fast access to individual images within the file. The program can monitor received video and automatically start recording upon detecting motion within the video stream. Motion detection is implemented by comparing color component values for pixels from different images.

Claims

1. A computer readable medium for use by a computer in providing an interface to multiple cameras via one or more video servers accessible to the computer via a network, comprising:

a digital storage device;

a user interface program stored on said digital storage device in computer readable form, said program being operable upon execution by the computer to access server data uniquely identifying each video server and to attempt access to the video servers over the network, said program also being operable to obtain from each of the accessible video servers a hardware address stored in the video server;

wherein said program is further operable to validate the hardware addresses received from the video servers using the server data and, for those video servers having valid hardware address, said program is operable to generate a user interface display on the computer that includes a display window for each of the cameras accessed via the validated servers over the network and to display in each of the display windows an image received from the camera associated with that display window.

2. A computer readable medium as defined in claim 1, wherein said program is operable to sequentially access each of the cameras that are accessible via the validated servers over the network and is further operable to update the image displayed in the window associated with each accessed camera using an updated image received from that camera.

3. A computer readable medium as defined in claim 2, wherein said program is operable to sequentially update the display windows using the updated images received from the cameras associated with those windows.

4. A computer readable medium as defined in claim 3, wherein said program is operable to sequentially highlight each display window as it is being updated.

5. A computer readable medium as defined in claim 1, wherein said program is operable to sequentially poll each of the cameras that are accessible via the validated servers over the network and to update the display window associated with a polled camera using an updated image received from the polled camera.

6. A computer readable medium as defined in claim 1, wherein said program is operable to scale the images received according to the number of display windows contained in the user interface display.

7. A computer readable medium as defined in claim 1, wherein said program is operable to permit a user of the computer to select the number of display windows included in the user interface display.

8. A computer readable medium as defined in claim 1, wherein said program is operable to receive the images as image files in which the images are represented using a graphic file format.

9. A computer readable medium as defined in claim 8, wherein said program includes a compression library and is operable to compress the size of the image files using said compression library.

10. A computer readable medium as defined in claim 9, wherein said program is operable to permit a user of the computer to select a level of compression used by said compression library.

11. A computer readable medium as defined in claim 1, wherein said program is operable to open a connection to a selected one of the cameras using a network protocol supported by the network and to initiate streaming of images from the selected camera, and wherein said program is operable to periodically pause and resume the streaming of images from the selected camera to thereby control the frame rate of images received from the selected camera.

12. A computer readable medium as defined in claim 11, wherein said program is operable to pause the streaming of images temporarily following each received image from the selected camera.

13. A computer readable medium as defined in claim 11, wherein said program is operable to pause the streaming of images during which time it accesses images from one or more other cameras.

14. A computer readable medium as defined in claim 11, wherein said program is operable to open a connection to a plurality of selected cameras using the network protocol and to initiate streaming of images from each of the selected cameras, and wherein the program is operable to sequentially cycle through each of the selected cameras, obtaining one or more of the streamed images from one selected camera, pausing the streaming from that selected camera, and then obtaining one or more streamed images from a next one of the selected cameras until all of the plurality of selected cameras have been accessed and paused and thereafter repeating the cycle one or more times for the selected cameras.

15. A computer readable medium as defined in claim 11, wherein the network is an Ethernet network and the network protocol is TCP/IP.

16. A computer readable medium as defined in claim 1, wherein said program is operable to launch a separate process that is itself operable to open a connection to a selected one of the cameras using a network protocol supported by the network and to initiate streaming of images from the selected camera.

17. A computer readable medium as defined in claim 1, wherein said program is operable to record sequences of individual images received by the computer from one or more of the cameras, with the individual images being stored together in a single file.

18. A computer readable medium as defined in claim 17, wherein said program is operable to build the single file by concatenating each image onto the end of the file upon receipt of the image by the computer.

19. A computer readable medium as defined in claim 18, wherein said program is operable to index the images within the single file and store the index as a separate index file.

20. A computer readable medium as defined in claim 18, further comprising an image viewer program that is operable to index the images within the single file.

21. A computer readable medium as defined in claim 20, wherein said image viewer program is operable to generate the index as a set of pointers to start-of-image and end-of-image markers contained within the single file.

22. A computer readable medium as defined in claim 20, wherein said image viewer program is operable upon user selection of the single file to use the index to locate individual images within said single file.

23. A computer readable medium as defined in claim 1, wherein said program is operable to access a plurality of remote cameras via a video server connected between the remote cameras and the network.

24. A computer readable medium as defined in claim 1, wherein said program is operable to store images received from a selected one of the cameras in a hierarchical directory structure on a digital storage device, with the images being stored within a subdirectory located below a directory unique to the camera from which the images being stored originated, and wherein the subdirectory is identified according to date information that relates to when the image was stored.

25. A computer readable medium as defined in claim 1, wherein said program is operable to detect motion in a sequence of images received by a camera by processing color component information contained in the images.

26. A computer readable medium as defined in claim 25, wherein said program is operable to store a first image from the sequence as a reference image and to compare color component values for pixels from one or more subsequent images in the sequence to the color component values of corresponding pixels in the reference image.

27. A computer readable medium as defined in claim 26, wherein said program is operable to maintain a count of the number of pixels for which the difference in color component values for corresponding pixels from the reference and selected images differ by more than the preselected amount.

28. A computer readable medium as defined in claim 1, wherein the hardware addresses received from the video servers are MAC addresses and the server data includes at least a MAC address for each of the video servers, and wherein said program is operable to validate the video servers using the MAC addresses received from the video servers along with the MAC addresses included in the server data.

29. A computer readable medium for use by a computer in providing an interface to multiple cameras accessible to the computer via a network, comprising:

a digital storage device;

a user interface program stored on said digital storage device in computer readable form, said program being operable upon execution by the computer to access camera data uniquely identifying the cameras and to attempt access to the cameras over the network using the camera data;

wherein said program is further operable to verify access to those ones of the cameras that are accessible over the network and to generate a user interface display on the computer that includes a display window for each of the cameras accessed over the network and to display in each of the display windows an image received from the camera associated with that display window; and

wherein said program is operable to monitor the network for a trigger event generated by one of the cameras.

30. A computer readable medium as defined in claim 29, wherein said program is operable to send control instructions to a selected one of the cameras defining conditions under which the trigger event is generated by the selected camera.

31. A computer readable medium for use by a computer in providing an interface to multiple cameras accessible to the computer via a network, comprising:

a digital storage device;

a user interface program stored on said digital storage device in computer readable form, said program being operable upon execution by the computer to access camera data uniquely identifying the cameras and to attempt access to the cameras over the network using the camera data;

wherein said program is further operable to verify access to those ones of the cameras that are accessible over the network and to generate a user interface display on the computer that includes a display window for each of the cameras accessed over the network and to display in each of the display windows an image received from the camera associated with that display window; and

wherein said program is operable in response to user selection of a display window to display a form that permits the user to initiate recording of images from the camera associated with that display window.

32. A computer readable medium for use by a computer in detecting motion in a sequence of color video images received by the computer, comprising:

a digital storage device;

a program stored on said digital storage device in computer readable form, said program being operable upon execution by the computer to select one of the video images as a reference image and another of the video images as a selected image to be compared to the reference image;

wherein said program is operable to access a plurality of different color component values for each of a number of pixels from the reference image and for each of a number of corresponding pixels from the selected image, with said program being operable to perform pixel companions using the color component values and to maintain a count of the number of pixels for which the difference in color component values for corresponding pixels from the reference and selected images differ by more than the preselected amount;

wherein said program is operable to maintain a separate counter for each of the different color component values and to increment the counter associated with a particular color component value if that color component value for a pixel from the selected image differs from that same color component value for the corresponding pixel from the reference image by more than a preselected offset associated with that color component; and

wherein said program is operable to generate a motion detect signal based on the counts stored in the counters.

33. A computer readable medium as defined in claim 32, wherein said program is operable to generate the motion detect signal when, for each of the counters, the count stored in that counter exceeds a predetermined minimum count associated with the color component value associated with that counter.

34. A computer readable medium as defined in claim 33, wherein said program is operable to permit user selection of the predetermined minimum counts.

35. A computer readable medium as defined in claim 32, wherein said program is operable during receipt of the video images by the computer to compare the video images with the reference image as those video images are received by the computer, whereby said program provides real time motion detection processing of the received images.

36. A computer readable medium as defined in claim 35, wherein said program is operable to periodically select a new reference image from the video images being received by the computer.

37. A computer readable medium as defined in claim 32, wherein said program is operable to perform the pixel comparisons only for those pixels located within a region of the reference and selected images.

38. A computer readable medium as defined in claim 37, wherein said program is operable to permit a user to specify the region using a mask.

39. A computer readable medium as defined in claim 38, wherein said program is operable to provide a display window containing one of the images and to enable the user to create the mask by erasing a portion of the image contained in the display window.

40. A computer readable medium as defined in claim 32, wherein said program is operable to begin recording of the video images in response to the motion detect signal.

41. A computer readable medium as defined in claim 40, wherein said program is operable to concatenate the video images and store them in a single file.

42. A computer readable medium as defined in claim 40, wherein said program is operable to continue the comparison of the reference image with selected images during recording of the video images.

43. A computer readable medium as defined in claim 42, wherein said program is operable to stop recording after no further motion is detected using the pixel comparisons.

44. A computer readable medium as defined in claim 43, wherein said program is operable to stop recording after no further motion is detected and a specified number of extra images have been recorded.

45. A computer readable medium for use by a computer in detecting motion in a sequence of color video images received by the computer, comprising:

a digital storage device;

a program stored on said digital storage device in computer readable form, said program being operable upon execution by the computer to select one of the video images as a reference image and another of the video images as a selected image to be compared to the reference image;

wherein said program is operable to access a color component value for each of a number of pixels from the reference image and for each of a number of corresponding pixels from the selected image, with said program being operable to perform pixel comparisons using the color component values and to generate a motion detect signal when the difference in color component values for corresponding pixels from the reference and selected images differ by more than a preselected amount;

wherein said program is operable during comparison of the selected and reference images to maintain a count of the number of pixels for which the difference in color component values for corresponding pixels from the reference and selected images differ by more than the preselected amount;

wherein each pixel has a plurality of different color component values associated therewith and wherein said program is operable to maintain a separate counter for each of the different color component values and to increment the counter associated with a particular color component value if that color component value for a pixel from the selected image differs from that same color component value for the corresponding pixel from the reference image by more than a preselected offset associated with that color component;

wherein said program is operable to generate the motion detect signal when, for each of the counters, the count stored in that counter exceeds a predetermined minimum count associated with the color component value associated with that counter; and

wherein the different color component values comprise RGB component values, each of which has its own offset and minimum count.

46. A computer readable medium as defined in claim 45, wherein said program is operable to permit user selection of the offsets and minimum counts for each of the RGB component values.

47. A computer readable medium for use by a computer in recording a sequence of images, each of which comprises an individual image file in which the image is represented in a graphics file format that includes a first marker identifying the start of the image and a second marker identifying the end of the image, comprising:

a digital storage device;

a program stored on said digital storage device in computer readable form, said program being operable to store the images together as a single file that comprises the images concatenated together in sequential order;

wherein said program is further operable to index the file using the first and second marker;

wherein said single file comprises a digital video file and said program is operable to store a plurality or digital video files on the data storage device; and

wherein said program is operable to automatically index the digital video files on a periodic basis.

48. A computer readable medium for use by a computer in recording a sequence of images, each of which comprises an individual image file in which the image is represented in a graphics file format that includes a first marker identifying the start of the image and a second marker identifying the end of the images, comprising:

a digital storage device;

a program stored on said digital storage device in computer readable form, said program being operable to store the images together as a single file that comprises the images concatenated together in sequential order;

wherein said program is further operable to index the file using the first and second markers;

wherein said program comprises an image viewer program that is operable upon user selection of the single file to use the index to locate individual images within said single file; and

wherein said program is operable upon user selection of the single file to search for the index and, if not found, is further operable to create the index.

49. A digital video system, comprising:

a client computer,

one or more video servers accessible by the client computer over a network;

a plurality of cameras connected to the video server(s), said video server(s) providing an interface between the cameras and client computer in which images from the cameras are accessed by the video server(s) and then sent to the client computer upon request by the client computer;

the video server(s) including a unique number that is stored in memory within the video server(s) and that is accessible to the client computer over the network;

wherein the client computer provides a user interface display on the computer that includes a display window for each of the cameras accessed over the network and that displays in each of the display windows an image received from the camera associated with that display window; and

wherein the client computer uses the unique number obtained from a particular video server to determine whether to enable or disable the display of images from one or more cameras connected to that particular video server.

50. The digital video system of claim 49, wherein the one or more video servers comprises a plurality of video servers.

51. The digital video system of claim 50, wherein the one or more video servers comprises a portion of a camera server which includes that video server together with one of the cameras.

52. The digital video system of claim 49, wherein the one or more video servers includes a multi-camera video server connected to at least two of the cameras.

53. The digital video system of claim 49, wherein the client computer determines whether to enable or disable the display of images accessible from the particular video server by validating the unique number using server data stored on the client computer.

54. The digital video system of claim 49, wherein the unique number(s) each comprises a MAC address.