Interface for browsing and viewing video from multiple cameras simultaneously that conveys spatial and temporal proximity

• US 20080088706A1
• Filed: 10/13/2006
• Published: 04/17/2008
• Est. Priority Date: 10/13/2006
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Dunnigan, Anthony Eric, Girgensohn, Andreas, Shipman, Frank M., Wilcox, Lynn D.
• Granted Patent: US 8,274,564 B2
• Time in Patent Office: Days
• US Class Current: 348/207.99
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An interface and display of video from multiple fixed-position cameras is provided. A main video stream captured by a camera is selected to be the main video stream and is displayed to the interface. Video streams captured by the set of cameras and the main camera that are temporally related to the displayed main video stream are selected, including playback positions from one or more of a first segment of time in each of their respective video streams at the time of the main video stream, a second segment of time in each of their respective video streams prior to the time of the main video stream, and a third segment of time in each of their respective video streams after the time of the main video stream. The selected video streams are displayed to the interface in temporal relation to the display of the main video stream.

Claims

1. An interface that provides a display of video from multiple cameras at fixed locations, the interface comprising:

a first interface portion to display a main video stream to the interface, wherein the video stream was captured by a main camera; and

one or more additional interface portions to display video streams that are temporally related to the displayed main video stream, wherein the video streams are captured by a set of cameras and the main camera, the video stream displays comprising one or more of;

a first set of video playback positions from a first segment of time in each of their respective video streams at the time of the main video stream;

a second set of video playback positions from a second segment of time in each of their respective video streams prior to the time of the main video stream; and

a third set of video playback positions from a third segment of time in each of their respective video streams after the time of the main video stream.

2. The interface of claim 1, wherein a video stream associated with one of the one or more video stream displays becomes the main video stream displayed to the interface with a corresponding video playback position when a user clicks on the video stream display.

3. The interface of claim 1, wherein one or more of the video stream displays are static keyframes, wherein a keyframe is a representative video frame from a video stream near the time of the video stream'"'"'s video playback position.

4. The interface of claim 3, wherein a keyframe is a representative video frame from a video stream sequence having the most activity within one of the first, second and third segments of time.

5. The interface of claim 4, wherein the video stream sequence that has the most activity comprises the largest number of pixels that change between video frames in the sequence.

6. The interface of claim 1, wherein:

the second set of video playback positions are from a first fixed point in time in each of their respective video streams prior to the time of the video stream of the main camera; and

the third set of video playback positions are from a second fixed point in time in each of their respective video streams after the time of the video stream of the main camera.

7. The interface of claim 1, wherein color coding of video stream displays in the interface display identifies video stream source cameras.

8. The interface of claim 1, wherein a map that shows a spatial view of camera positions and their corresponding angles of view for the set of cameras and the main camera is displayed alongside the interface.

9. The interface of claim 8, wherein one or more video streams are displayed next to their corresponding source cameras on the map.

10. The interface of claim 9, wherein the cameras and their corresponding angles of view displayed to the map are color coded to match video stream display color coding displayed to the interface that identifies source cameras.

11. The interface of claim 1 wherein the video stream displays associated with the second and third sets of video playback positions are each displayed within a band on opposite edges, respectively, of the main video stream display.

12. The interface of claim 11, further comprising a timeline displayed next to the bands and the main video stream display.

13. The interface of claim 1, wherein the multiple cameras have fixed view directions.

14. The interface of claim 1, wherein the multiple cameras perform one or more of pan, tilt and zoom when the interface displays live video streams.

15. The interface of claim 1, wherein the set of cameras is in spatially proximity to the main camera.

16. The interface of claim 1, wherein the video stream displays associated with the first, second, and third sets of video playback positions further comprise an assessment of one or more of activity within the video streams, motion close to points of interest in the space being videotaped and events from infrared motion sensors, face detection and radio frequency identification.

17. The interface of claim 1, wherein

video stream displays associated with the first, second, and third sets of video playback positions are displayed to the interface in spatial relation to the displayed video stream from the main camera,

wherein the spatial relation corresponds to relative physical positions of each of a set of cameras to the physical position of the main camera, and

wherein the video stream displays are from video streams of a set of cameras that are closest to the main camera.

18. The interface of claim 17, wherein a video stream associated with one of the one or more video stream displays becomes the main video stream displayed to the interface with a corresponding video playback position when a user clicks on the video stream display.

19. The interface of claim 17, wherein one or more of the video stream displays are static keyframes, wherein a keyframe is a representative video frame from a video stream near the time of the video streams'"'"' video playback position.

20. The interface of claim 19, wherein a keyframe is a representative video frame from a video stream sequence having the most activity within one of the first, second and third segments of time.

21. The interface of claim 20, wherein the video stream sequence that has the most activity comprises the largest number of pixels that change between video frames in the sequence.

22. The interface of claim 17, wherein a map that shows a spatial view of camera positions and their corresponding angles of view for the set of cameras and the main camera is displayed alongside the interface.

23. The interface of claim 22, wherein one or more video streams are displayed next to their corresponding source cameras on the map.

24. The interface of claim 23, wherein a video stream becomes the main video stream when a user clicks on the map on one of the video stream and its corresponding camera.

25. The interface of claim 17, wherein the video stream displays are animated to rotate around the new main video stream to their new positions spatially related to the new main video stream.

26. The interface of claim 22, wherein

a video stream associated with one of the one or more video stream displays becomes the main video stream displayed to the interface with a corresponding playback position when a user clicks on the video stream display;

and

the spatial map is animated to show rotation of the map as it is centered to a new viewpoint of the newly selected camera that corresponds to the new main video stream displayed to the interface.

27. The interface of claim 26, wherein the animation of the spatial map is synchronized to animated video stream displays that are rotated around the new main video stream to their new positions spatially related to the new main video stream.

28. The interface of claim 17, wherein the closeness of each of the set of cameras to the main camera comprises the average of the sum of distances between one or more focal points between each of the set of cameras and the main camera.

29. The interface of claim 17, wherein the spatial relation of a video stream display to the main video stream display further comprises a determination of the angle formed by a line from a focal point of the main camera to a focal point of the camera corresponding to the video stream display and a line from the focal point of the main camera to the main camera, wherein each video stream display is displayed at the determined angle in relation to the main video stream display.

30. The interface of claim 17, wherein color coding of video stream displays in the interface display identifies video stream display source cameras.

31. The interface of claim 22, wherein the cameras and their corresponding angles of view displayed to the map are color coded to match video stream display color coding displayed to the interface that identifies source cameras.

32. The interface of claim 17, wherein the multiple cameras have fixed view directions.

33. The interface of claim 17, wherein the multiple cameras perform one or more of pan, tilt and zoom when the interface displays live video streams.

34. The interface of claim 17, wherein the video stream displays associated with the first, second, and third sets of video playback positions further comprise an assessment of one or more of activity within the video streams, motion close to points of interest in the space being videotaped and events from infrared motion sensors, face detection and radio frequency identification.

35. A method for providing an interface and display of video from multiple fixed cameras, the method comprising:

selecting a video stream captured by a camera to be the main video stream;

displaying the main video stream to a first portion of the interface;

selecting video streams captured by the set of cameras and the main camera that are temporally related to the displayed main video stream, the selecting comprising one or more of;

selecting a first set of playback positions from a first segment of time in each of their respective video streams at the time of the main video stream;

selecting a second set of playback positions from a second segment of time in each of their respective video streams prior to the time of the main video stream;

selecting a third set of playback positions from a third segment of time in each of their respective video streams after the time of the main video stream; and

displaying to one or more additional interface portions the selected video streams in temporal relation to the display of the main video stream.

36. The method of claim 35, wherein selecting video streams further comprises:

selecting video streams captured by a set of cameras that are closest to the main camera.

37. The method of claim 36, wherein displaying to one or more additional interface portions comprises:

displaying the selected video streams in spatial relation to the displayed main video stream, wherein the spatial relation corresponds to relative physical positions of each of a set of cameras to the physical position of the main camera.

38. A program of instructions executable by a computer to perform a function for providing an interface and display of video from multiple fixed cameras, the function comprising:

selecting a video stream captured by a camera to be the main video stream;

displaying the main video stream to a first portion of the interface;

selecting video streams captured by the set of cameras and the main camera that are temporally related to the displayed main video stream, the selecting comprising one or more of;

selecting a first set of playback positions from a first segment of time in each of their respective video streams at the time of the main video stream;

selecting a second set of playback positions from a second segment of time in each of their respective video streams prior to the time of the main video stream;

selecting a third set of playback positions from a third segment of time in each of their respective video streams after the time of the main video stream; and

displaying to one or more additional interface portions the selected video streams in temporal relation to the display of the main video stream.

39. The function of claim 38, wherein selecting video streams further comprises:

selecting video streams captured by a set of cameras that are closest to the main camera.

40. The function of claim 39, wherein displaying to one or more additional interface portions comprises:

displaying the selected video streams in spatial relation to the displayed main video stream, wherein the spatial relation corresponds to relative physical positions of each of a set of cameras to the physical position of the main camera.