Video user interface system and method

• US 9,955,073 B2
• Filed: 07/19/2013
• Issued: 04/24/2018
• Est. Priority Date: 09/12/2003
• Status: Active Grant

• Current Assignee: Tyco Fire & Security GmbH (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Inventors: Mills, Lawrence R.
• Primary Examiner(s): FINDLEY, CHRISTOPHER G
• Application Number:
  US13/946,279
• Publication Number:
  US 20130300822A1
• Time in Patent Office: 1,740 Days
• Field of Search: None
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system providing a graphical user interface incorporating fisheye image data captured by a first camera using a fisheye lens is provided. The system includes a buffer configured to receive the fisheye image data corresponding to a monitored area. A processor is in communication with the buffer. The fisheye image data is transformed into panoramic view data corresponding to a panoramic view of the monitored area using a panoramic transformation process. The fisheye image data is transformed into virtual view data corresponding to a partial view of the panoramic view using a virtual view transformation process different from the panoramic view transformation process. The panoramic view data and the virtual view data are encoded for display in the graphical user interface.

Claims

1. A system providing a graphical user interface, the system comprising:

a first camera configured to capture fisheye image data using a fisheye lens;

a buffer configured to receive the fisheye image data corresponding to a monitored area; and

a processor in communication with the buffer, the processor configured to;

transform the fisheye image data into panoramic view data corresponding to an undistorted panoramic view of the monitored area using a panoramic transformation process;

transform the fisheye image data into virtual view data corresponding to a partial view of the panoramic view using a virtual view transformation process different from the panoramic view transformation process;

encode the panoramic view data and the virtual view data for display in the graphical user interface;

determine reference data corresponding to an area in the panoramic view represented by a virtual view, the virtual view corresponding to the virtual view data; and

overlay a reference window over the area in the panoramic view represented by the virtual view based on the reference data.

2. The system according to claim 1, further comprising:

a user input module configured to provide user command data to the processor; and

the processor being further configured to determine the partial view of the panoramic view based on the user command data.

3. The system according to claim 1, wherein a size and a position of the reference window is determined according to the user command data.

4. The system according to claim 1, further comprising a second camera in communication with the buffer and the processor, the second camera being configured to capture non-fisheye image data of a portion of the monitored area according to pan, tilt and zoom (PTZ) command data;

and

the processor being further configured to allow switching between the virtual view data and the captured non-fisheye image data for display.

5. The system according to claim 1, wherein the transforming of fisheye image data to panoramic data and virtual view data occurs in real time.

6. A system according to claim 1, wherein:

the panoramic view includes a first panoramic view and a second panoramic view, the first panoramic view corresponding to a first portion of the monitored area, and the second panoramic view corresponding to a remaining portion of the monitored area;

the virtual view includes a first virtual view and a second virtual view, the first virtual view corresponding to a first portion of the first panoramic view, and the second virtual view corresponding to a second portion of the second panoramic view; and

a combination of the first panoramic view and the second panoramic view provide a 360°

view of the monitored area relative to a vertical axis.

7. A method of generating a graphical user interface, the method comprising:

capturing fisheye image data with a first camera using a fisheye lens;

buffering fisheye image data corresponding to a wide-angle image of a monitored area;

transforming the fisheye image data into panoramic view data corresponding to an undistorted panoramic view of the monitored area;

transforming the fisheye image data into virtual view data corresponding to a portion of the panoramic view;

encoding the panoramic view data and the virtual view data for display in the graphical user interface;

determining reference data corresponding to an area in the panoramic view represented by a virtual view, the virtual view corresponding to the virtual view data; and

overlaying a reference window over an area in the panoramic view represented by the virtual view based on the reference data.

8. The method of claim 7, further comprising:

receiving user command data; and

determining the portion of the panoramic view based on the user command data.

9. The method of claim 7, wherein a size and a position of the reference window is determined according to the user command data.

10. The method according to claim 7, wherein the buffered fisheye image data is received from a first video camera system;

and

the method further comprising;

communicating pan, tilt and zoom commands to a second camera system;

receiving non-fisheye image data for the virtual view from the second camera system; and

switching from using the virtual view data based on the transformation of the fisheye image data to using the non-fisheye image data from the second camera system for display.

11. The method according to claim 7, further comprising encoding reference data, virtual view data and panoramic view data for output.

12. The method of claim 7, wherein the transforming of fisheye image data to panoramic data and virtual view data occurs in real time.

13. The method according to claim 7, wherein the panoramic view includes:

a first panoramic view corresponding to a first portion of the monitored area;

a second panoramic view corresponding to a remaining portion of the monitored area; and

the first panoramic view and the second panoramic view combine to provide a 360°

view of the monitored area relative to a vertical axis; and

the method further comprising encoding the first panoramic view, the second panoramic view, and virtual view of a portion of at least one of the first panoramic view and the second panoramic view for simultaneous display.

14. A system for providing a graphical user interface for display of a panoramic view and virtual view, the system comprising:

a first camera, the first camera configured to capture fisheye image data using a fisheye lens, the fisheye image data corresponding to a monitored area;

a second camera, the second camera configured to capture non-fisheye image data of a portion of the monitored area according to pan, tilt and zoom (PTZ) command data;

an image buffer, the image buffer configured to receive the fisheye image data and the non-fisheye image data; and

a processor in communication with the image buffer, the processor configured to;

transform the fisheye image data into panoramic view data corresponding to a panoramic view of the monitored area;

transform the fisheye image data into virtual view data corresponding to a partial view of the panoramic view;

encode the panoramic view data and the virtual view data for display in the graphical user interface;

determine reference data corresponding to an area in the panoramic view represented by a virtual view, the virtual view corresponding to the virtual view data;

overlay a reference window over an area in the panoramic view represented by the virtual view based on the reference data; and

switch from encoding virtual view data to encoding non-fisheye data for display with the panoramic view data in the graphical user interface.

15. The system of claim 14, further comprising:

a user input module, the user input module configured to provide user command data to the processor; and

the processor is further configured to determine the partial view of the panoramic view based on the user command data.

16. The system according to claim 15, wherein a size and a position of a reference window is determined according to the user command data.

17. The system according to claim 14, wherein:

the panoramic view includes a first panoramic view and a second panoramic view, the first panoramic view corresponding to a first portion of the monitored area, and the second panoramic view corresponding to a remaining portion of the monitored area;

the virtual view includes a first virtual view and a second virtual view, the first virtual view corresponding to a first portion of the first panoramic view, and the second virtual view corresponding to a second portion of the second panoramic view; and

the first panoramic view and the second panoramic view providing a 360°

view of the monitored area relative to a vertical axis.