Multi-imager video camera with frame-by-frame view switching

• US 8,754,941 B1
• Filed: 09/21/2010
• Issued: 06/17/2014
• Est. Priority Date: 09/22/2009
• Status: Active Grant

• Current Assignee: Gn Audio A/S (GN Store Nord A/S)
• Original Assignee: Altia Systems Inc. (GN Store Nord A/S)
• Inventors: Alam, Naveed, Mazhar, Khurshed, Sarwari, Atif M.
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): MIKESKA, NEIL R
• Application Number:
  US12/887,263
• Time in Patent Office: 1,365 Days
• Field of Search: 348/36, 348/143, 348/E7.086, 348/207.1
• US Class Current: 348/143
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

One embodiment relates to a method of outputting multiple views from a networked camera. Each imager of an array of imagers in the camera captures image frames and transmits the captured image frames to an associated image flow processor. Each image flow processor processes the captured image frames and transmits the processed image frames to a multi-imager video processor. An updating of parameters for said processing by each image flow processor is performed on a frame-by-frame basis. Another embodiment relates to a video camera including a plurality of imagers, a plurality of image flow processors, a multi-imager video processor, and a plurality of update queues. Other embodiments and features are also disclosed.

Claims

1. A video camera apparatus with frame-by-frame view switching, the apparatus comprising:

a plurality of imagers, each imager comprising a sensor array that is configured to capture a source rectangular image frame and perform a cropping of the source rectangular image frame to obtain a primary rectangular image frame that is in alignment with neighboring primary rectangular image frames;

a plurality of image flow processors, each image flow processor being associated with an imager and configured to receive and process the primary rectangular image frames captured by from the sensor array of its associated imager, resize the primary rectangular image frame to a re-sized image frame having a lower number of pixels than the primary rectangular image frame, and perform a horizontal cropping of the re-sized image frame to obtain a horizontally-cropped re-sized image frame; and

a multi-imager video processor configured to receive the horizontally-cropped re-sized image frames from the plurality of image flow processors, combine the horizontally-cropped re-sized image frames to obtain a combined image frame, perform a vertical cropping of the combined image frame to obtain a final image frame, and simultaneously provide different views to multiple users through frame-by-frame view switching enabled by sending resizing and horizontal cropping parameters to each image flow processor to control said resizing and horizontal cropping by said image flow processor on a frame-frame basis.

2. The apparatus of claim 1, further comprising:

a plurality of update queues, each update queue being associated with an image flow processor

wherein the multi-imager video processor sends horizontal cropping parameters from each update queue to the associated image flow processor.

3. The apparatus of claim 2, wherein the horizontal cropping parameters provide information for performing the horizontal cropping of the individual image frames which removes pixels from the re-sized image frame that are outside of top and/or bottom borders of the horizontal cropping.

4. The apparatus of claim 1, wherein vertical cropping removes pixels from the combined image frame that are outside of left and/or right borders of the vertical cropping.

5. The apparatus of claim 2, wherein the multi-imager video processor further sends re-sizing parameters from each update queue to the associated image flow processor, wherein the re-sizing parameters provide information for performing re-sizing of the primary rectangular image frames which reduces a number of pixels without changing image borders.

6. The apparatus of claim 1, wherein the imagers are arranged in a plurality of groups, and wherein the image flow processors are configured to receive the primary rectangular image frames from one said group at a time.

7. The apparatus of claim 6, wherein the imagers in a first group are optimized for daylight viewing conditions, and the imagers in a second group are optimized for nightlight viewing conditions.

8. The apparatus of claim 6, wherein the imagers in a first group are configured for lower resolution viewing, and the imagers in a second group are configured for higher resolution viewing.

9. The apparatus of claim 1, wherein the imagers in a first group are configured to have a combined field of view of 180 degrees, the imagers in a second group are configured to have a combined field of view of 180 degrees, and wherein the imagers in the first group are arranged above the imagers in the second group.

10. The apparatus of claim 1, further comprising:

a plurality of barrel lenses, each barrel lens configured to receive and focus light onto the sensor array of an associated imager.

11. A method of outputting multiple views from a networked camera, the method comprising:

each imager of an array of imagers in the camera capturing a source rectangular image frame and performing a cropping of the source rectangular image frame to obtain a primary rectangular image frame that is in alignment with neighboring primary rectangular image frames;

each imager transmitting the primary rectangular image frame to an associated image flow processor;

each image flow processor receiving the primary rectangular image frame from an associated image, resizing the primary rectangular image frame to a re-sized image frame having a lower number of pixels than the primary rectangular image frame, and performing a horizontal cropping of the re-sized image frame to obtain a horizontally-cropped re-sized image frame;

each image flow processor transmitting the horizontally-cropped re-sized image frame to a multi-imager video processor; and

the multi-imager video processor receiving the horizontally-cropped re-sized image frames from the plurality of image flow processors, combining the horizontally-cropped re-sized image frames to obtain a combined image frame and performing a vertical cropping of the combined image frame to obtain a final image frame;

wherein said multi-imager video processor simultaneously provides different views to multiple users through frame-by-frame view switching enabled by sending resizing and horizontal cropping parameters to each image flow processor to control said resizing and horizontal cropping by said image flow processor on a frame-frame basis.

12. The method of claim 11, further comprising:

the multi-imager video processor sending horizontal cropping parameters from a plurality of update queues to associated image flow processors.

13. The method of claim 12, wherein the horizontal cropping parameters provide information to perform the horizontal cropping of the re-sized image frame, the method further comprising:

applying the horizontal cropping to the captured re-sized image frame to remove pixels from the re-sized image frame that are outside of top and/or bottom borders of the horizontal cropping.

14. The method of claim 11, wherein the vertical cropping removes pixels from the combined image frame that are outside of left and/or right borders of the vertical cropping.

15. The method of claim 12, further comprising:

the multi-imager video processor further sending re-sizing parameters from a plurality of update queues to associated image flow processors, wherein the re-sizing parameters provide information to perform the resizing of the primary rectangular image frames which reduces a number of pixels without changing image borders.

16. The method of claim 11, wherein there are at least two arrays of imagers in the camera, and wherein the image flow processors receive the image frames from one said array at a time.

17. The method of claim 14, further comprising:

the multi-imager video processor encrypting and compressing the final image frame to generate an encoded image frame.

18. The method of claim 17, further comprising:

a packetizer receiving the encoded image frame; and

the packetizer generating data packets addressed to a plurality of user computers,

wherein the data packets addressed to a specific user computer contain the encoded image frames which are customized for that specific user computer.

19. The method of claim 15 further comprising:

switching views on a frame-by-frame basis using the horizontal cropping and re-sizing parameters and the vertical cropping.

20. The method of claim 19, wherein different views are provided simultaneously to multiple users by said switching views.