Networked image/video processing system

• US 8,948,253 B2
• Filed: 12/15/2011
• Issued: 02/03/2015
• Est. Priority Date: 12/15/2011
• Status: Active Grant

• Current Assignee: IMERJ LLC (Flex, Ltd.), Flextronics America LLC (Flextronics International Limited)
• Original Assignee: Flextronics America LLC (Flextronics International Limited)
• Inventors: Reckwerdt, Bill, Li, Wei
• Primary Examiner(s): WONG, ALLEN C
• Application Number:
  US13/327,663
• Publication Number:
  US 20130156095A1
• Time in Patent Office: 1,146 Days
• Field of Search: 375/240, 375/240.01, 375/240.02
• US Class Current: 375/240.02
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2335 involving reformatting operations of audio signals, e.g. by converting from one coding standard to another details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16

Abstract

A distributed image/video processing system is disclosed herein wherein one or more of digital image/video recorders (e.g., a digital cameras, video recorders, or smart phones, etc.) are in network communication with central network site for transmitting image or video data thereto. The recorders process their image/video data dependent upon an estimate of a measurement of network bandwidth that is available for transmitting image or video data to the central network site.

Claims

1. A method for processing data, comprising:

obtaining network data related to a transmission characteristic for a telecommunications network;

obtaining first data at a first device operably in communication with a network site via the telecommunications network;

obtaining data for compressing the first data, the data for compressing for use in identifying one of a first data compression and a second data compression;

wherein for the network data being indicative of a first result of the transmission characteristic, the data for compressing is for identifying the first data compression, and for the network data being indicative of a more constrained second result of the transmission characteristic, the data for compressing is for identifying the second data compression, wherein the first data compression compresses data less than the second data compression;

enhancing the first data, or data derived therefrom, for obtaining enhanced data;

wherein the step of enhancing includes performing a first noise reduction resulting in the enhanced data having a first reduction in noise content when the data for compressing is for identifying the first data compression, and

wherein the step of enhancing includes performing a second noise reduction resulting in the enhanced data having a second reduction in noise content when the data for compressing is for identifying the second data compression;

wherein, for the first data including video data, the first noise reduction reduces more noise in the enhanced data than the second noise reduction;

compressing the enhanced data with one of the first data compression and the second data compression according to the network data, thereby obtaining processed data;

obtaining history information indicative of processing of the first data to obtain the process data, wherein the history information includes compression description information related to the data for compressing, and including enhancement description information indicative of enhancements performed on the first data for obtaining the enhanced data; and

transmitting the processed data and the history information, via the network, to the network site;

wherein following the transmission, the transmitted history information is accessed for decompressing the transmitted processed data when the transmitted processed data is compressed.

2. The method of claim 1, wherein at least most of the steps claim 1 are performed by the first device, and each of the first and second data compressions includes one of M-JPEG, H.264 or JPEG.

3. The method of claim 1, wherein each of the first and second noise reductions includes a Gaussian Blur, wherein a Gaussian Blur value for the first noise reduction is greater than a Gaussian Blur value for the second noise reduction.

4. The method of claim 1, wherein the transmission characteristic is dependent upon one of:

an available network bandwidth, a network transmission time delay, a network data rate, and a maximum transmission time delay threshold.

5. The method of claim 4, wherein when the transmission characteristic is dependent upon the network transmission time delay, the first result is indicative of a lesser value for the network transmission time delay than the second result;

wherein the transmission characteristic is dependent upon the network data rate, the first result is indicative of a greater value for the network data rate than the second result;

wherein the transmission characteristic is dependent upon the maximum transmission time delay threshold, the first result is indicative of a greater value for the maximum transmission time delay threshold than the second result.

6. The method of claim 1, wherein the step of enhancing includes sharpening the first data or the data derived therefrom.

7. The method of claim 6, wherein the step of sharpening includes sharpening the first data or the data derived therefrom more when the second data compression compresses the enhanced data, as opposed to the first data compression compressing the enhanced data.

8. The method of claim 1, wherein the transmitted history information is accessed for enhancing the processed data by activating one or more of:

a de-noising process, a sharpening process, an auto-exposure process, an auto-white balance process, and an auto-focus process.

9. The method of claim 1, further including a step of determining whether one or more of an auto-exposure, auto-white balance, and auto-focus is to be performed prior to the step of obtaining the first data, wherein the step of determining prevents at least one of the auto-exposure, auto-white balance, and auto-focus from being performed when the network data is indicative of an available network bandwidth below a predetermined threshold.

10. The method of claim 1, wherein the step of enhancing includes activating one of first sharpening process and a second sharpening process, wherein the first sharpening process sharpens the first data or data derived therefrom less than the second sharpening process;

wherein the first sharpening process is activated when the first data compression is designated for compressing the enhanced data, or data derived therefrom; and

wherein the second sharpening process is activated when the second data compression is designated for compressing the enhanced data, or data derived therefrom.

11. The method of claim 1, wherein the network site does not perform further video or image enhancement of the processed data.

12. A non-transitory computer-readable medium having machine instructions stored thereon, the instructions comprising:

obtaining network data related to a transmission characteristic for a telecommunications network;

obtaining first data at a first device operably in communication with a network site via the telecommunications network;

obtaining data for compressing the first data, the data for compressing for use in identifying one of a first data compression and a second data compression;

wherein for the network data being indicative of a first result of the transmission characteristic, the data for compressing is for identifying the first data compression, and for the network data being indicative of a more constrained second result of the transmission characteristic, the data for compressing is for identifying the second data compression, wherein the first data compression compresses data less than the second data compression;

enhancing the first data, or data derived therefrom, for obtaining enhanced data;

wherein the step of enhancing includes performing a first noise reduction resulting in the enhanced data having a first reduction in noise content when the data for compressing is used in determining the first data compression, and

wherein the step of enhancing includes performing a second noise reduction resulting in the enhanced data having a second reduction in noise content when the data for compressing is used in determining the second data compression;

wherein, for first data including video data, the first noise reduction is identified for reducing more noise in the enhanced data than the second noise reduction;

compressing the enhanced data with one of the first data compression and the second data compression according to the network data, thereby obtaining processed data;

obtaining history information indicative of processing of the first data, wherein the history information includes compression description information related to the data for compressing, and including enhancement description information indicative of enhancements performed on the first data for obtaining the enhanced data; and

transmitting the processed data and the history information, via the network, to the network site;

wherein following the transmission, the transmitted history information is accessed for decompressing the transmitted processed data when the transmitted processed data is compressed.

13. The computer readable medium of claim 12, wherein for at least most of the steps of claim 8, the machine instructions for performing the step reside on the first device, and the first device is a hand-held mobile communications device.

14. The computer readable medium of claim 12, wherein the machine instructions include instructions for comparing the network data with data indicative of a bandwidth extent of the network.

15. The computer readable medium of claim 12, wherein the step of enhancing includes de-noising one of the processed data or the decompressed data, wherein the de-noising reduces one of:

a salt and pepper noise or a Gaussian noise.

16. A system for processing image or video data comprising:

an image or video recorder comprising;

(a) an image or video generator for generating image or video data;

(b) one or more enhancement processes for enhancing the image or video data, or a version derived therefrom resulting in enhanced data, wherein the enhancement processes include at least one noise reduction process;

(c) a compressor for compressing the enhanced data resulting in recorder processed data;

(d) an evaluator for determining whether (i) network data, related to a transmission characteristic for a network in operable communication with the recorder, is indicative of (ii) a first specification of the transmission characteristic or a second specification of the transmission characteristic;

wherein the first specification is indicative of more constrained operational condition for performing a transmission on the network;

wherein the first specification corresponds to a first data compression, and the second specification corresponds to a second data compression;

wherein the first data compression compresses data less than the second data compression;

wherein prior to the first data compression being performed on the enhanced data, a first noise reduction is performed by the at least one noise reduction process for obtaining the enhanced data having a first reduction in noise content, and

wherein prior to the second data compression being performed on the enhanced data, a second noise reduction is performed by the at least one noise reduction process for obtaining the enhanced data having a second reduction in noise content;

wherein the first noise reduction and the second noise reduction reduce different amounts of noise;

wherein history information indicative of processing of the image or video data to yield the recorder processed data is obtained, wherein the history information includes compression description information related to which of the first and second data compressions were performed, and including enhancement description information indicative of the enhancement processes performed to obtain the enhanced data; and

(e) an interface to the network for transmitting the recorder processed data and the history information, via the network, to a predetermined network site;

wherein following the network transmission, the transmitted history information is accessed for decompressing the transmitted recorder processed data.

17. The system of claim 16, wherein the evaluator compares the network data with data indicative of a predetermined available bandwidth for the network.

18. The system of claim 16, wherein the enhancement processes includes a sharpening process for sharpening the image or video data, or a version derived therefrom, wherein the sharpening is decreased when the first data compression is performed, and the sharpening is increased when the second data compression is performed.

19. The system of claim 16 further including a controller for preventing or reducing one of an auto-exposure, an auto-white balance, and an auto-focus from being performed at the image or video recorder when the network data is indicative of a reduced available network bandwidth.

20. The system of claim 16, wherein the at least one noise reduction process includes Gaussian Blur or a high pass filter.

21. The system of claim 16 further including a controller for outputting information related to selecting a noise reduction process to perform as one of the first and second noise reductions, wherein when the image or video data includes video data, the noise reduction process is selected as being inversely related to the amount of data compression for performing.

22. The system of claim 16 further including a controller for outputting information related to selecting a noise reduction process to perform as one of the first and second noise reductions, wherein when the image or video data includes non-video data, the noise reduction process is selected as being directly related to the amount of data compression for performing.