REAL-TIME PROCESSING OF VIDEO STREAMS CAPTURED USING MOBILE DEVICES

• US 20170111532A1
• Filed: 12/30/2016
• Published: 04/20/2017
• Est. Priority Date: 01/12/2012
• Status: Abandoned Application

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Amtrup, Jan W., Ma, Jiyong, Thompson, Stephen Michael, Shustorovich, Alexander, Thrasher, Christopher W., Macciola, Anthony
• Application Number:
  US15/396,306
• Publication Number:
  US 20170111532A1
• Time in Patent Office: Days
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00251 with an apparatus for taking photographic images, e.g. a camera
• CPC Class Codes: H04N 1/00801 according to characteristics of the original
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3872 Repositioning or masking
• CPC Class Codes: H04N 1/3878 Skew detection or correction
• CPC Class Codes: H04N 1/40 Picture signal circuits H04N1/387 takes precedence
• CPC Class Codes: H04N 2201/0075 by a user operated remote control device, e.g. receiving instructions from a user via a computer terminal or mobile telephone handset
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/0096 Portable devices
• CPC Class Codes: H04N 2201/0458 Additional arrangements for improving or optimising scanning resolution or quality
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04W 4/02 Services making use of location information

Abstract

The presently disclosed inventive concepts encompass capturing video data using a mobile device, streaming the captured video data to a server for processing of the video data in real-time or near-real time, and providing the server'"'"'s processing result to the mobile device for additional analysis and/or processing of the captured video data, the processing result, or both. In one embodiment an image processing server is configured to: process, in real time, input streamed to the server from a mobile device, the input comprising one or more frames of digital video data; and output a result of processing the input to the mobile device. In another embodiment, a method includes capturing video data using a mobile device, streaming the video data to an image processing server, receiving a processing result from the server, and further processing the captured video data and/or the processing result using the mobile device.

Claims

1. An image processing server, comprising at least one processor, and logic configured, upon execution thereof by the processor, to cause the server to:

process, in real time, input streamed to the server from a mobile device, the input comprising one or more frames of digital video data; and

output a result of processing the input to the mobile device.

2. The image processing server as recited in claim 1, wherein the input comprises a plurality of the one or more frames of digital video data, and processing the input comprises at least one of:

transforming a representation of an object depicted in the plurality of frames of digital video data from a native object representation to an improved object representation;

determining information of interest regarding the object from the plurality of frames of digital video data;

classifying the object depicted in the plurality of frames of digital video data;

extracting the information of interest regarding the object from the plurality of frames of digital video data; and

validating the information of interest extracted from the plurality of frames of digital video data.

3. The image processing server as recited in claim 1, wherein the input further comprises at least one of:

identifying information corresponding to the mobile device;

user-defined processing parameters; and

location information corresponding to the mobile device, the one or more frames of digital video data, or both; and

wherein processing the input comprises processing the one or more frames of digital video data further based at least in part on the identifying information, the user-defined processing parameters, or the location information.

4. The image processing server as recited in claim 1, wherein the logic is configured to cause the server to process each one of the one or more frames of the digital video data streamed to the server in real-time or near real-time.

5. The image processing server as recited in claim 1, comprising calculating a consensus processing result from among a plurality of processing results each respectively corresponding to one of the one or more frames of digital video data.

6. The image processing server as recited in claim 1, wherein the input comprises a plurality of successive ones of the one or more frames of digital video data.

7. The image processing server as recited in claim 1, wherein the input comprises a plurality of alternating ones of the one or more frames of digital video data.

8. A computer-implemented method, comprising:

capturing, using a camera of a mobile device, a plurality of frames of digital video data;

streaming at least some of the plurality of frames of digital video data to an image processing server configured to process frames of digital video data in real-time;

receiving, from the image processing server, a processing result corresponding to some or all of the plurality of frames of digital video data streamed to the image processing server; and

further processing, using a processor of the mobile device, according to one or more predetermined additional processing operations, some or all of the plurality of frames of digital video data captured using the camera of the mobile device, the processing result received from the image processing server, or both.

9. The computer-implemented method as recited in claim 8, further comprising pre-processing the plurality of frames of digital video data prior to streaming the plurality of frames of digital video data to the image processing server.

10. The computer-implemented method as recited in claim 8, wherein the processing result comprises at least one of:

an improved representation of an object depicted in the plurality of frames of digital video data, wherein the improved representation is generated by transforming at least some of the plurality of frames of digital video data from a native object representation to the improved representation of the object;

a determination, from the plurality of frames of digital video data, of information of interest regarding the object;

a classification of the object;

extracted information of interest regarding the object;

a validation status of some or all of the extracted information of interest; and

feedback guiding the capture of additional frames of the digital video data.

11. The computer-implemented method as recited in claim 8, wherein the streaming comprises streaming a plurality of successive ones of the plurality of frames of digital video data to the image processing server.

12. The computer-implemented method as recited in claim 8, wherein the streaming comprises either:

streaming a plurality of alternate ones of the plurality of frames of digital video data to the image processing server;

streaming a limited number of the plurality of frames of digital video data to the image processing server;

or both.

13. The computer-implemented method as recited in claim 8, further comprising transmitting to the image processing server, in association with the streamed ones of the plurality of frames of digital video data, one or more of:

identifying information corresponding to the mobile device;

user-defined processing parameters; and

location information corresponding to the mobile device, the one or more frames of digital video data, or both.

14. The computer-implemented method as recited in claim 8, wherein the one or more predetermined additional processing operations are selected from:

capturing additional video frames depicting a tracked object in the video data, the capture of the additional video frames being characterized by a modification of capture conditions selected from illumination, capture angle, capture distance, and capture device movement;

classifying an object whose location within the video data is indicated by the processing result received from the image processing server;

extracting information of interest from select locations within one or more frames of the digital video data, the select locations being based on the processing result received from the image processing server and the extraction employing extraction conditions based on the processing result received from the server; and

validating information of interest extracted from the digital video data based on the processing result received from the image processing server, wherein the extracted information of interest are indicated in the processing result received from the image processing server.

15. The computer-implemented method as recited in claim 14, wherein the processing result received from the image processing server comprises a consensus processing result.

16. A computer program product, comprising a computer readable medium having embodied therewith computer readable program code configured, upon execution thereof, to cause a mobile device to perform operations comprising:

capturing, using a camera of the mobile device, a plurality of frames of digital video data;

streaming at least some of the plurality of frames of digital video data to an image processing server configured to process frames of digital video data in real-time;

receiving, from the image processing server, a processing result corresponding to some or all of the plurality of frames of digital video data streamed to the image processing server; and

further processing, using a processor of the mobile device and according to one or more predetermined additional processing operations, some or all of the plurality of frames of digital video data captured using the camera of the mobile device, the processing result received from the image processing server, or both.

17. The computer program product as recited in claim 16, further comprising computer readable program code configured, upon execution thereof, to cause a mobile device to pre-process the plurality of frames of digital video data prior to streaming the plurality of frames of digital video data to the image processing server.

18. The computer program product as recited in claim 16, wherein the processing result comprises at least one of:

an improved representation of an object depicted in the plurality of frames of digital video data, wherein the improved representation is generated by transforming at least some of the plurality of frames of digital video data from a native object representation to the improved representation of the object;

a determination, from the plurality of frames of digital video data, of information of interest regarding the object;

a classification of the object;

extracted information of interest regarding the object;

a validation status of some or all of the extracted information of interest; and

feedback guiding the capture of additional frames of the digital video data.

19. The computer program product as recited in claim 16, wherein the one or more predetermined additional processing operations are selected from:

capturing additional video frames depicting a tracked object in the video data, the capture of the additional video frames being characterized by a modification of capture conditions selected from illumination, capture angle, capture distance, and capture device movement;

classifying an object whose location within the video data is indicated by the processing result received from the image processing server;

extracting information of interest from select locations within one or more frames of the digital video data, the select locations being based on the processing result received from the image processing server and the extraction employing extraction conditions based on the processing result received from the server; and

validating information of interest extracted from the digital video data based on the processing result received from the image processing server, wherein the extracted information of interest are indicated in the processing result received from the image processing server.

20. The computer program product as recited in claim 19, wherein the processing result received from the image processing server comprises a consensus processing result.