METHOD AND APPARATUS FOR VIDEO STREAMING

• US 20140104440A1
• Filed: 10/12/2012
• Published: 04/17/2014
• Est. Priority Date: 10/12/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sampathkumaran, Sriram, Neumann, Helmut, Yam, Eric, Hong, Seungwook, Agnihotri, Tanmay
• Granted Patent: US 9,001,216 B2
• Time in Patent Office: Days
• US Class Current: 348/207.1
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4415 using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries

Abstract

An apparatus and method are provided for a client application to provide a video stream as well as data regarding the video stream to an output device, where the data is generated from a process performed at a layer lower than the client application layer. By this, multiple client applications can simultaneously utilize the data generated by the lower-level process while at least one client application can effect streaming of the video stream to an output device.

Claims

1. A method comprising:

receiving at an arbitration module of a computing device from at least one client application at least a request for a video stream, a request for results of at least one process performed on the video stream, and a designated location within a video plane at which to stream the video stream;

receiving the video stream at the arbitration module;

performing by the arbitration module the at least one process on the video stream to generate the results of the at least one process;

outputting from the arbitration module the video stream, the results of the at least one process, and the designated location within the video plane at which to stream the video stream;

receiving at a video pipeline element at least the video stream and the designated location within the video plane at which to stream the video stream;

streaming by the video pipeline element at least the video stream to the video plane at the designated location; and

providing to an output device by the at least one client application at least additional visual information generated by the at least one client application and access to the video plane and access to the results of the at least one process performed on the video stream.

2. The method of claim 1 wherein performing by the arbitration module the at least one process on the video stream to generate the results of the at least one process further comprises adding at least one visual representation of the results to the video stream prior to outputting the video stream from the arbitration module, and wherein outputting from the arbitration module the results of the at least one process further comprises outputting the video stream with the added at least one visual representation of the results.

3. The method of claim 1 further comprising:

receiving at the at least one client application from the arbitration module the results of the at least one process;

and wherein providing to the output device by the at least one client application at least the additional visual information generated by the at least one client application and access to the results of the at least one process performed on the video stream further comprises generating by the at least one client application at least one visual representation of the results of the at least one process performed on the video stream and providing the at least one visual representation of the results of the at least one process performed on the video stream to the output device.

4. The method of claim 1 wherein providing to the output device by the at least one client application at least the additional visual information generated by the at least one client application further comprises providing at least one visual image residing in a graphics plane existing on top of the video plane, and wherein the at least one visual image comprises at least one void corresponding to the designated location within the video plane at which the stream the video stream.

5. The method of claim 1 wherein the video pipeline element comprises a gStreamer-compatible video pipeline element in a gStreamer-compatible video pipeline.

6. The method of claim 1 wherein the video pipeline element comprises an OpenMAX™

-compatible video pipeline element in an OpenMAX™

-compatible video pipeline.

7. The method of claim 1 wherein the computing device utilizes an Android™

-compatible operating system.

8. The method of claim 1 wherein performing by the arbitration module the at least one process on the video stream to generate the results of the at least one process further comprises a natural inputs engine of the arbitration module processes the video stream to determine at least one natural input present in the video stream.

9. The method of claim 8 wherein the natural inputs engine further comprises a face detection engine configured to determine at least one aspect relating to at least one face present in the video stream.

10. The method of claim 1 wherein receiving the video stream at the arbitration module further comprises receiving the video stream from a camera device.

11. An apparatus comprising:

an arbitration module configured to;

receive from at least one client application at least a request for a video stream, a request for at least one result of at least one process performed on the video stream, and at least one designated location within a video plane at which to stream the video stream;

receive the video stream;

perform the at least one process on the video stream to generate the at least one result of the at least one process; and

output the video stream, the at least one result of the at least one process, and the designated location within the video plane at which to stream the video stream;

a video pipeline element coupled to the arbitration module, the video pipeline element configured to;

receive at least the video stream and the designated location within the video plane at which to stream the video stream; and

stream at least the video stream to the video plane at the designated location;

the at least one client application coupled to the arbitration module, the at least one client application configured to;

generate additional visual information;

provide to an output device at least the additional visual information; and

provide to the output device access to the video plane and access to the results of the at least one process performed on the video stream;

the output device coupled to the video plane and the at least one client application; and

the video plane coupled to the video pipeline element and the output device.

12. The apparatus of claim 11 further comprising:

at least one camera coupled to the arbitration module, the at least one camera configured to generate the video stream.

13. The apparatus of claim 11 further comprising:

an Android™

-compatible operating system in which the at least one client application operates.

14. The apparatus of claim 11 wherein the video pipeline element further comprises a gStreamer-compatible video pipeline element in a gStreamer-compatible video pipeline.

15. The apparatus of claim 11 wherein the video pipeline element further comprises an OpenMAX™

-compatible video pipeline element in an OpenMAX™

-compatible video pipeline.

16. The apparatus of claim 11 wherein the arbitration module is further configured to:

perform the at least one process on the video stream to generate the at least one result by adding at least one visual representation of the at least one result to the video stream prior to outputting the video stream; and

output the at least one result of the at least one process by outputting the video stream with the added at least one visual representation of the at least one result.

17. The apparatus of claim 11 wherein the at least one client application is further configured to:

receive from the arbitration module the at least one result of the at least one process;

generate additional visual information by generating at least one visual representation of the at least one result of the at least one process performed on the video stream; and

provide to the output device at least the additional visual information and access to the at least one result of the at least one process performed on the video stream by providing the at least one visual representation of the at least one result of the at least one process performed on the video stream to the output device.

18. The apparatus of claim 11 wherein the at least one client application is further configured to:

generate the additional visual information by generating at least one visual image residing in a graphics plane existing on top of the video plane, wherein the at least one visual image comprises at least one void corresponding to the designated location within the video plane at which the stream the video stream.

19. The apparatus of claim 11 wherein the arbitration module further comprises a natural inputs engine configured to perform the at least one process on the video stream to generate the at least one result of the at least one process by processing the video stream to determine at least one natural input present in the video stream.

20. The apparatus of claim 19 wherein the natural inputs engine further comprises a face detection engine configured to determine at least one aspect relating to at least one face present in the video stream.