Media stream mixing

• US 7,907,638 B2
• Filed: 06/09/2004
• Issued: 03/15/2011
• Est. Priority Date: 06/19/2003
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Inventors: Norhammar, Björn, Lindquist, Tobias
• Primary Examiner(s): Ngo; Ricky
• Assistant Examiner(s): Liu; Ben H
• Application Number:
  US10/561,156
• Publication Number:
  US 20060149399A1
• Time in Patent Office: 2,470 Days
• Field of Search: 370/487, 370/493, 370/494, 370/495, 370/535, 370/538
• US Class Current: 370/493
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/44 Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

The present invention is directed towards a method and a portable communication device (200) for forming a real time output media stream by mixing a first real time media stream with a second media stream. Provision of mixing of media streams provides a number of attractive functions, for instance. This invention concerns how to increase the usage of a mobile VT application and how to be able to handle integrity issues when used in a portable communication device.

Claims

1. A method of forming an output media stream to be transmitted during a communication session from a portable communication device wherein said output media stream comprises signals of a first media type, the method comprising:

generating in real time a first media stream in the portable communication device,

combining in real time the first media stream with a second media stream to form the output media stream wherein combining comprises superposing, within the portable communication device, signals of the first media type from the first media stream on signals of the first media type from the second media stream to produce the output media stream, wherein the output media stream comprises portions of the first and second media streams which are configured to be presented in a substantially simultaneous time period, and

transmitting said output media stream,

wherein the communication session includes a video telephony session,

wherein the second media stream includes a prerecorded media stream that is user selectable, and

wherein at least one of generating and/or combining is dependent on input data from a user of said portable communication device.

2. Method according to claim 1, wherein said output media stream comprises signals of a second media type.

3. Method according to claim 1, further comprising:

establishing a connection with another device.

4. Method according to claim 3, wherein said connection is a circuit-switched connection.

5. Method according to claim 1, wherein combining comprises combining signals of the first media type from the first media stream with signals of a second media type from the second media stream.

6. Method according to claim 1, wherein combining further comprises combining signals of a second media type from the first media stream with the signals from the second media stream.

7. Method according to claim 1, wherein combining further comprises combining signals from the first media stream with signals of the second media type from the second media stream.

8. Method according to claim 7, wherein combining further comprises combining signals of the second media type from the first media stream with signals from the second media stream.

9. Method according to claim 8, wherein combining further comprises:

delaying, prior to combining, signals of one media type of the second media stream in relation to the other media type of signals of the same stream to provide synchronized signals from the second media stream within the output media stream.

10. Method according to claim 7, wherein combining further comprises independently combining signals of the first media type and signals of the second media type.

11. Method according to claim 6, wherein combining further comprises delaying signals of one media type within the output media stream, in relation to the other media type of signals of the same stream to provide synchronized signals from the first media stream within the output media stream.

12. Method according to claim 1, wherein the signals of the first media type are audio signals so that the signals of the first media type from the first media stream comprise first audio signals and the signals of the first media type from the second media stream comprise second audio signals, wherein combining further comprises superposing the first and second audio signals of the first and second media streams.

13. Method according to claim 12, wherein superposing comprises weighting properties of the audio signals from the first media stream and the second media stream.

14. Method according to claim 1, wherein the signals of the first media type are image signals so that the signals of the first media type from the first media stream comprise first image signals and the signals of the first media type from the second media stream comprise second image signals, wherein combining further comprises blending the first and second image signals of the first and second media streams.

15. Method according to claim 14, wherein blending comprises weighting properties of the image signals from the first media stream and the second media stream.

16. Method according to claim 13, wherein weighting properties includes varying the proportion of signals from the first media stream in relation to the proportion of signals from the second media stream.

17. Method according to claim 16, wherein weighting properties is dependent on input data of a user of said portable communication device.

18. Method according to claim 16, wherein varying said proportions comprises varying of each proportion within the range between 0 and 100%.

27. A Method according to claim 1, wherein generating the first media stream comprises generating real time image signals from a first camera in the portable communication device, further comprising generating the second media stream by generating real time image signals from a second camera in the portable communication device, wherein the first camera is arranged to capture image signals in a first direction and the second camera is arranged to capture image signals in a second direction that is different from the first direction.

19. Portable communication device configured to form an output media stream to be transmitted during a communication session from said portable communication device, wherein said output media stream comprises signals of a first media type, said portable communication device comprising:

at least one generating unit configured to generate a first media stream,

a first combining unit, connected to said generating unit, wherein the first combining unit is configured to combine in real time the first media stream with a second media stream and wherein the first combining unit is configured to superpose, within the portable communication device, signals of the first media type from the first media stream on signals of the first media type from the second media stream to form the output media stream, wherein the output media stream comprises portions of the first and second media streams which are configured to be presented in a substantially simultaneous time period,

a control unit configured to control the generating unit and the combining unit, in dependence of user input, and

a transmitter configured to transmit said output media stream,

wherein the communication session includes a video telephony session,

wherein the second media stream includes a prerecorded media stream that is user selectable, and

wherein at least one of the generating unit and/or the combining unit is configured to function in response to input data from a user of said portable communication device.

20. Portable communication device according to claim 19 configured to form an output media stream to be transmitted during a communication session from said portable communication device, wherein the first combining unit is configured to combine signals of the first media type of both the first and the second media streams, wherein the output media stream comprises signals of the first media type and a second media type, wherein the portable device further comprises:

a second combining unit configured to combine signals of the second media type of the first media stream and signals of the second media type of the second media stream by using the second combining unit.

21. Portable communication device according to claim 19, further comprising:

a memory unit configured to provide storage for the second media stream.

22. Portable communication device according to claim 19, further comprising:

a user input interface configured to provide user input.

23. Portable communication device according to claim 20, wherein said device further comprises:

a multiplexing unit configured to provide synchronization of signals of one media type from the first media stream in relation to signals of the other media type from the same first media stream, within the output media stream.

24. Portable communication device according to claim 20, further comprising:

a delaying unit configured to provide synchronized signals within the output media stream.

25. Portable communication device according to claim 24, where the delaying unit provides synchronization of signals from the second media stream, prior to combining with the first stream.

26. Portable communication device according to claim 25, where the delaying unit provides synchronization of signals of one media type in relation to signals of the other media type from the same second media stream.

28. A portable communication device according to claim 19, wherein the a least one generating unit comprises a first generating unit that is configured to generate the first media stream that includes a first image signal corresponding to a first direction from the portable communication device, further comprising a second generating unit that is configured to generate the second media stream that includes a second image signal corresponding to a second direction from the portable communication device that is different from the first direction.