Apparatus and methods for handling interactive applications in broadcast networks

• US 8,370,892 B2
• Filed: 05/17/2010
• Issued: 02/05/2013
• Est. Priority Date: 02/11/2003
• Status: Active Grant

• Current Assignee: Synamedia Limited
• Original Assignee: NDS Limited (Cisco Systems, Inc.)
• Inventors: Shelton, Ian R., Costello, Michael B, Dan, Ronnie Israel, Thexton, Nicholas, Walker, James Geoffrey, Ashby, Stuart P, Cormican, Neil, Chakkara, Rahul, Hall, Nicholas Ashton
• Primary Examiner(s): Saltarelli, Dominic D
• Application Number:
  US12/781,007
• Publication Number:
  US 20100225742A1
• Time in Patent Office: 995 Days
• Field of Search: 725/135, 725/136, 725/139, 463/40
• US Class Current: 725/136
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics

Abstract

An interactive method includes receiving background video including a multiplicity of video frames, at least one of the multiplicity of video frames include a plurality of sub-pictures, each of the plurality of sub-pictures representing an alternative background (1200), and switching between a first sub-picture of the plurality of sub-pictures and a second sub-picture of the plurality of sub-pictures (1210).

Claims

1. An interactive method comprising:

receiving, at a display device, background video comprising encoded video frames, at least one encoded video frame comprising a plurality of sub-pictures, each of the plurality of sub-pictures comprising a different alternative background representing a possible outcome of an interactive video application;

decoding said at least one encoded video frame to produce a decoded video frame;

upsampling a first sub-picture of the plurality of sub-pictures of said decoded video frame to a first full upsampled video frame and displaying the first full upsampled video frame as a first alternative background;

switching, at the display device, between the first sub-picture of the plurality of sub-pictures and a second sub-picture of the plurality of sub-pictures of said decoded video frame; and

upsampling the second sub-picture to a second full upsampled video frame after the switching and displaying said second full upsampled video frame as a second alternative background instead of the first full upsampled video frame,

wherein the switching between the first and second sub-pictures comprises dynamic switching, thereby creating a simulation of on-going motion and fluid change.

2. The method according to claim 1 and wherein the upsampling comprises upsampling for displaying on a display.

3. The method according to claim 1 and wherein the switching comprises seamlessly switching between the first sub-picture and the second sub-picture.

4. The method according to claim 1 and wherein at least one of the first sub-picture and the second sub-picture comprises a 3-dimensional (3D) scene.

5. The method according to claim 1 and also comprising:

displaying a display effect in response to the switching.

6. The method according to claim 1 and wherein the switching comprises switching between outputs of two audio decoders.

7. The method according to claim 1 and wherein the switching comprises switching between the first sub-picture and the second sub-picture in response to at least one of the following:

timing information;

user input;

an instruction from a headend;

an instruction from a broadcast source; and

an instruction from an interactive application.

8. The method according to claim 1 and also comprising, prior to the switching:

receiving object information at the display device, the object information comprising information determining an object; and

overlaying, at the display device, the object over the background video.

9. The method according to claim 8 and wherein the switching comprises switching between the first sub-picture and the second sub-picture in response to the object approaching or entering a border of the first sub-picture.

10. The method according to claim 8 and wherein the object comprises a game object.

11. The method according to claim 8 and wherein the background video and the object information are related to an interactive game application.

12. The method according to claim 1 and also comprising:

receiving assistance information at the display device, the assistance information comprising area assistance information related to the background video;

preparing, at the display device, graphics for display at a display location, the preparing being based, at least in part, on at least a portion of the assistance information associated with the display location; and

spatially synchronizing the prepared graphics with the background video.

13. The method according to claim 12 and wherein the at least a portion of the assistance information comprises non-boundary assistance information.

14. The method according to claim 13 and wherein the non-boundary assistance information comprises at least one of the following:

physics information;

lighting information;

at least one alpha mask;

information on at least one target zone;

information on at least one dead zone;

information regarding at least one game specific area;

control information; and

preferred track information.

15. The method according to claim 12 and wherein the preparing also comprises:

temporally synchronizing the prepared graphics with the background video.

16. The method according to claim 12 and also comprising:

displaying the prepared graphics and the background video in a synchronized form.

17. The method according to claim 12 and wherein said preparing also comprises the step of generating, at the display device, audio for rendering in association with the graphics.

18. The method according to claim 17 and wherein said generating comprises preparing the audio for rendering in association with the graphics based, at least in part, on the at least a portion of the assistance information associated with the display location.

19. The method according to claim 12 and wherein the generating also comprises synchronizing the prepared audio with the graphics and the background video.

20. The method according to claim 12 and wherein the background video, the assistance information, and the graphics are related to an interactive game application.

21. The method according to claim 1 and wherein the first sub-picture corresponds to a first perspective view of a scene and the second sub-picture corresponds to a second perspective view of the scene.

22. The method according to claim 21 and wherein the first perspective view is different from the second perspective view.

23. The method according to claim 8 and also comprising:

receiving, at the display device, distinguishing information distinguishing portions of the background video by at least one characteristic; and

determining, at the display device, whether an action is to be performed on the object based on the distinguishing information.

24. The method according to claim 23 and wherein the determining comprises determining whether the action is to be performed on the object when the object moves from a first one of the portions of the background video towards a second one of the portions of the background video distinguished from the first one of the portions of the background video by the at least one characteristic.

25. The method according to claim 24 and also comprising using the distinguishing information to trigger an audio signal when the object moves from the first one of the portions of the background video towards the second one of the portions of the background video.

26. The method according to claim 23 and wherein the at least one characteristic comprises at least one of the following:

a lighting characteristic;

a physics characteristic;

an alpha mask characteristic;

a target characteristic;

a dead zone characteristic;

a characteristic of a game specific area;

a preferred track characteristic; and

a track-control characteristic.

27. The method according to claim 23 and wherein the distinguishing information comprises at least one of the following:

at least one in-video mask; and

interactive application data.

28. The method according to claim 27 and wherein the at least one in-video mask distinguishes between at least one of the following:

boundary regions in the background video; and

collision regions in the background video.

29. The method according to claim 27 and wherein the at least one in-video mask comprises at least one of the following:

at least one alpha mask;

at least one target;

at least one dead zone;

tracking information; and

control information.

30. The method according to claim 23 and wherein the background video, the object information and the distinguishing information are related to an interactive game application, and the action comprises an action in the interactive game application.

31. The method according to claim 23 and wherein the step of receiving background video comprises receiving the background video from a broadcast transmission, the step of receiving object information comprises receiving the object information from a broadcast transmission, and the step of receiving distinguishing information comprises receiving the distinguishing information from one of the following:

a broadcast transmission; and

a storage device of a PVR.

32. The method according to claim 1 and also comprising:

receiving object information at the display device, the object information comprising information determining an object;

computing, at the display device, an adjustable sliding window display frame around the object;

upsampling a portion of the background video comprised in the adjustable sliding window display frame; and

overlaying, at the display device, the object over the upsampled portion of the background video.

35. The method according to claim 1, wherein the background video is received from one or more of the following:

a broadcast;

a video-on-demand (VOD) transmission; and

a near video-on-demand (NVOD) transmission, and

the background video is used directly, and not from storage local to the display device.

33. A display device comprising:

a receiver receiving background video comprising encoded video frames, at least one of the multiplicity of encoded video frame comprising a plurality of sub-pictures, each of the plurality of sub-pictures comprising a different alternative background representing a possible outcome of an interactive video application;

a decoder decoding said at least one encoded video frame to produce a decoded video frame;

an upsampler which upsamples a first sub-picture of the plurality of sub-pictures of said decoded video frame to a first full upsampled video frame;

a renderer rendering the first full upsampled video frame for display as a first alternative background; and

a processor switching between the first sub-picture of the plurality of sub-pictures and a second sub-picture of the plurality of sub-pictures of said decoded video frame;

the upsampler upsampling the second sub-picture to a second full upsampled video frame after the switching, and the renderer rendering the second full upsampled video frame for display as a second alternative background instead of the first full upsampled video frame,

wherein the switching between the first and second sub-pictures comprises dynamic switching, thereby creating a simulation of on-going motion and fluid change.

34. A display device comprising:

means for receiving background video comprising encoded video frames, at least one of the multiplicity of encoded video frame comprising a plurality of sub-pictures, each of the plurality of sub-pictures comprising a different alternative background representing a possible outcome of an interactive video application;

means for decoding said at least one encoded video frame to produce a decoded video frame;

means for upsampling a first sub-picture of the plurality of sub-pictures of said decoded video frame to a first full upsampled video frame;

means for rendering the first full upsampled video frame for display as a first alternative background; and

means for switching between the first sub-picture of the plurality of sub-pictures and a second sub-picture of the plurality of sub-pictures of said decoded video frame, and rendering the second sub-picture for display; and

the means for upsampling upsampling the second sub-picture to a second full upsampled video frame after the switching, and the means for rendering rendering the second full upsampled video frame for display as a second alternative background instead of the first full upsampled video frame,

wherein the switching between the first and second sub-pictures comprises dynamic switching, thereby creating a simulation of on-going motion and fluid change.

36. An interactive method comprising:

receiving, at a display device, background video comprising encoded video frames, at least one encoded video frame comprising a plurality of sub-pictures, each of the plurality of sub-pictures comprising a different alternative background representing a possible outcome of an interactive video application;

decoding said at least one encoded video frame to produce a decoded video frame;

upsampling a first sub-picture of the plurality of sub-pictures of said decoded video frame to a first full upsampled video frame and displaying the first full upsampled video frame as a first alternative background;

switching, at the display device, between the first sub-picture of the plurality of sub-pictures and a second sub-picture of the plurality of sub-pictures of said decoded video frame; and

upsampling the second sub-picture to a second full upsampled video frame after the switching and displaying said second full upsampled video frame as a second alternative background instead of the first full upsampled video frame,

wherein the switching comprises a seamless transition between the first and second sub-pictures, thereby creating an illusion of fluid and continuous movement.

37. A display device comprising:

a receiver receiving background video comprising encoded video frames, at least one of the multiplicity of encoded video frame comprising a plurality of sub-pictures, each of the plurality of sub-pictures comprising a different alternative background representing a possible outcome of an interactive video application;

a decoder decoding said at least one encoded video frame to produce a decoded video frame;

an upsampler which upsamples a first sub-picture of the plurality of sub-pictures of said decoded video frame to a first full upsampled video frame;

a renderer rendering the first full upsampled video frame for display as a first alternative background; and

a processor switching between the first sub-picture of the plurality of sub-pictures and a second sub-picture of the plurality of sub-pictures of said decoded video frame;

the upsampler upsampling the second sub-picture to a second full upsampled video frame after the switching, and the renderer rendering the second full upsampled video frame for display as a second alternative background instead of the first full upsampled video frame,

wherein the switching comprises a seamless transition between the first and second sub-pictures, thereby creating an illusion of fluid and continuous movement.

38. A display device comprising:

means for receiving background video comprising encoded video frames, at least one of the multiplicity of encoded video frame comprising a plurality of sub-pictures, each of the plurality of sub-pictures comprising a different alternative background representing a possible outcome of an interactive video application;

means for decoding said at least one encoded video frame to produce a decoded video frame;

means for upsampling a first sub-picture of the plurality of sub-pictures of said decoded video frame to a first full upsampled video frame;

means for rendering the first full upsampled video frame for display as a first alternative background; and

means for switching between the first sub-picture of the plurality of sub-pictures and a second sub-picture of the plurality of sub-pictures of said decoded video frame, and rendering the second sub-picture for display; and

the means for upsampling upsampling the second sub-picture to a second full upsampled video frame after the switching, and the means for rendering rendering the second full upsampled video frame for display as a second alternative background instead of the first full upsampled video frame,

wherein the switching comprises a seamless transition between the first and second sub-pictures, thereby creating an illusion of fluid and continuous movement.