System for enabling a handheld device to capture video of an interactive application
First Claim
1. A method for enabling a handheld device to capture video of an interactive session of an interactive application presented on a main display, the method comprising:
- initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application;
determining an initial position and orientation of a handheld device operated by a spectator;
determining a current state of the interactive application based on the interactivity between the user and the interactive application;
tracking the position and orientation of the handheld device during the interactive session;
generating a spectator video stream of the interactive session based on the current state of the interactive application and the tracked position and orientation of the handheld device;
rendering the spectator video stream on a handheld display of the handheld device wherein the position and orientation of the handheld device define a position and orientation of a virtual viewpoint within a virtual space defined by the interactive application; and
wherein the spectator video stream includes images of the virtual space captured from the perspective of the virtual viewpoint.
2 Assignments
0 Petitions
Accused Products
Abstract
Methods and systems for enabling a handheld device to capture video of an interactive session of an interactive application presented on a main display are provided. An interactive session of the interactive application defines interactivity between a user and the interactive application. An initial position and orientation of a handheld device operated by a spectator are determined. A current state of the interactive application based on the interactivity between the user and the interactive application is determined. The position and orientation of the handheld device are tracked during the interactive session. A spectator video stream of the interactive session based on the current state of the interactive application and the tracked position and orientation of the handheld device is generated. The spectator video stream is rendered on a handheld display of the handheld device.
24 Citations
31 Claims
-
1. A method for enabling a handheld device to capture video of an interactive session of an interactive application presented on a main display, the method comprising:
-
initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application; determining an initial position and orientation of a handheld device operated by a spectator; determining a current state of the interactive application based on the interactivity between the user and the interactive application; tracking the position and orientation of the handheld device during the interactive session; generating a spectator video stream of the interactive session based on the current state of the interactive application and the tracked position and orientation of the handheld device; rendering the spectator video stream on a handheld display of the handheld device wherein the position and orientation of the handheld device define a position and orientation of a virtual viewpoint within a virtual space defined by the interactive application; and wherein the spectator video stream includes images of the virtual space captured from the perspective of the virtual viewpoint. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for enabling a handheld device to capture video of an interactive session of an interactive application presented on a main display, the method comprising:
-
initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application; determining an initial position and orientation of a handheld device operated by a spectator; determining a current state of the interactive application based on the interactivity between the user and the interactive application; tracking the position and orientation of the handheld device during the interactive session; generating a spectator video stream of the interactive session based on the current state of the interactive application and the tracked position and orientation of the handheld device; rendering the spectator video stream on a handheld display of the handheld device; determining application state data based on the current state of the interactive application; transmitting the application state data to the handheld device; wherein the spectator video stream is generated at the handheld device based on the application state data and the tracked position and orientation of the handheld device.
-
-
11. A method for enabling a handheld device to capture video of an interactive session of an interactive application presented on a main display, the method comprising:
-
initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application; determining an initial position and orientation of a handheld device operated by a spectator; determining a current state of the interactive application based on the interactivity between the user and the interactive application; tracking the position and orientation of the handheld device during the interactive session; generating a spectator video stream of the interactive session based on the current state of the interactive application and the tracked position and orientation of the handheld device; rendering the spectator video stream on a handheld display of the handheld device; capturing an environmental video stream of an interactive environment in which the interactivity between the user and the interactive application occurs; wherein the spectator video stream is generated based on the environmental video stream; wherein the spectator video stream is generated by augmenting the environmental video stream with a virtual element; wherein the generating of the spectator video stream includes, detecting the user within the environmental video stream, and replacing at least a portion of the detected user in the environmental video stream with a virtual element. - View Dependent Claims (12)
-
-
13. A method for enabling a handheld device to capture video of an interactive session of an interactive application presented on a main display, the method comprising:
-
initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application; determining an initial position and orientation of a handheld device operated by a spectator; determining a current state of the interactive application based on the interactivity between the user and the interactive application; tracking the position and orientation of the handheld device during the interactive session; generating a spectator video stream of the interactive session based on the current state of the interactive application and the tracked position and orientation of the handheld device; rendering the spectator video stream on a handheld display of the handheld device; determining an initial position of the user; tracking the position of the user during the interactive session; wherein the spectator video stream is generated based on the position and orientation of the handheld device relative to the position of the user; mapping an object within a virtual environment of the interactive application to the position of the user; wherein the position and orientation of the handheld device relative to the position of the user define a position and orientation of a virtual viewpoint within a virtual space defined by the interactive application; and wherein the spectator video stream includes images of the virtual space captured from the perspective of the virtual viewpoint, the object being included in the images of the spectator video stream when the handheld device is oriented towards the position of the user. - View Dependent Claims (14, 15, 16)
-
-
17. A system for capturing video of an interactive session of an interactive application presented on a main display, the system comprising:
-
(a) a device for presenting the interactive application, the device including, (i) a session module for initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application, (ii) a handheld device module for determining and tracking a position and orientation of a handheld device operated by a spectator during the interactive session, (iii) a spectator video stream module for generating a spectator video stream of the interactive session based on a current state of the interactive application and the tracked position and orientation of the handheld device; and (b) a handheld device, the handheld device including a handheld display for rendering the spectator video stream; wherein the position and orientation of the handheld device define a position and orientation of a virtual viewpoint within a virtual space defined by the interactive application; and wherein the spectator video stream includes images of the virtual space captured from the perspective of the virtual viewpoint. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24, 25)
-
-
26. A system for capturing video of an interactive session of an interactive application presented on a main display, the system comprising:
-
(a) a device for presenting the interactive application, the device including, (i) a session module for initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application, (ii) a handheld device module for determining and tracking a position and orientation of a handheld device operated by a spectator during the interactive session, (iii) a spectator video stream module for generating a spectator video stream of the interactive session based on a current state of the interactive application and the tracked position and orientation of the handheld device; and (b) a handheld device, the handheld device including a handheld display for rendering the spectator video stream; wherein the handheld device further includes an image capture device for capturing an environmental video stream of an interactive environment in which the interactivity between the user and the interactive application occurs; and wherein the spectator video stream module generates the spectator video stream based on the environmental video stream; wherein the spectator video stream module generates the spectator video stream by augmenting the environmental video stream with a virtual element; wherein the spectator video stream module generates the spectator video stream by detecting the user within the environmental video stream, and replacing at least a portion of the detected user in the environmental video stream with a virtual element. - View Dependent Claims (27)
-
-
28. A system for capturing video of an interactive session of an interactive application presented on a main display, the system comprising:
-
(a) a device for presenting the interactive application, the device including, (i) a session module for initiating the interactive session of the interactive application, the interactive session defining interactivity between a user and the interactive application, (ii) a handheld device module for determining and tracking a position and orientation of a handheld device operated by a spectator during the interactive session, (iii) a spectator video stream module for generating a spectator video stream of the interactive session based on a current state of the interactive application and the tracked position and orientation of the handheld device; and (b) a handheld device, the handheld device including a handheld display for rendering the spectator video stream; a user tracking module, the user tracking module determining an initial position of the user and tracking the position of the user during the interactive session; and wherein the spectator video stream module generates the spectator video stream based on the position and orientation of the handheld device relative to the position of the user; wherein the interactive application maps an object within a virtual environment of the interactive application to the position of the user; wherein the position and orientation of the handheld device relative to the position of the user define a position and orientation of a virtual viewpoint within a virtual space defined by the interactive application; and wherein the spectator video stream includes images of the virtual space captured from the perspective of the virtual viewpoint, the object being included in the images of the spectator video stream when the handheld device is oriented towards the position of the user. - View Dependent Claims (29, 30, 31)
-
Specification