Supplemental video content on a mobile device

• US 9,814,977 B2
• Filed: 07/13/2010
• Issued: 11/14/2017
• Est. Priority Date: 07/13/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Stafford, Jeffrey R., Zhu, Yunpeng, Osman, Steven
• Primary Examiner(s): Flynn, Nathan
• Assistant Examiner(s): Huerta, Alexander Q
• Application Number:
  US12/835,657
• Publication Number:
  US 20120017236A1
• Time in Patent Office: 2,681 Days
• Field of Search: 725 32, 725 37- 61, 725112-113
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/27 characterised by a large display in a public venue, e.g. in a movie theatre, stadium or game arena
• CPC Class Codes: A63F 13/323 between game devices with different hardware characteristics, e.g. hand-held game devices connectable to game consoles or arcade machines
• CPC Class Codes: A63F 13/332 using wireless networks, e.g. cellular phone networks
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/843 involving concurrently two or more players on the same game device, e.g. requiring the use of a plurality of controllers or of a specific view of game data for each player
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8088 involving concurrently several players in a non-networked game, e.g. on the same game console
• CPC Class Codes: G07F 17/3202 Hardware aspects of a gaming system, e.g. components, construction, architecture thereof
• CPC Class Codes: G07F 17/3211 Display means
• CPC Class Codes: G07F 17/323 wherein the player is informed, e.g. advertisements, odds, instructions
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/4223 : Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content

Abstract

Methods, systems, devices, and software are described for providing supplemental content for display on a mobile device. In one embodiment, a system includes a video display that displays video content. A mobile device with an integrated display may track the video display, with a video camera or other methods, and identify the video content in the tracked video display. The mobile device may access supplemental content related to the identified video content, and display the accessed supplemental content on the integrated display.

Claims

1. A system for augmented video, the system comprising:

a video source configured to;

provide video content to a video display, the video content including video segments and a time variable marker anchored to a portion of the video content, wherein the time variable marker comprises a presented pattern in the video content, wherein;

the presented pattern changes over time,

a presented change to the presented pattern corresponds to a time code, identifies the video content, identifies a progression of the video content based on the time code, and does not identify supplemental content, and

the progression indicates a temporal position of a video segment relative to another video segment within the video content; and

a mobile device with an integrated display, the mobile device configured to;

track the video content on the video display using the time variable marker as an anchor;

determine the temporal position of the video segment from the presented change to the presented pattern of the time variable marker;

access, based on the video content and the temporal position, the supplemental content that corresponds to the video segment; and

display the supplemental content on the integrated display, wherein the display of the supplemental content is time synchronized with the display of the video segment based on the presented change to the presented pattern of the time variable marker.

2. The system of claim 1, wherein the mobile device further comprises an integrated video camera configured to capture the video content for display with the supplemental content on the integrated display.

3. The system of claim 2, wherein the mobile device identifies the video content by capturing the time variable marker using the integrated video camera.

4. The system of claim 1, wherein the supplemental content comprises content requested by a user with the mobile device.

5. The system of claim 1, wherein the video source comprises a video game console, the video content is a video game, and the supplemental content comprises first player-specific information for the video game for a user of the mobile device.

6. The system of claim 5, further comprising a second mobile device with an integrated display, the second mobile device configured to:

identify the video content displayed on the video display;

track the video content on the video display using the time variable marker as an anchor;

determine a game status of the video game based on the time variable marker;

access second player-specific information for the video game for a user of the second mobile device based on the game status, the second player-specific information different from the first information; and

display the second player-specific information on the integrated display of the second mobile device.

7. The system of claim 1, wherein the mobile device is configured to track the video display by identifying a position and orientation of the video content relative to the mobile device.

8. The system of claim 1, further comprising:

a supplemental content datastore, remote from the video display and mobile device and in communication with the mobile device, configured to store the supplemental content, wherein the mobile device accesses the supplemental content from the supplemental content datastore.

9. The system of claim 1, wherein the mobile device is remote from the video display, and wherein the supplemental content is accessed based on a view angle between the mobile device and the video display.

10. The system of claim 1, wherein the mobile device and a second mobile device are remote from the video display, wherein a view angle between the mobile device and the video display identifies a user profile associated with a user of the mobile device, wherein a second view angle between the mobile device and the video display identifies a second user profile associated with a second user of the second mobile device, wherein the supplemental content is accessed based on the user profile being identified according to the view angle, and wherein the supplemental content is different from second supplemental content that is accessed based on the second user profile.

11. The system of claim 1, wherein the mobile device has a stored low-resolution version of the video content, and wherein the display of the supplemental content is time synchronized with the display of the video content by at least performing image matching based on a hash of the stored low-resolution version of the video content.

12. The system of claim 1, wherein the time variable marker is presented at a frequency not perceptible to the human eye, and wherein the time variable marker is identified based on an application of a Fourier transform to the video segment.

13. The system of claim 1, wherein the time variable marker further comprises an audio marker, and wherein the temporal position is identified based on the audio marker.

14. A computer-implemented method for augmented video on a mobile device, the method comprising:

identifying video content displayed on a video display, wherein the video content comprises video segments and a time variable marker anchored to a portion of the video content, wherein the time variable marker comprises a presented pattern in the video content, wherein;

the presented pattern changes over time,

a presented change to the presented pattern corresponds to a time code identifies the video content, identifies a progression of the video content based on the time code, and does not identify supplemental content, and

the progression indicates a temporal position of a video segment relative to another video segment within the video content;

identifying the temporal position in the identified video content using the time variable marker;

tracking the video content on the video display using the time variable marker as an anchor;

accessing the supplemental content based on the video content and the temporal position, the supplemental content corresponding to the video segment; and

displaying the supplemental content on the mobile device, wherein the display of the supplemental content is time synchronized with the display of the video segment based on the presented change to the presented pattern of the time variable marker.

15. The computer-implemented method of claim 14, further comprising:

displaying the video content on the mobile device;

scaling and positioning the supplemental content on the mobile device to avoid overlaying the video content, wherein the supplemental content includes at least one of text, animations, graphic enhancements, advertisements, hyperlinks, and other information.

16. The method of claim 14, wherein the video content is available from a content source and is presented on a display according to a display rate, wherein the mobile device is configured to identify at time intervals the presented change to the presented pattern of the time variable marker on the display, wherein the time intervals are known to the mobile device and correspond to predefined time intervals that are different from the display rate and that the content source uses to update the time variable marker in the video presentation on the display.

17. A computer-implemented method for augmented video, the method comprising:

identifying video game content displayed on a video display, wherein the video game content comprises video game segments and a time variable marker anchored to a portion of the video game content, wherein the time variable marker comprises a presented pattern in the video game content, wherein;

the presented pattern changes over time,

a presented change to the presented pattern corresponds to a time code, identifies the video game content, identifies a progression of the video game content based on the time code, and does not identify player-specific data,

the progression indicates a temporal position of a video game segment relative to another video game segment within the video game content, and

the video display is tracked by a first mobile device associated with a first player using the time variable marker as an anchor and is tracked by a second mobile device associated with a second player using the time variable marker as an anchor;

accessing first player-specific video data based on the time variable marker in the identified video game content such that the first player-specific video data is time synchronized with the video game segment based on the temporal position, wherein the accessed first player-specific video data is supplemental to and associated with the video game segment, for the first player;

transmitting the accessed first player-specific video data directed to the first mobile device for display based on a time synchronization from the presented change to the presented pattern of the time variable marker;

accessing second player-specific video data based on the time variable marker in the identified video game content such that the second player-specific video data is time synchronized with the video game segment based on the temporal position, wherein the accessed second player-specific video data is supplemental to and associated with the video game segment, for the second player; and

transmitting the accessed second player-specific video data directed to the second mobile device for display based on the time synchronization from the presented change to the presented pattern of the time variable marker.

18. The computer-implemented method of claim 17, further comprising;

generating the first player-specific video data comprising text to be displayed on the first mobile device, the text to be positioned adjacent to captured images of the video game content on the first mobile device; and

generating the second player-specific video data comprising graphics to be displayed on the second mobile device, the graphics to be positioned as overlay content on captured images of the video game content on the second mobile device.

19. The computer-implemented method of claim 18, wherein,

the first player-specific video data is information available to the first player and unavailable to the second player;

and

the second player-specific video data is information available to the first player and the second player.

20. The computer-implemented method of claim 17, wherein,

accessing first player-specific video data comprises accessing secret video data available to the first player and unavailable to the second player;

and

accessing second player-specific video data comprises accessing a view requested by the second player.