Methods and apparatus for accessing peripheral content

• US 9,597,590 B2
• Filed: 03/28/2012
• Issued: 03/21/2017
• Est. Priority Date: 04/01/2011
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Bove, Victor Michael, Alfaro, Santiago, Lee, Vincent
• Primary Examiner(s): Mehmood, Jennifer
• Assistant Examiner(s): GUPTA, PARUL H
• Application Number:
  US13/433,020
• Publication Number:
  US 20120249424A1
• Time in Patent Office: 1,819 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/6661 for changing the position of the virtual camera

Abstract

In exemplary implementations of this invention, a main content feed is displayed on a main screen. A user may select one or more auxiliary feeds of content to display simultaneously on a second screen. The second screen is located on a handheld device. The user makes the selection by changing the orientation of the handheld device relative to the main screen. For example, the user may select which auxiliary feed to display by pointing the device at different areas that are around the periphery of the main screen. The handheld device includes one or more sensors for gathering data, and one or more processors for (a) processing the sensor data to calculate the orientation of the handheld device relative to the main screen and (b) based at least in part on that orientation, selecting which of the auxiliary feeds to display.

Claims

1. A method comprising, in combination:

(a) using one or more sensors to gather sensor data;

(b) using one or more computer processors

(i) to process the sensor data to make an estimation of an orientation of a handheld device, which orientation specifies a vertical angle and a horizontal angle of the handheld device relative to a particular location in the environment of the handheld device, and

(ii) to select, based at least in part on that estimation, a first channel of content, such that which particular channel is selected as the first channel depends on the orientation relative to the particular location; and

(c) visually displaying the first channel on a first screen, while simultaneously displaying a second, different channel of content on a second screen, the first screen being part of the handheld device and the second screen being separate from the handheld device;

wherein the first and second channels comprise television channels or are streamed, transmitted over the Internet or broadcast.

2. The method of claim 1, wherein:

(a) the particular location has been determined by prior calibration; and

(b) the orientation is not merely a tilt, which tilt does not require the handheld device to be pointed at the particular location.

3. The method of claim 1, wherein, during a prior calibration step, a reference compass heading of the second screen is determined by a magnetometer onboard the handheld device taking magnetometer readings while the handheld device is pointed at the second screen and by the one or more processors analyzing visual data to identify a visual tag that is at or adjacent to the second screen, the visual data being captured by a camera mounted in the handheld device.

4. The method of claim 1, wherein the method further comprises:

(a) a zoom out step in which the first screen displays a view that drops back revealing a plurality of windows; and

(b) a zoom in step in which the first screen displays a view that zooms in to center on an item selected by a user after the zoom out step.

5. The method of claim 1, wherein the first screen displays a feed and simultaneously displays an icon that shows where a camera supplying the feed is located.

6. The method of claim 1, wherein:

(a) the first screen displays a virtual view from the vantage point of a player of a video game; and

(b) the second screen displays a channel that is accessible to other players of the video game.

7. The method of claim 1, wherein:

(a) the first screen displays a first image and the second screen simultaneously displays a second image; and

(b) the first image extends the second image beyond the frame of the second screen, such that the first and second images depict non-overlapping portions of a single scene as seen from a single observation point.

8. The method of claim 1, wherein the vertical angle is tilt and the horizontal angle is compass heading.

9. The method of claim 1, wherein:

(a) the first screen displays a first image and the second screen simultaneously displays a second image; and

(b) the second image displays a main plot and the first image displays an additional subplot.

10. The method of claim 1, wherein the first channel comprises a television channel and the second channel comprises a different television channel.

11. The method of claim 1, wherein different orientations of the handheld device relative to the second display screen correspond to selections of different channels, respectively.

12. The method of claim 11, wherein each of the different orientations is along a line that intersects a different area, respectively, and each of the different areas is located outside of and adjacent to the perimeter of the second display screen.

13. The method of claim 11, wherein each of the different channels, respectively, displays different viewing angle.

14. The method of claim 11, wherein:

(a) at least a subset of the different channels each display a different portion of a specific scene, which specific scene is, or would be if real, visible from a single vantage point that is the same for all of the subset; and

(b) the different portions of the specific scene are each at a different viewing angle from the vantage point.

15. The method of claim 1, wherein at least one of the computer processors modifies an image from a feed to change a viewing angle or other apparent geometric property of the image.

16. A method comprising, in combination:

(a) visually displaying a first channel of content in a first area and simultaneously visually displaying a second channel of content in a second area, the first area being a screen that is part of a handheld device and the second area being separate from the handheld device, and the first and second channels being different from each other;

(b) using one or more sensors to gather sensor data; and

(c) using one or more processors

(i) to process the sensor data to determine an orientation of the handheld device, which orientation specifies a vertical angle and a horizontal angle of the handheld device relative to a particular location in the environment of the handheld device; and

(ii) to select which particular channel to display as the first channel, such that which particular channel is selected to display as the first channel depends on the orientation of the handheld device relative to the particular location;

wherein the first and second channels comprise television channels or are streamed, transmitted over the Internet or broadcast.

17. The method of claim 16, wherein the one or more particular channels are visually projected onto the second area and the second area is remote from the handheld device.

19. The method of claim 16, wherein selection of the first channel occurs when the handheld device is pointed towards a location that is outside of and adjacent to the second display screen.

20. The method of claim 16, wherein, during a prior calibration step, a reference compass heading of the second screen is determined by a magnetometer onboard the handheld device taking magnetometer readings while the handheld device is pointed at the second screen and by the one or more processors analyzing visual data to identify a visual tag that is at or adjacent to the second screen, the visual data being captured by a camera mounted in the handheld device.

18. A handheld device comprising, in combination:

(a) a first display screen for visually displaying a first channel;

(b) one or more sensors onboard the handheld device, which sensors include a magnetometer; and

(c) one or more processors adapted, or means for communicating with one or processors adapted,

(i) to analyze data gathered by the one or more sensors to determine orientation of the handheld device, and

(ii) to select which channel to visually display as the first channel, such that

(A) which channel is selected to display as the first channel depends on orientation of the handheld device, which orientation specifies a vertical angle and a horizontal angle of the handheld device relative to a particular location in the environment of the handheld device, which location is at or adjacent to a perimeter of a second visual display screen, the second screen being separate from the handheld device, and

(B) the first channel is displayed by the first display screen while the second display screen displays a second channel;

wherein the first and second channels comprise television channels or are streamed, transmitted over the Internet or broadcast.