Methods and apparatus for controlling a viewing position

• US 9,832,449 B2
• Filed: 01/28/2016
• Issued: 11/28/2017
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: Cole, David, Moss, Alan McKay
• Primary Examiner(s): Zhao, Daquan
• Application Number:
  US15/009,531
• Publication Number:
  US 20160227190A1
• Time in Patent Office: 670 Days
• Field of Search: 386248, 386239, 386224, 348 42, 348 47
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/158 Switching image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/368 for two or more viewers
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

Methods and apparatus for allowing a user to switch viewing positions and/or perspective while viewing an environment, e.g., as part of a 3D playback/viewing experience, are described. In various embodiments images of the environment are captured using cameras placed at multiple camera positions. During viewing a user can select which camera position he/she would like to experience the environment from. While experiencing the environment from the perspective of a first camera position the user may switch from the first to a second camera position by looking at the second position. A visual indication is provided to the user to indicate that the user can select the other camera position as his/her viewing position. If a user input indicates a desired viewing position change, a switch to the alternate viewing position is made and the user is presented with images captured from the perspective of the user selected alternative viewing position.

Claims

1. A method of presenting content corresponding to an environment, the method comprising:

displaying to a user content corresponding to a first camera position in said environment;

determining if the user is viewing another camera position in said environment, wherein determining if the user is viewing another camera position in said environment includes determining from i) a user head angle of rotation, ii) user head tilt angle or iii) both the user head angle of rotation and user head tilt angle, if said user is viewing said another camera position; and

monitoring, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position.

2. The method of claim 1, further comprising:

determining which of a plurality of alternative camera positions the user is viewing based on at least one of said user head angle of rotation or user head tilt angle.

3. The method of claim 1, further comprising:

when it is determined that the user is viewing another camera position in said environment, presenting the user an indication that the user can switch to viewing the environment from a perspective of said another camera position.

4. The method of claim 3, wherein presenting the user an indication that the user can switch to viewing the environment from a perspective of said another camera position includes providing a visual indication corresponding to the another camera position that a switch to viewing the environment from the perspective of said another camera position is possible.

5. The method of claim 1, wherein said environment includes an audience seating area and a performance area, said environment including at least two camera positions corresponding to different audience seating positions.

6. The method of claim 1, further comprising:

in response to said monitoring detecting user input indicating a user selected switch to said another camera position, switching from displaying to the user content corresponding to a first camera position in said environment to displaying to the user content corresponding to the another camera position in said environment.

7. The method of claim 1, wherein monitoring to detect user input indicating a user selected switch to said another camera position includes monitoring for user input on a control input interface while said user is viewing said another camera position.

8. The method of claim 7, wherein determining if the user is viewing another camera position in said environment includes:

monitoring a current head orientation of the user; and

determining if the head orientation indicates whether the user is viewing the another camera position.

9. The method of claim 8, further comprising:

determining, prior to presenting the user an indication that the user can switch to viewing the environment from a perspective of said another camera position, if the user was viewing said another camera position for a predetermined time; and

wherein said presenting the user said indication is contingent on determining that the user was viewing said another camera position for the predetermined time.

10. The method of claim 1, wherein said monitoring, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position includes:

monitoring for a user touch input indicating a user selected switch to said another camera position.

11. The method of claim 1, further comprising:

making a decision as to what action to take in response to detecting that a user is viewing another camera position in the environment based on an amount of time the user has been viewing the another camera position.

12. A method of presenting content corresponding to an environment, the method comprising:

displaying to a user content corresponding to a first camera position in said environment;

determining if the user is viewing another camera position in said environment;

when it is determined that the user is viewing another camera position in said environment, presenting the user an indication that the user can switch to viewing the environment from a perspective of said another camera position, wherein presenting the user an indication that the user can switch to viewing the environment from a perspective of said another camera position includes providing a visual indication corresponding to the another camera position that a switch to viewing the environment from the perspective of said another camera position is possible, wherein said visual indication is one of at least a change in color or shading at the another camera position; and

monitoring, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position.

13. A method of presenting content corresponding to an environment, the method comprising:

displaying to a user content corresponding to a first camera position in said environment;

determining if the user is viewing another camera position in said environment;

when it is determined that the user is viewing another camera position in said environment, presenting the user an indication that the user can switch to viewing the environment from a perspective of said another camera position, wherein presenting the user an indication that the user can switch to viewing the environment from a perspective of said another camera position includes providing a visual indication corresponding to the another camera position that a switch to viewing the environment from the perspective of said another camera position is possible, wherein said visual indication includes text in the area of said another camera position putting the user on notice of the potential to switch to viewing from the perspective of said another camera position; and

monitoring, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position.

14. A method of presenting content corresponding to an environment, the method comprising:

displaying to a user content corresponding to a first camera position in said environment;

determining if the user is viewing another camera position in said environment; and

monitoring, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position; and

in response to said monitoring detecting user input indicating a user selected switch to said another camera position, switching from displaying to the user content corresponding to a first camera position in said environment to displaying to the user content corresponding to the another camera position in said environment, wherein switching from displaying to the user content corresponding to a first camera position in said environment to displaying to the user content corresponding to the another camera position in said environment includes switching from receiving and displaying a first broadcast content stream corresponding to an output of a first stereoscopic camera pair located at said first camera position to receiving and displaying a second broadcast content stream corresponding to an output of a second stereoscopic camera pair located at said another camera position.

15. A method of presenting content corresponding to an environment, the method comprising:

displaying to a user content corresponding to a first camera position in said environment;

determining if the user is viewing another camera position in said environment; and

monitoring, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position, wherein monitoring to detect user input indicating a user selected switch to said another camera position includes monitoring for user input on a control input interface while said user is viewing said another camera position; and

if said monitoring, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position fails to detect said user input indicating a user selected switch within a predetermined time of the start of presenting the user the indication, eliminating the indication that the user can switch to viewing the environment from a perspective of said another camera position.

16. An interactive system for presenting content corresponding to an environment, comprising:

a display device;

a display controller configured to control displaying, to a user on said display device, of content corresponding to a first camera position in said environment;

a user view determination module configured to determine if the user is viewing another camera position in said environment, wherein said user view determination module is configured to determine from i) a user head angle of rotation, ii) user head tilt angle or iii) both the user head angle of rotation and user head tilt angle, if said user is viewing said another camera position; and

a monitoring module configured to monitor, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position.

17. The interactive system of claim 16, further comprising:

a camera position determination module configured to determine which of a plurality of alternative camera positions the user is viewing based on at least one of said user head angle of rotation or user head tilt angle.

18. The interactive system of claim 16, wherein said display controller is further configured to control presenting, on said display device, of an indication to the user that the user can switch to viewing the environment from a perspective of said another camera position when it is determined that the user is viewing another camera position in said environment.

19. The interactive system of claim 16, further comprising:

a switch controller configured to control the system to switch from displaying, on said display device, content corresponding to a first camera position in said environment to displaying to the user content corresponding to the another camera position in said environment, in response to said monitoring module detecting user input indicating a user selected switch to said another camera position.

20. The interactive system of claim 16, wherein said monitoring module is configured to monitor a sensor for one or more user taps indicating a user selected switch to said another camera position.

21. A non-transitory machine readable medium comprising processor executable instructions, which when executed by a processor, control a system to:

display to a user, on a display, content corresponding to a first camera position in an environment;

determine if the user is viewing another camera position in said environment, wherein determining if the user is viewing another camera position in said environment includes determining from i) a user head angle of rotation, ii) user head tilt angle or iii) both the user head angle of rotation and user head tilt angle, if said user is viewing said another camera position; and

monitor, while said user is viewing said another camera position, to detect user input indicating a user selected switch to said another camera position.