Zoom related methods and apparatus

• US 9,955,147 B2
• Filed: 01/28/2016
• Issued: 04/24/2018
• Est. Priority Date: 01/28/2015
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: Cole, David, Moss, Alan McKay
• Primary Examiner(s): Zhao, Daquan
• Application Number:
  US15/009,320
• Publication Number:
  US 20160219262A1
• Time in Patent Office: 817 Days
• Field of Search: 348 42, 348 47, 348 51, 348 53, 34824099, 3482401, 3482402, 3482403, 386242, 386248, 386239
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

Methods and apparatus for implementing user controlled zoom operations during a stereoscopic, e.g., 3D, presentation are described. While viewing a 3D presentation of a scene environment, a user may switch to a zoom mode allowing the user to zoom in on a particular portion of the environment being displayed. In order to maintain the effect of being physically present at the event, and also to reduce the risk of making the user sick from sudden non-real world like changes in views of the environment, the user in response to initiating a zoom mode of operation is presented with a view which is the same or similar to that which might be expected as the result of looking through a pair of binoculars. In some embodiments the restriction in view is achieved by applying masks to enlarged version of left and right eye views to be displayed.

Claims

1. A method of operating an interactive system, the method comprising:

displaying a scene on a first portion of a display;

detecting user input indicating a zoom mode of operation; and

in response to the detected user input indicating the zoom operation, simulating viewing of a portion of said scene through binoculars, said simulating viewing of the portion of the scene through binoculars including using a second portion of said display to display a zoomed version of said portion of said scene while a third portion of said display is blocked from displaying a portion of said scene, said second portion of the display being smaller than said first portion of the display, said second and third portions being different portions of said first portion of the display.

2. The method of claim 1, wherein displaying a scene on the display includes:

outputting stereoscopic image pairs on the display to provide a 3D presentation of said scene, said scene corresponding to an event, each stereoscopic image pair including a left eye view and a right eye view.

3. The method of claim 1, wherein simulating viewing of the portion of the scene through binoculars includes displaying a mask on said third portion of the display.

4. A method of operating an interactive system, the method comprising:

displaying a scene on a display;

detecting user input indicating a zoom mode of operation; and

in response to the detected user input indicating the zoom operation, simulating viewing of a portion of said scene through binoculars, wherein simulating viewing of a portion of said scene includes;

performing a zoom operation on a first left eye image; and

applying a mask to zoomed first left eye image simulating a viewing constraint corresponding to viewing through binoculars to generate a masked zoomed left eye image.

5. The method of claim 4, further comprising:

displaying the masked zoomed left eye image.

6. The method of claim 5, wherein simulating viewing of a portion of said scene further includes:

performing a zoom operation on a first right eye image corresponding to the first left eye image;

applying a mask to the zoomed right eye image simulating a viewing constraint corresponding to viewing through binoculars to generate a masked zoomed right eye image; and

displaying the masked zoomed right eye image while said masked zoomed left eye image is being displayed.

7. The method of claim 6, further comprising:

applying zooming and masking operations on additional left and right eye images prior to displaying said additional left and right eye images for a period of time in which said interactive system is in said zoom mode of operation.

8. The method of claim 7, further comprising:

detecting user input indicating a switch from said zoom mode of operation to a normal mode of operation; and

ceasing said zooming and masking operations prior to displaying left and right eye images in response to detecting input indicating a switch from said zoom mode of operation to said normal mode of operation.

9. A method of operating an interactive system, the method comprising:

displaying a scene on a display;

detecting user input indicating a zoom mode of operation; and

in response to the detected user input indicating the zoom operation, simulating viewing of a portion of said scene through binoculars;

monitoring user head position;

detecting a change in user head position; and

changing a center portion of the displayed scene by an amount which is a function of the detected change in head position and a zoom factor used during said zoom mode of operation.

10. The method of claim 9, further comprising:

receiving user input indicating the zoom factor to be used during said zoom mode of operation; and

wherein changing the center portion of the displayed scene by an amount which is a function of the detected change in head position and the zoom factor includes moving the center of the displayed scene by an amount which is the zoom factor times the amount the displayed image would be moved in response to the detected change position if the change in head position were detected during normal mode operation.

11. An interactive system, comprising:

a display device configured to display images corresponding to a scene on a first portion of the display device;

a user input interface configured to detect user input indicating a zoom mode of operation; and

a simulator configured to simulate viewing of a portion of said scene through binoculars in response to the detected user input indicating the zoom operation, said simulating viewing of the portion of the scene through binoculars including using a second portion of said display device to display a zoomed version of said portion of said scene while a third portion of said display is blocked from displaying a portion of said scene, said second portion of the display device being smaller than said first portion of the display, said second and third areas being different portions of said first portion of the display device.

12. The system of claim 11, wherein said display device is configured, as part of being configured to display images corresponding to a scene, to:

output stereoscopic image pairs on the display device to provide a 3D presentation of said scene, said scene corresponding to an event, each stereoscopic image pair including a left eye view and a right eye view.

13. An interactive system, comprising:

a display device configured to display images corresponding to a scene;

a user input interface configured to detect user input indicating a zoom mode of operation; and

a simulator configured to simulate viewing of a portion of said scene through binoculars in response to the detected user input indicating the zoom operation, wherein said simulator is configured, as part of being configured to simulate viewing of a portion of said scene through binoculars, to;

perform a zoom operation on a first left eye image; and

apply a mask to zoomed first left eye image simulating a viewing constraint corresponding to viewing through binoculars to generate a masked zoomed left eye image.

14. The system of claim 13, wherein said display device is further configured to:

display the masked zoomed left eye image.

15. The system of claim 14,

wherein said simulator is further configured, as part of being configured to simulate viewing of a portion of said scene through binoculars, to:

perform a zoom operation on a first right eye image corresponding to the first left eye image; and

apply a mask to the zoomed right eye image simulating a viewing constraint corresponding to viewing through binoculars to generate a masked zoomed right eye image; and

wherein said display device is further configured to display the masked zoomed right eye image while said masked zoomed left eye image is being displayed.

16. The system of claim 15, wherein said simulator is further configured to perform zooming and masking operations on additional left and right eye images prior to said additional left and right eye images being displayed by said display device for a period of time in which said interactive system is in said zoom mode of operation.

17. The system of claim 16,

wherein said user input interface is configured to detect user input indicating a switch from said zoom mode of operation to a normal mode of operation;

and

wherein said simulator is further configured to cease said zooming and masking operations in response to said user input interface detecting said input indicating a switch from said zoom mode of operation to said normal mode of operation.

18. An interactive system, comprising:

a display device configured to display images corresponding to a scene;

a user input interface configured to detect user input indicating a zoom mode of operation;

a simulator configured to simulate viewing of a portion of said scene through binoculars in response to the detected user input indicating the zoom operation;

a head position monitoring module configured to monitor user head position for detecting a change in user head position; and

a display controller configured to change a center portion of the displayed scene by an amount which is a function of the detected change in head position and a zoom factor used during said zoom mode of operation.

19. The system of claim 18, further comprising:

wherein said user input interface includes a receiver configured to receive user input indicating the zoom factor to be used during said zoom mode of operation; and

wherein said display controller is configured to move the center of the displayed scene by an amount which is the zoom factor times the amount the displayed image would be moved in response to the detected change position if the change in head position were detected during normal mode operation, as part of being configured to change the center portion of the displayed scene by an amount which is a function of the detected change in head position and the zoom factor.

20. A non-transitory machine readable medium comprising processor executable instructions, which when executed by a processor, control a playback system to:

display a scene on a display device;

detect user input indicating a zoom mode of operation; and

simulate viewing of a portion of said scene through binoculars in response to said user input indicating a zoom mode of operation, wherein simulating viewing of a portion of said scene includes;

performing a zoom operation on a first left eye image; and

applying a mask to zoomed first left eye image simulating a viewing constraint corresponding to viewing through binoculars to generate a masked zoomed left eye image.