Method and system for adaptive viewport for a mobile device based on viewing angle

• US 10,009,603 B2
• Filed: 06/23/2014
• Issued: 06/26/2018
• Est. Priority Date: 11/16/2009
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Inventors: Kellerman, Marcus, Chen, Xuemin, Hulyalkar, Samir, Klebanov, Ilya
• Primary Examiner(s): Rahaman, Mohammed
• Application Number:
  US14/312,379
• Publication Number:
  US 20150015671A1
• Time in Patent Office: 1,464 Days
• Field of Search: None
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

A 2D and/or 3D video processing device comprising a camera and a display captures images of a viewer as the viewer observes displayed 2D and/or 3D video content in a viewport. Face and/or eye tracking of viewer images is utilized to generate a different viewport. Current and different viewports may comprise 2D and/or 3D video content from a single source or from different sources. The sources of 2D and/or 3D content may be scrolled, zoomed and/or navigated through for generating the different viewport. Content for the different viewport may be processed. Images of a viewer'"'"'s positions, angles and/or movements of face, facial expression, eyes and/or physical gestures are captured by the camera and interpreted by face and/or eye tracking. The different viewport may be generated for navigating through 3D content and/or for rotating a 3D object. The 2D and/or 3D video processing device communicates via wire, wireless and/or optical interfaces.

Claims

1. A method comprising:

capturing, by one or more processors, one or more images of a viewer while the viewer observes a first set of one or more content images representing a first set of one or more contents;

performing at least one of face tracking or eye tracking on the one or more images of the viewer to generate tracking data, the tracking data indicating a first content that the viewer is attempting to see;

determining, by the one or more processors, whether the first content is within the first set of one or more contents;

generating, by the one or more processors, a second set of one or more content images using the tracking data, the step of generating the second set of one or more content images comprising;

modifying at least one image of the first set of content images in response to determining the first content is within the first set of one or more contents; and

responsive to determining that the first content is not within the first set of one or more contents,

determining at least one new content image that is not within the first set of content images and associated with the first content; and

replacing at least one image of the first set of content images with the at least one new content image; and

displaying the second set of one or more content images.

2. The method of claim 1, wherein the first set of content images and the second set of content images represent at least partially different portions of a single source of content.

3. The method of claim 1, wherein the first set of content images is from a first source of content, and the at least one new content image is from a second source of content, wherein the first source of content is different from the second source of content.

4. The method of claim 1, wherein generating the second set of content images comprises at least one of scrolling, zooming, or navigating through at least one of a source of currently available content or a different source of content.

5. The method of claim 1, wherein capturing the one or more images of the viewer comprises capturing images of at least one of one or more positions of a face of the viewer, one or more angles of the face of the viewer, one or more movements of the face of the viewer, a facial expression of the viewer, eyes of the viewer, or other physical gestures of a body of the viewer using a camera.

6. The method of claim 1, further comprising interpreting captured images of at least one of one or more positions of a face of the viewer, one or more angles of the face of the viewer, one or more movements of the face of the viewer, a facial expression of the viewer, eyes of the viewer, or other physical gestures of a body of the viewer for performing at least one of a scrolling operation, a zooming operation, or a navigating operation on the first set of content images.

7. The method of claim 1, wherein generating the second set of content images comprises navigating, based on the tracking data, through three-dimensional content by varying viewing angles or depths of view in the three-dimensional content.

8. A system comprising:

circuitry configured to;

capture one or more images of a viewer while the viewer observes a first set of one or more content images representing a first set of one or more contents;

perform at least one of face tracking or eye tracking on the one or more images of the viewer to generate tracking data, the tracking data indicating a first content that the viewer is attempting to see;

determine whether the first content is within the first set of one or more contents;

generate a second set of one or more content images using the tracking data by;

modifying at least one image of the first set of content images in response to determining the first content is within the first set of one or more contents;

replacing at least one image of the first set of content images with at least one new content image in response to determining the first content is not within the first set of one or more contents, wherein the at least one new content image is not within the first set of one or more content images and the at least one new content image is associated with the first content; and

display the second set of one or more content images.

9. The system of claim 8, wherein the first set of content images and the second set of content images represent at least partially different portions of a single source of content.

10. The system of claim 8, wherein the second set of content images comprises content from a different source of content than the first set of content images.

11. The system of claim 8, wherein the circuitry is configured to generate the second set of content images by at least one of scrolling, zooming, or navigating through at least one of a source of currently available content or a different source of content.

12. The system of claim 8, wherein the circuitry is configured to capture images of at least one of one or more positions of a face of the viewer, one or more angles of the face of the viewer, one or more movements of the face of the viewer, a facial expression of the viewer, eyes of the viewer, or other physical gestures of a body of the viewer using a camera.

13. The system of claim 8, wherein the circuitry is further configured to interpret captured images of at least one of one or more positions of a face of the viewer, one or more angles of the face of the viewer, one or more movements of the face of the viewer, a facial expression of the viewer, eyes of the viewer, or other physical gestures of a body of the viewer for performing at least one of a scrolling operation, a zooming operation, or a navigating operation on the first set of content images.

14. The system of claim 8, wherein the circuitry is configured to navigate, based on the tracking data, through three-dimensional content by varying viewing angles or depths of view in the three-dimensional content.

15. A method comprising:

capturing, by one or more processors, one or more images of a viewer while the viewer observes a first viewport comprising a first set of one or more content images representing a first set of one or more contents;

performing at least one of face tracking or eye tracking on the one or more images of the viewer to generate tracking data, the tracking data indicating a first content that the viewer is attempting to see;

determining whether the first content is within the first set of one or more contents;

generating, by the one or more processors, a second viewport comprising a second set of one or more content images using the tracking data, the step of generating the second viewport comprising;

modifying at least one image of the first set of content images in response to determining the first content is within the first set of contents;

replacing at least one image of the first set of content images with at least one new content image in response to determining the first content is not within the first set of one or more contents, wherein the at least one new content image is not within the first set of one or more content images and the at least one new content image is associated with the first content; and

displaying the second viewport.

16. The method of claim 15, wherein the first viewport and the second viewport represent at least partially different portions of a single source of content.

17. The method of claim 15, wherein the second viewport comprises content from a different source of content than the first viewport.

18. The method of claim 15, wherein generating the second viewport comprises at least one of scrolling, zooming, or navigating through at least one of a source of currently available content or a different source of content.

19. The method of claim 15, wherein capturing the one or more images of the viewer comprises capturing images of at least one of one or more positions of a face of the viewer, one or more angles of the face of the viewer, one or more movements of the face of the viewer, a facial expression of the viewer, eyes of the viewer, or other physical gestures of a body of the viewer using a camera.

20. The method of claim 15, further comprising interpreting captured images of at least one of one or more positions of a face of the viewer, one or more angles of the face of the viewer, one or more movements of the face of the viewer, a facial expression of the viewer, eyes of the viewer, or other physical gestures of a body of the viewer for performing at least one of a scrolling operation, a zooming operation, or a navigating operation on the first set of content images.