Using involuntary orbital movements to stabilize a video

• US 8,736,692 B1
• Filed: 07/09/2012
• Issued: 05/27/2014
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wong, Adrian, Geiss, Ryan
• Primary Examiner(s): Giles, Nicholas
• Assistant Examiner(s): Tissire, Abdelaaziz
• Application Number:
  US13/544,245
• Time in Patent Office: 687 Days
• Field of Search: 348/208.4
• US Class Current: 348/208.4
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 9/3173 wherein the projection device is specially adapted for enhanced portability
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Methods and systems for using eye-tracking data to stabilize a video are provided. An example method may involve receiving a video captured using a body-mountable camera and receiving gaze-direction data from one or more head-mountable eye trackers. Additionally, the method may include analyzing the received gaze-direction data to detect at least one involuntary orbital eye-movement. According to the method, the received video may be stabilized based at least in part on the detected at least one involuntary orbital eye-movement, and the stabilized video may be sent In some instances, camera-movement data may also be received and utilized to detect involuntary orbital eye-movements or stabilize the received video.

Claims

1. A method comprising:

receiving, at a computing device, a video captured using a body-mountable camera;

receiving, at the computing device, gaze-direction data from one or more head-mountable eye trackers;

analyzing the received gaze-direction data using the computing device to detect at least one involuntary orbital eye-movement;

stabilizing the received video based at least in part on the detected at least one involuntary orbital eye-movement; and

sending the stabilized video.

2. The method of claim 1, wherein the body-mountable camera comprises a head-mountable camera.

3. The method of claim 1, wherein the received video comprises point-of-view video.

4. The method of claim 1, wherein the gaze-direction data is taken with respect to a forward-facing direction, and wherein the body-mountable camera captures at least a portion of the video in a direction other than the forward-facing direction.

5. The method of claim 1, wherein analyzing the received gaze-direction data to detect the at least one involuntary orbital eye-movement comprises:

detecting, from the received gaze-direction data, one or more orbital eye-movements; and

determining that at least one of the one or more orbital eye-movements is involuntary.

6. The method of claim 5, wherein the at least one of the one or more orbital eye-movements is determined to be involuntary, based on a speed of the at least one of the one or more orbital eye-movements being less than a nonzero threshold speed.

7. The method of claim 5, wherein the at least one of the one or more orbital eye-movements is determined to be involuntary, based on a size of the at least one of the one or more orbital eye-movements being less than a nonzero threshold size.

8. The method of claim 5, further comprising determining a gaze depth associated with the gaze-direction data, wherein the at least one of the one or more orbital eye-movements is determined to be involuntary based on the determined gaze depth.

9. The method of claim 1, further comprising receiving camera-movement data from one or more of motion detectors and orientation detectors, wherein stabilizing the received video is further based on the received camera-movement data.

10. The method of claim 1, wherein stabilizing the received video, based at least in part on the detected at least one involuntary orbital eye-movement, is performed in real-time.

11. The method of claim 1, wherein stabilizing the received video, based at least in part on the detected at least one involuntary orbital eye-movement comprises:

changing a recorded portion of the received video in response to detecting involuntary orbital eye-movements; and

refraining from changing the recorded portion of the received video in response to detecting voluntary orbital eye-movements.

12. The method of claim 11, wherein the received video comprises a wide-field-of-view (wide-FOV) video, and wherein changing the recorded portion of the received video comprises dynamically selecting a portion of the wide-FOV video that is recorded.

13. The method of claim 1, further comprising varying an optical path of the body-mountable camera to move an image that is projected on a camera sensor plane of the body-mountable camera.

14. A system comprising:

a non-transitory computer-readable medium;

program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to;

receive a video captured by a body-mountable camera;

receive gaze-direction data from one or more head-mountable eye trackers;

analyze the received gaze-direction data to detect at least one involuntary orbital eye-movement;

stabilize the received video based at least in part on the at least one involuntary orbital eye-movement; and

send the stabilized video.

15. The system of claim 14, wherein the system is head-mountable.

16. The system of claim 14, wherein analyzing the received gaze-direction data to detect the at least one involuntary orbital eye-movement comprises:

detecting, from the received gaze-direction data, one or more orbital eye-movements; and

determining that at least one of the one or more orbital eye-movements is involuntary.

17. The system of claim 16, wherein the program instructions are further executable by the at least one processor to receive camera-movement data from one or more motion detectors and/or orientation detectors indicative of movement of the body-mountable camera, and wherein the at least one of the one or more orbital eye-movements is determined to be involuntary based at least in part on the received camera-movement data.

18. The system of claim 14, wherein the program instructions are further executable by the at least one processor to receive camera-movement data from one or more motion detectors and/or orientation detectors indicative of movement of the body-mountable camera, and wherein stabilizing the received video is further based on the received camera-movement data.

19. The system of claim 14, wherein the program instructions are executable by the at least one processor to stabilize the received video in real-time.

20. A non-transitory computer-readable medium having stored thereon program instructions executable by at least one processor to:

receive a video captured using a body-mountable camera;

receive gaze-direction data from one or more head-mountable eye trackers;

analyze the received gaze-direction data to detect at least one involuntary orbital eye-movement;

stabilize the received video based at least in part on the detected at least one involuntary orbital eye-movement; and

send the stabilized video.

21. The non-transitory computer-readable medium of claim 20, wherein analyzing the received gaze-direction data to detect the at least one involuntary orbital eye-movement comprises:

detecting, from the received gaze-direction data, one or more orbital eye-movements; and

determining that at least one of the one or more orbital eye-movements is involuntary.

22. The non-transitory computer-readable medium of claim 21, wherein the program instructions are further executable by the at least one processor to receive camera-movement data from one or more of motion detectors and orientation detectors that is indicative of movement of the body-mountable camera, and wherein the at least one of the one or more orbital eye-movements is determined to be involuntary, based at least in part on the received camera-movement data.

23. The non-transitory computer-readable medium of claim 20, wherein the program instructions are further executable by the at least one processor to receive camera-movement data from one or more of motion detectors and orientation detectors that is indicative of movement of the body-mountable camera, and wherein stabilizing the received video is further based on the received camera-movement data.