Adaptive modes of operation based on user intention or activity

• US 10,498,964 B2
• Filed: 12/28/2017
• Issued: 12/03/2019
• Est. Priority Date: 12/28/2017
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Gohl, Pascal, Gubler, Thomas, Murguet, Axel, Bruneau, Garance
• Primary Examiner(s): Segura, Cynthia
• Application Number:
  US15/856,706
• Publication Number:
  US 20190208101A1
• Time in Patent Office: 705 Days
• Field of Search: None
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/685 performed by mechanical compensation

Abstract

The disclosure describes systems and methods for stabilizing an imaging device with an image stabilization device. The image stabilization device includes sensors, one or more arms, one or more motors, and a control unit. The sensors provide sensor data including orientation data, angular velocity data, and acceleration data. Each of the motors is associated with a respective arm. The control unit is configured to set a setpoint of the imaging device to a default setpoint, receive sensor data from the sensors, determine whether a flip condition exists, in response to determining that the flip condition exists, fix the setpoint of the imaging device, and in response to determining that the flip condition does not exists, maintain the default setpoint of the imaging device by moving at least one of respective arms with one of the respective motors.

Claims

1. An image stabilization device for stabilizing an imaging device, comprising:

sensors for providing sensor data comprising orientation data, angular velocity data, and acceleration data;

one or more arms;

one or more motors, each motor associated with a respective arm; and

a control unit, the control unit configured to;

set a setpoint of the imaging device to a default setpoint, the setpoint corresponding to an orientation of the imaging device;

receive sensor data from the sensors;

determine whether a flip condition exists;

in response to determining that the flip condition exists, stop operating at least one of the motors of the image stabilization device so that the default setpoint is not maintained and the imaging device is not stabilized; and

in response to determining that the flip condition does not exist, maintain the default setpoint to stabilize the imaging device by moving at least one of the one or more arms with one of the respective motors.

2. The image stabilization device of claim 1, wherein to determine whether the flip condition exists comprises to:

detect a g-force exceeding a threshold; and

determine that the flip condition exists based on the g-force exceeding the threshold.

3. The image stabilization device of claim 2,

wherein to determine whether the flip condition exists further comprises to:

receive, by the image stabilization device, an image stream from the imaging device; and

detect an activity indicative of the flip condition based on the image stream; and

wherein to determine that the flip condition exists is further based on detecting the activity.

4. The image stabilization device of claim 3, wherein to detect, based on the image stream, the activity, comprises to:

detect an object indicative of the flip condition in the image stream.

5. The image stabilization device of claim 2, wherein to determine whether the flip condition exists is further based on a user input indicating that the imaging device is to be flipped.

6. An image stabilization device for stabilizing an imaging device, comprising:

sensors for providing sensor data comprising orientation data, angular velocity data, and acceleration data;

one or more arms;

one or more motors, each motor associated with a respective arm; and

a control unit, the control unit configured to;

set a setpoint of the imaging device to a default setpoint;

receive sensor data from the sensors;

in response to the sensor data, maintain the default setpoint of the imaging device by moving at least one of the one or more arms with at least one of the respective motors;

identify an exceptional movement of the image stabilization device,

wherein the exceptional movement is identified based on one or more movements of the image stabilization device over time, and

wherein the exceptional movement indicates to the control unit that the image stabilization device is to stop maintaining the default setpoint;

identify, based on the exceptional movement in at least one of a yaw direction, a pitch direction, and a roll direction of the image stabilization device, an activity, wherein the image stabilization device is worn by or carried by a person performing the activity; and

operate the one or more motors of the image stabilization device according to an activity profile associated with the activity.

7. The image stabilization device of claim 6, wherein to identify the activity is further based on a user input indicating that the activity is to be identified.

8. The image stabilization device of claim 6,

wherein the exceptional movement comprises an acceleration in the yaw direction,

wherein the activity is a panning activity, and

wherein the activity profile includes operating the motors of the image stabilization device to follow, instead of to stabilize, the imaging device in the yaw direction.

9. The image stabilization device of claim 6,

wherein the exceptional movement is identified based on a g-force measured by the sensors of the image stabilization device, and

wherein the activity profile includes to operate the one or more motors of the image stabilization device to fix, instead of to stabilize, the imaging device in the pitch direction.

10. The image stabilization device of claim 9, wherein to operate the one or more motors of the image stabilization device to fix, instead of to stabilize, the imaging device in the pitch direction comprises to:

disable flipping of the imaging device.

11. The image stabilization device of claim 10, wherein the activity includes a flip activity.

12. The image stabilization device of claim 6,

wherein the exceptional movement is identified based on a rotational rate of the image stabilization device, and

wherein the activity profile includes operating the motors to follow the imaging device in the roll direction.

13. The image stabilization device of claim 12, wherein the activity includes a leaning activity.

14. The image stabilization device of claim 6, wherein the identifying the activity is further based on an image stream received from the imaging device by the image stabilization device.

15. A method for stabilizing an imaging device with an image stabilization device, comprising:

operating, based on sensor data, motors of the image stabilization device to stabilize the imaging device to a default setpoint;

identifying an exceptional movement of the image stabilization device, wherein the exceptional movement indicates that the image stabilization device is to stop maintaining the default setpoint;

identifying, based on the exceptional movement of the image stabilization device in at least one of a yaw direction, a pitch direction, and a roll direction, an activity, wherein the image stabilization device is worn by or carried by a person performing the activity; and

operating the motors of the image stabilization device according to an activity profile associated with the activity.

16. The method of claim 15,

wherein the exceptional movement comprises an acceleration in the yaw direction,

wherein the activity is a panning activity, and

wherein operating the motors of the image stabilization device according to the activity profile associated with the activity, comprises:

operating the motors of the image stabilization device to follow, instead of to stabilize, the imaging device in the yaw direction.

17. The method of claim 15,

wherein the exceptional movement is identified based on a g-force of the image stabilization device, and

wherein operating the motors of the image stabilization device according to the activity profile associated with the activity, comprises:

operating the motors of the image stabilization device to fix, instead of to stabilize, the imaging device in the pitch direction.

18. The method of claim 17, wherein operating the motors of the image stabilization device to fix, instead of to stabilize, the imaging device in the pitch direction comprises:

disabling flipping of the imaging device.

19. The method of claim 15,

wherein the exceptional movement is identified based on a rotational rate of the image stabilization device, and

wherein operating the motors of the image stabilization device according to the activity profile associated with the activity, comprises:

operating the motors of the image stabilization device to follow, instead of to stabilize, the imaging device in the roll direction.

20. The method of claim 15, wherein the identifying the activity is further based on an image stream received from the imaging device by the image stabilization device.