System and method for supporting photography with different effects

• US 10,104,300 B2
• Filed: 11/10/2016
• Issued: 10/16/2018
• Est. Priority Date: 09/16/2015
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Guo, Zhuo, Su, Guanhua, Tao, Ye, Cao, Zisheng, Bao, Linchao, Wang, Mingyu
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): PHAM, QUAN L
• Application Number:
  US15/348,549
• Publication Number:
  US 20170078587A1
• Time in Patent Office: 705 Days
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

System and method can support photography. A controller can configure a carrier to move an imaging device along a moving path. Furthermore, the controller can apply a time-dependent configuration on the imaging device, and use the imaging device for capturing a set of image frames along the moving path based on the one or more time-dependent parameters.

Claims

1. A method for supporting photography, comprising:

acquiring a target in an image view based on one or more images that are captured by an imaging device associated with a carrier;

obtaining characterization information of the target as indicated in said one or more images;

determining a moving path having at least two sections for tracking the target with the imaging device;

determining a corresponding speed for each section of the moving path;

directing the carrier to move the imaging device, at the corresponding speeds, along each of the at least two sections of the moving path to track the target based on the obtained characterization information of the target; and

capturing one or more images using the imaging device along the moving path based on a time-dependent configuration of the imaging device.

2. The method of claim 1, further comprising:

receiving, from an application, one or more signals to configure the imaging device and the carrier, wherein the application provides a user interface via a display on a terminal.

3. The method of claim 1, further comprising:

displaying the image view on a terminal that operates to receive one or more signals from the imaging device.

4. The method of claim 3, wherein:

said one or more signals contains real-time images that are captured by the imaging device.

5. The method of claim 3, further comprising:

allowing a user to designate the target in the image view that is displayed on the terminal.

6. The method of claim 5, further comprising:

receiving information that is indicative of a position of the target in the image view based on said one or more images.

7. The method of claim 1, further comprising:

determining a position of the target in the image view as the target moves, and

detecting whether the target is outside a designated area in the image view.

8. The method of claim 7, wherein:

the designated area in the image view is configurable.

9. The method of claim 8, wherein:

the designated area in the image view is in a rectangular shape with an up boundary, a down boundary, a left boundary and a right boundary.

10. The method of claim 7, wherein:

the carrier includes a gimbal that operates to rotate about one or more axes to maintain the target within the designated area in the image view.

11. The method of claim 10, further comprising:

directing the carrier to rotate about a yaw axis when the target moves outside a left or a right boundary of the designated area; and

directing the carrier to rotate about a pitch axis when the target moves outside an up or a down boundary of the designated area.

12. The method of claim 1, further comprising:

receiving one or more signals to switch target.

13. The method of claim 12, further comprising:

directing the carrier to move the imaging device tracking another target based on obtained characterization information of the another target.

14. The method of claim 1, further comprising:

capturing one or more images and/or recording a video of the target while tracking the target.

15. The method of claim 1, wherein the carrier includes a gimbal that operates to rotate the imaging device around one or more axes, the rotating causing the imaging device to move along the at least two sections of the moving path at the corresponding speeds.

16. The method of claim 1, wherein the carrier couples the imaging device to a portable device.

17. The method of claim 16, wherein the portable device includes one or more mechanisms for moving a gimbal along the at least two sections of the moving path at the corresponding speeds.

18. A system for supporting photography, comprising:

one or more microprocessors; and

a controller running on the one or more microprocessors, wherein the controller operates to;

acquire a target in an image view based on one or more images that are captured by an imaging device associated with a carrier;

obtain characterization information of the target as indicated in said one or more images;

determine a moving path having at least two sections for tracking the target with the imaging device;

determine a corresponding speed for each section of the moving path;

direct the carrier to move the imaging device, at the corresponding speeds, along each of the at least two sections of the moving path to track the target based on the obtained characterization information of the target; and

capture one or more images using the imaging device along the moving path based on a time-dependent configuration of the imaging device.

19. The system of claim 18, wherein:

the controller operates to receive, from an application, one or more signals to configure the imaging device and the carrier, wherein the application provides a user interface via a display on a terminal.

20. The system of claim 18, wherein:

the controller operates to display the image view on a terminal that operates to receive one or more signals from the imaging device.

21. The system of claim 20, wherein:

said one or more signals contains real-time images that are captured by the imaging device.

22. The system of claim 20, wherein:

a user is allowed to designate the target in the image view that is displayed on the terminal.

23. The system of claim 22, further comprising:

the controller operates to receive information that is indicative of a position of the target in the image view based on said one or more images.

24. The system of claim 18, wherein:

the controller operates to

determine a position of the target in the image view as the target moves, and

detect whether the target is outside a designated area in the image view.

25. The system of claim 24, wherein:

the designated area in the image view is configurable.

26. The system of claim 25, wherein:

the designated area in the image view is in a rectangular shape with an up boundary, a down boundary, a left boundary and a right boundary.

27. The system of claim 24, wherein:

the carrier includes a gimbal that operates to rotate about one or more axes to maintain the target within the designated area in the image view.

28. The system of claim 27, wherein:

the controller operates to

direct the carrier to rotate about a yaw axis when the target moves outside a left or a right boundary of the designated area; and

direct the carrier to rotate about a pitch axis when the target moves outside an up or a down boundary of the designated area.

29. The system of claim 18, wherein:

the controller operates to receive one or more signals to switch to another target.

30. The system of claim 29, wherein:

the controller operates to direct the carrier to move the imaging device tracking said another target based on obtained characterization information of said another target.

31. The system of claim 18, wherein:

the controller operates to capture one or more images and/or recording a video of the target while tracking the target.

32. The system of claim 18, wherein the carrier includes a gimbal that operates to rotate the imaging device around one or more axes, the rotating causing the imaging device to move along the at least two sections of the moving path at the corresponding speeds.

33. The system of claim 18, wherein the carrier couples the imaging device to a portable device.

34. The system of claim 33, wherein the portable device includes one or more mechanisms for moving a gimbal along the at least two sections of the moving path at the corresponding speeds.

35. A non-transitory computer-readable medium with instructions stored thereon, that when executed by a processor, perform the steps comprising:

acquiring a target in an image view based on one or more images that are captured by an imaging device associated with a carrier;

obtaining characterization information of the target as indicated in said one or more images;

determining a moving path having at least two sections for tracking the target with the imaging device;

determining a corresponding speed for each section of the moving path;

directing the carrier to move the imaging device, at the corresponding speeds, along each of the at least two sections of the moving path to track the target based on the obtained characterization information of the target; and

capturing one or more images using the imaging device along the moving path based on a time-dependent configuration of the imaging device.

36. A system for supporting photography, comprising:

an imaging device, which is associated with an carrier; and

a control device, wherein the control device operates to;

acquire a target in an image view based on one or more images that are captured by an imaging device associated with a carrier;

obtain characterization information of the target as indicated in said one or more images;

determine a moving path having at least two sections for tracking the target with the imaging device;

determine a corresponding speed for each section of the moving path;

direct the carrier to move the imaging device, at the corresponding speeds, along each of the at least two sections of the moving path to track the target based on the obtained characterization information of the target; and

capture one or more images using the imaging device along the moving path based on a time-dependent configuration of the imaging device.