Computing device and method for controlling unmanned aerial vehicle in flight space

• US 8,761,964 B2
• Filed: 11/13/2012
• Issued: 06/24/2014
• Est. Priority Date: 03/26/2012
• Status: Active Grant

• Current Assignee: Beijing Jingdong Century Trading Co., Ltd.
• Original Assignee: Hon Hai Precision Industry Co., Ltd.
• Inventors: Lee, Hou-Hsien, Lee, Chang-Jung, Lo, Chih-Ping
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Lee, Tyler J
• Application Number:
  US13/674,959
• Publication Number:
  US 20130253733A1
• Time in Patent Office: 588 Days
• Field of Search: 701/23, 701 2- 18, 701/120, 244/189, 244/190, 244/195, 244/196, 244/222, 382/103, 382115-128, 702/152, 702/153, 702/166, 700/59, 700/245, 700/258, 700/259
• US Class Current: 701/2
• CPC Class Codes: B64C 19/00 Aircraft control not otherwise provided for
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/17 Helicopters flying platforms B64U10/13
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0202 specially adapted to aircraft
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C

Abstract

In a method for controlling an unmanned aerial vehicle (UAV) in a flight space using a computing device, a 3D sample database is created and store in a storage device of the computing device. The computing device includes a depth-sensing camera that captures a 3D scene image of a scene in front of a user, and senses a depth distance between the user and the depth-sensing camera. A 3D person image of the user is detected from the 3D scene image, and gesture information of the user is obtained by comparing the 3D person image with human gesture data stored in the 3D sample database. The method converts the gesture information of the user into one or more flight control commands, and drives a driver of the UAV to control the UAV to fly in a flight space according to the flight control commands.

Claims

1. A computing device, comprising:

a depth-sensing camera;

a storage device;

at least one processor; and

one or more programs stored in the storage device comprising one or more programs and executable by the at least one processor, the one or more programs comprising;

an image detecting module that captures a 3D scene image of a scene in front of a user using the depth-sensing camera, obtains a depth distance between the user and the depth-sensing camera using the depth-sensing camera, detects a 3D person image of the user from the 3D scene image, and compares the 3D person image with human gesture data stored in a 3D sample database to obtain gesture information of the user from the 3D person image, wherein the 3D sample database is created by capturing the human gesture data using the depth-sensing camera, building a data relationship between each of the human gesture data and a flight control command of an unmanned aerial vehicle (UAV), and storing the human gesture data in the 3D sample database; and

a flight control module that converts the gesture information of the user into one or more flight control commands, sends the flight control commands to the UAV through a wireless network, and driving a driver of the UAV to control the UAV to fly in a flight space or adjust a flying mode of the UAV according to the flight control commands.

2. The computing device according to claim 1, wherein the depth-sensing camera is a time of flight (TOF) camera device having a 3D image capturing functionality.

3. The computing device according to claim 1, wherein the depth-sensing camera senses the depth distance between the depth-sensing camera and the user according to a pulse time of a short light illuminated at the user and a reflected time of a reflected light of the short light.

4. The computing device according to claim 1, wherein the 3D person image of the user is detected from the 3D scene image by performing steps of:

converting a depth distance between each point of the user and the depth-sensing camera into a pixel value, and storing all of the pixel value in a data array; and

comparing the pixel values of the data array with characteristic data of the user stored in the 3D sample database to obtain the 3D person image from the 3D scene image.

5. The computing device according to claim 1, wherein the flight control commands comprise a first control command for controlling the UAV to fly upwards, downwards, leftwards or rightwards, and a second adjustment command for adjusting a flying speed or a flying direction of the UAV.

6. A method for controlling an unmanned aerial vehicle (UAV) in a flight space using a computing device, the method comprising:

capturing a 3D scene image of a scene in front of a user using a depth-sensing camera of the computing device, and obtaining a depth distance between the user and the depth-sensing camera using the depth-sensing camera;

detecting a 3D person image of the user from the 3D scene image;

comparing the 3D person image with gesture data stored in a 3D sample database to obtain gesture information of the user from the 3D person image, wherein the 3D sample database is created by capturing the human gesture data using the depth-sensing camera, building a data relationship between each of the human gesture data and a flight control command of the UAV, and storing the human gesture data in the 3D sample database;

converting the gesture information of the user to one or more flight control commands, and sending the flight control commands to the UAV through a wireless network; and

driving a driver of the UAV to control the UAV to fly in a flight space or adjust a flying mode of the UAV according to the flight control commands.

7. The method according to claim 6, wherein the depth-sensing camera is a time of flight (TOF) camera device having a 3D image capturing function.

8. The method according to claim 6, wherein the depth-sensing camera senses the depth distance between the depth-sensing camera and the user according to a pulse time of a short light illuminated at the user and a reflected time of a reflected light of the short light.

9. The method according to claim 7, wherein the 3D person image of the user is detected from the 3D scene image by performing steps of:

converting a depth distance between each point of the user and the depth-sensing camera into a pixel value, and storing all of the pixel values in a data array; and

comparing the pixel values of the data array with characteristic data of the user stored in the 3D sample database to obtain the 3D person image from the 3D scene image.

10. The method according to claim 6, wherein the flight control commands comprise a first control command for controlling the UAV to fly upwards, downwards, leftwards or rightwards, and a second adjustment command for adjusting a flying speed or a flying direction of the UAV.

11. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor of a computing device, cause the computing device to perform a method for controlling an unmanned aerial vehicle (UAV) in a flight space, the method comprising:

capturing a 3D scene image of a scene in front of a user using a depth-sensing camera of the computing device, and obtaining a depth distance between the user and the depth-sensing camera using the depth-sensing camera;

detecting a 3D person image of the user from the 3D scene image;

comparing the 3D person image with human gesture data stored in a 3D sample database to obtain gesture information of the user from the 3D person image, wherein the 3D sample database is created by capturing the human gesture data using the depth-sensing camera, building a data relationship between each of the human gesture data and a flight control command of the UAV, and storing the human gesture data in the 3D sample database;

converting the gesture information of the user to one or more flight control commands, and sending the flight control commands to the UAV through a wireless network; and

driving a driver of the UAV to control the UAV to fly in a flight space or adjust a flying mode of the UAV according to the flight control commands.

12. The storage medium according to claim 11, wherein the depth-sensing camera is a time of flight (TOF) camera device having a 3D image capturing functionality.

13. The storage medium according to claim 11, wherein the depth-sensing camera senses the depth distance between the depth-sensing camera and the user according to a pulse time of a short light illuminated at the user and a reflected time of a reflected light of the short light.

14. The storage medium according to claim 11, wherein the 3D person image of the user is detected from the 3D scene image by performing steps of:

converting a depth distance between each point of the user and the depth-sensing camera into a pixel value, and storing all of the pixel value in a data array; and

comparing the pixel values of the data array with characteristic data of the user stored in the 3D sample database to obtain the 3D person image from the 3D scene image.

15. The storage medium according to claim 11, wherein the flight control commands comprise a first control command for controlling the UAV to fly upwards, downwards, leftwards or rightwards, and a second adjustment command for adjusting a flying speed or a flying direction of the UAV.