AERIAL DRONE COMPANION DEVICE AND A METHOD OF OPERATING AN AERIAL DRONE COMPANION DEVICE

• US 20180082682A1
• Filed: 09/16/2016
• Published: 03/22/2018
• Est. Priority Date: 09/16/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: ERICKSON, THOMAS D., Pickover, Clifford A., Weldemariam, Komminist
• Granted Patent: US 10,140,987 B2
• Time in Patent Office: Days
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2021/02085 Periodic noise

Abstract

A method of operating an aerial drone companion device includes detecting a first voice command spoken by a first user. The aerial drone companion device is autonomously oriented such that an image capture device faces the first user in response to detecting the first voice command. A second voice command spoken by the first user is detected while the image capture device faces the first user. The second voice command is transmitted from the aerial drone companion device to a computer located remotely from the aerial drone companion device. A task signal is received indicating a task to be performed. The task signal is generated by the computer based on the second voice command, and the task signal is transmitted by the computer and received by the aerial drone companion device. The method includes autonomously executing the task by the aerial drone companion device.

Claims

1. A method of operating an aerial drone companion device, comprising:

detecting a first voice command spoken by a first user by at least one microphone disposed on the aerial drone companion device;

autonomously orientating the aerial drone companion device such that an image capture device disposed on the aerial drone companion device faces the first user in response to detecting the first voice command;

detecting a second voice command spoken by the first user by the at least one microphone while the image capture device faces the first user and while the first user looks at the image capture device;

transmitting the second voice command from the aerial drone companion device to a computer located remotely from the aerial drone companion device;

receiving a task signal indicating a task to be performed, wherein the task signal is generated by the computer based on the second voice command, and the task signal is transmitted by the computer and received by the aerial drone companion device; and

autonomously executing the task by the aerial drone companion device.

2. The method of claim 1, wherein the first voice command is a wakeup command that causes the aerial drone companion device to transition from a low-power operation mode to a normal operation mode.

3. The method of claim 2, wherein the aerial drone companion device is stationed on ground while in the low-power operation mode, autonomously flies above the ground and follows the first user while in the normal operation mode and while the user is moving, and is stationed on the ground while in the normal operation mode and while the user is not moving.

4. The method of claim 2, wherein the wakeup command is predefined by the first user.

5. The method of claim 1, wherein autonomously orientating the aerial drone companion device comprises at least one of autonomously flying the aerial done companion device toward the first user and autonomously rotating the aerial drone companion device such that the image capture device faces the first user.

6. The method of claim 1, wherein the second voice command indicates a destination at which the task is to be executed, and the aerial drone companion device autonomously flies to the destination to execute the task in response to receiving the task signal.

7. The method of claim 6, wherein the second voice command further indicates a time at which the task is to be executed, and the aerial drone companion device autonomously flies to the destination at the indicated time to execute the task in response to receiving the task signal.

8. The method of claim 1, further comprising:

generating a voice model of the first user using at least one of the first voice command, the second voice command, and additional voice commands spoken by the first user;

storing the voice model of the first user in a first user record corresponding to the first user in a user database;

generating a list of personalized preferences of the first user using the second voice command and previous voice commands spoken by the first user; and

storing the list of personalized preferences of the first user in the first user record.

9. The method of claim 8, wherein the task signal generated by the computer is further based on the list of personalized preferences of the first user.

10. The method of claim 8, further comprising:

creating a voice model of a second user using at least one voice command spoken by the second user;

storing the voice model of the second user in a second user record corresponding to the second user in the user database;

generating a list of personalized preferences of the second user using the at least one voice command spoken by the second user;

storing the list of personalized preferences of the second user in the second user record; and

determining that the first and second user are present, using the at least one microphone, when the second voice command spoken by the first user is detected by the at least one microphone,

wherein the task signal generated by the computer is further based on the list of personalized preferences of the first user and the second user.

11. The method of claim 10, wherein the second voice command directs the aerial drone companion device to play music, and the task signal indicates a type of music to be played based on music preferences of the first user stored in the list of personalized preference of the first user and on music preferences of the second user stored in the list of personalized preferences of the second user.

12. The method of claim 1, further comprising:

capturing at least one image of the first user by the image capture device;

transmitting the at least one image of the first user from the aerial drone companion device to the computer located remotely from the aerial drone companion device;

storing the at least one image of the first user in a first user record corresponding to the first user in a user database;

generating a list of personalized preferences of the first user using the second voice command and previous voice commands spoken by the first user; and

storing the list of personalized preferences of the first user in the first user record.

13. The method of claim 12, wherein the task signal generated by the computer is further based on the list of personalized preferences of the first user.

14. The method of claim 1, further comprising:

capturing at least one image of the first user by the image capture device; and

determining a current cognitive state of the first user using the at least one image,

wherein the task signal generated by the computer is further based on the current cognitive state of the first user.

15. The method of claim 1, wherein the task comprises communicating with an appliance located remotely from the aerial drone companion device to execute a function of the appliance.

16. The method of claim 1, wherein the task comprises placing an order for goods or services by relaying the order for goods or services from the aerial drone companion device to a seller or provider of goods or services located remotely from the aerial drone companion device.

17. The method of claim 1, further comprising:

filtering noise generated by a propeller of the aerial drone companion device to identify the first and second voice commands spoken by the first user.

18. A method of operating an aerial drone companion device, comprising:

capturing at least one registration image of a user by an image capture device disposed on the aerial drone companion device;

registering the user as a target user of the aerial drone companion device using the at least one registration image;

autonomously flying the aerial drone companion device from a charging station to the target user, wherein the aerial drone companion device locates the target user by capturing images using the image capture device while flying, and by comparing people in the captured images to the at least one registration image until the target user is identified,

wherein the aerial drone companion device cycles between autonomously docking in the charging station and autonomously flying to the target user at a predetermined interval;

determining a current cognitive state of the target user by capturing at least one image of the target user using the image capture device and comparing the at least one image to a plurality of predefined images in an image database; and

autonomously executing a task based on the cognitive state of the target user without receiving input from the user.

19. The method of claim 18, wherein the current cognitive state indicates that the target user requires emergency assistance, and the task comprises the aerial drone companion device contacting a third party.

20. An aerial drone companion device, comprising:

a processor;

at least one microphone that detects a first voice command and a second voice command spoken by a user under control of the processor;

an image capture device;

a plurality of propellers that autonomously orientate the aerial drone companion device, under control of the processor, such that the image capture device faces the user in response to the at least one microphone detecting the first voice command,

wherein the second voice command is detected by the at least one microphone while the image capture device faces the first user and while the first user looks at the image capture device; and

a network adapter that transmits the second voice command from the aerial drone companion device to a computer located remotely from the aerial drone companion device under control of the processor, and that receives a task signal from the computer indicating a task to be performed, wherein the task signal is generated by the computer based on the second voice command,

wherein the aerial drone companion device autonomously executes the task.