Apparatus and methods for controlling attention of a robot

• US 10,545,074 B2
• Filed: 12/24/2018
• Issued: 01/28/2020
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Meier, Philip
• Primary Examiner(s): Figueroa, Jaime
• Application Number:
  US16/231,762
• Publication Number:
  US 20190137366A1
• Time in Patent Office: 400 Days
• Field of Search: 700245, 700250
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1656 characterised by programming, planning systems for manipulators
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: B60K 11/02 with liquid cooling
• CPC Class Codes: B60Y 2400/432 Diesel Engines
• CPC Class Codes: B60Y 2400/433 Gas Engines, e.g. using LPG, natural gas or gasifiers
• CPC Class Codes: F01P 11/16 concerning coolant temperature F01P11/20 takes precedence
• CPC Class Codes: F01P 3/20 Cooling circuits not specific to a single part of engine or machine F01P3/22 takes precedence
• CPC Class Codes: F01P 7/16 by thermostatic control
• CPC Class Codes: G01N 1/2273 Atmospheric sampling
• CPC Class Codes: G01N 1/44 Sample treatment involving radiation, e.g. heat
• CPC Class Codes: G01N 2001/2288 Filter arrangements
• CPC Class Codes: G01N 2035/00475 Filters
• CPC Class Codes: G01N 30/72 Mass spectrometers mass spectrometers per se H01J49/00
• CPC Class Codes: G01N 33/0016 by regulating a physical variable, e.g. pressure, temperature
• CPC Class Codes: G01N 33/0047 for organic compounds
• CPC Class Codes: G05B 19/42 Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
• CPC Class Codes: G05B 2219/31048 Project on workpiece, image of finished workpiece, info or a spot
• CPC Class Codes: G05B 2219/36447 Project light on path to be followed, keep also distance constant
• CPC Class Codes: G05B 2219/40103 Show object with laser pointer, give oral command for action on, with object
• CPC Class Codes: Y02P 90/02 : Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A method includes receiving, from an imaging device associated with a robotic apparatus, an image signal including an object of interest represented in the image signal; receiving, by the robotic apparatus from an external agent, a task indication related to the object of interest, the task indication representative of a task to be performed by the robotic apparatus with respect to the object of interest; and responsive to receipt of the task indication: detecting salient features of the object of interest within the image; storing, by the robotic apparatus, a task context, the task context comprising data pertaining to the salient features and data pertaining to the task indication; and commencing, by the robotic apparatus, the task with respect to the object of interest.

Claims

1. A method, comprising:

receiving, from an imaging device associated with a robotic apparatus, an image signal including an object of interest represented in the image signal;

receiving, by the robotic apparatus from an external agent, a task indication related to the object of interest, the task indication representative of a task to be performed by the robotic apparatus with respect to the object of interest; and

responsive to receipt of the task indication;

storing, by the robotic apparatus, a task context, the task context comprising data pertaining to the task indication and being configured to cause the robotic apparatus to perform the task;

commencing, by the robotic apparatus, the task with respect to the object of interest; and

providing, by the robotic apparatus to the external agent, a confirmation signal that the robotic apparatus has commenced task execution.

2. The method of claim 1, wherein the task includes at least one of approaching the object of interest, exploring an area of interest around the object of interest, or capturing an image of the object of interest.

3. The method of claim 1, wherein the task indication is provided by the external agent through physical input to an input/output interface and comprises a touch-screen input, a speech input, a single click input to a button, or a double click input to the button.

4. The method of claim 1, wherein the confirmation signal includes a feedback indicator configured to provide the external agent confirmation that the task context was received by the robotic apparatus and wherein the feedback indicator comprises at least one of a vibration, an audible indication, and a visual indication.

5. A method, comprising:

receiving, from an imaging device associated with a robotic apparatus, an image signal including an object of interest represented in the image signal;

receiving, by the robotic apparatus from an external agent, a task indication related to the object of interest, the task indication representative of a task to be performed by the robotic apparatus with respect to the object of interest; and

responsive to receipt of the task indication;

detecting salient features of the object of interest within the image signal;

storing, by the robotic apparatus, a task context, the task context comprising data pertaining to the salient features and being configured to cause the robotic apparatus to perform the task; and

commencing, by the robotic apparatus, the task with respect to the object of interest.

6. The method of claim 5, wherein the task includes at least one of approaching the object of interest, exploring an area of interest around the object of interest, or capturing an image of the object of interest.

7. The method of claim 5, wherein the task indication includes an encoded task identifier that is detectable within the image signal.

8. The method of claim 5, wherein the task indication is irradiation of the object of interest by a beam of visible or invisible electromagnetic radiation.

9. The method of claim 5, wherein the task indication includes a task identifier in the form of a physical input executed by the external agent.

10. The method of claim 9, wherein the physical input comprises a touch-screen input, a speech input, a single click input to a button, or a double click input to the button.

11. The method of claim 5, wherein the salient features are detected using processing techniques comprising at least one of contrast enhancement, edge tracing, spectral transforms, spatial transforms, and spatio-temporal methodologies.

12. The method of claim 5, wherein data pertaining to the salient features includes a size of the salient features and a location of the salient features within the image signal.

13. The method of claim 5, responsive to receipt of the task indication, further comprising:

providing, by the robotic apparatus to the external agent, a confirmation signal that the robotic apparatus has commenced task execution.

14. The method of claim 13, wherein the confirmation signal includes a feedback indicator configured to provide the external agent confirmation that the task context was received by the robotic apparatus.

15. The method of claim 14, wherein the feedback indicator comprises at least one of a vibration, an audible indication, and a visual indication.

16. The method of claim 15, wherein the visual indication includes at least one of an LED light and an indication visible to the external agent on a mini-display.

17. An interface apparatus for a robot operable in an environment, comprising:

an input interface that receives a sequence of images of the environment; and

an input/output interface that receives a physical input executed by an external agent,

wherein the physical input identifies a task to perform in reference to an object of interest present in the sequence of images of the environment, and

wherein the physical input provides a specific robotic context configured to cause the robot to perform the task using previous training for the task over a plurality of different contexts.

18. The interface apparatus of claim 17, wherein the task includes at least one of approaching the object of interest, exploring an area of interest around the object of interest, or capturing an image of the object of interest.

19. The interface apparatus of claim 17, wherein the physical input comprises a touch-screen input, a speech input, a single click input to a button, or a double click input to the button.

20. The interface apparatus of claim 17, wherein the input/output interface includes a feedback indicator configured to provide the external agent confirmation that the robot has commenced task execution.