Training of autonomous robots

• US 20020183895A1
• Filed: 04/29/2002
• Published: 12/05/2002
• Est. Priority Date: 04/30/2001
• Status: Active Grant

• Current Assignee: Sony France SA (Sony Group Corp.)
• Original Assignee: Sony France SA (Sony Group Corp.)
• Inventors: Kaplan, Frederic, Oudeyer, Pierre-Yves
• Granted Patent: US 6,760,645 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: A63H 11/00 Self-movable toy figures
• CPC Class Codes: A63H 2200/00 Computerized interactive toys, e.g. dolls

Abstract

A clicker-training technique developed for animal training is adapted for training robots, notably autonomous animal-like robots. In this robot-training method, a behaviour (for example, [DIG]) is broken down into smaller achievable responses ([SIT]—[HELLO]—[DIG]) that will eventually lead to the desired final behaviour. The robot is guided progressively to the correct behaviour through the use, normally the repeated use, of a secondary reinforcer. When the correct behaviour has been achieved, a primary reinforcer is applied so that the desired behaviour can be “captured”. This method can be used for training a robot to perform, on command, rare behaviours or a sequence of behaviours (typically actions). This method can also be used to ensure that a robot is focusing its attention upon a desired object.

Claims

1. A method of programming a robot to perform a desired behaviour, the method comprising the application of a clicker-training technique.

12. An autonomous robot adapted to be programmable by a method according to any previous claim, wherein the robot comprises:

means for recognising at least one stimulus as a primary reinforcer, and

means enabling at least one further stimulus to be identified as a secondary reinforcer.

15. The autonomous robot according to any one of claims 12 to 14, wherein said enabling means is adapted to be able to recognise at least one further stimulus as a secondary reinforcer when the robot perceives said further stimulus repeatedly presented thereto in association with a primary stimulus.

16. The autonomous robot according to any one of claims 12 to 14, adapted, in response to perception of a primary reinforcer, to await presentation of, and register, a command indication.

18. The autonomous robot according to any one of claims 12 to 17, adapted to undergo a transition from one behaviour to another behaviour thereof according to a respective probability, wherein the probability of a transition taking place between a particular pair of behaviours is increased if the exhibition of said transition occurs and prompts appearance of a secondary or primary reinforcer.

2. A method of programming a robot to perform a desired behaviour, the method comprising the steps of:

providing a robot adapted to recognise at least one stimulus as a primary reinforcer and;

conditioning the robot to recognise at least one further stimulus as a secondary reinforcer;

guiding the robot to the desired behaviour by presenting the robot with a secondary reinforcer when the robot exhibits a behaviour approaching the desired behaviour and presenting the robot with a primary reinforcer when the robot exhibits the desired behaviour.

3. The robot programming method of claim 2, wherein the providing step comprises providing a robot adapted, in response to perception of a secondary reinforcer, to exhibit a behaviour related to the exhibited behaviour that prompted appearance of the secondary reinforcer and adapted, in response to perception of a primary reinforcer, to register one or more of the exhibited behaviours that prompted appearance of the primary reinforcer.

4. The robot programming method of claim 3, wherein the providing step comprises providing a robot adapted, in response to perception of a primary reinforcer, to repeat the one or more registered behaviours, and the method further comprises the step of presenting the robot with a primary reinforcer if the repeated one or more behaviours corresponds to the desired behaviour.

5. A method according to claim 2, 3 or 4, for programming a robot to perform a sequence of desired behaviours, the method comprising the step of presenting the robot with a secondary reinforcer when the robot exhibits a desired behaviour of said sequence and presenting the robot with a primary reinforcer after the robot has exhibited the sequence of desired behaviours.

6. The robot programming method of claim 2, 3 or 4, wherein the desired behaviour is the directing of the robot'"'"'s attention on a particular subject, and the guiding step comprises presenting the robot with a secondary reinforcer as the robot directs the visual apparatus thereof more and more precisely towards said particular subject and presenting the robot with a primary reinforcer when the robot directs the visual apparatus thereof at said particular subject.

7. The robot programming method of any one of claims 2 to 6, wherein the step of adapting the robot to recognise at least one further stimulus as a secondary reinforcer comprises repeatedly presenting the robot with said further stimulus in association with a primary reinforcer.

8. The robot programming method of any one of claims 2 to 7, and comprising the step of providing the robot with a command indication immediately after provision of a primary reinforcer.

9. The robot programming method of claim 8, wherein the providing step comprises providing a robot adapted to provide feedback enabling the command indication to be confirmed.

10. The robot programming method of claim 8 or 9, wherein the command indication is a spoken word or hand signal.

11. The robot programming method of any one of claims 2 to 9, wherein the providing step comprises providing a robot adapted to undergo a transition from one behaviour to another behaviour thereof according to a respective probability, wherein the probability of a transition taking place between a particular pair of behaviours is increased if the exhibition of said transition occurs and prompts appearance of a secondary or primary reinforcer.

13. The autonomous robot according to claim 12, adapted, in response to perception of a secondary reinforcer, to exhibit a behaviour related to the exhibited behaviour that prompted appearance of the secondary reinforcer and adapted, in response to perception of a primary reinforcer, to register one or more of the exhibited behaviours that prompted appearance of the primary reinforcer.

14. The autonomous robot according to claim 13, adapted, in response to perception of a primary reinforcer, to repeat the one or more registered behaviours, and to confirm registration of said one or more behaviours if the repetition prompts appearance of a primary reinforcer.

17. The autonomous robot according to claim 16, adapted to provide feedback enabling the command indication to be confirmed.