Training of autonomous robots

• US 6,760,645 B2
• Filed: 04/29/2002
• Issued: 07/06/2004
• Est. Priority Date: 04/30/2001
• Status: Expired due to Fees

• Current Assignee: Sony France SA (Sony Group Corp.)
• Original Assignee: Sony France SA (Sony Group Corp.)
• Inventors: Oudeyer, Pierre-Yves, Kaplan, Frédéric
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): Marc, McDieunel, Marc, McDieunel
• Application Number:
  US10/134,909
• Publication Number:
  US 20020183895A1
• Time in Patent Office: 799 Days
• Field of Search: 700/245, 700/31, 700/259, 700/248, 700/258, 318/568.1, 318/568.11, 318/568.12, 318/568.2, 318/569, 318/16, 318/565, 901/1, 901/15, 901/47, 704/207, 704/209, 704/270, 348/121
• US Class Current: 700/245
• CPC Class Codes: A63H 11/00 Self-movable toy figures
• CPC Class Codes: A63H 2200/00 Computerized interactive toys, e.g. dolls

Abstract

A clicker-training technique developed for animal training is adapted for training robots, notably autonomous animal-like robots. In this robot-training method, a behaviour (for example, (DIG)) is broken down into smaller achievable responses ((SIT)-(HELLO)-(DIG)) that will eventually lead to the desired final behaviour. The robot is guided progressively to the correct behaviour through the use, normally the repeated use, of a secondary reinforcer. When the correct behaviour has been achieved, a primary reinforcer is applied so that the desired behaviour can be “captured”. This method can be used for training a robot to perform, on command, rare behaviours or a sequence of behaviours (typically actions). This method can also be used to ensure that a robot is focusing its attention upon a desired object.

Claims

1. A method of programming a robot to perform a desired behaviour, the method comprising the steps of:

providing a robot for recognizing at least one stimulus as a primary reinforcer and;

conditioning the robot to recognize at least one further stimulus as a secondary reinforcer;

guiding the robot to the desired behaviour by presenting the robot with a secondary reinforcer when the robot exhibits a behaviour approaching the desired behaviour and presenting the robot with a primary reinforcer when the robot exhibits the desired behaviour.

2. The robot programming method of claim 1, wherein the providing step comprises providing a robot which, in use, in response to perception of a secondary reinforcer, exhibits a behaviour related to the exhibited behaviour that prompted appearance of the secondary reinforcer and, in response to perception of a primary reinforcer, registers one or more of the exhibited behaviours that prompted appearance of the primary reinforcer.

3. The robot programming method of claim 2, wherein the providing step comprises providing a robot which, in use, in response to perception of a primary reinforcer, repeats the one or more registered behaviours, and the method further comprises the step of presenting the robot with a primary reinforcer if the repeated one or more behaviours corresponds to the desired behaviour.

4. A method according to claim 1, for programming a robot to perform a sequence of desired behaviours, the method comprising the step of presenting the robot with a secondary reinforcer when the robot exhibits a desired behaviour of said sequence and presenting the robot with a primary reinforcer after the robot has exhibited the sequence of desired behaviours.

5. The robot programming method of claim 1, wherein the desired behaviour is the directing of the robot'"'"'s attention on a particular subject, and the guiding step comprises presenting the robot with a secondary reinforcer as the robot directs the visual apparatus thereof more and more precisely towards said particular subject and presenting the robot with a primary reinforcer when the robot directs the visual apparatus thereof at said particular subject.

6. The robot programming method of claim 1, wherein the step of conditioning the robot to recognize at least one further stimulus as a secondary reinforcer comprises repeatedly presenting the robot with said further stimulus in association with a primary reinforcer.

7. The robot programming method of claim 1, and comprising the step of providing the robot with a command indication immediately after provision of a primary reinforcer.

8. The robot programming method of claim 7, wherein the providing step comprises providing a robot which, in use, provides feedback enabling the command indication to be confirmed.

9. The robot programming method of claim 7, wherein the command indication is a spoken word or hand signal.

10. The robot programming method of claim 1, wherein the providing step comprises providing a robot which, in use, undergoes a transition from one behaviour to another behaviour thereof according to a respective probability, wherein the probability of a transition taking place between a particular pair of behaviours is increased if the exhibition of said transition occurs and prompts appearance of a secondary or primary reinforcer.

11. An autonomous robot programmable by a method according to claim 1, wherein the robot comprises:

means for recognizing at least one stimulus as a primary reinforcer, and

means for enabling at least one further stimulus to be identified as a secondary reinforcer.

12. The autonomous robot according to claim 11, which, in use, in response to perception of a secondary reinforcer, exhibits a behaviour related to to the exhibited behaviour that prompted appearance of the secondary reinforcer and, in response to perception of a primary reinforcer, registers one or more of the exhibited behaviours that prompted appearance of the primary reinforcer.

13. The autonomous robot according to claim 12, which, in use, in response to perception of a primary reinforcer, repeats the one or more registered behaviours, and confirms registration of said one or more behaviours if the repetition prompts appearance of a primary reinforcer.

14. The autonomous robot according to claim 11, wherein said enabling means for recognizing at least one further stimulus as a secondary reinforcer when the robot perceives said further stimulus repeatedly presented thereto in association with a primary stimulus.

15. The autonomous robot according to claim 11, which, in use, in response to perception of a primary reinforcer, awaits presentation of, and registers, a command indication.

16. The autonomous robot according to claim 15, which, in use, provides feedback enabling the command indication to be confirmed.

17. The autonomous robot according to claim 11, which, in use, undergoes a transition from one behaviour to another behaviour thereof according to a respective probability, wherein the probability of a transition taking place between a particular pair of behaviours is increased if the exhibition of said transition occurs and prompts appearance of a secondary or primary reinforcer.