State determination system, state determination method, and movable robot

• US 10,740,611 B2
• Filed: 12/19/2014
• Issued: 08/11/2020
• Est. Priority Date: 12/26/2013
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Saito, Fuminori
• Primary Examiner(s): Bali, Vikkram
• Application Number:
  US14/577,298
• Publication Number:
  US 20150186729A1
• Time in Patent Office: 2,062 Days
• Field of Search: None
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

An state determination system comprises an information acquiring portion that acquires an environment information, a movable body, an instruction portion that performs an action instruction to the movable body, and an estimation portion that estimates an abnormality of the object based on at least one of the information acquired by the information acquiring portion and the environment information acquired by the movable body. In case the estimation portion estimates the abnormality of the object, the instruction portion performs the action instruction for moving to the neighborhood of the object and requiring a response, to the movable body. At least one of the information acquiring portion and the movable body acquires the response state of the object to the action of the movable body instructed by the instruction portion. The state determination system comprises a determination portion that the abnormality of the object based on the acquired response state.

Claims

1. A state determination system comprising:

a first detector that is provided in at least one of a predetermined environment and an object in the predetermined environment, and acquires an environment information of the object;

a movable body that moves autonomously and includes a second detector that acquires a circumferential environment information around the movable body, the circumferential environment information including at least one of an image or a voice; and

a processor that

sends an action instruction to the movable body, estimates an abnormality of the object based on at least one of the environment information acquired by the first detector and the circumferential environment information acquired by the movable body, and

determines a position of the object estimated as the abnormality, wherein in a case where the processor estimates (a) that the abnormality of the object exists based on at least one of the environment information acquired by the first detector and the circumferential environment information acquired by the movable body and (b) that the position of the object is in a predetermined private area where an obstacle is between the moveable body and the object to block the object, the predetermined private area being a bathroom, a toilet, a bedroom, or a kitchen,

(i) the processor sends the action instruction to the movable body for the movable body to move to a neighborhood of the position of the object, to take an action that elicits a response from the object, and to require a response from the object,

(ii) the moveable body opens a door between the moveable body and the object in response to the action instruction,

(iii) at least one of the first detector and the movable body directly acquires at least one of an image or a voice of a response state of the object in response to the action of the movable body instructed by the processor that elicits the response from the object, and

(iv) the processor determines the abnormality of the object based on the directly acquired image or voice of the response state of the object, and

when the processor determines that the object being estimated to be

abnormal is at a position within a predetermined area which is set in advance, the action Instruction includes an instruction for the movable body to move to the position.

2. The state determination system according to claim 1,

wherein the processor estimates the abnormality of the object also based on an action pattern information of the object.

3. The state determination system according to claim 1, further comprising a reporting device that reports in case the processor determines that the object is abnormal.

4. A state determination method for a state determination

system which comprises:

a first detector that is provided in at least one of a predetermined

environment and an object in the predetermined environment, and acquires an environment information of the object;

a movable body that moves autonomously and includes a second detector that acquires a circumferential environment information around the movable body,

the circumferential environment information including at least one of an image or a voice, the method comprising;

estimating an abnormality of the object based on at least one of the environment information acquired by the first detector and the circumferential environment information acquired by the movable body;

determining a position of the object;

sending an action instruction to the movable body for the movable body to move to a neighborhood of the position of the object, to take an action that elicits a response from the object, and to require a response from the object, when it is estimated that the abnormality of the object exists in a predetermined private area where an obstacle is between the moveable body and the object to block the object, the predetermined private area being a bathroom, a toilet, a bedroom, or a kitchen;

at least one of the first detector and the movable body acquiring directly at least one of an image or a voice of a response state of the object in response to the instructed action of the movable body that elicits the response from the object; and

determining the abnormality of the object based on the directly acquired image or voice of the response state of the object,

wherein sending an action instruction includes instructing the moveable body to opens a door between the movable body and the object, and

when the processor determines that the object being estimated to be abnormal is at a position within a predetermined area which is set in advance, the action instruction includes an instruction for the movable body to move to the position.

5. A movable robot comprising:

a first sensor that acquires a circumferential environment information around the movable robot, the circumferential environment information including at least one of an image or a voice, and

a processor that estimates an abnormality of an object in a predetermined environment based on at least one of an environment information of the object acquired by a second sensor provided in at least one of the predetermined environment and the object, and the circumferential environment information acquired by the first sensor,

wherein in a case where the processor estimates that the abnormality of the object exists in a predetermined private area where an obstacle is between the moveable robot and the object to block the object, the predetermined private area being a bathroom, a toilet, a bedroom, or a kitchen,

(i) the processor sends an action instruction for the movable robot to perform an action of moving to a neighborhood of the position of the object, of taking an action that elicits a response from the object, and requiring a response from the object,

(ii) the moveable robot opens a door between the moveable body and the object in response to the action instructions,

(iii) at least one of the first sensor and second sensor directly acquires at least one of an image or a voice of a response state of the object in response to the action of the movable robot that elicits the response from the object, and

(iv) the processor determines the abnormality of the object based on the image or voice of the response state of the object directly acquired by at least one of the first sensor and second sensor, and

when the processor determines that the object being estimated to be abnormal is at a position within a predetermined area which is set in advance, the action Instruction includes an instruction for the movable body to move to the position.

6. A state determination system comprising:

an information acquiring portion that is provided in at least one of a predetermined environment and an object in the predetermined environment, and acquires an environment information of the object;

a movable body that moves autonomously and includes a circumferential environment information acquiring portion that acquires a circumferential environment information around the movable body, the circumferential environment information including at least one of an image or a voice;

an instruction portion that sends an action instruction to the movable body;

and

an estimation portion that estimates an abnormality of the object based on at least one of the environment information acquired by the information acquiring portion and the circumferential environment information acquired by the movable body,

wherein in a case where the estimation portion estimates that the abnormality of the object exists in a private area where an obstacle is between the moveable robot and the object to block the object, the predetermined private area being a bathroom, a toilet, a bedroom, or a kitchen,

(i) the instruction portion sends the action instruction to the movable body for the movable body to move to a neighborhood of the position of the object, take an action that elicits a response from the object, and require a response from the object,

(ii) the moveable body opens a door between the moveable body and the object in response to the action instructions;

(iii) at least one of the information acquiring portion and the movable body directly acquires at least one of an image or a voice of a response state of the object in response to the action of the movable body instructed by the instruction portion that elicits the response from the object, and

(iv) a determination portion determines the abnormality of the object based on the directly acquired image or voice of the response state, and

when the processor determines that the object being estimated to be abnormal is at a position within a predetermined area which is set in advance, the action instruction includes an instruction for the movable body to move to the position.