Robots comprising projectors for projecting images on identified projection surfaces

• US 9,943,965 B2
• Filed: 02/04/2015
• Issued: 04/17/2018
• Est. Priority Date: 08/03/2012
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Moore, Douglas Alan
• Primary Examiner(s): McPherson, James M
• Application Number:
  US14/613,518
• Publication Number:
  US 20150148960A1
• Time in Patent Office: 1,168 Days
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: H04N 9/3179 Video signal processing therefor
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Robots including projectors for projecting images on identified projection surfaces are disclosed. A robot includes a housing, an electronic control unit coupled to the housing, a projector coupled to the housing, a human recognition module coupled to the housing, and a projection surface identification module coupled to the housing. The projector, the human recognition module, and the surface identification module are communicatively coupled with the electronic control unit. The electronic control unit includes a non-transitory memory that stores a set of machine readable instructions and a processor for executing the machine readable instructions. When executed by the processor, the machine readable instructions cause the robot to recognize a human using the human recognition module, identify a projection surface using the projection surface recognition module, and project an image on the identified projection surface with the projector.

Claims

1. A robot comprising:

a housing;

an electronic control unit coupled to the housing, wherein the electronic control unit comprises a non-transitory memory that stores a set of machine readable instructions and a processor for executing the machine readable instructions;

a projector coupled to the housing and communicatively coupled with the electronic control unit;

a movement mechanism coupled to the housing and communicatively coupled with the electronic control unit;

a human recognition module coupled to the housing and communicatively coupled with the electronic control unit;

a camera configured to output image data to the electronic control unit; and

a projection surface identification module coupled to the housing and communicatively coupled with the electronic control unit, wherein when executed by the processor, the machine readable instructions cause the robot to;

identify a location of a human using the human recognition module;

detect movement of the human by analyzing the image data across frames of the image data;

predict a future location of the human based on the detected movement; and

project an image on an area of a projection surface based on the predicted future location.

2. The robot of claim 1, wherein the movement mechanism comprises:

a drive wheel rotatably coupled to the housing;

a drive motor coupled to the drive wheel and communicatively coupled with the electronic control unit; and

a steering mechanism coupled to the housing and communicatively coupled with the electronic control unit.

3. The robot of claim 1, wherein the movement mechanism comprises a rotor and a drive motor coupled to the rotor.

4. The robot of claim 1 further comprising a wireless communicator coupled to the housing and communicatively coupled with the electronic control unit, wherein when executed by the processor, the machine readable instructions further cause the robot to:

receive data with the wireless communicator; and

generate the projected image based on the received data.

5. The robot of claim 1, wherein when executed by the processor, the machine readable instructions further cause the robot to:

determine a distance between the robot and the area of the projection surface;

determine a size of the area of the projection surface; and

scale the projected image based on the determined distance and the determined size, such that the entire projected image is displayed on the area of the projection surface.

6. The robot of claim 1 wherein when executed by the processor, the machine readable instructions further cause the robot to:

identify the location and an orientation of the human using the human recognition module; and

identify the area of the projection surface based on the location and orientation.

7. The robot of claim 1, wherein the human recognition module comprises the camera, wherein the human recognition module recognizes the human based on facial recognition of an image received from the camera.

8. The robot of claim 1, wherein the human recognition module comprises a microphone communicatively coupled to the electronic control unit, wherein the human recognition module recognizes the human based on a mechanical vibration received by the microphone and transformed into a signal indicative of human speech that is transmitted to the electronic control unit.

9. The robot of claim 1, wherein the projection surface identification module comprises the camera.

10. The robot of claim 1, wherein the projector is a pico projector.

11. The robot of claim 1, wherein the projected image substantially resembles a human.

12. The robot of claim 1, wherein the projected image comprises a face.

13. The robot of claim 1 further comprising a speaker for providing an audible indication of received information to a human, wherein the speaker is communicatively coupled to the electronic control unit.

14. A robot comprising:

a housing;

an electronic control unit coupled to the housing, wherein the electronic control unit comprises a non-transitory memory that stores a set of machine readable instructions and a processor for executing the machine readable instructions;

a projector coupled to the housing and communicatively coupled with the electronic control unit;

a movement mechanism coupled to the housing and communicatively coupled with the electronic control unit; and

a camera coupled to the housing and communicatively coupled with the electronic control unit,

wherein when executed by the processor, the machine readable instructions cause the robot to;

identify a location of a human using the camera;

identify a projection area based on the location of the human using the camera;

cause the movement mechanism to automatically move the robot to a position based on the identified projection area; and

project only a portion of a single image on the identified projection area with the projector from the position,

wherein the movement mechanism comprises;

a drive wheel rotatably coupled to the housing;

a drive motor coupled to the drive wheel and communicatively coupled with the electronic control unit; and

a steering mechanism coupled to the housing and communicatively coupled with the electronic control unit, wherein when executed by the processor, the machine readable instructions further cause the robot to;

cause the drive motor to rotate the drive wheel and the steering mechanism to steer the robot to the position.

15. The robot of claim 14 further comprising a wireless communicator coupled to the housing and communicatively coupled with the electronic control unit, wherein when executed by the processor, the machine readable instructions further cause the robot to:

receive data with the wireless communicator; and

generate the projected portion of the single image based on the received data.

18. The robot of claim 14, wherein when executed by the processor, the machine readable instructions cause the robot to

project only the portion of the single image on the identified projection area with the projector from the position such that the single image is formed by combining the portion and one or more other portions of the single image that are projected adjacent to the identified projection area.

20. A projection system comprising:

the robot of claim 14; and

a second robot comprising

a second electronic control unit, wherein the second electronic control unit comprises a second non-transitory memory that stores a second set of machine readable instructions and a second processor for executing the second set of machine readable instructions;

a second projector communicatively coupled with the second electronic control unit;

a second movement mechanism communicatively coupled with the second electronic control unit; and

a second camera communicatively coupled with the second electronic control unit,

wherein when executed by the second processor, the second set of machine readable instructions cause the second robot to;

identify a second location of the human using the second camera;

identify a second projection area based on the second location of the human using the second camera;

cause the second movement mechanism to automatically move the second robot to a second position based on the identified second projection area; and

project one or more other portions of the single image on the second projection area from the second position, wherein the second projection area is adjacent to the projection area identified by the robot of claim 14.

16. A robot comprising:

a housing;

an electronic control unit coupled to the housing, wherein the electronic control unit comprises a non-transitory memory that stores a set of machine readable instructions and a processor for executing the machine readable instructions;

a projector coupled to the housing and communicatively coupled with the electronic control unit;

a movement mechanism coupled to the housing and communicatively coupled with the electronic control unit;

a microphone coupled to the housing and communicatively coupled with the electronic control unit; and

a camera coupled to the housing and communicatively coupled with the electronic control unit, wherein when executed by the processor, the machine readable instructions cause the robot to;

recognize a human based on a mechanical vibration received by the microphone and transforming the mechanical vibration into a signal indicative of human speech that is transmitted to the electronic control unit;

identify a location of the human using at least one of the camera and the microphone;

identify a projection area based on the location of the human using the camera;

cause the movement mechanism to automatically move the robot to a position based on the identified projection area; and

project only a portion of a single image on the identified projection area with the projector from the position,

wherein the movement mechanism comprises;

a drive wheel rotatably coupled to the housing;

a drive motor coupled to the drive wheel and communicatively coupled with the electronic control unit; and

a steering mechanism coupled to the housing and communicatively coupled with the electronic control unit, wherein when executed by the processor, the machine readable instructions further cause the robot to;

cause the drive motor to rotate the drive wheel and the steering mechanism to steer the robot to the position.

17. The robot of claim 16 further comprising a wireless communicator coupled to the housing and communicatively coupled with the electronic control unit, wherein when executed by the processor, the machine readable instructions further cause the robot to:

receive data with the wireless communicator; and

generate the projected portion of the single image based on the received data.

19. The robot of claim 16, wherein when executed by the processor, the machine readable instructions cause the robot to

project only the portion of the single image on the identified projection area with the projector from the position such that the single image is formed by combining the portion and one or more other portions of the single image that are projected adjacent to the identified projection area.

21. A projection system comprising:

the robot of claim 16; and

a second robot comprising;

a second electronic control unit, wherein the second electronic control unit comprises a second non-transitory memory that stores a second set of machine readable instructions and a second processor for executing the second set of machine readable instructions;

a second projector communicatively coupled with the second electronic control unit;

a second movement mechanism communicatively coupled with the second electronic control unit;

a second camera communicatively coupled with the second electronic control unit; and

a second microphone communicatively coupled with the electronic control unit,

wherein when executed by the second processor, the second set of machine readable instructions cause the second robot to;

recognize the human based on a second mechanical vibration received by the second microphone and transforming the second mechanical vibration into a second signal indicative of human speech that is transmitted to the second electronic control unit;

identify a second location of the human using at least one of the second camera and the second microphone;

identify a second projection area based on the second location of the human using the second camera;

cause the second movement mechanism to automatically move the second robot to a second position based on the identified projection area; and

project one or more other portions of the single image on the second projection area from the second position, wherein the second projection area is adjacent to the projection area identified by the robot of claim 16.