ROBOT DEVICE, METHOD OF CONTROLLING ROBOT DEVICE, COMPUTER PROGRAM, AND PROGRAM STORAGE MEDIUM

• US 20160236352A1
• Filed: 04/29/2016
• Published: 08/18/2016
• Est. Priority Date: 05/25/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: SAWADA, Tsutomu
• Granted Patent: US 9,802,320 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 11/0085 Cleaning
• CPC Class Codes: B25J 19/0058 Means for cleaning manipulators, e.g. dust removing means
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1674 characterised by safety, monitoring, diagnostic
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39053 Probe, camera on hand scans many points on own robot body, no extra jig
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor

Abstract

Provided is an excellent robot device capable of preferably detecting difference between dirt and a scratch on a lens of a camera and difference between dirt and a scratch on a hand.

A robot device 100 detects a site in which there is the dirt or the scratch using an image of the hand taken by a camera 305 as a reference image. Further, this determines whether the detected dirt or scratch is due to the lens of the camera 305 or the hand by moving the hand. The robot device 100 performs cleaning work assuming that the dirt is detected, and then this detects the difference between the dirt and the scratch depending on whether the dirt is removed.

Claims

1. A device, comprising:

circuitry configured to

control an image capturing device to capture a first image, and

determine whether there is dirt on the image capturing device by comparing the first image with a reference image,

wherein, in a case that it is determined that there is dirt on the image capturing device, the circuitry instructs cleaning work to be performed on the image capturing device, and further compares a second image captured by the image capturing device after the cleaning work on the image capturing device with at least one of the reference image or the first image.

2. The device according to claim 1, wherein the circuitry is configured to

calculate a correlation between the first image and the reference image;

determine whether a site exists in which the correlation between the first image and the reference image is below a predetermined value;

determine that there is no dirt on the image capturing device when the correlation between the first image and the reference image is higher than the predetermined value for an entirety of the first image and the reference image; and

determine that there is dirt on the image capturing device when it is determined that the site exists in which the correlation is lower than the predetermined value.

3. The device according to claim 2, wherein the circuitry is configured to determine that there is no dirt on the image capturing device by comparing a third image, captured by the image capturing device after an object represented in the first image is moved, with the first image.

4. The device according to claim 3, wherein the circuitry is configured to calculate a correlation between the first image and the third image, and

wherein the circuitry is configured to determine that there is the dirt on the image capturing device in a case that a correlation between the third image and the first image is higher than a predetermined value in a low-correlation site and determine that there is the dirt on the object in the case that the correlation between the third image and the first image is lower than a predetermined value in the low-correlation site determined by comparison between the second image with the first image, the low-correlation site is a site for which the correlation between the second image and the first image is lower than a predetermined value.

5. The device according to claim 4,

wherein in the case that there is the dirt on the object, the circuitry instructs cleaning work to clean the object.

6. The device according to claim 1, wherein the cleaning work to be performed on the lens of the image capturing device includes covering the image capturing device with water and/or using a moveable part to clean the image capturing device.

7. The device according to claim 1, wherein the circuitry is configured to

calculate a correlation between a plurality of sites in the first image and the plurality of sites in the reference image, and

determine whether any site of the plurality of sites exists in which the correlation between the first image and the reference image is below a predetermined value by determining that there is no dirt on the image capturing device when the correlation between the first image and the reference image is higher than the predetermined value for each site of the plurality of sites, and determining that there is dirt on the image capturing device when it is determined that any site of the plurality of sites exists in which the correlation is lower than the predetermined value.

8. The device according to claim 1, wherein the determination of whether there is dirt on the image capturing device is a determination of whether there is dirt on a lens of the image capturing device.

9. The device according to claim 1, wherein in a case that it is determined that there is dirt on the image capturing device, the circuitry outputs a notification in response to a result of the comparison of the second image, captured by the image capturing device after the cleaning work on the image capturing device, with the at least one of the reference image or the first image.

10. A method of controlling a device, comprising:

controlling an image capturing device to capture a first image; and

determining whether there is dirt on an image capturing device by comparing the first image with a reference image,

wherein, in a case that it is determined that there is dirt on the image capturing device, the method further comprises instructing cleaning work to be performed on the image capturing device, and comparing a second image captured by the image capturing device after the cleaning work on the image capturing device with at least one of the reference image or the first image.

11. The method according to claim 10, further comprising:

calculating a correlation between the first image and the reference image;

determining whether a site exists in which the correlation between the first image and the reference image is below a predetermined value;

determining that there is no dirt on the image capturing device when the correlation between the first image and the reference image is higher than the predetermined value for an entirety of the first image and the reference image; and

determining that there is dirt on the image capturing device when it is determined that the site exists in which the correlation is lower than the predetermined value.

12. The method according to claim 11, further comprising:

determining that there is no dirt on the image capturing device by comparing a third image, captured by the image capturing device after an object represented in the first image is moved, with the first image.

13. The method according to claim 12, further comprising:

calculating a correlation between the first image and the third image; and

determining that there is the dirt on the image capturing device in a case that a correlation between the third image and the first image is higher than a predetermined value in a low-correlation site and determine that there is the dirt on the object in the case that the correlation between the third image and the first image is lower than a predetermined value in the low-correlation site determined by comparison between the second image with the first image, the low-correlation site is a site for which the correlation between the second image and the first image is lower than a predetermined value.

14. The method according to claim 13, further comprising:

in the case that there is the dirt on the object, instructing cleaning work to clean the object.

15. The method according to claim 10, wherein the cleaning work to be performed on the lens of the image capturing device includes covering the image capturing device with water and/or using a moveable part to clean the image capturing device.

16. The method according to claim 10, further comprising:

calculating a correlation between a plurality of sites in the first image and the plurality of sites in the reference image; and

determining whether any site of the plurality of sites exists in which the correlation between the first image and the reference image is below a predetermined value by determining that there is no dirt on the image capturing device when the correlation between the first image and the reference image is higher than the predetermined value for each site of the plurality of sites, and determining that there is dirt on the image capturing device when it is determined that any site of the plurality of sites exists in which the correlation is lower than the predetermined value.

17. The method according to claim 10, wherein the determination of whether there is dirt on the image capturing device is a determination of whether there is dirt on a lens of the image capturing device.

18. The method according to claim 10, wherein in a case that it is determined that there is dirt on the image capturing device, outputting a notification in response to a result of the comparison of the second image, captured by the image capturing device after the cleaning work on the image capturing device, with the at least one of the reference image or the first image.

19. A non-transitory program storage medium storing thereon control program that when executed by a computer causes the computer to implement a method comprising:

controlling an image capturing device to capture a first image; and

determining whether there is dirt on an image capturing device by comparing the first image with a reference image,

wherein, in a case that it is determined that there is dirt on the image capturing device, the method further comprises instructing cleaning work to be performed on the image capturing device, and comparing a second image captured by the image capturing device after the cleaning work on the imaging capturing device with at least one of the reference image or the first image.