Robot device, method of controlling robot device, computer program, and program storage medium

• US 9,358,690 B2
• Filed: 04/16/2012
• Issued: 06/07/2016
• Est. Priority Date: 05/25/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sawada, Tsutomu
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US14/114,333
• Publication Number:
  US 20140074292A1
• Time in Patent Office: 1,513 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 11/0085 Cleaning
• CPC Class Codes: B25J 19/0058 Means for cleaning manipulators, e.g. dust removing means
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1674 characterised by safety, monitoring, diagnostic
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39053 Probe, camera on hand scans many points on own robot body, no extra jig
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor

Abstract

Provided is an excellent robot device capable of preferably detecting difference between dirt and a scratch on a lens of a camera and difference between dirt and a scratch on a hand.

A robot device 100 detects a site in which there is the dirt or the scratch using an image of the hand taken by a camera 305 as a reference image. Further, this determines whether the detected dirt or scratch is due to the lens of the camera 305 or the hand by moving the hand. The robot device 100 performs cleaning work assuming that the dirt is detected, and then this detects the difference between the dirt and the scratch depending on whether the dirt is removed.

Claims

1. A device, comprising:

circuitry configured to

control an image capturing device to capture a first image of an object, and

determine whether there is dirt on a lens of the image capturing device or whether there is dirt on the object by comparing the first image with a reference image of the object, which is separate from the lens;

wherein, in a case that it is determined that there is dirt on the lens of the image capturing device, the circuitry controls a movable part to clean the lens of the image capturing device, and outputs a notification in response to a result of comparison between a second image of the object captured by the image capturing device after the movable part cleans the lens and at least one of the reference image or the first image.

2. The device according to claim 1, wherein the circuitry is configured to

calculate a correlation between the first image and the reference image of the object;

determine whether a site exists in which the correlation between the first image and the reference image is below a predetermined value;

determine that there is no dirt on the lens of the image capturing device when the correlation between the first image and the reference image is higher than the predetermined value for an entirety of the first image and the reference image; and

determine that there is dirt on the lens of the image capturing device when it is determined that the site exists in which the correlation is lower than the predetermined value.

3. The device according to claim 1, wherein the circuitry is configured to determine on which of the lens of the image capturing device and the object the dirt is by comparing a third image captured by the image capturing device after the object is moved from a position at which the first image is captured with the first image of the object in the case that there is the dirt on the lens of the image capturing device or the object.

4. The device according to claim 3,

wherein in the case that there is the dirt on the object, the circuitry controls the movable part to clean the object and outputs a notification in response to a result of comparison between the second image and the reference image of the object.

5. The device according to claim 1, wherein

the circuitry controls the movable part to wipe the lens of the image capturing device.

8. The device according to claim 3, wherein

the circuitry is configured to determine that there is the dirt on the lens of the image capturing device in a case that a correlation between the third image and the first image is higher than a predetermined value in a low-correlation site and determine that there is the dirt on the object in the case that the correlation between the third image and the first image is lower than a predetermined value in the low-correlation site determined by comparison between the second image with the first image of the object, the low-correlation site is a site for which the correlation between the second image and the first image is lower than a predetermined value.

10. The device according claim 1, wherein the circuitry is configured to determine whether there is the dirt on the lens of the image capturing device or the object by comparing the first image with the reference image of the object.

11. The device according claim 10,

wherein the circuitry is configured to determine that there is no dirt on the lens of the image capturing device and the object in a case that a correlation between the first image and the reference image is higher than a predetermined value for an entirety of the first image and the reference image and determine that there is the dirt on the lens of the image capturing device or the object in a case that the correlation is lower than the predetermined value for a site of the first image and the reference image.

6. A method of controlling a device, comprising:

controlling an image capturing device to capture a first image of an object, and

determining whether there is dirt on a lens of an image capturing device or whether there is dirt on the object by comparing the first image with a reference image of the object, which is separate from the lens;

wherein, in a case that it is determined that there is dirt on the lens of the image capturing device, the method further comprises controlling a movable part to clean the lens of the image capturing device, and outputting a notification in response to a result of comparison between a second image of the object captured by the image capturing device after the movable part cleans the lens and at least one of the reference image or the first image.

7. A non-transitory program storage medium storing a control program of a robot device, which allows a computer to execute:

controlling an image capturing device to capture a first image of an object, and

determining whether there is dirt on a lens of an image capturing device or whether there is dirt on the object by comparing the first image with a reference image of the object, which is separate from the lens;

wherein, in a case that it is determined that there is dirt on the lens of the image capturing device, the method further comprises controlling a movable part to clean the lens of the image capturing device, and outputting a notification in response to a result of comparison between a second image of the object captured by the image capturing device after the movable part cleans the lens and at least one of the reference image or the first image.

9. A system, comprising:

an image capturing device;

a movable part; and

circuitry configured to

control the image capturing device to capture a first image of an object, and determine whether there is dirt on a lens of the image capturing device or whether there is dirt on the object by comparing the first image with a reference image of the object which is separate from the lens;

wherein, when it is determined that there is dirt on the lens of the image capturing device, the circuitry controls a movable part to clean the lens of the image capturing device, and outputs a notification in response to a result of comparison between a second image of the object captured by the image capturing device after the movable part cleans the lens and at least one of the reference image or the first image.