ROBOT DEVICE, METHOD OF CONTROLLING ROBOT DEVICE, COMPUTER PROGRAM, AND PROGRAM STORAGE MEDIUM

• US 20140074292A1
• Filed: 04/16/2012
• Published: 03/13/2014
• Est. Priority Date: 05/25/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sawada, Tsutomu
• Granted Patent: US 9,358,690 B2
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: B25J 11/0085 Cleaning
• CPC Class Codes: B25J 19/0058 Means for cleaning manipulators, e.g. dust removing means
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1674 characterised by safety, monitoring, diagnostic
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39053 Probe, camera on hand scans many points on own robot body, no extra jig
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor

Abstract

Provided is an excellent robot device capable of preferably detecting difference between dirt and a scratch on a lens of a camera and difference between dirt and a scratch on a hand.

A robot device 100 detects a site in which there is the dirt or the scratch using an image of the hand taken by a camera 305 as a reference image. Further, this determines whether the detected dirt or scratch is due to the lens of the camera 305 or the hand by moving the hand. The robot device 100 performs cleaning work assuming that the dirt is detected, and then this detects the difference between the dirt and the scratch depending on whether the dirt is removed.

Claims

1. A robot device, comprising:

a camera;

a hand; and

a controller, which processes an image taken by the camera and controls operation of the hand, wherein

the controller

obtains a reference image obtained by photographing the hand set in a specific position relative to the camera,

detects whether there is dirt on a lens of the camera or the hand by comparing a first detection image obtained by photographing the hand set in the specific position with the reference image at the time of dirt detection, and

determines on which of the lens of the camera and the hand the dirt is by comparing a second detection image obtained by photographing the hand moved from the specific position with the first detection image when detecting the dirt on the lens of the camera or the hand.

2. The robot device according to claim 1, wherein

the controller determines that there is no dirt on the lens of the camera and the hand when correlation between the first detection image and the reference image is high over an entire image and detects the dirt on the lens of the camera or the hand in a low-correlation site when there is the low-correlation site with low correlation on the image.

3. The robot device according to claim 2, wherein

the controller determines that there is the dirt on the lens of the camera when the correlation is high in the low-correlation site on the image and determines that there is the dirt on the hand when the correlation is low in the low-correlation site on the image when comparing the second detection image with the first detection image.

4. The robot device according to claim 1, wherein

the controller wipes a dirty portion on the lens of the camera and then compares a third detection image obtained by photographing the hand set in the specific position with the reference image to detect whether there is a scratch on the lens of the camera when determining that there is the dirt on the lens of the camera.

5. The robot device according to claim 1, wherein

the controller wipes a dirty portion on the hand and then compares a fourth detection image obtained by photographing the hand set in the specific position with the reference image to detect whether there is a scratch on the hand when determining that there is the dirt on the hand.

6. A method of controlling a robot device, comprising:

a step of obtaining a reference image by photographing a hand set in a specific position relative to a camera of a robot;

a dirt detecting step of detecting whether there is dirt on a lens of the camera or the hand by comparing a first detection image obtained by photographing the hand set in the specific position with the reference image at the time of dirt detection; and

a determining step of determining on which of the lens of the camera and the hand the dirt is by comparing a second detection image obtained by photographing the hand moved from the specific position with the first detection image when detecting the dirt on the lens of the camera or the hand at the dirt detecting step.

7. A computer program for controlling a robot device, which allows a computer to execute:

a step of obtaining a reference image by photographing a hand set in a specific position relative to a camera of a robot;

a dirt detecting step of detecting whether there is dirt on a lens of the camera or the hand by comparing a first detection image obtained by photographing the hand set in the specific position with the reference image at the time of dirt detection; and

a determining step of determining on which of the lens of the camera and the hand the dirt is by comparing a second detection image obtained by photographing the hand moved from the specific position with the first detection image when detecting the dirt on the lens of the camera or the hand at the dirt detecting step.

8. A program storage medium storing a control program of a robot device, which allows a computer to execute:

a step of obtaining a reference image by photographing a hand set in a specific position relative to a camera of a robot;

a dirt detecting step of detecting whether there is dirt on a lens of the camera or the hand by comparing a first detection image obtained by photographing the hand set in the specific position with the reference image at the time of dirt detection; and

a determining step of determining on which of the lens of the camera and the hand the dirt is by comparing a second detection image obtained by photographing the hand moved from the specific position with the first detection image when detecting the dirt on the lens of the camera or the hand at the dirt detecting step.