Moving robot and method of recognizing location of a moving robot

• US 10,303,179 B2
• Filed: 04/08/2016
• Issued: 05/28/2019
• Est. Priority Date: 04/08/2015
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Shin, Yongmin, Nam, Kyungmin
• Primary Examiner(s): Dager, Jonathan M
• Application Number:
  US15/093,974
• Publication Number:
  US 20160299508A1
• Time in Patent Office: 1,145 Days
• Field of Search: None
• CPC Class Codes: B25J 13/08 by means of sensing devices, e.g. viewing or touching devices
• CPC Class Codes: B25J 19/021 Optical sensing devices
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0219 ensuring the processing of the whole working surface
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/242 Means based on the reflection of waves generated by the vehicle using passive navigation aids external to the vehicle G05D1/244; using signals provided by artificial sources external to the vehicle G05D1/247
• CPC Class Codes: G05D 1/243 Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249
• CPC Class Codes: G05D 1/246 using environment maps, e.g. simultaneous localisation and mapping [SLAM]
• CPC Class Codes: G05D 2105/10 for cleaning, vacuuming or polishing
• CPC Class Codes: G05D 2109/10 Land vehicles
• CPC Class Codes: G05D 2111/10 Optical signals
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A moving robot may include a first location recognition module configured to extract a straight line in a driving process and recognize a location using the extracted straight line and a second location recognition module configured to confirm the recognized location by reflecting the location recognized by the first location recognition module based on the image information obtained from the surroundings by way of an image capturing device.

Claims

1. A moving robot comprising:

a controller, the controller comprising;

a first location recognition module configured to extract a straight line of an obstacle based on information detected by a distance sensor during a driving process and recognize a location using the extracted straight line; and

a second location recognition module configured to confirm the recognized location based on the location recognized by the first location recognition module based on image information obtained from surroundings,

wherein the moving robot further comprises a memory configured to store the straight line extracted by the first location recognition module,

wherein the first location recognition module includes;

a straight line information extractor configured to extract the straight line information from values provided by the distance sensor;

a straight line information matching module configured to determine whether the straight line extracted from the straight line information extractor is the same as the previously extracted straight line stored in the memory;

a straight line information registry configured to register or confirm that the information regarding the straight line exists in the memory when the extracted straight line is a new straight line and extend the previously extracted straight line when the extracted straight line is the same as the previously extracted straight line; and

a first location recognizing module configured to recognize a current location using the information registered by the straight line information registry.

2. The moving robot according to claim 1, further including a capturing device configured to obtain the image information, wherein when the capturing device is usable, the second location recognition module confirms the recognized location with the image information, and when the capturing device is unusable, the second location recognition module determines the location recognized by the first location recognition module as a final location.

3. The moving robot according to claim 2, wherein the second location recognition module includes:

an image characteristic extractor configured to extract a plurality of characteristic points by performing image-processing on the image information obtained from the capturing device;

an image characteristic matching module configured to match the plurality of characteristic points based on information on at least two successively captured images;

an image characteristic registry configured to extract characteristic lines using line components extracted from the characteristic points shown as 3-dimensional coordinates and the image information and store the characteristic lines in the memory; and

a second location recognizing module configured to recognize a location of the moving robot based on the characteristic lines extracted from the image characteristic registry.

4. The moving robot according to claim 2, further including a moving device to drive the moving robot and a driving information sensor configured to sense driving information when the moving robot is driven by the moving device, wherein the first location recognition module extracts the straight line using the result value output from the distance sensor and based on information output from the driving information sensor.

5. The moving robot according to claim 4, wherein the moving device includes at least one wheel, and the driving information sensor includes a wheel sensor configured to sense the number of rotations of the at least one wheel.

6. The moving robot according to claim 2, further including an illumination level sensor configured to sense an illumination level, wherein the capturing device is able to be used when the illumination level detected by the illumination level sensor is equal to or greater than a predetermined illumination level that allows the capturing device to capture an image.

7. A moving robot comprising:

a housing having a distance sensor configured to measure a distance to a surrounding object;

a moving device configured to move the housing;

a driving information sensor configured to detect driving information when the housing is driven by the moving device; and

a controller, the controller comprising;

a first location recognition module configured to recognize a current location by using information output from the driving information sensor and extracting a straight line of an obstacle using a result value output from the distance sensor; and

a memory configured to store the straight line extracted by the first location recognition module,

wherein the first location recognition module includes;

a straight line information extractor configured to extract the straight line information from values output from the distance sensor;

a straight line information matching module configured to determine whether the straight line extracted from the straight light information extractor is the same as a previously extracted straight line that is stored in the memory;

a straight line information registry configured to register or confirm that the information regarding the straight line exists in the memory when the extracted straight line is a new straight line and extend the existing straight line when the extracted straight line is the same as the previously extracted straight line; and

a first location recognizing module configured to recognize the current location using the information registered by the straight line information registry.

8. The moving robot according to claim 7, further including a capturing device configured to obtain image information;

and a second location recognition module configured to confirm the recognized location by reflecting the location recognized by the first location recognition module based on the image information obtained from the capturing device.

9. The moving robot according to claim 8, wherein when the capturing device is usable, the second location recognition module confirms the recognized location with the image information, and when the capturing device is unusable, the second location recognition module determines the recognized location as a final location.

10. The moving robot according to claim 7, wherein the moving device includes at least one wheel, and the driving information sensor detects a number of rotations of the at least one wheel.

11. A method of recognizing a location of a moving robot including a controller, the method comprising:

recognizing a location of the moving robot using a first location recognition module within the controller, wherein the first location recognition module is configured to extract a straight line of an obstacle based on information detected by a distance sensor in a driving process and recognize a location using the extracted straight line;

determining whether a location is able to be confirmed using a second location recognition module within the controller, wherein the second location recognition module is configured to confirm the recognized location by verifying the location recognized by the first location recognition module based on image information obtained from surroundings; and

by the second location recognition module, confirming and determining a location by reflecting the location recognized by the first location recognition module when a location is able to be recognized by the second location recognition module, and recognizing and determining the location recognized by the first location recognition module as a final location when a location is not able to be recognized by the second location recognition module,

wherein the first location recognition module includes;

a straight line information extractor configured to extract the straight line information from values output from the distance sensor;

a straight line information matching module configured to determine whether a straight line extracted from the straight light information extractor is the same as a previously extracted straight line stored in the memory;

a straight line information registry configured to register or confirm that the information regarding the straight line exists in the memory when the extracted straight line is a new straight line and extend the existing straight line when the extracted straight line is the same as the previously extracted straight line; and

a first location recognizing module configured to recognize the current location using the information registered by the straight line information registry.

12. The method according to claim 11, wherein the second location recognition module recognizes a location by extracting a characteristic line from image information obtained from a capturing device.