Movable body circumstance monitoring apparatus

• US 7,379,564 B2
• Filed: 12/18/2003
• Issued: 05/27/2008
• Est. Priority Date: 12/18/2002
• Status: Expired due to Fees

• Current Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Original Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Inventors: Yamamoto, Shuji, Kakinami, Toshiaki
• Primary Examiner(s): Carter; Aaron
• Application Number:
  US10/738,097
• Publication Number:
  US 20050163343A1
• Time in Patent Office: 1,622 Days
• Field of Search: 382/103, 382/104, 382/153, 340/435, 340/436, 340/932.2, 701/1, 701/28, 701/36, 701/41, 701/200, 348/116, 348/148
• US Class Current: 382/104
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/586 of parking space
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A movable body circumstance monitoring apparatus includes element for capturing an image of a surrounding of a movable body depending upon movement of the movable body, deriving at least four feature points in a specified flat at the image, tracking the feature points in response to movement of the movable body from a first state to a second state, and specifying relative position and posture of the element for capturing the image at the first and second states in response to first flat coordinates of the feature points at the first state and second flat coordinates of the feature points at the second state based upon a tracking result.

Claims

1. A movable body circumstance monitoring apparatus comprising:

at least one means for capturing an image of a surrounding of a movable body depending upon movement of the movable body;

means for deriving at least four feature points in a specified flat at the image captured by the at least one means for capturing the image;

means for tracking the at least four feature points in response to movement of the movable body from a first state to a second state; and

means for specifying a moving condition including relative position and posture of the at least one means for capturing the image at the first and second states in response to first flat coordinates of the feature points at the image captured at the first state and second flat coordinates of the feature points at the image at the second state based upon a tracking result by the means for tracking the at least four feature points, the specified flat having the four feature points being at least one of a road surface and a wall surface, the feature points being derived from at least one of a white line, a concave portion, a convex portion, a step, a pattern, a surface crack, an edge, a point on a profile line of a pattern, a shadow of an object on at least one of the road surface and the wall surface, a gravel stone and an edge of a curb stone on the road surface.

2. A movable body circumstance monitoring apparatus according to claim 1, further comprising:

means for estimating three-dimensional coordinates of the feature points based upon the relative position and posture of the means for capturing the image at the first and second states specified by the means for specifying the moving condition, the first flat coordinates of the feature points at the image at the first state, and the second flat coordinates of the feature points at the image at the second state.

3. A movable body circumstance monitoring apparatus according to claim 2, further comprising:

means for creating a specific view based upon the three-dimensional coordinates of the feature points estimated by the means for estimating the three-dimensional coordinates; and

means for displaying the specific view created by the means for creating the specific view; and

means for transmitting a command related to the specified view from a user of the movable body, and

the means for creating the specific view comprising;

means for changing the specified view in response to the command from the user of the movable body.

4. A movable body circumstance monitoring apparatus according to claim 2, further comprising:

means for creating a specific view based upon the three-dimensional coordinates of the feature points estimated by the means for estimating the three-dimensional coordinates;

means for displaying the specific view created by the means for creating the specific view; and

means for detecting a substance approaching to the surrounding of the movable body based upon information of the image captured by the means for capturing the image, wherein the means for creating the specific view creates the specific view which includes the substance detected by the means for detecting the substance.

5. A movable body circumstance monitoring apparatus according to claim 2 further comprising:

means for creating a specific view based upon the three-dimensional coordinates of the feature points estimated by the means for estimating the three-dimensional coordinates;

means for displaying the specific view created by the means for creating the specific view; and

means for estimating a moving track of the movable body based upon information from the means for specifying the moving condition, and

the means for creating the specific view comprising;

means for overlapping the moving track estimated by the means for estimating the moving track on the specified view.

6. A movable body circumstance monitoring apparatus according to claim 1, wherein the movable body includes a vehicle and a robot.

7. A movable body circumstance monitoring apparatus comprising:

at least one camera mounted at a movable body and configured to capture an image of a surrounding of a movable body depending upon movement of the movable body; and

an electronic control unit supplied with information of the image captured by the at least one camera, wherein position of the posture of the movable body can be specified;

wherein the electronic control unit is configured to derive at least four feature points in a specified flat at the image captured by the at least one camera, tracks the at least four feature points in response to movement of the movable body from a first state to a second state, and specifies moving condition such as relative position and posture of the at least one camera at the first and second states in response to first flat coordinates of the feature points at the image captured at the first state and second flat coordinates of the feature points at the image at the second state based upon a tracking result;

the specified flat having the four feature points being at least one of a road surface and a wall surface, the feature points being derived from at least one of a white line, a concave portion, a convex portion, a step, a pattern, a surface crack, an edge, a point on a profile line of a pattern, a shadow of an object on at least one of the road surface and the wall surface, a gravel stone and an edge of a curb stone on the road surface.

8. A movable body circumstance monitoring apparatus according to claim 7, further comprising:

a monitor being visible from a user of the movable body and configured to display the image captured by the camera and a three-dimensional graphic; and

the electronic control unit comprising;

a central processing unit;

an image recognizing module connected to the central processing unit and configured to recognize a signal of the image captured by the camera;

a graphic drawing module connected to the central processing unit and configured to draw a three-dimensional graphic; and

a superimposing module connected to the graphic drawing module and configured to superimpose the three-dimensional graphic drawn by the graphic drawing module on the image captured by the camera.

9. A movable body circumstance monitoring apparatus according to claim 8, wherein the electronic control unit further estimates three-dimensional coordinates of the feature points based upon the relative position and posture of the camera at the first and second states, the first flat coordinates of the feature points at the image at the first state, and the second flat coordinates of the feature points at the image at the second state.

10. A movable body circumstance monitoring apparatus according to claim 9, wherein the electronic control unit further creates a specific view based upon the estimated three-dimensional coordinates of the feature points, and the monitor displays the created specific view, the movable body circumstance monitoring apparatus further comprising:

a member configured to transmit a command related to the specified view from a user of the movable body to the electronic control unit, wherein the specific view is changed in response to the command from the user of the movable body.

11. A movable body circumstance monitoring apparatus according to claim 10, wherein the member includes a mechanical switch and a touch panel portion at the monitor screen.

12. A movable body circumstance monitoring apparatus according to claim 9, wherein the electronic control unit further detects a substance approaching the surrounding of the movable body based upon information of the image captured by the camera, and creates a specific view, which includes the detected substance, based upon the estimated three-dimensional coordinates of the feature points, and the monitor displays the created specific view.

13. A movable body circumstance monitoring apparatus according to claim 9, wherein the electronic control unit further estimates a moving track of the movable body, and overlaps the estimated moving track on the specific view, and the monitor displays an image in which the estimated moving track overlaps the specific view.

14. A movable body circumstance monitoring apparatus comprising:

at least one means for capturing an image of a surrounding of a movable body depending upon movement of the movable body;

means for deriving at least four feature points in a specified flat at the image captured by the at least one means for capturing the image;

means for tracking the at least four feature points in response to movement of the movable body from a first state to a second state; and

means for specifying a moving condition including relative position and posture of the at least one means for capturing the image at the first and second states in response to first flat coordinates of the feature points at the image captured at the first state and second flat coordinates of the feature points at the image at the second state based solely upon a tracking result by the means for tracking the at least four feature points.