Vehicle, image processing system image processing method, image processing program, method for configuring image processing system, and server

• US 8,135,175 B2
• Filed: 05/22/2006
• Issued: 03/13/2012
• Est. Priority Date: 05/27/2005
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Aoki, Tomoyoshi, Kobayashi, Sachio, Mori, Naoki, Nakamori, Takuma
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US11/918,674
• Publication Number:
  US 20090041303A1
• Time in Patent Office: 2,122 Days
• Field of Search: 382100-104, 340907-910, 340933-937, 348113-116
• US Class Current: 382/104
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An image processing system and the like capable of accurately recognizing lane edges defined by dotted lane marks are provided. According to an image processing system (100) of the present invention, a first processing unit (110) searches a road surface image captured by a vehicle-mounted camera for a “small area,” which is composed of high- or low-luminance pixels and satisfies “eligibility conditions” on the “size,” “shape,” and “arrangement” in the road surface image. Additionally, a second processing unit (120) recognizes “lane edges” of the lane along which the vehicle travels on the basis of the “small area.” Furthermore, a third processing unit (130) sets a “search range” of the small area searched by the first processing unit (110) on the basis of a result of the foregoing recognition of the lane edges by the second processing unit (120).

Claims

1. A vehicle equipped with:

an imaging means;

an image processing system which performs image processing on the basis of a road surface image captured by the imaging means; and

a vehicle running condition control system which controls a running condition of the vehicle on the basis of a result of the image processing performed by the image processing system,

wherein the image processing system includes;

a first processing unit which searches for a first small area composed of high-luminance pixels and satisfying eligibility conditions and a second small area composed of low-luminance pixels and satisfying eligibility conditions; and

a second processing unit which recognizes lane edges of a lane along which the vehicle travels on the basis of the first and second small areas found by the first processing unit;

wherein the high-luminance pixels with a luminance equal to or higher than a light threshold value have a higher luminance than a first luminance of the road surface image, and the low-luminance pixels with a luminance equal to or lower than a dark threshold value have a lower luminance than a second luminance of the road surface image, and

wherein the eligibility conditions are those on the size, the size and shape, the size and arrangement, or the size, shape, and arrangement in the road surface image.

2. The vehicle according to claim 1, further comprising a third processing unit which sets a search range of the first and second small areas searched by the first processing unit on the basis of:

a result of a previous search for the first and second small areas performed by the first processing unit,

a result of previous recognition of the lane edges performed by the second processing unit, or

standard arrangement information on dotted lane marks stored in a storage unit.

3. The vehicle according to claim 2, wherein the third processing unit sets the search range of the first and second small areas searched by the first processing unit on the basis of the standard arrangement information on the dotted lane marks stored in the storage unit if the first and second small areas are not previously found by the first processing unit or if the lane edges are not previously recognized by the second processing unit though the first and second small areas are previously found by the first processing unit.

4. The vehicle according to claim 1, further comprising a third processing unit which sets the search range of the first and second small areas searched by the first processing unit to be smaller than in the case of the first and second small areas not previously found by the first processing unit if the first and second small areas are previously found by the first processing unit, or which sets the search range of the first and second small areas searched by the first processing unit to be smaller than in the case of the lane edges not previously recognized by the second processing unit if the lane edges are previously recognized by the second processing unit.

5. The vehicle according to claim 1, which is equipped with a speed sensor and a yaw rate sensor, further comprising a third processing unit which sets the search range of the first and second small areas searched by the first processing unit on the basis of one or both of a vehicle speed measured by the speed sensor and a vehicle yaw rate measured by the yaw rate sensor.

6. The vehicle according to claim 2, wherein the first processing unit sets one or both of a threshold value defining a high luminance level of a pixel and a threshold value defining a low luminance level of the pixel on the basis of a luminance average of pixels in the search range set by the third processing unit.

7. The vehicle according to claim 1, wherein the first processing unit performs one or both of a search for the first small area composed of high-luminance pixels by performing a tophat filtering process and a search for the second small area composed of low-luminance pixels by performing a bothat filtering process in the road surface image.

8. The vehicle according to claim 1, wherein the first processing unit corrects the eligibility conditions on the basis of a running speed of the vehicle and exposure time of the imaging means before searching for the first and second small areas satisfying the eligibility conditions.

9. The vehicle according to claim 1, wherein the first processing unit searches for only the first small area composed of high-luminance pixels in the road surface image if the luminance average of a part or all of the pixels in the road surface image is equal to or lower than a night reference value.

10. The vehicle according to claim 1, wherein the second processing unit recognizes the outline of the dotted lane mark on the basis of the first and second small areas found by the first processing unit and a shading pattern of the dotted lane mark stored in the storage unit before recognizing the lane edge.

11. The vehicle according to claim 10, wherein the second processing unit preferentially recognizes a point close to the center of the vehicle traffic lane among points on the outline of the dotted lane mark as a part of the lane edge.

27. The vehicle according to claim 1, wherein the first processing unit is configured to correct the eligibility conditions for adjusting the road surface image in connection with a reflectivity of the lane along which the vehicle travels.

28. The vehicle according to claim 1, wherein the first processing unit is configured to correct at least a portion of the eligibility conditions on the size, the eligibility conditions on the shape, and the eligibility conditions on the arrangement based at least in part on the running speed of the vehicle and an exposure time of a camera of the imaging means.

29. The vehicle according to claim 1, wherein the eligibility conditions at least comprise a condition that a number of overflow pixels from a reference area is n1 or less, or that a number of missing pixels in the reference area is n2 or less.

12. An image processing system comprising:

a first processing unit which searches for a first small area composed of high-luminance pixels and satisfying eligibility conditions and a second small area composed of low-luminance pixels and satisfying eligibility conditions in a road surface image captured by imaging means mounted on a vehicle; and

a second processing unit which recognizes lane edges of a lane along which the vehicle travels on the basis of the first and second small areas found by the first processing unit,

wherein the high-luminance pixels with a luminance equal to or higher than a light threshold value have a higher luminance than a first luminance of the road surface image, and the low-luminance pixels with a luminance equal to or lower than a dark threshold value have a lower luminance than a second luminance of the road surface image,

wherein the eligibility conditions are those on the size, the size and shape, the size and arrangement, or the size, shape, and arrangement in the road surface image.

13. The image processing system according to claim 12, further comprising a third processing unit which sets a search range of the first and second small areas searched by the first processing unit on the basis of a result of a previous search for the first and second small areas performed by the first processing unit, a result of previous recognition of the lane edges performed by the second processing unit, or standard arrangement information on dotted lane marks stored in a storage unit.

14. The image processing system according to claim 13, wherein the third processing unit sets the search range of the first and second small areas searched by the first processing unit on the basis of the standard arrangement information on the dotted lane marks stored in the storage unit if the first and second small areas are not previously found by the first processing unit, or if the lane edges are not previously recognized by the second processing unit though the first and second small areas are previously found by the first processing unit.

15. The image processing system according to claim 12, further comprising a third processing unit which sets the search range of the first and second small areas searched by the first processing unit smaller than in the case of the first and second small areas not previously found by the first processing unit if the first and second small areas are previously found by the first processing unit or which sets the search range of the first and second small areas searched by the first processing unit to be smaller than in the case of the lane edges not previously recognized by the second processing unit if the lane edges are previously recognized by the second processing unit.

16. The image processing system according to claim 12, further comprising a third processing unit which sets the search range of the first and second small areas searched by the first processing unit on the basis of one or both of a vehicle speed measured by a speed sensor mounted on the vehicle and a vehicle yaw rate measured by a yaw rate sensor mounted on the vehicle.

17. The image processing system according to claim 13, wherein the first processing unit sets one or both of a threshold value defining a high luminance level of a pixel and a threshold value defining a low luminance level of the pixel on the basis of a luminance average of pixels in the search range set by the third processing unit.

18. The image processing system according to claim 12, wherein the first processing unit performs one or both of the search for the first small area composed of high-luminance pixels by performing a tophat filtering process and the search for the second small area composed of low-luminance pixels by performing a bothat filtering process in the road surface image.

19. The image processing system according to claim 12, wherein the first processing unit corrects the eligibility conditions on the basis of a running speed of the vehicle and exposure time of the imaging means before searching for the first and second small areas satisfying the eligibility conditions.

20. The image processing system according to claim 12, wherein the first processing unit searches for only the first small area composed of high-luminance pixels in the road surface image if the luminance average of a part or all of the pixels in the road surface image is equal to or lower than a night reference value.

21. The image processing system according to claim 12, wherein the second processing unit recognizes the outline of the dotted lane mark on the basis of the first and second small areas found by the first processing unit and a shading pattern of the dotted lane mark stored in the storage unit before recognizing the lane edge.

22. The image processing system according to claim 21, wherein the second processing unit preferentially recognizes a point close to the center of a vehicle traffic lane among points on the outline of the dotted lane mark as a part of the lane edge.

30. The image processing system according to claim 12, wherein the first processing unit is configured to correct the eligibility conditions for adjusting the road surface image in connection with a reflectivity of the lane along which the vehicle travels.

31. The image processing system according to claim 12, wherein the first processing unit is configured to correct at least a portion of the eligibility conditions on the size, the eligibility conditions on the shape, and the eligibility conditions on the arrangement based at least in part on the running speed of the vehicle and an exposure time of a camera of the imaging means.

32. The image processing system according to claim 12, wherein the eligibility conditions at least comprise a condition that a number of overflow pixels from a reference area is n1 or less, or that a number of missing pixels in the reference area is n2 or less.

23. An image processing method comprising:

a first process of searching for a first small area composed of high-luminance pixels and satisfying eligibility conditions and a second small area composed of low-luminance pixels and satisfying eligibility conditions in a road surface image captured by imaging means mounted on a vehicle; and

a second process of recognizing lane edges of a lane along which the vehicle travels on the basis of the first and second small areas found in the first process,

wherein the high-luminance pixels with a luminance equal to or higher than a light threshold value have a higher luminance than a first luminance of the road surface image, and the low-luminance pixels with a luminance equal to or lower than a dark threshold value have a lower luminance than a second luminance of the road surface image,

wherein the eligibility conditions are those on the size, the size and shape, the size and arrangement, or the size, shape, and arrangement in the road surface image.

33. The image processing method according to claim 23, wherein the first process further comprises correcting the eligibility conditions for adjusting the road surface image in connection with a reflectivity of the lane along which the vehicle travels.

34. The image processing method according to claim 23, wherein the first process further comprises correcting at least a portion of the eligibility conditions on the size, the eligibility conditions on the shape, and the eligibility conditions on the arrangement based at least in part on the running speed of the vehicle and an exposure time of a camera of the imaging means.

35. The image processing method according to claim 23, wherein the eligibility conditions at least comprise a condition that a number of overflow pixels from a reference area is n1 or less, or that a number of missing pixels in the reference area is n2 or less.

24. An image processing program embedded in a non-transitory medium, the image processing program causing a computer to perform:

a first processing function of searching for a first small area composed of high-luminance pixels and satisfying eligibility conditions and a second small area composed of low-luminance pixels and satisfying eligibility conditions in a road surface image captured by imaging means mounted on a vehicle; and

a second processing function of recognizing lane edges of a lane along which the vehicle travels on the basis of the first and second small areas found by the first processing function,

wherein the high-luminance pixels with a luminance equal to or higher than a light threshold value have a higher luminance than a first luminance of the road surface image, and the low-luminance pixels with a luminance equal to or lower than a dark threshold value have a lower luminance than a second luminance of the road surface image,

wherein the eligibility conditions are those on the size, the size and shape, the size and arrangement, or the size, shape, and arrangement in the road surface image.

25. The image processing program according to claim 24, wherein part of all of the program is downloaded to an in-vehicle computer.

26. The image processing program according to claim 24, wherein the program is hosted on a server.

36. The image processing program according to claim 24, wherein the first processing function further comprises correcting the eligibility conditions for adjusting the road surface image in connection with a reflectivity of the lane along which the vehicle travels.

37. The image processing program according to claim 24, wherein the first processing function further comprises correcting at least a portion of the eligibility conditions on the size, the eligibility conditions on the shape, and the eligibility conditions on the arrangement based at least in part on the running speed of the vehicle and an exposure time of a camera of the imaging means.

38. The image processing program according to claim 24, wherein the eligibility conditions at least comprise a condition that a number of overflow pixels from a reference area is n1 or less, or that a number of missing pixels in the reference area is n2 or less.