Travel distance detection device and travel distance detection method

• US 8,854,456 B2
• Filed: 06/20/2011
• Issued: 10/07/2014
• Est. Priority Date: 06/21/2010
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Tsuchiya, Chikao, Furushou, Hiroyuki, Tanaka, Shinya
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): KALAPODAS, DRAMOS
• Application Number:
  US13/164,404
• Publication Number:
  US 20110310245A1
• Time in Patent Office: 1,205 Days
• Field of Search: 348/135, 348/E07.085, 348/106, 348/218.1, 348/222, 348/363, 348/148, 382/106, 382/103, 382/104, 382/107, 382/284, 382/154, 340/995.1, 345/634, 345/633
• US Class Current: 348/135
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A travel distance detection device includes: an image-capturing unit configured to capture an image; an alignment unit configured to align images in position of bird'"'"'s-eye view image, the images captured at different times by the image-capturing unit; and a solid object detection unit configured to detect a solid object based on difference image data between data of the images captured at the different times and aligned by the alignment unit. The solid object detection unit counts the number of pixels which each express a certain difference on the difference image data and which are arranged in a direction in which the solid object is inclined when the image of the predetermined region is converted into the bird'"'"'s-eye view image, then produces a frequency distribution of the counted pixels to generate a difference waveform, and calculates a travel distance of the solid object based on time variation of the difference waveform.

Claims

1. A travel distance detection device comprising:

an image-capturing unit mounted on a rear part of a vehicle and configured to capture, at a predetermined downward view angle, an image of a predetermined region comprising at least one detection region;

an alignment unit configured to align images of the predetermined region in a position of a bird'"'"'s-eye view image, the images being captured at different times by the image-capturing unit; and

a solid object detection unit configured to detect a solid object based on difference image data for the at least one detection region between data of the images captured at the different times and aligned by the alignment unit,

wherein the solid object detection unit is configured to

define at least one line in a direction in which a solid object is inclined when the image of the predetermined region is converted into the bird'"'"'s-eye view image,

count a number of pixels on the at least one line, each pixel expressing a certain difference with respect to the difference image data,

locate an intersection between the at least one line and a reference line in a running direction of the vehicle,

determine a position on an axis, the position corresponding to the counted number of pixels on the at least one line,

produce a frequency distribution of the counted number of pixels along the reference line in the running direction of the vehicle,

generate a difference waveform in the inclined direction in which the solid object is inclined due to conversion to the bird'"'"'s-eye view image, and

calculate a travel distance of the solid object based on time variation of the difference waveform.

2. The travel distance detection device according to claim 1, further comprising a viewpoint conversion unit configured to perform viewpoint conversion to convert the image of the predetermined region captured by the image-capturing unit into the bird'"'"'s-eye view image, wherein:

the alignment unit is configured to perform position alignment to coincide with each other position of bird'"'"'s-eye view image data at different times after viewpoint conversion by the viewpoint conversion unit, and

based on difference image data of the bird'"'"'s-eye view image data at the different times as aligned by the alignment unit, the solid object detection unit is configured to

detect the solid object.

3. The travel distance detection device according to claim 2, further comprising

a smear detection unit configured to detect an occurrence region of a smear from any of the images captured by the image-capturing unit,

wherein the solid object detection unit is configured to set a counted number of the frequency distribution equal to zero for a position in the difference waveform within the occurrence region of the smear detected by the smear detection unit.

4. The travel distance detection device according to claim 2, wherein the solid object detection unit is configured to calculate the travel distance of the solid object in accordance with an offset amount of the difference waveform, based on a difference waveform overlapped with another difference waveform generated at a different time with a minimum difference.

5. The travel distance detection device according to claim 4, wherein the solid object detection unit is configured to

divide each of the difference waveforms generated at the different times into a plurality of regions,

calculate an offset amount of each of the difference waveforms for each of the regions, based on the difference waveform overlapped with another one of the difference waveforms with a minimum difference,

produce a histogram by counting the offset amounts calculated for each of the regions, and

calculate therefrom the travel distance of the solid object.

6. The travel distance detection device according to claim 5, wherein the solid object detection unit is configured to

assign a weight to each of the plurality of regions, and

count the offset amounts calculated for each of the regions depending on the weights to produce the histogram.

7. The travel distance detection device according to claim 6, wherein the solid object detection unit is configured to increase the weight for each of the regions of the difference waveforms as a difference between a maximum counted value and a minimum counted value of the number of pixels expressing a certain difference increases.

8. The travel distance detection device according to claim 5, wherein the solid object detection unit is configured to calculate the travel distance of the solid object based on a local maximum value in the histogram obtained by counting offset amounts calculated for each of the regions.

9. The travel distance detection device according to claim 8, wherein the solid object detection unit is configured to

find a traveling speed of the image-capturing unit to calculate offset amounts of a stationary object based on the found traveling speed, and

calculate the travel distance of the solid object by ignoring the offset amounts of the stationary object in a local maximum value in the histogram obtained by counting the offset amounts calculated for each of the regions.

10. The travel distance detection device according to claim 9, wherein the solid object detection unit is configured to abort calculation of the travel distance of the solid object if there are a plurality of local maximum values even after the offset amounts of the stationary object are ignored.

11. A travel distance detection method comprising:

performing viewpoint conversion to convert an image of a predetermined region comprising at least one detection region into a bird'"'"'s-eye view image, the image captured by an image-capturing unit mounted on a rear part of a vehicle and configured to capture the image of the predetermined region at a predetermined downward view angle;

aligning positions in bird'"'"'s-eye view image data at different times after the viewpoint conversion; and

detecting a solid object based on difference image data for the at least one detection region between the aligned bird'"'"'s-eye view image data, wherein

the detecting of the solid object based on difference image data between the aligned bird'"'"'s-eye view image data comprises

defining at least one line in a direction in which a solid object is inclined when the image of the predetermined region is converted into the bird'"'"'s-eye view image,

counting a number of pixels on the at least one line, each pixel expressing a certain difference with respect to the difference image data,

locating an intersection between the at least one line and a reference line in a running direction of the vehicle,

determining a position on an axis, the position corresponding to the counted number of pixels on the at least one line,

producing a frequency distribution of the counted pixels along the reference line in the running direction of the vehicle,

generating a difference waveform in the inclined direction in which the solid object is inclined due to conversion to the bird'"'"'s-eye view image, and

calculating a travel distance of the solid object based on time variation of the difference waveform.

12. The travel distance detection method according to claim 11, further comprising:

detecting an occurrence region of a smear from any of the images captured by the image-capturing unit,

wherein the detecting of the solid object based on difference image data between the aligned bird'"'"'s-eye view image data further comprises

setting the counted number of the frequency distribution equal to zero for a position in the difference waveform within the occurrence region of the smear.

13. The travel distance detection method according to claim 11, wherein the calculating the travel distance of the solid object based on time variation of the difference waveform comprises:

calculating the travel distance of the solid object in accordance with an offset amount of the difference waveform, based on a difference waveform overlapped with another difference waveform generated at a different time with a minimum difference.

14. The travel distance detection method according to claim 13, wherein the calculating the travel distance of the solid object in accordance with the offset amount of the difference waveform, based on the difference waveform overlapped with another difference waveform generated at a different time with a minimum difference comprises:

dividing each of the difference waveforms generated at the different times into a plurality of regions;

calculating an offset amount of each of the difference waveforms for each of the regions, based on the difference waveform overlapped with another one of the difference waveforms with a minimum difference; and

producing a histogram by counting the offset amounts calculated for each of the regions to calculate the travel distance of the solid object.

15. The travel distance detection method according to claim 14, wherein the producing the histogram by counting the offset amounts calculated for each of the regions to calculate the travel distance of the solid object comprises:

assigning a weight to each of the plurality of regions; and

counting the offset amounts calculated for each of the regions depending on the weights to produce the histogram.

16. The travel distance detection method according to claim 15, wherein the assigning the weight to each of the plurality of regions comprises:

increasing the weight for each of the regions of the difference waveforms as a difference between a maximum counted value and a minimum counted value of the number of pixels expressing a certain difference increases.

17. The travel distance detection method according to claim 14, wherein the producing the histogram by counting the offset amounts calculated for each of the regions to calculate the travel distance of the solid object comprises:

calculating the travel distance of the solid object based on a local maximum value in the histogram obtained by counting the offset amounts calculated for each of the regions.

18. The travel distance detection method according to claim 17, wherein the calculating the travel distance of the solid object by using the local maximum value in the histogram obtained by counting the offset amounts calculated for each of the regions comprises:

finding a traveling speed of the image-capturing unit to calculate offset amounts of a stationary object based on the found traveling speed; and

calculating the travel distance of the solid object by ignoring the offset amounts of the stationary object in a local maximum value in the histogram obtained by counting the offset amounts calculated for each of the regions.

19. The travel distance detection method according to claim 18, wherein the calculating the travel distance of the solid object based on the local maximum value in the histogram obtained by counting the offset amounts calculated for each of the regions further comprises:

aborting calculation of the travel distance of the solid object if there are a plurality of local maximum values even after the offset amounts of the stationary object are ignored.

20. A travel distance detection device comprising:

means for capturing an image of a predetermined region comprising at least one detection region at a predetermined downward view angle, the means for capturing the image being mounted on a rear part of a vehicle;

means for aligning images of the predetermined region in a position of a bird'"'"'s-eye view image, the images captured at different times by the means for capturing the image; and

means for detecting a solid object based on difference image data for the at least one detection region between data of the images captured at the different times and aligned by the means for aligning images,

wherein the means for detecting the solid object

defines at least one line in a direction in which a solid object is inclined when the image of the predetermined region is converted into the bird'"'"'s-eye view image,

counts the number of pixels on the at least one line, each pixel expressing a certain difference with respect to the difference image data,

locates an intersection between the at least one line and a reference line in a running direction of the vehicle,

determines a position on an axis, the position corresponding to the counted number of pixels on the at least one line,

produces a frequency distribution of the counted pixels along the reference line in the running direction of the vehicle,

generates a difference waveform in the inclined direction in which the solid object is inclined due to conversion to the bird'"'"'s-eye view image, and

calculates a travel distance of the solid object based on time variation of the difference waveform.

21. A travel distance detection device comprising:

an image-capturing unit mounted on a vehicle and configured to capture an image of a predetermined region;

an alignment unit configured to align images of the predetermined region in a position of a bird'"'"'s-eye view image, the images being captured at different times by the image-capturing unit; and

a solid object detection unit configured to detect a solid object based on difference image data between data of the images captured at the different times and aligned by the alignment unit,

wherein the solid object detection unit is configured to

count a number of pixels each expressing a certain difference with respect to the difference image data and which are arranged in a direction in which the solid object is inclined when the image of the predetermined region is converted into the bird'"'"'s-eye view image,

then produce a frequency distribution of the counted number of pixels in a running direction of the vehicle to generate a difference waveform,

divide each of difference waveforms generated at different times into a plurality of regions,

calculate an offset amount of each of the difference waveforms for each of the regions, based on a difference waveform overlapped with another one of the difference waveforms with a minimum difference,

produce a histogram by counting offset amounts calculated for each of the regions, and

calculate the travel distance of the solid object based on the histogram.