Picture synthesizing apparatus

• US 20020110262A1
• Filed: 02/08/2002
• Published: 08/15/2002
• Est. Priority Date: 02/09/2001
• Status: Active Grant

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Mizusawa, Kazufumi, Miwa, Michio, Iida, Ryosuke
• Granted Patent: US 7,058,207 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B62D 15/0295 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

There is disclosed a picture synthesizing apparatus for superimposing an auxiliary image upon an image converted as if the image were photographed with a virtual viewpoint, displaying the image in a monitor, and providing the image such that an approach to a surrounding solid object can easily be predicted from the monitor image. The picture synthesizing apparatus includes: an image pickup unit 101 disposed in a car; a viewpoint change image synthesizing unit 102 for changing a viewpoint of the image obtained by the image pickup unit 101 and synthesizing the image; a car straight locus line generation unit 103 for generating a locus line at an arbitrary height of the car, when the car linearly advances; and a car locus line drawing unit 104 for drawing the locus line generated by the car straight locus line generation unit 103 on the image synthesized by the viewpoint change image synthesizing unit 102.

Claims

1. A picture synthesizing apparatus comprising:

image pickup means disposed in a car;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

car locus line generation means for generating at least one of a locus line at an arbitrary height of said car and a vertical line; and

car locus line drawing means for drawing the locus line generated by said car locus line generation means on the image synthesized by said viewpoint change image synthesizing means.

2. The picture synthesizing apparatus according to claim 1 wherein said car locus line generation means comprises three-dimensional locus line generation means, road surface projection means, and synthesized image projection means.

3. The picture synthesizing apparatus according to claim 1 wherein said car locus line generation means generates a locus line in a case in which said car linearly advances.

4. The picture synthesizing apparatus according to claim 1, further comprising steering angle information output means for outputting a steering wheel angle of said car, wherein said car locus line generation means generates the locus line in accordance with steering angle information outputted by said steering angle information output means.

5. The picture synthesizing apparatus according to claim 1 which has a function of interpolating a locus line on a road surface of said car and the locus line at the arbitrary height with a straight line or a curved line, and drawing a line vertical to said road surface on said synthesized image.

6. The picture synthesizing apparatus according to claim 1 which has a function of drawing a locus line of a bumper end of said car or a locus line of a car height on said synthesized image.

7. The picture synthesizing apparatus according to claim 1 which has a function of changing a color or a thickness of said locus line in accordance with a distance from said car and drawing the locus line.

8. The picture synthesizing apparatus according to claim 4 which has a function of drawing a section of said car moved apart from a rear end of said car along said locus line with an elapse of time on said synthesized image.

9. The picture synthesizing apparatus according to claim 4 which has a function of drawing a solid diagram of said car moved apart from a rear end of said car along said locus line with an elapse of time on said synthesized image.

10. The picture synthesizing apparatus according to claim 4 wherein said car locus line generation means comprises three-dimensional shape storage means, three-dimensional locus region generation means, road surface projection means, and synthesized image projection means.

11. The picture synthesizing apparatus according to claim 10 wherein said three-dimensional shape storage means stores a shape of said car.

12. The picture synthesizing apparatus according to claim 10 wherein said three-dimensional shape storage means stores a shape of a rectangular parallelepiped inscribed by said car.

13. The picture synthesizing apparatus according to claim 10 wherein said three-dimensional shape storage means stores a shape of a wheel of said car.

14. The picture synthesizing apparatus according to claim 10 wherein said three-dimensional shape storage means stores a shape of a bumper of said car.

15. The picture synthesizing apparatus according to claim 4, further comprising obstacle collision prediction means for detecting an obstacle present around said car, and predicting a possibility of collision of said car with said obstacle.

16. The picture synthesizing apparatus according to claim 15 wherein said car locus line drawing means does not draw the locus line of said car ahead of a collision place, when said obstacle collision prediction means predicts the collision of said car with said obstacle.

17. The picture synthesizing apparatus according to claim 15 wherein said car locus line drawing means emphasizes and displays a collision place, when said obstacle collision prediction means predicts the collision of said car with said obstacle.

18. The picture synthesizing apparatus according to claim 4, further comprising multi-screen generation means for displaying the image synthesized by said viewpoint change image synthesizing means in a multiplicity of divided screens.

19. The picture synthesizing apparatus according to claim 18 wherein said car locus line drawing means draws a locus of the car in each screen generated by said multi-screen generation means.

20. The picture synthesizing apparatus according to claim 18 wherein said image pickup means includes means for picking up an image behind said car, and means for picking up an image beside said car.

21. The picture synthesizing apparatus according to claim 18 wherein said car locus line drawing means draws a locus line of a rear end of said car on an image beside said car, or an image obtained by converting said image beside the car.

22. The picture synthesizing apparatus according to claim 19 wherein said car locus line drawing means draws the locus line or a car frame indicating the same position in the same color in different screens, when said locus line is drawn in a plurality of screens.

24. The picture synthesizing apparatus according to claim 23, further comprising multi-screen generation means for displaying the image synthesized by said viewpoint change image synthesizing means in a multiplicity of divided screens, wherein said auxiliary line drawing means draws the generated auxiliary line in each screen generated by said multi-screen generation means.

25. The picture synthesizing apparatus according to claim 23 wherein said auxiliary line generation means generates an auxiliary line indicating a position of a rear end of said car.

26. The picture synthesizing apparatus according to claim 23 wherein said auxiliary line generation means generates an auxiliary line indicating a constant distance from a rear end of said car.

27. The picture synthesizing apparatus according to claim 23 wherein said auxiliary line generation means generates an auxiliary line indicating a width of a general car.

29. The picture synthesizing apparatus according to claim 28 which has a function of superimposing an auxiliary line upon a rear edge of said car, and providing an image emphasizing/indicating the corresponding position.

30. The picture synthesizing apparatus according to claim 28 which has a function of providing an image showing a three-dimensional illustration prepared as if the image of said car were picked up by an actually disposed image pickup unit and converted/synthesized.

31. The picture synthesizing apparatus according to claim 28 which has a function of representing said car by an illustration of a skeleton or a wire frame, and providing an image explicitly indicating a tire position.

32. The picture synthesizing apparatus according to claim 31 which has a function of providing an image obtained by transforming/synthesizing an image actually obtained by said image pickup means in a region corresponding to a bumper of the illustration.

33. The picture synthesizing apparatus according to claim 28 which has a function of superimposing an illustration of two wall surfaces disposed vertically to a road surface in a rear end position of said car, and on an inner side of the rear end position of said car, and having the same width as a width of said car upon a displayed image, and providing an image metaphorically representing said car as a solid object.

34. The picture synthesizing apparatus according to claim 28 which has a function of providing an image showing a mirror confirmation line behind a rear end of a bumper of said car by a constant distance and horizontally with said bumper.

35. The picture synthesizing apparatus according to claim 28 which has a function of providing an image including a road surface passage locus indicating a position obtained by projecting a position passed by a body end of said car onto a road surface, and a bumper end passage locus indicating a position passed by a bumper end of said car, when said car moves backwards, and a solid auxiliary line for connecting the loci to produce a solid sense.

36. The picture synthesizing apparatus according to claim 35 which has a function of providing an image showing said road surface passage locus like a tire trace, and indicating the bumper end passage locus connected to a bumper end of an actual image or an illustration of said car.

37. The picture synthesizing apparatus according to claim 35 which has a function of providing an image showing said road surface passage locus drawn from a tire explicitly indicated in an illustration.

38. The picture synthesizing apparatus according to claim 37 which has a function of providing an image showing a mirror confirmation line behind a rear end of a bumper of said car by a constant distance and horizontally with said bumper.

39. The picture synthesizing apparatus according to claim 35 which further comprises a locus calculation unit to calculate a predicted locus from a steering angle signal inputted from the outside, and which has a function of providing an image including a road surface passage locus corresponding to a steering angle of said car, a bumper end passage locus corresponding to the steering angle of said car, and a solid auxiliary line for connecting the loci to produce a solid sense.

40. The picture synthesizing apparatus according to claim 28 which has a function of providing an image simultaneously showing a road surface passage locus, a bumper end passage locus, a passage locus indicating a position passed by an appropriate height portion of a body of said car, and an illustration imitating a rear part of the car, when said car moves backwards.

41. The picture synthesizing apparatus according to claim 40 which has a function of providing an image showing said road surface passage locus like a tire trace, and indicating said bumper end passage locus connected to a bumper end of an actual image or an illustration.

42. The picture synthesizing apparatus according to claim 40 which has a function of providing an image showing said road surface passage locus drawn from a tire explicitly indicated in an illustration.

43. The picture synthesizing apparatus according to claim 42 which has a function of providing an image showing a mirror confirmation line behind a rear end of a bumper of said car by a constant distance and horizontally with said bumper.

44. The picture synthesizing apparatus according to claim 40 which further comprises a locus calculation unit to calculate a predicted locus from a steering angle signal inputted from the outside, and which has a function of providing an image simultaneously showing said road surface passage locus corresponding to a steering angle of said car, said bumper end passage locus corresponding to the steering angle of said car, a passage locus indicating a position passed by an appropriate height portion of a body of said car, and an illustration imitating a rear part of said car.

45. The picture synthesizing apparatus according to claim 28 which has a function of providing an image simultaneously showing a road surface passage locus, a bumper upper surface passage locus indicating a position passed by an end of a bumper upper surface of said car, a bumper lower surface passage locus indicating a position passed by an end of a bumper lower surface of said car, and an illustration imitating a rear part of said car, when said car moves backwards.

46. The picture synthesizing apparatus according to claim 45 which has a function of providing an image showing said road surface passage locus like a tire trace, and indicating the bumper end passage locus connected to a bumper end of an actual image or an illustration of the car.

47. The picture synthesizing apparatus according to claim 45 which has a function of providing an image showing said road surface passage locus drawn from a tire explicitly indicated in an illustration.

48. The picture synthesizing apparatus according to claim 47 which has a function of providing an image showing a mirror confirmation line behind a rear end of a bumper of said car by a constant distance and horizontally with said bumper.

49. The picture synthesizing apparatus according to claim 45 which further comprises a locus calculation unit to calculate a predicted locus from a steering angle signal inputted from the outside, and which has a function of providing an image simultaneously showing said road surface passage locus corresponding to a steering angle of said car, said bumper upper surface passage locus corresponding to the steering angle of said car, said bumper lower surface passage locus corresponding to the steering angle of said car, and an illustration imitating a bumper of said car.

23. A picture synthesizing apparatus comprising:

image pickup means disposed in a car;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

auxiliary line generation means for generating an auxiliary line of an arbitrary position from said car; and

auxiliary line drawing means for drawing the auxiliary line generated by said auxiliary line generation means on the image synthesized by said viewpoint change image synthesizing means.

28. A picture synthesizing apparatus comprising:

image pickup means disposed in a car so that a rear part of said car is positioned in a view field;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image including an image of said car;

storage means for storing predetermined data beforehand; and

drawing means for superimposing predetermined auxiliary data upon the image synthesized by said viewpoint change image synthesizing means based on the data read from said storage means.

50. An image synthesis/display apparatus comprising:

a picture synthesizing apparatus comprising;

image pickup means disposed in a car;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

car locus line generation means for generating at least one of a locus line at an arbitrary height of said car and a vertical line; and

car locus line drawing means for drawing the locus line generated by said car locus line generation means on the image synthesized by said viewpoint change image synthesizing means;

display means for displaying the image synthesized by said picture synthesizing apparatus; and

display data conversion means for converting said image to be displayed into data suitable for said display means.

51. An image synthesis/display apparatus comprising:

a picture synthesizing apparatus comprising;

image pickup means disposed in a car;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

auxiliary line generation means for generating an auxiliary line of an arbitrary position from said car; and

auxiliary line drawing means for drawing the auxiliary line generated by said auxiliary line generation means on the image synthesized by said viewpoint change image synthesizing means;

display means for displaying the image synthesized by said picture synthesizing apparatus; and

display data conversion means for converting said image to be displayed into data suitable for said display means.

52. An image synthesis/display apparatus comprising:

a picture synthesizing apparatus comprising;

image pickup means disposed in a car so that a rear part of said car is positioned in a view field;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image including an image of said car;

storage means for storing predetermined data beforehand; and

drawing means for superimposing predetermined auxiliary data upon the image synthesized by said viewpoint change image synthesizing means based on the data read from said storage means;

display means for displaying the image synthesized by said picture synthesizing apparatus; and

display data conversion means for converting said image to be displayed into data suitable for said display means.

53. An image acquirement warning apparatus comprising:

detection means for detecting an approaching state of a connection object connected to a car;

a picture synthesizing apparatus comprising;

image pickup means disposed in said car and/or said connection object;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

car locus line generation means for generating at least one of a locus line at an arbitrary height of said car and a vertical line; and

car locus line drawing means for drawing the locus line generated by said car locus line generation means on the image synthesized by said viewpoint change image synthesizing means; and

warning means for generating a warning signal from said approaching state obtained by said detection means and/or a relation between said car and said connection object in the image synthesized by said picture synthesizing apparatus.

56. The image acquirement warning apparatus according to any one of claims 53 to 55, further comprising warning signal generation condition setting means for a user to arbitrarily set a condition for generating the warning signal by said warning means.

54. An image acquirement warning apparatus comprising:

detection means for detecting an approaching state of a connection object connected to a car;

a picture synthesizing apparatus comprising;

image pickup means disposed in said car and/or said connection object;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

auxiliary line generation means for generating an auxiliary line of an arbitrary position from said car; and

auxiliary line drawing means for drawing the auxiliary line generated by said auxiliary line generation means on the image synthesized by said viewpoint change image synthesizing means; and

warning means for generating a warning signal from said approaching state obtained by said detection means and/or a relation between said car and said connection object in the image synthesized by said picture synthesizing apparatus.

55. An image acquirement warning apparatus comprising:

detection means for detecting an approaching state of a connection object connected to a car;

a picture synthesizing apparatus comprising;

image pickup means disposed in said car and/or said connection object so that a rear part of said car is positioned in a view field;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image including an image of said car;

storage means for storing predetermined data beforehand; and

drawing means for superimposing predetermined auxiliary data upon the image synthesized by said viewpoint change image synthesizing means based on the data read from said storage means; and

warning means for generating a warning signal from said approaching state obtained by said detection means and/or a relation between said car and said connection object in the image synthesized by said picture synthesizing apparatus.

57. A car position recognition apparatus comprising:

a picture synthesizing apparatus comprising;

a plurality of image pickup means disposed in a car, and including rear image pickup means for picking up an image behind said car;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

car locus line generation means for generating at least one of a locus line at an arbitrary height of said car and a vertical line; and

car locus line drawing means for drawing the locus line generated by said car locus line generation means on the image synthesized by said viewpoint change image synthesizing means;

image detection means for detecting an image of an arbitrary object from the image obtained by said rear image pickup means or the image synthesized by said picture synthesizing apparatus;

recognition means for recognizing a position relation between the image detected by said image detection means and the image of said car; and

comparison means for comparing said position relation recognized by said recognition means with a predetermined position relation, and detecting a deviation amount between the position relations from the position relations.

58. A car position recognition apparatus comprising:

a picture synthesizing apparatus comprising;

a plurality of image pickup means disposed in a car, and including rear image pickup means for picking up an image behind said car;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image;

auxiliary line generation means for generating an auxiliary line of an arbitrary position from said car; and

auxiliary line drawing means for drawing the auxiliary line generated by said auxiliary line generation means on the image synthesized by said viewpoint change image synthesizing means;

image detection means for detecting an image of an arbitrary object from the image obtained by said rear image pickup means or the image synthesized by said picture synthesizing apparatus;

recognition means for recognizing a position relation between the image detected by said image detection means and the image of said car; and

comparison means for comparing said position relation recognized by said recognition means with a predetermined position relation, and detecting a deviation amount between the position relations from the position relations.

59. A car position recognition apparatus comprising:

a picture synthesizing apparatus comprising;

a plurality of image pickup means disposed in a car, including rear image pickup means for picking up an image behind said car, and disposed so that a rear part of said car is positioned in a view field;

viewpoint change image synthesizing means for changing a viewpoint of an image obtained by said image pickup means and synthesizing the image including an image of said car;

storage means for storing predetermined data beforehand; and

drawing means for superimposing predetermined auxiliary data upon the image synthesized by said viewpoint change image synthesizing means based on the data read from said storage means;

image detection means for detecting an image of an arbitrary object from the image obtained by said rear image pickup means or the image synthesized by said picture synthesizing apparatus;

recognition means for recognizing a position relation between the image detected by said image detection means and the image of said car; and

comparison means for comparing said position relation recognized by said recognition means with a predetermined position relation, and detecting a deviation amount between the position relations from the position relations.