Cruise-assist image generation device, cruise-assist image generation method, in-vehicle camera and equipment-control assist image generation device

• US 9,533,709 B2
• Filed: 04/04/2013
• Issued: 01/03/2017
• Est. Priority Date: 04/25/2012
• Status: Active Grant

• Current Assignee: Sony Group Corp.
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Oba, Eiji
• Primary Examiner(s): Algahaim, Helal A
• Assistant Examiner(s): WALLACE, DONALD JOSEPH
• Application Number:
  US13/856,663
• Publication Number:
  US 20130325205A1
• Time in Patent Office: 1,370 Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/002 specially adapted for covering the peripheral part of the vehicle, e.g. for viewing tyres, bumpers or the like integrated in the windows B60R1/001; combined with rear-view mirrors B60R1/08
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 2300/108 using 'non-standard' camera systems, e.g. camera sensor used for additional purposes i.a. rain sensor, camera sensor split in multiple image areas
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B62D 15/0275 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: B62D 15/0295 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A cruise-assist image generation device includes a steering-angle information acquisition unit, an additional line generation unit generating a cruise assist additional line corresponding to acquired steering angle information and formed by combination of a given number of segments, and an additional line superimposing unit superimposing the generated cruise assist additional line on a taken image of the vehicle periphery to obtain a composite image. The additional line generation unit generates respective curved lines forming at least some of the segments by quadratic curve approximation using coordinate data corresponding to the acquired steering angle information and including coordinates of points representing each segment, and generating respective straight lines forming any remaining segments by collinear approximation using coordinate data corresponding to the acquired steering angle information.

Claims

1. A cruise-assist image generation device comprising:

a steering angle information acquisition circuitry configured to acquire, repeatedly at a time interval, steering angle information of a steering wheel;

a memory configured to store a plurality of data sets in a lookup table, each data set of the plurality of data sets includes coordinate data corresponding to a different steering angle of a plurality of steering angles;

an additional line generation circuitry configured to;

select a data set from the plurality of data sets in accordance with the steering angle information,

generate a cruise assist additional line formed by a combination of a number of vertical segments and a number of horizontal segments based on the data set selected from the plurality of data sets, and

update the cruise assist additional line based on a change of the steering angle information; and

an additional line superimposing circuitry configured to superimpose the cruise assist additional line on an image of a vehicle periphery, wherein

the each data set of the plurality of data sets includes a common coordinate between at least one of the vertical segments and at least one of the horizontal segments that form the cruise assist additional line.

2. The cruise-assist image generation device according to claim 1, wherein, to generate the cruise assist additional line formed by the combination of the number of vertical segments and the number of horizontal segments, the additional line generation circuitry is further configured to

generate one or more curved lines to represent at least one of the vertical segments or at least one of the horizontal segments by quadratic curve approximation using the coordinate data from the data set selected from the plurality of data sets, wherein the coordinate data includes coordinates of at least two end points representing the at least one of the vertical segments or the at least one of the horizontal segments, and

generate one or more straight lines to represent at least a second one of the vertical segments or at least a second one of the horizontal segments by collinear approximation using the coordinate data from the data set selected from the plurality of data sets.

3. The cruise-assist image generation device according to claim 1,

wherein the image of the vehicle periphery is an image of a back region of a vehicle.

4. The cruise-assist image generation device according to claim 1,

wherein the image of the vehicle periphery is an image of a side region of a vehicle.

5. The cruise-assist image generation device according to claim 1, wherein, to acquire repeatedly at the time interval, the steering angle information of the steering wheel, the steering angle information acquisition circuitry is further configured to acquire the steering angle information of the steering wheel from an in-vehicle network.

6. The cruise-assist image generation device according to claim 1, wherein the memory is configured to store the coordinate data of the each data set of the plurality of data sets corresponding to a discrete value associated with the different steering angle of the plurality of steering angles, and

wherein the steering angle information acquisition circuitry is further configured to

acquire the steering angle information of a target steering angle, and

generate target coordinate data corresponding to the target steering angle by interpolation processing of the coordinate data corresponding to the different steering angle of the plurality of steering angles in a vicinity of the target steering angle.

7. The cruise-assist image generation device according to claim 6,

wherein the target coordinate data corresponding to the target steering angle is generated by linear interpolation of the coordinate data corresponding to the different steering angle of the plurality of steering angles in the vicinity of the target steering angle, and wherein the memory is configured to store the target coordinate data in a data set of the plurality of data sets corresponding to the target steering angle.

8. The cruise-assist image generation device according to claim 2,

wherein one middle point is a point positioned in the middle between the two end points in a horizontal direction or a vertical direction.

9. A cruise-assist image generation method comprising:

acquiring, repeatedly at a time interval, steering angle information of a steering wheel;

selecting a data set from a plurality of data sets stored in a lookup table in accordance with the steering angle information, wherein each data set of the plurality of data sets includes coordinate data corresponding to a different steering angle of a plurality of steering angles;

generating a cruise assist additional line formed by a combination of a number of vertical segments and a number of horizontal segments based on the selection of the data set from the plurality of data sets;

superimposing the cruise assist additional line on an image of a vehicle periphery; and

updating the cruise assist additional line based on a change of the steering angle information, wherein

the each data set of the plurality of data sets includes a common coordinate between at least one of the vertical segments and at least one of the horizontal segments that form the cruise assist additional line.

10. The cruise-assist image generation method according to claim 9, wherein generating the cruise assist additional line the combination of the number of vertical segments and the number of horizontal segments based on the selection of the data set from the plurality of data sets further includes

generating one or more curved lines to represent at least one of the vertical segments or at least one of the horizontal segments by quadratic curve approximation using the coordinate data from the data set selected from the plurality of data sets, wherein the coordinate data includes coordinates of at least two end points representing the at least one of the vertical segments or the at least one of the horizontal segments;

and

generating one or more straight lines to represent at least a second one of the vertical segments or at least a second one of the horizontal segments by collinear approximation using the coordinate data from the data set selected from the plurality of data sets.

11. An in-vehicle camera comprising:

an image sensor configured to capture an image of a vehicle periphery; and

a cruise assist image generation circuitry including;

a steering angle information acquisition circuitry configured to acquire repeatedly at a time interval, steering angle information of a steering wheel;

an additional line generation circuitry configured to

select a data set from a plurality of data sets stored in a lookup table in accordance with the steering angle information, wherein each data set of the plurality of data sets includes coordinate data corresponding to a different steering angle of a plurality of steering angles,

generate a cruise assist additional line formed by a combination of a number of vertical segments and a number of horizontal segments based on the selection of the data set from the plurality of data sets, and

update the cruise assist additional line based on a change of the steering angle information; and

an additional line superimposing circuitry configured to superimpose the cruise assist additional line on the image of the vehicle periphery, wherein

the each data set the plurality of data sets includes a common coordinate between at least one of the vertical segments and at least one of the horizontal segments that form the cruise assist additional line.

12. The in-vehicle camera according to claim 11, wherein, to generate the cruise assist additional line formed by the combination of the number of vertical segments and the number of horizontal segments, the additional line generation circuitry is further configured to

generate one or more curved lines to represent at least one of the vertical segments or at least one of the horizontal segments by quadratic curve approximation using the coordinate data from the data set selected from the plurality of data sets, wherein the coordinate data includes coordinates of at least two end points representing the at least one of the vertical segments or the at least one of the horizontal segments, and

generate one or more straight lines to represent at least a second one of the vertical segments or at least a second one of the horizontal segments by collinear approximation using the coordinate data from the data set selected from the plurality of data sets.

13. A cruise-assist image generation device comprising:

an additional line generation circuitry configured to;

receive steering angle information of a steering wheel,

retrieve a data set from a plurality of data sets stored in a lookup table, each data set of the plurality of data sets includes coordinate data corresponding to a different steering angle of a plurality of steering angles,

generate a cruise assist additional line formed by a combination of a number of vertical segments and a number of horizontal segments based on the data set retrieved from the plurality of data sets, and

update the cruise assist additional line based on a change of the steering angle information; and

an additional line superimposing circuitry configured to superimpose the cruise assist additional line on an image of a vehicle periphery, wherein

the each data set of the plurality of data sets includes a common coordinate between at least one of the vertical segments and at least one of the horizontal segments that form the cruise assist additional line.

14. The cruise-assist image generation device according to claim 13, wherein, to generate the cruise assist additional line formed by the combination of the number of vertical segments and the number of horizontal segments, the additional line generation circuitry is further configured to

generate one or more curved lines to represent at least one of the vertical segments or at least one of the horizontal segments by quadratic curve approximation using the coordinate data from the data set selected from the plurality of data sets, wherein the coordinate data includes coordinates of at least two end points representing the at least one of the vertical segments or the at least one of the horizontal segments.

15. An equipment-control assist image generation device comprising:

a device configured to

monitor an external periphery environment using an imaging device mounted on a driving equipment, and

superimpose a track on an image in which a coordinate change occurs along with a driving operation on the image obtained by the device;

an additional line generation circuitry configured to

receive steering angle information of a steering wheel,

retrieve a data set from a plurality of data sets stored in a lookup table, each data set of the plurality of data sets includes coordinate data corresponding to a different steering angle of a plurality of steering angles,

generate a track assist additional line formed by a combination of a number of vertical segments and a number of horizontal segments based on the data set retrieved from the plurality of data sets, and

update the track assist additional line based on a change of the steering angle information; and

an additional line superimposing circuitry configured to superimpose the track assist additional line on a background image, wherein

the each data set of the plurality of data sets includes a common coordinate between at least one of the number of vertical segments and at least one of the horizontal segments that form the track assist additional line.

16. The equipment-control assist image generation device according to claim 15, wherein, to generate the track assist additional line formed by the combination of the number of vertical segments and the number of horizontal segments, the additional line generation circuitry is further configured to:

generate one or more curved lines to represent at least one of the vertical segments or at least one of the horizontal segments by quadratic curve approximation using the coordinate data from the data set selected from the plurality of data sets, wherein the coordinate data includes coordinates of at least two end points representing the at least one of the vertical segments or the at least one of the horizontal segments, and

generate one or more straight lines to represent at least a second one of the vertical segments or at least a second one of the horizontal segments by collinear approximation using the coordinate data from the data set selected from the plurality of data sets.

17. A cruise-assist image generation device comprising:

an additional line generation circuitry configured to;

generate coordinate data in accordance with a variable state of a visual field range,

store the coordinate data in a plurality of data sets in a lookup table, each data set of the plurality of data sets includes coordinate data corresponding to a different steering angle of a plurality of steering angles,

receive steering angle information of a steering wheel,

select a data set from the plurality of data sets stored in the lookup table based on the steering angle information,

generate a cruise assist additional line formed by a combination of a number of vertical segments and a number of horizontal segments based on the selection of the data set from the plurality of data sets, and

update the cruise assist additional line based on a change of the steering angle information,

wherein the each data set of the plurality of data sets includes a common coordinate between at least one of the vertical segments and at least one of the horizontal segments that form the cruise assist additional line; and

an additional line superimposing circuitry configured to superimpose the cruise assist additional line on an image of a vehicle periphery.

18. The cruise-assist image generation device according to claim 17, wherein, to generate the cruise assist additional line formed by the combination of the number of vertical segments and the number of horizontal segments, the additional line generation circuitry is further configured to

generate one or more curved lines to represent at least one of the vertical segments or at least one of the horizontal segments by quadratic curve approximation using the coordinate data from the data set selected from the plurality of data sets, wherein the coordinate data includes coordinates of at least two end points representing the at least one of the vertical segments or the at least one of the horizontal segments, and

generate one or more straight lines to represent at least a second one of the vertical segments or at least a second one of the horizontal segments by collinear approximation using the coordinate data from the data set selected from the plurality of data sets.