Image processing apparatus and image processing method for generating synthetic image and changing synthetic image

• US 10,328,866 B2
• Filed: 11/05/2013
• Issued: 06/25/2019
• Est. Priority Date: 01/30/2013
• Status: Active Grant

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Original Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Inventors: Shimizu, Kenta, Yamada, Masahiro, Sawada, Yasuyoshi, Hayashida, Takumi, Matsuoka, Azusa, Okada, Naoyuki, Fujioka, Minoru, Shidochi, Yuka, Esaka, Toshinori
• Primary Examiner(s): Werner, David N
• Application Number:
  US14/655,918
• Publication Number:
  US 20150356735A1
• Time in Patent Office: 2,058 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/101 using cameras with adjustable capturing direction
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/304 using merged images, e.g. merging camera image with stored images
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image processing apparatus for processing an image according to one embodiment includes a generating unit, a display control unit, and a change receiving unit. The display control unit generates a synthetic image providing a view of a vehicle from a virtual viewpoint, based on a plurality of onboard camera images. The display control unit displays the generated synthetic image on a display unit. The change receiving unit receives a change in the relative positional relation between an image region that is based on one of the camera images and image regions that are based on the other camera images in the synthetic image. The generating unit generates a synthetic image, based on a changed positional relation every time a change in the positional relation is received.

Claims

1. An image processing apparatus for processing an image, the image processing apparatus comprising a processor configured to:

generate a synthetic image (i) providing, based on a plurality of onboard camera images, a view of an environment surrounding a vehicle positioned on a predetermined parking zone from a virtual viewpoint and (ii) including images of markers provided at predetermined positions with respect to the parking zone that correspond to the camera images;

superimpose frame-shaped indicators at predetermined fixed positions in the synthetic image;

display the generated synthetic image and the frame-shaped indicators on a display;

when among the marker images and the frame-shaped indicators, one marker image and a corresponding frame-shaped indicator on the display do not match each other, receive a change of an image region that is based on one camera image including the one marker image, a of the camera images, while the vehicle is stopping on the parking zone; and

generate a changed synthetic image based on the changed image region every time a change in the image region is received without moving the vehicle.

2. The image processing apparatus according to claim 1, wherein when a position is selected in the synthetic image displayed on the display, the processor is further configured to enlarge a part of the synthetic image including the position and display the enlarged part on the display.

3. An image processing method comprising:

generating a synthetic image (i) providing, based on a plurality of onboard camera images, a view of an environment surrounding a vehicle positioned on a predetermined parking zone from a virtual viewpoint and (ii) including images of first markers positioned at predetermined positions with respect to the parking zone that correspond to the camera images on a parking plane on which the vehicle is parked;

superimposing frame-shaped indicators at predetermined fixed positions in the synthetic image;

displaying the generated synthetic image and the frame-shaped indicators;

when among the marker images and the frame-shaped indicators, one marker image and a corresponding frame-shaped indicator on the display do not match each other, receiving a change of an image region that is based on one camera image including the image of the first marker of the camera images, while the vehicle is stopping on the parking zone; and

generating a changed synthetic image based on the image region every time a change in the image region is received without moving the vehicle.

4. The image processing method according to claim 3, further comprising

changing the relative positional relation and generating the synthetic image so that an image of a second marker included in the synthetic image forms a certain shape, the second marker being positioned within angles of view of two onboard cameras that are adjacently disposed along an outer periphery of the vehicle on the parking plane.

5. An image processing apparatus for processing an image, the image processing apparatus comprising a processor configured to:

generate a synthetic image (i) providing, based on a single onboard camera image including the image of the marker, a view of an environment surrounding a vehicle positioned on a predetermined parking zone from a virtual viewpoint and (ii) including an image of a marker provided at a predetermined position with respect to the parking zone that correspond to the camera image;

superimpose a frame-shaped indicator at a predetermined fixed position in the synthetic image;

display the generated synthetic image and the frame-shaped indicator on a display;

when the marker image and the frame-shaped indicator on the display do not match each other, receive a change of an image region that is based on the camera image, while the vehicle is stopping on the parking zone; and

generate a changed synthetic image based on the changed image region every time a change in the image region is received without moving the vehicle.