EXTENDED VIEW METHOD, APPARATUS, AND SYSTEM

• US 20160332574A1
• Filed: 11/12/2015
• Published: 11/17/2016
• Est. Priority Date: 05/11/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: PARK, Chang-soon, HONG, Young-jun
• Granted Patent: US 9,884,590 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/50 characterised by the display information being shared, e.g. external display, data transfer to other traffic participants or centralised traffic controller
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Provided is an extended view method and apparatus, the method including generating a virtual viewpoint image of a vehicle based on a surrounding view of the vehicle, generating an extended virtual viewpoint image based on the virtual viewpoint image of the vehicle and a received virtual viewpoint image of another vehicle, and displaying the extended virtual viewpoint image to a user.

Claims

1. A surround viewing method, the method comprising:

generating, by a first vehicle, a first virtual viewpoint image of the first vehicle based on at least one captured surrounding view image of the first vehicle;

receiving a second virtual viewpoint image, for a surrounding view image of a second vehicle, from the second vehicle;

generating, by the first vehicle, an extended virtual viewpoint image based on a combining of image information from the first virtual viewpoint image and the second virtual viewpoint image; and

outputting the generated extended virtual viewpoint image.

2. The method of claim 1, wherein the outputting of the generated extended virtual viewpoint image comprises outputting the generated extended virtual viewpoint image to a display and controlling the display to display the extended virtual viewpoint image with additional information received from the second vehicle regarding the second vehicle and/or an object in a region in the received second virtual viewpoint image.

3. The method of claim 1, further comprising receiving the at least one captured surrounding view image of the first vehicle from a road side equipment (RSE) separate from the first vehicle.

4. The method of claim 1, wherein the generating of the extended virtual viewpoint image comprises:

adjusting a position of at least one of a virtual viewpoint of the first virtual viewpoint image and a virtual viewpoint of the second virtual viewpoint image; and

generating the extended virtual viewpoint image based on a result of the adjusting.

5. The method of claim 1, wherein the generating of the extended viewpoint image comprises:

adjusting an image size of at least one of the first virtual viewpoint image and the second virtual viewpoint image; and

generating the extended virtual viewpoint image based on a result of the adjusting.

6. The method of claim 5, wherein the adjusting of the image size further comprises adjusting the image size of at least one of the first virtual viewpoint image and the second virtual viewpoint image based on at least one of position information of respective virtual viewpoints of the first and second virtual viewpoint images, angle information of the respective virtual viewpoints, and respective scale information of the first and second virtual viewpoint images.

7. The method of claim 1, wherein the generating of the extended virtual viewpoint image comprises aligning the first virtual viewpoint image and the second virtual viewpoint image based on a determined correlation between the first virtual viewpoint image and the second virtual viewpoint image or a determined location relationship between the first vehicle and the second vehicle.

8. The method of claim 7, wherein the generating of the extended virtual viewpoint image comprises adjusting, based on a result of the aligning, a position of a virtual viewpoint of the second virtual viewpoint image to be at a same position as a virtual viewpoint of the first virtual viewpoint image.

9. The method of claim 7, wherein the generating of the extended virtual viewpoint image comprises:

selecting between one of the first and second virtual viewpoint images to use to generate an overlapping area of the extended virtual viewpoint image, the overlapping area representing a same area that is included in both the first virtual viewpoint image and the second virtual viewpoint image; and

generating the extended virtual viewpoint image by combining the first virtual viewpoint image and the second virtual viewpoint image based on a result of the selecting.

10. The method of claim 9, wherein the selecting comprises selecting between one of the first and second virtual viewpoint images to use to generate the overlapping area based on at least one of an image resolution and a quality of communication with respective regard to each of the first virtual viewpoint image and the second virtual viewpoint image.

11. The method of claim 1, further comprising:

receiving, from road side equipment (RSE), a surrounding view image of a surrounding region of the RSE;

wherein the generating of the extended virtual viewpoint image comprises generating the extended virtual viewpoint image by combining the first virtual viewpoint image, the second virtual viewpoint image, and the surrounding view image received from the RSE.

12. The method of claim 1, further comprising:

adjusting a position of a virtual viewpoint of the first virtual viewpoint image, based on a location of the second vehicle, to generate an adjusted first virtual viewpoint image; and

transmitting, to the second vehicle, the adjusted first virtual viewpoint image.

13. The method of claim 1, further comprising:

transmitting additional information to the second vehicle,

wherein the additional information comprises at least one of identification (ID) information of the first vehicle, speed information of the first vehicle, height information of a virtual viewpoint of the first virtual viewpoint image, angle information of the virtual viewpoint of the first virtual viewpoint image, resolution information of the first virtual viewpoint image, scale information computed by the first vehicle, location information of the first vehicle, and traffic information obtained by the first vehicle.

14. The method of claim 1, wherein the generating of the first virtual viewpoint image comprises generating a top-view image of the first vehicle based on the at least one captured surrounding view image.

15. The method of claim 14, wherein the generating of the first virtual viewpoint image comprises generating the top-view image by reducing respective distances between pixels corresponding to a short-range region of the at least one captured surrounding view image relative to a location of the first vehicle and increasing respective distances between pixels corresponding to a long-range region of the at least one captured surrounding view image relative to a location of the first vehicle.

16. The method of claim 1, further comprising:

receiving additional information from the second vehicle,

wherein the additional information comprises at least one of ID information of the second vehicle, speed information of the second vehicle, height information of a virtual viewpoint of the second virtual viewpoint image, angle information of the virtual viewpoint of the second virtual viewpoint image, resolution information of the second virtual viewpoint image, scale information computed by the second vehicle, location information of the second vehicle, and traffic information obtained using the second vehicle.

17. The method of claim 16, wherein the generating of the extended virtual viewpoint image includes combining image information from the first virtual viewpoint image and the second virtual viewpoint image with image information representing one or more of the additional information received from the second vehicle.

18. The method of claim 1, wherein the outputting of the generated extended virtual viewpoint image comprises incorporating at least one of traffic information and obstacle information in the extended virtual viewpoint image.

19. The method of claim 1, wherein the extended virtual viewpoint image omnidirectionally represents a region surrounding the first vehicle that is larger than a region surrounding the first vehicle represented by the first virtual viewpoint image of the first vehicle.

20. A non-transitory recording medium comprising processor readable code to control at least one processing device to implement the method of claim 1.

21. A surround viewing method performed by a first vehicle, the method comprising:

receiving, from a second vehicle, a virtual viewpoint image of the second vehicle, for a surrounding view of the second vehicle;

adjusting at least one of an image size of a generated virtual image of the first vehicle, an image size of the received virtual image of the second vehicle, a position of a virtual viewpoint of the generated virtual viewpoint image of the first vehicle, and a position of a virtual viewpoint of the received virtual viewpoint image of the second vehicle, to respectively generate an adjusted virtual image of the first vehicle and/or adjusted virtual image of the second vehicle;

generating an extended virtual viewpoint image by combining one of the generated virtual image of the first vehicle and the adjusted virtual viewpoint image of the first vehicle with one of the received virtual viewpoint image of the second vehicle and the adjusted virtual viewpoint image of the second vehicle, based on a result of the adjusting; and

outputting the generated extended virtual viewpoint image.

22. The method of claim 21, further comprising transmitting the generated virtual viewpoint image of the first vehicle or the adjusted virtual viewpoint image of the first vehicle to the second vehicle with additional information regarding the generated virtual viewpoint image of the first vehicle.

23. A non-transitory recording medium comprising processor readable code to control at least one processing device to implement the method of claim 21.

24. A surround viewing system, the system comprising:

a virtual viewpoint image generator configured to generate a first virtual viewpoint image of a first vehicle based on at least one captured surrounding view image of the first vehicle;

a receiver configured to receive, from a second vehicle, a second virtual viewpoint image of the second vehicle for a surrounding view image of the second vehicle;

an extended virtual viewpoint image generator configured to generate an extended virtual viewpoint image by combining image information from the generated first virtual viewpoint image and the received second virtual viewpoint image; and

a display configured to display the generated extended virtual viewpoint image.

25. The apparatus of claim 24, wherein the extended virtual viewpoint image generator comprises:

an image converter configured to adjust at least one of an image size of the generated first virtual viewpoint image, an image size of the received second virtual viewpoint image, a position of a virtual viewpoint of the generated first virtual viewpoint image, and a position of a virtual viewpoint of the received second virtual viewpoint image, to respectively generate an adjusted virtual image of the first vehicle and/or adjusted virtual image of the second vehicle;

an image aligner configured to align one of the generated first virtual viewpoint image and the adjusted first viewpoint image with one of the received second virtual viewpoint image and the adjusted second viewpoint image, to generate aligned first and second virtual viewpoint images, based on a result of the adjusting;

an image selector configured to select between one of the aligned first and second virtual viewpoint images to use to generate an overlapping area of the extended viewpoint image, the overlapping area representing a same area that is included in both the aligned first and second virtual viewpoint images; and

an image generator configured to generate the extended virtual viewpoint image by combining the aligned first and second virtual viewpoint images based on a result of the selecting.