OBJECT VISUALIZATION IN BOWL-SHAPED IMAGING SYSTEMS

• US 20160080699A1
• Filed: 09/17/2014
• Published: 03/17/2016
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Buerkle, Cornelius, Natroshvili, Koba, Scholl, Kay-Ulrich
• Granted Patent: US 10,442,355 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2215/12 Shadow map, environment map
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens

Abstract

Technologies for visualizing moving objects on a bowl-shaped image include a computing device to receive a first fisheye image generated by a first fisheye camera and capturing a first scene and a second fisheye image generated by a second fisheye camera and capturing a second scene overlapping with the first scene at an overlapping region. The computing device identifies a moving object in the overlapping region and modifies a projected overlapping image region to visualize the identified moving object on a virtual bowl-shaped projection surface. The projected overlapping image region is projected on the virtual bowl-shaped projection surface and corresponds with the overlapping region captured in the first and second fisheye images.

Claims

1. A computing device for visualizing moving objects on a bowl-shaped image, the computing device comprising:

a first fisheye camera to capture a first fisheye image including a first scene;

a second fisheye camera to capture a second fisheye image including a second scene overlapping with the first scene at an overlapping region;

an object identification module to identify a moving object in the overlapping region; and

an object visualization module to modify a projected overlapping image region to visualize the identified moving object on a virtual bowl-shaped projection surface, wherein the projected overlapping image region is projected on the virtual bowl-shaped projection surface and corresponds with the overlapping region of the first and second fisheye images.

2. The computing device of claim 1, wherein to modify the projected overlapping image region comprises to:

select a fisheye image from which to project the identified moving object to the virtual bowl-shaped projection surface, wherein the fisheye image is selected from one of the first fisheye image or the second fisheye image;

determine a contour of the identified moving object on the selected fisheye image;

blend the projected overlapping image region to generate a blended overlapping image region; and

visualize the identified moving object'"'"'s contour in the blended overlapping image region.

3. The computing device of claim 2, wherein to visualize the identified moving object'"'"'s contour comprises to project the identified moving object'"'"'s contour from the selected fisheye image to the blended overlapping image region of the virtual bowl-shaped projection surface.

4. The computing device of claim 2, wherein to visualize the identified moving object'"'"'s contour comprises to:

identify an image region of the selected fisheye image within the identified moving object'"'"'s contour; and

project the identified image region to the blended overlapping image region of the virtual bowl-shaped projection surface to overlay the identified image region on the blended overlapping image region.

5. The computing device of claim 2, wherein the blended overlapping image region of the virtual bowl-shaped projection surface is transparent.

6. The computing device of claim 1, wherein to modify the projected overlapping image region comprises to:

determine a junction between the first and second fisheye images within the overlapping region of the first and second fisheye images based on a location of the identified moving object relative to the first and second fisheye cameras, wherein the junction is a border within the overlapping image region at which to combine the first and second fisheye images;

modify the first and second fisheye images within the overlapping region based on the determined junction;

combine the modified first and second fisheye images at the determined junction; and

project the combined first and second fisheye images to the virtual bowl-shaped projection surface.

7. The computing device of claim 6, wherein to determine the junction between the first and second fisheye images comprises to:

determine a corresponding viewing threshold of each of the first and second fisheye cameras;

select a fisheye image from which to project the identified moving object to the virtual bowl-shaped projection surface based on the identified moving object'"'"'s relative location and the viewing threshold, wherein the fisheye image is selected from one of the first fisheye image or the second fisheye image; and

determine a corresponding region of each of the first and second fisheye images to project to the virtual bowl-shaped projection surface based on the identified moving object'"'"'s relative location and the viewing threshold.

8. The computing device of claim 7, wherein the corresponding viewing threshold indicates a maximum and minimum viewing angle that each of the corresponding first and second fisheye cameras is able to capture.

9. The computing device of claim 1, further comprising an overlap determination module to determine an overlapping image region of the first and second fisheye images corresponding with the overlapping image region based on a geometry of a camera system including the first and second fisheye cameras.

10. The computing device of claim 1, further comprising:

a bowl generation module to generate a bowl-shaped image based on the projection of the modified projected overlapping image region to the virtual bowl-shaped projection surface; and

a display module to display the generated bowl-shaped image on a display of the computing device.

11. The computing device of claim 1, wherein the computing device is embodied as an in-vehicle computing system.

12. One or more machine-readable storage media comprising a plurality of instructions stored thereon that, in response to execution by a computing device, cause the computing device to:

receive (i) a first fisheye image generated by a first fisheye camera and capturing a first scene and (ii) a second fisheye image generated by a second fisheye camera and capturing a second scene overlapping with the first scene at an overlapping region;

identify a moving object in the overlapping region; and

modify a projected overlapping image region to visualize the identified moving object on a virtual bowl-shaped projection surface, wherein the projected overlapping image region is projected on the virtual bowl-shaped projection surface and corresponds with the overlapping region captured in the first and second fisheye images.

13. The one or more machine-readable storage media of claim 12, wherein to modify the projected overlapping image region comprises to:

select a fisheye image from which to project the identified moving object to the virtual bowl-shaped projection surface, wherein the fisheye image is selected from one of the first fisheye image or the second fisheye image;

determine a contour of the identified moving object on the selected fisheye image;

blend the projected overlapping image region to generate a blended overlapping image region; and

visualize the identified moving object'"'"'s contour in the blended overlapping image region.

14. The one or more machine-readable storage media of claim 13, wherein to visualize the identified moving object'"'"'s contour comprises to project the identified moving object'"'"'s contour from the selected fisheye image to the blended overlapping image region of the virtual bowl-shaped projection surface.

15. The one or more machine-readable storage media of claim 13, wherein to visualize the identified moving object'"'"'s contour comprises to:

identify an image region of the selected fisheye image within the identified moving object'"'"'s contour; and

project the identified image region to the blended overlapping image region of the virtual bowl-shaped projection surface to overlay the identified image region on the blended overlapping image region.

16. The one or more machine-readable storage media of claim 12, wherein to modify the projected overlapping image region comprises to:

determine a junction between the first and second fisheye images within the overlapping region of the first and second fisheye images based on a location of the identified moving object relative to the first and second fisheye cameras, wherein the junction is a border within the overlapping image region at which to combine the first and second fisheye images;

modify the first and second fisheye images within the overlapping region based on the determined junction;

combine the modified first and second fisheye images at the determined junction; and

project the combined first and second fisheye images to the virtual bowl-shaped projection surface.

17. The one or more machine-readable storage media of claim 16, wherein to determine the junction between the first and second fisheye images comprises to:

determine a corresponding viewing threshold of each of the first and second fisheye cameras;

select a fisheye image from which to project the identified moving object to the virtual bowl-shaped projection surface based on the identified moving object'"'"'s relative location and the viewing threshold, wherein the fisheye image is selected from one of the first fisheye image or the second fisheye image; and

determine a corresponding region of each of the first and second fisheye images to project to the virtual bowl-shaped projection surface based on the identified moving object'"'"'s relative location and the viewing threshold.

18. The one or more machine-readable storage media of claim 17, wherein the corresponding viewing threshold indicates a maximum and minimum viewing angle that each of the corresponding first and second fisheye cameras is able to capture.

19. The one or more machine-readable storage media of claim 12, wherein the plurality of instructions further cause the computing device to determine an overlapping image region of the first and second fisheye images corresponding with the overlapping image region based on a geometry of a camera system including the first and second fisheye cameras.

20. The one or more machine-readable storage media of claim 12, wherein the plurality of instructions further cause the computing device to:

generate a bowl-shaped image based on the projection of the modified projected overlapping image region to the virtual bowl-shaped projection surface; and

display the generated bowl-shaped image on a display of the computing device.

21. A method for visualizing moving objects on a bowl-shaped image generated by a computing device, the method comprising:

receiving, by the computing device, (i) a first fisheye image generated by a first fisheye camera and capturing a first scene and (ii) a second fisheye image generated by a second fisheye camera and capturing a second scene overlapping with the first scene at an overlapping region;

identifying, by the computing device, a moving object in the overlapping region; and

modifying, by the computing device, a projected overlapping image region to visualize the identified moving object on a virtual bowl-shaped projection surface, wherein the projected overlapping image region is projected on the virtual bowl-shaped projection surface and corresponds with the overlapping region captured in the first and second fisheye images.

22. The method of claim 21, wherein modifying the projected overlapping image region comprises:

selecting a fisheye image from which to project the identified moving object to the virtual bowl-shaped projection surface, wherein the fisheye image is selected from one of the first fisheye image or the second fisheye image;

determining a contour of the identified moving object on the selected fisheye image;

blending the projected overlapping image region to generate a blended overlapping image region; and

visualizing the identified moving object'"'"'s contour in the blended overlapping image region.

23. The method of claim 22, wherein visualizing the identified moving object'"'"'s contour comprises:

identifying an image region of the selected fisheye image within the identified moving object'"'"'s contour; and

projecting the identified image region to the blended overlapping image region of the virtual bowl-shaped projection surface to overlay the identified image region on the blended overlapping image region.

24. The method of claim 21, wherein modifying the projected overlapping image region comprises:

determining a junction between the first and second fisheye images within the overlapping region of the first and second fisheye images based on a location of the identified moving object relative to the first and second fisheye cameras, wherein the junction is a border within the overlapping image region at which to combine the first and second fisheye images;

modifying the first and second fisheye images within the overlapping region based on the determined junction;

combining the modified first and second fisheye images at the determined junction; and

projecting the combined first and second fisheye images to the virtual bowl-shaped projection surface.

25. The method of claim 24, wherein determining the junction between the first and second fisheye images comprises:

determining a corresponding viewing threshold of each of the first and second fisheye cameras;

selecting a fisheye image from which to project the identified moving object to the virtual bowl-shaped projection surface based on the identified moving object'"'"'s relative location and the viewing threshold, wherein the fisheye image is selected from one of the first fisheye image or the second fisheye image; and

determining a corresponding region of each of the first and second fisheye images to project to the virtual bowl-shaped projection surface based on the identified moving object'"'"'s relative location and the viewing threshold.