Filling in surround view areas blocked by mirrors or other vehicle parts
First Claim
1. An imaging system filling in blind spot regions in peripheral areas of an associated vehicle, the system comprising:
- a processor;
a non-transient memory operatively coupled with the processor, the non-transient memory storing intrinsic blind spot data representative of a blind spot region of a peripheral area of the associated vehicle, and movement data representative of a speed of movement of the associated vehicle;
an image obtaining unit operatively coupled with the processor, the image obtaining unit being configured to receive first image data representative of a first image of the peripheral area of the associated vehicle captured at a first time, and to receive second image data representative of a second image of the peripheral area of the associated vehicle captured at a second time after the first time;
a predicted blind spot region determining unit operatively coupled with the processor, the predicted blind spot region determining unit being configured to determine, at the first time, a portion of the first image predicted to be in the blind spot region at the second time in accordance with the intrinsic blind spot data and the movement data; and
a current blind spot region restoration unit operatively coupled with the processor, the current blind spot region restoration unit being configured to generate, at the second time, restored second image data by merging;
i) a selected portion of the first image data corresponding to the portion of the first image predicted to be in the blind spot region at the second time, with ii) the second image data, wherein the generated restored second image data is representative of a restored second image of the peripheral area of the associated vehicle at the second time unobstructed by the blind spot region.
2 Assignments
0 Petitions
Accused Products
Abstract
An imaging system, method, and computer readable medium filling in blind spot regions in images of peripheral areas of a vehicle. Intrinsic or extrinsic blind spot data is used together with vehicle movement data including vehicle speed and steering angle information to determine one or more portions of a series of images of the peripheral areas that include or will include one or more blind spot obstructions in the images. Portions of the images predicted to be obstructed at a future time, portions of overlapping images obtained concurrently from plural sources, or both, are obtained and used as an image patch. A blind spot region restoration unit operates to stitch together a restored image without the blind spot obstruction by merging one or more image patches into portions of the images that include the one or more blind spot obstructions.
-
Citations
32 Claims
-
1. An imaging system filling in blind spot regions in peripheral areas of an associated vehicle, the system comprising:
-
a processor; a non-transient memory operatively coupled with the processor, the non-transient memory storing intrinsic blind spot data representative of a blind spot region of a peripheral area of the associated vehicle, and movement data representative of a speed of movement of the associated vehicle; an image obtaining unit operatively coupled with the processor, the image obtaining unit being configured to receive first image data representative of a first image of the peripheral area of the associated vehicle captured at a first time, and to receive second image data representative of a second image of the peripheral area of the associated vehicle captured at a second time after the first time; a predicted blind spot region determining unit operatively coupled with the processor, the predicted blind spot region determining unit being configured to determine, at the first time, a portion of the first image predicted to be in the blind spot region at the second time in accordance with the intrinsic blind spot data and the movement data; and a current blind spot region restoration unit operatively coupled with the processor, the current blind spot region restoration unit being configured to generate, at the second time, restored second image data by merging;
i) a selected portion of the first image data corresponding to the portion of the first image predicted to be in the blind spot region at the second time, with ii) the second image data, wherein the generated restored second image data is representative of a restored second image of the peripheral area of the associated vehicle at the second time unobstructed by the blind spot region. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. An imaging system filling in blind spot regions in peripheral areas of an associated vehicle, the system comprising:
-
a processor; a non-transient memory operatively coupled with the processor, the non-transient memory storing intrinsic blind spot data representative of a blind spot region of a peripheral area of the associated vehicle; an image obtaining unit operatively coupled with the processor and configured to receive first image data representative of a first image of the peripheral area of the associated vehicle captured at a first time, and to receive auxiliary image data representative of an auxiliary image of the peripheral area of the associated vehicle captured at the first time; a blind spot overlap region determining unit operatively coupled with the processor and configured to determine, at the first time in accordance with the intrinsic blind spot data; a first portion of the first image in the blind spot region at the first time; and a first portion of the auxiliary image overlapping the first portion of the first image in the blind spot region at the first time; and a current blind spot region restoration unit operatively coupled with the processor and configured to generate restored first image data by merging;
i) a first portion of the auxiliary image data corresponding to the first portion of the auxiliary image overlapping the first portion of the first image in the blind spot region at the first time, with ii) the first image data at a first portion of the first image data corresponding to the first portion of the first image in the blind spot region at the first time, wherein the generated restored first image data is representative of a restored first image of the peripheral area of the associated vehicle at the first time unobstructed by the blind spot region. - View Dependent Claims (11, 12, 13, 14)
-
-
15. A method in an associated imaging system for filling in a blind spot region in a peripheral area of an associated vehicle, the method comprising:
-
storing intrinsic blind spot data representative of a blind spot region of a peripheral area of the associated vehicle in a non-transient memory operatively coupled with a processor of the associated imaging system; storing movement data representative of a speed of movement of the associated vehicle in the non-transient memory; obtaining, at a first time, first image data using an image obtaining unit operatively coupled with the processor of the associated imaging system, the first image data being representative of a first image of the peripheral area of the associated vehicle at the first time; obtaining, at a second time after the first time, second image data using the image obtaining unit, the second image data being representative of a second image of the peripheral area of the associated vehicle at a second time after the first time; determining, at the first time by a predicted blind spot region determining unit operatively coupled with the processor of the associated imaging system, a portion of the first image predicted to be in the blind spot region at the second time in accordance with the intrinsic blind spot data and the movement data; and generating, at the second time by a current blind spot region restoration unit operatively coupled with the processor of the associated imaging system, restored second image data by merging;
i) a selected portion of the first image data corresponding to the portion of the first image predicted to be in the blind spot region at the second time, with ii) the second image data, wherein the generated restored second image data is representative of a restored second image of the peripheral area of the associated vehicle at the second time unobstructed by the blind spot region. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. A method in an associated imaging system for filling in a blind spot region in a peripheral area of an associated vehicle, the method comprising:
-
storing intrinsic blind spot data representative of a blind spot region of a peripheral area of the associated vehicle in a non-transient memory operatively coupled with a processor of the associated imaging system; obtaining, at a first time, first image data using an image obtaining unit operatively coupled with the processor of the associated imaging system, the first image data being representative of a first image of the peripheral area of the associated vehicle captured at a first time; obtaining, at the first time, auxiliary image data using the image obtaining unit operatively coupled with the processor of the associated imaging system, the auxiliary image data being representative of an auxiliary image of the peripheral area of the associated vehicle captured at the first time; determining, by a blind spot overlap region determining unit operatively coupled with the processor and in accordance with the intrinsic blind spot data; a first portion of the first image in the blind spot region at the first time; and a first portion of the auxiliary image overlapping the first portion of the first image in the blind spot region at the first time; and generating, by a current blind spot region restoration unit operatively coupled with the processor, restored first image data by merging;
i) a first portion of the auxiliary image data corresponding to the first portion of the auxiliary image overlapping the first portion of the first image in the blind spot region at the first time, with ii) the first image data at a first portion of the first image data corresponding to the first portion of the first image in the blind spot region at the first time, wherein the generated restored first image data is representative of a restored first image of the peripheral area of the associated vehicle at the first time unobstructed by the blind spot region. - View Dependent Claims (25, 26, 27, 28)
-
-
29. A non-transitory computer readable storage medium storing one or more sequences of instructions for filling in a blind spot region in a peripheral area of an associated vehicle, wherein said instructions, when executed by one or more processors, cause the one or more processors to execute steps comprising:
-
storing intrinsic blind spot data representative of a blind spot region of a peripheral area of the associated vehicle in a non-transient memory operatively coupled with a processor of the associated imaging system; storing movement data representative of a speed of movement of the associated vehicle in the non-transient memory; obtaining, at a first time, first image data using an image obtaining unit operatively coupled with the processor of the associated imaging system, the first image data being representative of a first image of the peripheral area of the associated vehicle at the first time; obtaining, at a second time after the first time, second image data using the image obtaining unit, the second image data being representative of a second image of the peripheral area of the associated vehicle at a second time after the first time; determining, at the first time by a predicted blind spot region determining unit operatively coupled with the processor of the associated imaging system, a portion of the first image predicted to be in the blind spot region at the second time in accordance with the intrinsic blind spot data and the movement data; and generating, at the second time by a current blind spot region restoration unit operatively coupled with the processor of the associated imaging system, restored second image data by merging;
i) a selected portion of the first image data corresponding to the portion of the first image predicted to be in the blind spot region at the second time, with ii) the second image data, wherein the generated restored second image data is representative of a restored second image of the peripheral area of the associated vehicle at the second time unobstructed by the blind spot region. - View Dependent Claims (30, 31, 32)
-
Specification