Composite image generation to remove obscuring objects

• US 10,424,098 B2
• Filed: 04/10/2017
• Issued: 09/24/2019
• Est. Priority Date: 02/18/2014
• Status: Expired due to Fees

• Current Assignee: Xinctec Technologies LLC (IP Edge LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Li, Daqi, Fang, Jun
• Primary Examiner(s): Gray, Ryan M
• Application Number:
  US15/483,418
• Publication Number:
  US 20170213376A1
• Time in Patent Office: 897 Days
• Field of Search: None
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Technologies are generally described for methods and systems effective to generate a composite image. The methods may include receiving first image data that includes object data corresponding to an object and receiving second image data that includes obscuring data. The obscuring data, if displayed on a display, may obscure at least a portion of the. The methods may also include identifying a first region that may include the object data, in the first image data, The methods may also include identifying a second region, that may include the obscuring data, in the second image data. The methods may also include replacing at least part of the second region with at least part of the first region to generate the composite image data that may include at least some of the object data. The methods may also include displaying the composite image on a display.

Claims

1. A method to generate composite image data by a first vehicle, the method comprising:

identifying a first region in first image data, wherein the first region includes object data, and wherein the object data corresponds to an object that is obscured, at least in part, by a second vehicle;

identifying a second region in second image data, wherein the second region includes obscuring data that corresponds to at least a part of the second vehicle;

generating first outline data based on an outline of the obscuring data;

generating second outline data based on an outline of the object data; and

generating the composite image data by replacing at least a part of the second region in the second image data with at least the first outline data and the second outline data.

2. The method of claim 1, further comprising receiving the first image data from the second vehicle.

3. The method of claim 1, further comprising generating the second image data, prior to identification of the first region in the first image data.

4. The method of claim 1, further comprising storing the composite image data in a memory.

5. The method of claim 1, further comprising storing the composite image data in a memory that stores data related to dimensions and/or appearances of one or more models of one or more vehicles.

6. The method of claim 1, further comprising storing the first image data, the object data, the second image data, the obscuring data, the first outline data, the second outline data, and the composite image data in a memory.

7. The method of claim 1, further comprising displaying a composite image, based on the composite image data, on a display.

8. The method of claim 1, wherein identifying the second region comprises at least one of identifying differences between the first image data and the second image data, and comparing the second image data with one or more stored images in a database stored in a memory.

9. The method of claim 1, wherein generating the first outline data comprises comparing the obscuring data with the stored image data of one or more vehicles stored in a memory.

10. The method of claim 1, further comprising,

removing the first outline data and the second outline data from the composite image data to generate virtual corridor image data, wherein the virtual corridor image data corresponds to a virtual corridor image that includes other objects that surround the object, the first vehicle, and the second vehicle.

11. The method of claim 1, wherein:

the first vehicle is a trailing vehicle,

the second vehicle is a leading vehicle, and

the first image data and the second image data are captured via a camera facing a front direction of the second vehicle and the first vehicle, respectively.

12. The method of claim 1, wherein identifying the first region further comprises identifying the object data by comparing the first image data and the second image data.

13. The method of claim 1, wherein the identifying the first region comprises receiving a voice command from a user in the first vehicle.

14. The method of claim 1, wherein the object includes a first object, the method further comprising, prior to replacing at least the part of the second region in the second image data:

determining a scaling factor based on one or more features of a second object that is part of the first region and the second region, wherein the scaling factor is a ratio of an area of the one or more features of the second object in the first region to an area of the one or more features of the second object in the second region; and

scaling pixels in the first region based on the scaling factor.

15. A first vehicle configured to generate composite image data, the first vehicle comprising:

a processor configured to;

identify a first region in first image data, wherein the first region includes object data, and wherein the object data corresponds to an object that is obscured, at least in part, by a second vehicle;

identify a second region in second image data, wherein the second region includes obscuring data that corresponds to at least a part of the second vehicle;

generate first outline data based on an outline of the obscuring data;

generate second outline data based on an outline of the obscuring data; and

generate the composite image data by replacing at least a part of the second region in the second image data with at least the first outline data and the second outline data; and

a memory configured to be in communication with the processor and to store the composite image data.

16. The first vehicle of claim 15, further comprising a transmitter configured to transmit the composite image data to a third vehicle that trails the first vehicle.

17. The first vehicle of claim 15, further comprising a receiver configured to receive the first image data from the second vehicle.

18. The first vehicle of claim 15, further comprising an image processing module configured to store instructions related to the generation of the composite image data.

19. The first vehicle of claim 15, further comprising a display configured to display a composite image based on the composite image data.

20. The first vehicle of claim 15, wherein to identify the first region, the processor is configured to identify differences between the first image data and the second image data, or compare the first image data with a database stored in the memory.

21. The first vehicle of claim 15, wherein to generate the first outline data, the processor is configured to compare the obscuring data with the stored image data of one or more vehicles stored in a memory.

22. The first vehicle of claim 15, wherein the memory is further configured to store the first image data, the object data, the second image data, the obscuring data, the first outline data, and the second outline data.

23. A system, comprising:

a first vehicle; and

a second vehicle configured to be in communication with the first vehicle and to;

receive first image data from the first vehicle;

identify a first region in the first image data, wherein the first region includes object data, and wherein the object data corresponds to an object that is obscured, at least in part, by the first vehicle;

identify a second region in second image data, wherein the second region includes obscuring data that corresponds to at least a part of the first vehicle;

generate first outline data based on an outline of the obscuring data;

generate second outline data based on an outline of the object data; and

generate composite image data by replacing at least a part of the second region in the second image data with at least the first outline data and the second outline data.

24. The system of claim 23, wherein the second vehicle is further configured to display a composite image, based on the composite image data, on a display of the second vehicle.

25. The system of claim 23, wherein the second vehicle is further configured to transmit the composite image data to a third vehicle that trails the second vehicle.