COMPOSITE IMAGE GENERATION TO REMOVE OBSCURING OBJECTS

• US 20160012574A1
• Filed: 02/18/2014
• Published: 01/14/2016
• Est. Priority Date: 02/18/2014
• Status: Active Grant

• Current Assignee: Xinctec Technologies LLC (IP Edge LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC), Daqi Li, Jun Fang
• Inventors: LI, DAQI, FANG, JUN
• Granted Patent: US 9,406,114 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Technologies are generally described for methods and systems effective to generate a composite image. The methods may include receiving first image data that includes object data corresponding to an object and receiving second image data that includes obscuring data. The obscuring data, if displayed on a display, may obscure at least a portion of the. The methods may also include identifying a first region that may include the object data, in the first image data. The methods may also include identifying a second region, that may include the obscuring data, in the second image data. The methods may also include replacing at least part of the second region with at least part of the first region to generate the composite image data that may include at least some of the object data. The methods may also include displaying the composite image on a display.

Claims

1. A method to generate a composite image, the method comprising, by a first device:

receiving, from a second device, first image data that includes object data, wherein the object data corresponds to an object;

receiving second image data that includes obscuring data, wherein the obscuring data corresponds to at least a part of the second device, the obscuring data, if displayed on a display, would obscure at least a portion of the object;

identifying a first region in the first image data, wherein the first region includes the object data;

identifying a second region in the second image data, wherein the second region includes the obscuring data;

replacing at least part of the second region in the second image data with at least part of the first region, to generate the composite image data, where the composite image data includes at least some of the object data; and

displaying the composite image on a display.

2. The method of claim 1, wherein the first device includes a vehicle and the display is inside the vehicle.

3. The method of claim 1, wherein the first device includes a first vehicle, and the second device includes a second vehicle.

4. The method of claim 1, further comprising, prior to replacing at least part of the second region with at least part of the first region:

identifying a feature in the first image data;

identifying the feature in the second image data;

determining a scaling factor based on feature; and

scaling pixels in the first region based on the scaling factor.

5. The method of claim 1, further comprising, prior to generating the composite image data:

identifying an outline of the obscuring data; and

generating outline data based on the outline,

wherein the composite image data includes the outline data.

6. The method of claim 5, wherein identifying the outline includes comparing the obscuring data to a database that includes images of vehicles;

and

the outline data includes an outline of a vehicle.

7. The method of claim 6, further comprising, prior to replacing at least part of the second region with at least part of the first region:

identifying a feature in the first image data;

identifying the feature in the second image data;

determining a scaling factor based on feature; and

scaling pixels in the first region based on the scaling factor.

8. The method of claim 1, wherein:

the first device includes a first vehicle;

the second device includes a second vehicle

the composite image data is first composite image data;

the object data is first object data;

the display is a first display;

the obscuring data is first obscuring data; and

the method further comprises, at the second device;

receiving the first composite image data;

receiving third image data that includes second obscuring data, wherein the second obscuring data, if displayed on a second display in the second vehicle, would obscure at least a portion of the object;

identifying a third region in the first composite image data, wherein the third region includes at least a portion of the object data;

identifying a fourth region in the third image data, wherein the fourth region includes the second obscuring data;

replacing at least part of the fourth region in the third image data with at least part of the third region to generate second composite image data, wherein the second composite image data includes at least some of the object data; and

displaying the second composite image data on the second display.

9. A first device configured to generate a composite image, the first device comprising:

a camera;

a display;

a processor configured to be in communication with the camera and the display, the processor being configured to;

receive first image data from a second device, wherein the first image data includes object data, and the object data corresponds to an object;

receive second image data from the camera, wherein the second image data includes obscuring data that corresponds to at least a part of the second device, and the obscuring data, if displayed on the display, would obscure at least a portion of the object;

identify a first region in the first image data, wherein the first region includes the object data;

identify a second region in the second image data, wherein the second region includes the obscuring data;

replace at least part of the second region in the second image data with at least part of the first region to generate the composite image data, where the composite image data includes at least some of the object data; and

display the composite image on the display.

10. The first device of claim 9, wherein the display is a three-dimensional display.

11. The first device of claim 9, wherein the first device includes a vehicle, the camera and the display are inside of the vehicle.

12. The first device of claim 9, wherein the first device includes a first vehicle, the camera and the display are inside of the first vehicle, and the second device includes a second vehicle.

13. The first device of claim 9, wherein the processor is further configured to:

prior to replacement of at least part of the second region with at least part of the first region;

identify a feature in the first image data;

identify the feature in the second image data;

determine a scaling factor based on the feature; and

scale pixels in the first region based on the scaling factor.

14. The first device of claim 9, wherein:

the second device is further configured to identify an outline of the obscuring data and generate outline data based on the outline; and

the composite image data includes the outline data.

15. The first device of claim 9, wherein identification of the outline includes comparison of the obscuring data to a database that includes images of vehicles;

and

the outline data includes an outline of a vehicle.

16. A system configured to generate a composite image, the system comprising:

a first device, wherein the first device includes;

a first camera;

a transmitter;

a first processor configured to be in communication with the first camera and transmitter; and

a second device, wherein the second device includes;

a display;

a second camera;

a receiver; and

a second processor configured to be in communication with the second camera, the receiver and the display; and

the first camera configured to generate first image data, wherein the first image data includes object data, and the object data corresponds to an object;

the first processor configured to transmit the first image data through the transmitter from the first device to the second device;

the receiver configured to receive the first image data;

the second camera configured to generate second image data, wherein the second image data includes obscuring data that corresponds to at least a part of the first device, and the obscuring data, if displayed on the display, would obscure at least a portion of the object;

the second processor configured to;

receive the first image data;

receive the second image data;

identify a first region in the first image data, wherein the first region includes the object data;

identify a second region in the second image data, wherein the second region includes the obscuring data;

replace at least part of the second region in the second image data with at least part of the first region to generate the composite image data, where the composite image data includes at least part of the object data; and

display the composite image on the display.

17. The system of claim 16, wherein:

the first device includes a first vehicle;

the second device includes a second vehicle; and

the obscuring data corresponds to the first vehicle.

18. The system of claim 16, wherein the processor is further configured to, prior to replacement of at least part of the second region with at least part of the first region:

identify a feature in the first image data;

identify the feature in the second image data;

determine a scaling factor based on feature; and

scale pixels in the first region based on the scaling factor.

19. The system of claim 16, wherein:

the processor is further configured to identify an outline of the obscuring data and generate outline data based on the outline; and

the composite image data includes the outline data.

20. The system of claim 19, wherein identification of the outline includes comparison of the obscuring data to a database that includes images of vehicles;

and

the outline data includes an outline of a vehicle.