METHOD AND APPARATUS OF IMAGE PROCESSING

• US 20180343430A1
• Filed: 05/25/2017
• Published: 11/29/2018
• Est. Priority Date: 05/25/2017
• Status: Active Grant

• Current Assignee: Advanced Micro Devices, Inc.
• Original Assignee: Advanced Micro Devices, Inc.
• Inventors: Schmit, Michael L., Giduthuri, Radhakrishna, Nagesh Gowda, Kiriti
• Granted Patent: US 10,455,211 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A method and apparatus of precomputing includes capturing a first image by a first image capturing device. An image space for the first image is defined and pixels in the image space are analyzed for validity. Valid pixels are stored as valid pixel groups and the valid pixel groups are processed.

Claims

1. A method of precomputing, comprising:

capturing a first image by a first image capturing device of a plurality of image capturing devices;

capturing a second image by a second image capturing device of the plurality of image capturing devices;

defining a first image space for the first image that is a subset of the first image;

defining a second image space for the second image that is a subset of the second image;

analyzing pixels in the first image space for validity for processing of pixels captured by the first image capturing device;

analyzing pixels in the second image space for validity for processing of pixels captured by the second image capturing device;

storing valid pixels for the first and second image spaces as a valid pixel group; and

receiving and processing, in a graphics pipeline, the valid pixel group.

2. The method of claim 1 wherein defining the first and second image spaces includes defining the first and second image spaces as a subset of a total image to be projected into a three-hundred sixty degree space.

3. The method of claim 2 wherein the projection is an equirectangular projection.

4. The method of claim 2 wherein the defined first image space includes image space in a view of the first image capturing device and the defined second image space includes image space in a view of the second image capturing device.

5. The method of claim 2, further comprising generating a mask that identifies the defined image spaces and excludes from processing pixels in areas of the total image outside the defined image spaces for the respective first or second image devices.

6. The method of claim 1 wherein a boundary of the defined first image space is located in an overlap area between the first image and one or more adjacent images, and a boundary of the defined second image space is located in an overlap area between the second image and one or more adjacent images.

7. The method of claim 6 wherein the boundary is defined by one or more seams formed in the overlap areas where the first image or second image and respective adjacent images are combined during a stitching process.

8. The method of claim 6 wherein analyzing pixels in the first or second image spaces for validity includes determining if the pixel is within the boundary for that image space.

9. The method of claim 6, further comprising storing a pixel within the boundary in the valid pixel group.

10. The method of claim 6 wherein analyzing pixels in the first or second image spaces for validity includes determining pixels at the edge of the boundary and generating a mirror pixel for each pixel at the edge of the boundary.

11. The method of claim 9, further comprising storing a mirror pixel at the edge of the boundary in the valid pixel group.

17. The apparatus of claim 11 wherein a boundary of the defined first image space is located in an overlap area between the first image and one or more adjacent images, and a boundary of the defined second image space is located in an overlap area between the second image and one or more adjacent images.

18. The apparatus of claim 17 wherein the boundary is defined by one or more seams formed in the overlap areas where the first image or second image and respective adjacent images are combined during a stitching process.

19. The apparatus of claim 17 wherein the computing device determines if the pixel is within the boundary and stores pixels within the boundary in the valid pixel group.

12. An apparatus, comprising:

a computing device in communication with a first image capturing device and a second image capturing device, wherein the computing device receives a first image captured by the first image capturing device, receives a second image captured by the second image capturing device, the computing device defines a first image space for the first image that is a subset of the first image, defines a second image space for the second image that is a subset of the second image analyzes pixels in the first image space for validity for processing of pixels captured by the first image capturing device, analyzes pixels in the second image space for validity for processing of pixels captured by the second image capturing device, stores valid pixels for the first and second image spaces as a valid pixel group, and processes, in a graphics pipeline of the computing device, the valid pixel group.

13. The apparatus of claim 12 wherein the computing device defines the first and second image spaces as a subset of a total image to be projected into a three-hundred sixty degree space.

14. The apparatus of claim 13 wherein the projection is an equirectangular projection.

15. The apparatus of claim 13 wherein the defined first image space includes image space in a view of the first image capturing device and the defined second image space includes image space in a view of the second image capturing device.

16. The apparatus of claim 13, further comprising the computing device generating a mask that identifies the defined image spaces and excludes from processing pixels in areas of the total image outside the defined image spaces for the respective first or second image devices.

20. A non-transitory computer-readable medium having instructions recorded thereon, that when executed by a computing device, cause the computing device to perform operations including:

receiving a first image captured by a first image capturing device of a plurality of image capturing devices;

receiving a second image captured by a second image capturing device of the plurality of image capturing devices;

defining a first image space for the first image that is a subset of the first image;

defining a second image space for the second image that is a subset of the second image;

analyzing pixels in the first image space for validity for processing of pixels captured by the first image capturing device;

analyzing pixels in the second image space for validity for processing of pixels captured by the second image capturing device;

storing valid pixels for the first and second image spaces as a valid pixel group; and

receiving and processing, in a graphics pipeline, the valid pixel group.