Low power DMA snoop and skip

• US 10,181,175 B2
• Filed: 12/17/2014
• Issued: 01/15/2019
• Est. Priority Date: 12/17/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Haraden, Ryan Scott, Tubbs, Matthew Ray, Muff, Adam James, Shearer, Robert Allen
• Primary Examiner(s): Sun, Hai Tao
• Application Number:
  US14/574,100
• Publication Number:
  US 20160180494A1
• Time in Patent Office: 1,490 Days
• Field of Search: 345537
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

Methods for preprocessing pixel data using a Direct Memory Access (DMA) engine during a data transfer of the pixel data from a first memory (e.g., a DRAM) to a second memory (e.g., an SRAM) are described. The pixel data may derive from a color camera or a depth camera in which individual pixel values are not a multiple of eight bits. In some cases, the DMA engine may perform a variety of image processing operations on the pixel data prior to the pixel data being written into the second memory. In one embodiment, the DMA engine may be configured to determine whether one or more pixels corresponding with the pixel data may be invalidated or skipped based on a minimum pixel value threshold and a maximum pixel value threshold and to embed pixel skipping information within unused bits of the pixel data.

Claims

1. A method for processing a depth map using a direct memory access engine, comprising:

initiating a data transfer of the depth map from a first memory to a second memory using the direct memory access engine, wherein the depth map includes a plurality of pixels, and wherein individual pixels store depth values representing linear distances to one or more objects within a captured scene;

acquiring a set of pixels associated with a portion of the depth map;

preprocessing the set of pixels during the data transfer to determine whether the set of pixels includes a valid depth value that is less than or equal to a maximum depth value threshold of a depth value range and is greater than or equal to a minimum depth value threshold of the depth value range;

generating an updated pixel by updating an individual pixel to embed metadata that indicates that the set of pixels does not include the valid depth value that is within the depth value range, wherein updating the individual pixel includes identifying unused bits within the individual pixel and embedding the metadata within the unused bits of the individual pixel while the depth map is transferred from the first memory to the second memory using the direct memory access engine such that the individual pixel includes both a depth value corresponding with the individual pixel and the metadata;

writing the updated pixel to the second memory;

reading the depth value corresponding to the individual pixel from the updated pixel prior to reading other depth values corresponding to other pixels of the set of pixels;

subsequent to the reading the updated pixel, determining that the other depth values of the set of pixels should be read based on the metadata that is embedded within the unused bits of the updated pixel, wherein the updated pixel includes the metadata and the depth value that corresponds to the individual pixel;

reading the other depth values of the set of pixels;

generating a new depth map by applying one or more processing operations to the depth value included within the updated pixel and the other depth values of the set of pixels; and

storing the new depth map.

2. The method of claim 1, wherein:

the metadata is a metadata code comprising a plurality of bits.

3. The method of claim 1, wherein:

the depth value is not a multiple of eight bits.

4. The method of claim 1, wherein:

the maximum depth value threshold corresponds with a first distance from a capture device used for generating the depth map and the minimum depth value threshold corresponds with a second distance from the capture device that is less than the first distance; and

the depth value corresponding with the individual pixel is not updated while the depth map is transferred from the first memory to the second memory.

5. The method of claim 1, wherein:

the first memory comprises a DRAM and the second memory comprises an SRAM.

6. The method of claim 1, wherein:

the set of pixels corresponds with a two-dimensional region within the depth map.

7. The method of claim 6, wherein:

the generating the updated pixel includes embedding a metadata code within the individual pixel indicating that the two-dimensional region within the depth map does not include the valid depth value that is within the depth value range; and

the updated pixel is a last pixel within the set of pixels.

8. A system for implementing a direct memory access engine, the system comprising:

one or more processors;

at least one memory in communication with the one or more processors, the at least one memory having computer-readable instructions stored thereupon that, when executed by the one or more processors, cause the one or more processors to;

initiate a data transfer of a depth map from a first memory to a second memory using the direct memory access engine, wherein the depth map includes a plurality of pixels, and wherein individual pixels store depth values representing linear distances to one or more objects within a captured scene;

acquire a set of pixels associated with a two-dimensional region of the depth map and preprocess the set of pixels during the data transfer to determine whether the set of pixels, that is associated with the two-dimensional region of the depth map, includes a valid depth value that is less than or equal to a maximum depth value threshold of a depth value range and is greater than or equal to a minimum depth value threshold of the depth value range;

identify unused bits within an individual pixel of the set of pixels;

generate an updated pixel by updating the individual pixel of the depth map to embed, within the unused bits, an identification that the set of pixels does not include the valid depth value that is within the depth value range, wherein embedding the identification within the unused bits of the individual pixel occurs while the depth map is transferred from the first memory to the second memory using the direct memory access engine, and wherein the updated pixel includes both a depth value for the individual pixel and the identification;

cause the updated pixel to be written to the second memory;

cause the depth value of the updated pixel to be read from the second memory prior to other depth values corresponding to other pixels of the set of pixels associated with the two-dimensional region of the depth map;

determine that the other depth values of the set of pixels associated with the two-dimensional region of the depth map should be read based on the identification that is embedded within the unused bits of the updated pixel;

read the other depth values of the set of pixels;

generate a new depth map via application of one or more processing operations to the depth value of the updated pixel and the other depth values of the other pixels of the set of pixels; and

store the new depth map.

9. The system of claim 8, wherein the updated pixel is a last pixel within the set of pixels.

10. The system of claim 8, wherein the first memory comprises a DRAM and the second memory comprises an SRAM.

11. A computer-implemented method, comprising:

initiating a data transfer of a depth map from a first memory to a second memory, wherein the depth map includes pixels that store depth values representing linear distances to one or more objects within a captured scene;

acquiring a first set of the pixels associated with a first portion of the depth map and a second set of the pixels associated with a second portion of the depth map;

preprocessing the first set of the pixels during the data transfer to determine that the first set of the pixels does not include at least one valid depth value that is within a depth value range, wherein the depth value range corresponds to a maximum depth value threshold of a first distance from a capture device used for generating the depth map and a minimum depth value threshold of a second distance from the capture device that is less than the first distance;

generating a first updated pixel by embedding first metadata within first unused bits of a first pixel that stores a first depth value of the first portion of the depth map, wherein the first metadata indicates that the first portion of the depth map does not include the at least one valid depth value that is less than or equal to the maximum depth value threshold of the first distance from the capture device and is greater than or equal to the minimum depth value threshold of the second distance from the capture device;

preprocessing the second set of the pixels during the data transfer to determine that the second set of the pixels does include at least one valid depth value that is within the depth value range;

generating a second updated pixel by embedding second metadata within second unused bits of a second pixel that stores a second depth value of the second portion of the depth map, wherein the second metadata indicates that the second portion of the depth map does include the at least one valid depth value that is less than or equal to the maximum depth value threshold of the first distance from the capture device and is greater than or equal to the minimum depth value threshold of the second distance from the capture device; and

writing, to the second memory, the first updated pixel and the second updated pixel.

12. The computer-implemented method of claim 11, further comprising:

determining that the first set of the pixels associated with the first portion of the depth map should not be read based on the first metadata; and

determining that the second set of the pixels associated with the second portion of the depth map should be read based on the second metadata.

13. The computer-implemented method of claim 12, further comprising:

based on the second metadata, performing at least one processing operation with respect to the second set of the pixels associated with the second portion of the depth map; and

based on the first metadata, skipping the at least one processing operation with respect to the first set of the pixels associated with the first portion of the depth map.