SCANNING WINDOW IN HARDWARE FOR LOW-POWER OBJECT-DETECTION IN IMAGES

• US 20160092735A1
• Filed: 09/25/2015
• Published: 03/31/2016
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: GOVIL, Alok, RANGAN, Venkat, SEO, Hae-Jong, RASQUINHA, Nelson
• Granted Patent: US 9,940,533 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 25/443 by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array

Abstract

An apparatus includes a hardware sensor array including a plurality of pixels arranged along at least a first dimension and a second dimension of the array, each of the pixels capable of generating a sensor reading. A hardware scanning window array includes a plurality of storage elements arranged along at least a first dimension and a second dimension of the hardware scanning window array, each of the storage elements capable of storing a pixel value based on one or more sensor readings. Peripheral circuitry for systematically transfers pixel values, based on sensor readings, into the hardware scanning window array, to cause different windows of pixel values to be stored in the hardware scanning window array at different times. Control logic coupled to the hardware sensor array, the hardware scanning window array, and the peripheral circuitry, provides control signals to the peripheral circuitry to control the transfer of pixel values.

Claims

1. An apparatus for isolating pixel values in hardware comprising:

a hardware sensor array comprising a plurality of pixels arranged along at least a first dimension and a second dimension of the hardware sensor array, each of the pixels capable of generating a sensor reading based on environmental conditions;

a hardware scanning window array comprising a plurality of storage elements arranged along at least a first dimension and a second dimension of the hardware scanning window array, each of the storage elements capable of storing a pixel value based on one or more sensor readings from the hardware sensor array;

peripheral circuitry for systematically transferring pixel values, based on sensor readings from the hardware sensor array, into the hardware scanning window array, to cause different windows of pixel values to be stored in the hardware scanning window array at different times; and

control logic coupled to the hardware sensor array, the hardware scanning window array, and the peripheral circuitry, the control logic operable to provide control signals to the peripheral circuitry to control the transfer of pixel values into the hardware scanning window array.

2. The apparatus of claim 1, wherein the hardware sensor array further comprises:

configurable combining circuitry coupled to the plurality of pixels, the configurable combining circuitry operable to combine, in hardware, multiple sensor readings from the plurality of pixels to generate the pixel values, according to at least one combining pattern, wherein the configurable combining circuitry comprises;

a first plurality of controllable connections operable to connect adjacent pixels along the first dimension of the hardware sensor array; and

a second plurality of controllable connections operable to connect adjacent pixels along the second dimension of the hardware sensor array,

wherein connecting adjacent pixels generates averaged pixel values at each of the adjacent pixels being connected.

3. The apparatus of claim 2, wherein the sensor reading is an analog sensor reading, wherein the pixel value is an analog pixel value, and wherein the configurable combining circuitry is configurable analog combining circuitry.

4. The apparatus of claim 1, wherein the peripheral circuitry comprises:

a line buffer comprising a plurality of storage elements arranged along at least a first dimension and a second dimension of the line buffer, the first dimension of the line buffer being smaller than the first dimension of the hardware sensor array, the second dimension of the line buffer being equal to the second dimension of the hardware sensor array,

wherein the line buffer is capable of storing selected rows of pixel values from the hardware sensor array.

5. The apparatus of claim 4, wherein the control logic is capable of storing a next row of pixel values from the hardware sensor array into the line buffer, by replacing an oldest row of pixel values in the line buffer, thereby introducing a discontinuity along a row direction in an image represented by the pixel values stored in the line buffer.

6. The apparatus of claim 5, wherein the peripheral circuitry further comprises a multiplexer circuit positioned between the line buffer and the hardware scanning window array, the multiplexer circuit capable of switching row ordering while transferring pixel values from the line buffer to the hardware scanning window array, thereby removing the discontinuity along a row direction in an image represented by the pixel values stored in the line buffer.

7. The apparatus of claim 5, wherein the peripheral circuitry further comprises a multiplexer circuit positioned between the hardware scanning window array and a hardware computer vision feature computation block, the multiplexer circuit capable of switching column ordering while transferring pixel values from the hardware scanning window array to the hardware computer vision feature computation block, thereby removing the discontinuity along a column direction in an image represented by the pixel values stored in the hardware scanning window array.

8. The apparatus of claim 4, wherein the control logic is capable of storing a next column of pixel values from the line buffer into the hardware scanning window array, by replacing an oldest column of pixel values in the hardware scanning window array, thereby introducing a discontinuity along a column direction in an image represented by the pixel values stored in the hardware scanning window array.

9. The apparatus of claim 4, wherein the peripheral circuitry further comprises a hardware computer vision feature computation block operable to perform, in hardware, computer vision computations based on values stored in the hardware scanning window array.

10. The apparatus of claim 9, further comprising configurable combining circuitry coupled to the plurality of storage elements of the hardware scanning window, wherein the configurable combining circuitry is capable of performing combining, in hardware, of some multiple pixel values to generate a first set of combined pixel values according to a first combining pattern, while maintaining the multiple pixel values for subsequent combining of some other multiple pixel values to generate a second set of combined pixel values according to a second combining pattern, and wherein the first set of combined pixel values and the second set of combined pixel values are used by the hardware computer vision feature computation block to compute a multi-block local binary pattern computer vision feature.

11. The apparatus of claim 9, wherein the peripheral circuitry further comprises an integration block capable of computing an integral image based on the sensor readings from the hardware sensor array and storing the computed integral image in the scanning window array, and wherein combined pixel values generated from the integral image stored in the scanning window array are used by the hardware computer vision feature computation block to compute a multi-block local binary pattern computer vision feature.

12. The apparatus of claim 9, wherein the configurable combining circuitry comprises:

a first plurality of controllable connections operable to connect adjacent circuit elements along the first dimension of the hardware scanning window array; and

a second plurality of controllable connections operable to connect adjacent pixels along the second dimension of the hardware scanning window array,

wherein connecting adjacent pixels generates averaged pixel values at each of the adjacent pixels being connected.

13. The apparatus of claim 9, wherein the different ones of the first and second combining patterns allow for selection of the plurality of storage elements at different locations and dimensions within the hardware scanning window array.

14. The apparatus of claim 9, wherein the hardware computer vision feature computation block comprises circuitry operable to perform corner detection.

15. The apparatus of claim 14, further comprising a second hardware scanning window array coupled to the control logic, the control logic operable to provide control signals to control the transfer of pixel values from the hardware scanning window array to the second hardware scanning window array based on detection of a corner by the hardware computer vision feature computation block in an image represented by the pixel values stored in the hardware scanning window array, the peripheral circuitry further comprising a multiplexer circuit positioned between the hardware scanning window array and the second hardware scanning window array, the multiplexer circuit capable of switching column ordering while transferring the pixel values from the hardware scanning window array to the second hardware scanning window array, thereby removing a discontinuity along a column direction in the image.

16. The apparatus of claim 15, further comprising a second hardware computer vision feature computation block operable to perform, in hardware, computer vision computations based on values stored in the second hardware scanning window array.

17. The apparatus of claim 1, wherein at least one pixel of the plurality of pixels comprises a sensor element and in-pixel circuitry.

18. The apparatus of claim 1, wherein the pixel values each based on the one or more sensor readings from the hardware sensor array comprise raw pixel values each based on one or more raw sensor readings.

19. The apparatus of claim 1, wherein no image signal processing circuitry is disposed between the hardware sensor array and the hardware scanning window array.

20. A method for isolating pixel values in hardware, comprising:

generating, via a plurality of pixels within a hardware sensor array, a sensor reading based on environmental conditions, wherein the plurality of pixels are arranged along at least a first dimension and a second dimension of the hardware sensor array;

storing, within a plurality of storage elements within a hardware scanning window array, a pixel value based on one or more sensor readings from the hardware sensor array, wherein the plurality of storage elements are arranged along at least a first dimension and a second dimension of the hardware scanning window array;

causing, via peripheral circuitry for systematically transferring pixel values, different windows of pixel values to be stored, based on sensor readings from the hardware sensor array, in the hardware scanning window array at different times; and

providing, via control logic coupled to the hardware sensor array, the hardware scanning window array, and the peripheral circuitry, signals to the peripheral circuitry to control the transfer of pixel values into the hardware scanning window array.

21. The method of claim 20, further comprising combining, in hardware, via configurable combining circuitry coupled to the plurality of pixels, multiple sensor readings from the plurality of pixels to generate the pixel values according to at least one combining pattern, wherein the configurable combining circuitry comprises:

a first plurality of controllable connections operable to connect adjacent pixels along the first dimension of the hardware sensor array; and

a second plurality of controllable connections operable to connect adjacent pixels along the second dimension of the hardware sensor array,

wherein connecting adjacent pixels generates averaged pixel values at each of the adjacent pixels being connected.

22. The method of claim 21, wherein the sensor reading is an analog sensor reading, wherein the pixel value is an analog pixel value, and wherein the configurable combining circuitry is configurable analog combining circuitry.

23. The method of claim 20, wherein the causing, via the peripheral circuitry for systematically transferring pixel values, the different windows of the pixel values to be stored in the hardware scanning window array comprises storing selected rows of pixel values from the hardware sensor array in a line buffer comprising a plurality of storage elements arranged along at least a first dimension and a second dimension of the line buffer, the first dimension of the line buffer being smaller than the first dimension of the hardware sensor array, the second dimension of the line buffer being equal to the second dimension of the hardware sensor array,

wherein the line buffer is capable of storing selected rows of pixel values from the hardware sensor array.

24. The method of claim 23, further comprising storing, via the control logic, a next row of pixel values from the hardware sensor array into the line buffer, by replacing an oldest row of pixel values in the line buffer, thereby introducing a discontinuity along a row direction in an image represented by the pixel values stored in the line buffer.

25. The method of claim 24, further comprising switching row ordering, via a multiplexer circuit positioned between the line buffer and the hardware scanning window array, while transferring pixel values from the line buffer to the hardware scanning window array, thereby removing the discontinuity along a row direction in an image represented by the pixel values stored in the line buffer.

26. The method of claim 24, further comprising switching column ordering, via a multiplexer circuit positioned between the hardware scanning window array and a computer vision feature computation block, while transferring pixel values from the hardware scanning window array to the computer vision feature computation block, thereby removing the discontinuity along a column direction in an image represented by the pixel values stored in the hardware scanning window array.

27. The method of claim 23, further comprising storing, via the control logic, a next column of pixel values from the line buffer into the hardware scanning window array, by replacing an oldest column of pixel values in the hardware scanning window array, thereby introducing a discontinuity along a column direction in an image represented by the pixel values stored in the hardware scanning window array.

28. The method of claim 23, further comprising performing, via computer vision computation circuitry, corner detection based on pixel values stored in the hardware scanning window array.

29. The method of claim 28, wherein, based on the detection of a corner by the hardware computer vision computation circuitry in an image represented by pixel values stored in the hardware scanning window array, causing, via the peripheral circuitry for systematically transferring pixel values, pixel values to be transferred from the hardware scanning window array to a second hardware scanning window array, the peripheral circuitry further comprising a multiplexer circuit positioned between the hardware scanning window array and the second hardware scanning window array, the multiplexer circuit capable of switching column ordering while transferring the pixel values from the hardware scanning window array to the second hardware scanning window array, thereby removing a discontinuity along a column direction in the image.

30. The method of claim 29, further comprising performing, via second hardware computer vision computation circuitry, computer vision computations based on values stored in the second hardware scanning window array.

31. The method of claim 20, further comprising performing combining, via configurable combining circuitry coupled to the plurality of storage elements, of some multiple pixel values to generate a first set of combined pixel values according to a first combining pattern, while maintaining the multiple pixel values for subsequent combining of some other multiple pixel values to generate a second set of combined pixel values according to a second combining pattern.

32. The method of claim 31, further comprising performing, via computer vision computation circuitry coupled to the configurable combining circuitry, computer vision computations.

33. The method of claim 32, wherein performing, via the computer vision computation circuitry coupled to the configurable combining circuitry, computer vision computations comprises performing a first computer vision computation based on the first set of combined pixel values and performing a second computer vision computation based on the second set of combined pixel values.

34. The method of claim 33, wherein the computer vision computation circuitry is capable of performing the first and second computer vision computations as part of a multi-block local binary (LBP) operation.

35. The method of claim 31, wherein the pixel values stored in the hardware scanning window array represent an image having a high resolution and the first combining pattern represents an averaging of neighboring pixel values to generate an image having a first lower resolution lower than the high resolution.

36. The method of claim 35, wherein the second combining pattern represents an averaging of neighboring pixel values to generate an image having a second lower resolution lower than the first lower resolution.

37. The method of claim 20, wherein at least one pixel of the plurality of pixels comprises a sensor element and in-pixel circuitry.

38. The method of claim 20, wherein the pixel values each based on the one or more sensor readings from the hardware sensor array comprise raw pixel values each based on one or more raw sensor readings.

39. The method of claim 20, wherein no image signal processing circuitry is performed between generating the sensor reading within the hardware sensor array and storing the pixel value within the hardware scanning window array.

40. An apparatus for isolating pixel values in hardware, comprising:

hardware means for generating a plurality of sensor readings based on environmental conditions;

hardware means for storing, within a plurality of storage elements within the hardware means for storing, pixel values each based on one or more sensor readings from the hardware means for generating the sensor reading;

hardware means for causing different windows of pixel values to be stored in the hardware means for storing pixel values at different times; and

hardware means for providing control signals to the hardware means for causing different windows of pixel values to be stored to control the transfer of pixel values into the hardware means for storing pixel values.

41. The apparatus of claim 40, wherein the hardware means for generating a plurality of sensor readings further comprises hardware means for combining multiple sensor readings from the plurality of pixels to generate the pixel values.

42. The apparatus of claim 41, wherein the hardware means for combining multiple sensor readings comprises configurable combining circuitry coupled to the plurality of pixels, the configurable comprising circuitry comprising:

hardware means for connecting adjacent pixels along the first dimension of the hardware means for generating sensor readings; and

hardware means for connecting adjacent pixels along the second dimension of the hardware means for generating sensor readings,

wherein connecting adjacent pixels generates averaged pixel values at each of the adjacent pixels being connected.

43. The apparatus of claim 46, wherein the sensor reading is an analog sensor reading, wherein the pixel value is an analog pixel value, and wherein the configurable combining circuitry is configurable analog combining circuitry.

44. The apparatus of claim 40, further comprising:

hardware means for buffering comprising a plurality of storage elements arranged along at least a first dimension and a second dimension of the hardware means for buffering, the first dimension of the hardware means for buffering being smaller than the first dimension of the hardware means for generating sensor readings, the second dimension of the hardware means for buffering being equal to the second dimension of the hardware means for generating sensor readings,

wherein the hardware means for buffering is capable of storing selected rows of pixel values from the hardware means for generating sensor readings.

45. The apparatus of claim 44, wherein the hardware means for providing control signals provides signals for storing a next row of pixel values from the hardware means for generating a plurality of sensor readings into the hardware means for buffering, by replacing an oldest row of pixel values in the hardware means for buffering thereby introducing a discontinuity along a row direction in an image represented by the pixel values stored in the hardware means for buffering.

46. The apparatus of claim 45, wherein the hardware means for causing different windows of pixel values to be stored comprises hardware means for switching row ordering while transferring pixel values from the hardware means for buffering to the hardware hardware means for storing pixel values, thereby removing the discontinuity along a row direction in an image represented by the pixel values stored in the hardware means for buffering.

47. The apparatus of claim 45, wherein the hardware means for causing different windows of pixel values to be stored comprises hardware means for switching column ordering while transferring pixel values from the hardware means for storing pixel values to hardware means for performing computer vision computations, thereby removing the discontinuity along a column direction in an image represented by the pixel values stored in the hardware means for storing pixel values.

48. The apparatus of claim 44, further comprising hardware means for storing a next column of pixel values from the hardware means for buffering into the hardware means for storing pixel values, by replacing an oldest column of pixel values in the hardware means for storing pixel values, thereby introducing a discontinuity along a column direction in an image represented by the pixel values stored in the hardware means for storing pixel values.

49. The apparatus of claim 44, further comprising hardware means for performing corner detection.

50. The apparatus of claim 49, wherein a second hardware means for storing pixel values is coupled to the hardware means for providing control signals, the hardware means for providing control signals to control the transfer of pixel values from the hardware means for storing pixel values to the second hardware means for storing pixel values based on detection of a corner by the hardware means for performing corner detection in an image represented by the pixel values stored in the hardware means for storing pixel values, the hardware means for causing different windows of pixel values to be stored further comprising hardware means for switching column ordering while transferring pixel values from the hardware means for storing pixel values to the second hardware means for storing pixel values positioned between the hardware means for storing pixel values and the second hardware means for storing pixel values, thereby removing a discontinuity along a column direction in the image.

52. The apparatus of claim 40, further comprising hardware means for combining some multiple pixel values to generate a first set of combined pixel values according to a first combining pattern, while maintaining the multiple pixel values for subsequent combining of some other multiple pixel values to generate a second set of combined pixel values according to a second combining pattern.

53. The apparatus of claim 52, further comprising hardware means for performing computer vision computations.

54. The apparatus of claim 53, wherein the computer vision computations comprise a first computer vision computation based on the first set of combined pixel values and a second computer vision computation based on the second set of combined pixel values.

55. The apparatus of claim 54, wherein the hardware means for performing computer vision computations comprises hardware means for performing the first and second computer vision computations as part of a multi-block local binary (LBP) operation.

56. The apparatus of claim 52, wherein the different ones of the first and second combining patterns allow for selection of the plurality of storage elements at different locations and dimensions within the hardware means for storing pixel values.

57. The apparatus of claim 52, wherein the pixel values stored in the hardware means for storing pixel values represent an image having a high resolution and the first combining pattern represents an averaging of neighboring pixel values to generate an image having a first lower resolution lower than the high resolution.

58. The apparatus of claim 57, wherein the second combining pattern represents an averaging of neighboring pixel values to generate an image having a second lower resolution lower than the first lower resolution.

59. The apparatus of claim 40, wherein at least one pixel of the plurality of pixels comprises a sensor element and in-pixel circuitry.

60. The apparatus of claim 40, wherein the pixel values each based on the one or more sensor readings from the hardware means for generating sensor readings comprise raw pixel values each based on one or more raw sensor readings.

61. The apparatus of claim 40, wherein no means for image signal processing is disposed between the hardware means for generating sensor readings and the hardware means for storing pixel values.

62. One or more non-transitory computer-readable media storing computer-executable instructions for isolating pixel values in hardware that, when executed, cause one or more computing devices to:

cause, via peripheral circuitry for systematically transferring pixel values, different windows of pixel values to be stored, based on sensor readings from a hardware sensor array, in a hardware scanning window array at different times; and

provide, via control logic coupled to the hardware sensor array, the hardware scanning window array, and the peripheral circuitry, signals to the peripheral circuitry to control the transfer of pixel values into the hardware scanning window array.

51. The apparatus of claim 69, further comprising hardware means for performing computer vision computations based on values stored in the second hardware means for storing pixel values.

63. The non-transitory computer-readable media of claim 62, wherein the instructions, when executed, further cause the one or more computing devices to provide, via the control logic, signals to configurable combining circuitry for combining multiple sensor readings from the hardware sensor array to generate the pixel values according to at least one combining pattern, wherein the configurable combining circuitry comprises:

a first plurality of controllable connections operable to connect adjacent pixels along a first dimension of the hardware sensor array; and

a second plurality of controllable connections operable to connect adjacent pixels along a second dimension of the hardware sensor array,

wherein connecting adjacent pixels generates averaged pixel values at each of the adjacent pixels being connected.

64. The non-transitory computer-readable media of claim 63, wherein the pixel values are an analog pixel values, and wherein the configurable combining circuitry is configurable analog combining circuitry.

65. The non-transitory computer-readable media of claim 62, wherein the causing, via the peripheral circuitry for systematically transferring pixel values, the different windows of the pixel values to be stored in the hardware scanning window array comprises storing selected rows of pixel values from the hardware sensor array in a line buffer comprising a plurality of storage elements arranged along at least a first dimension and a second dimension of the line buffer, the first dimension of the line buffer being smaller than the first dimension of the hardware sensor array, the second dimension of the line buffer being equal to the second dimension of the hardware sensor array,

wherein the line buffer is capable of storing selected rows of pixel values from the hardware sensor array.

66. The non-transitory computer-readable media of claim 65, wherein a hardware computer vision feature computation block coupled to the control logic comprises circuitry operable to perform corner detection.

67. The non-transitory computer-readable media of claim 66, wherein the instructions, when executed, further cause the one or more computing devices to provide, via control logic coupled to a second hardware scanning window array, control signals to control the transfer of pixel values from the hardware scanning window array to the second hardware scanning window array based on detection of a corner by the hardware computer vision feature computation block in an image represented by the pixel values stored in the hardware scanning window array, the peripheral circuitry further comprising a multiplexer circuit positioned between the hardware scanning window array and the second hardware scanning window array, the multiplexer circuit capable of switching column ordering while transferring the pixel values from the hardware scanning window array to the second hardware scanning window array, thereby removing a discontinuity along a column direction in the image.

68. The non-transitory computer-readable media of claim 67, wherein the instructions, when executed, further cause the one or more computing devices to perform, in hardware, via a second hardware computer vision feature computation block, computer vision computations based on values stored in the second hardware scanning window array.

69. The non-transitory computer-readable media of claim 65, wherein the instructions, when executed, further cause the one or more computing devices to store, via the control logic, a next row of pixel values from the hardware sensor array into the line buffer, by replacing an oldest row of pixel values in the line buffer, thereby introducing a discontinuity along a row direction in an image represented by the pixel values stored in the line buffer.

70. The non-transitory computer-readable media of claim 69, wherein the instructions, when executed, further cause the one or more computing devices to switch, via a multiplexer circuit positioned between the line buffer and the hardware scanning window array, row ordering while transferring pixel values from the line buffer to the hardware scanning window array, thereby removing the discontinuity along a row direction in an image represented by the pixel values stored in the line buffer.

71. The non-transitory computer-readable media of claim 69, wherein the instructions, when executed, further cause the one or more computing devices to switch, via a multiplexer circuit positioned between the hardware scanning window array and a hardware computer vision feature computation block, column ordering while transferring pixel values from the hardware scanning window array to the hardware computer vision feature computation block, thereby removing the discontinuity along a column direction in an image represented by the pixel values stored in the hardware scanning window array.

72. The non-transitory computer-readable media of claim 65, wherein the instructions, when executed, further cause the one or more computing devices to store, via the control logic, a next column of pixel values from the line buffer into the hardware scanning window array, by replacing an oldest column of pixel values in the hardware scanning window array, thereby introducing a discontinuity along a column direction in an image represented by the pixel values stored in the hardware scanning window array.

73. The non-transitory computer-readable media of claim 62, wherein the instructions, when executed, further cause the one or more computing devices to combine, via configurable combining circuitry, some multiple pixel values to generate a first set of combined pixel values according to a first combining pattern, while maintaining the multiple pixel values for subsequent combining of some other multiple pixel values to generate a second set of combined pixel values according to a second combining pattern.

74. The non-transitory computer-readable media of claim 73, wherein the instructions, when executed, further cause the one or more computing devices to perform, via computer vision computation circuitry coupled to the configurable combining circuitry, computer vision computations.

75. The non-transitory computer-readable media of claim 74, wherein the computer vision computations comprise a first computer vision computation based on the first set of combined pixel values and a second computer vision computation based on the second set of combined pixel values.

76. The non-transitory computer-readable media of claim 75, wherein the computer vision computation circuitry is capable of performing the first and second computer vision computations as part of a multi-block local binary (LBP) operation.

77. The non-transitory computer-readable media of claim 73, wherein the different ones of the first and second combining patterns allow for selection of the plurality of storage elements at different locations and dimensions within the hardware scanning window array.

78. The non-transitory computer-readable media of claim 73, wherein the pixel values stored in the hardware scanning window array represent an image having a high resolution and the first combining pattern represents an averaging of neighboring pixel values to generate an image having a first lower resolution lower than the high resolution.

79. The non-transitory computer-readable media of claim 78, wherein the second combining pattern represents an averaging of neighboring pixel values to generate an image having a second lower resolution lower than the first lower resolution.

80. The non-transitory computer-readable media of claim 62, wherein the pixel values each based on the sensor readings from the hardware sensor array comprise raw pixel values each based on raw sensor readings.

81. The non-transitory computer-readable media of claim 62, wherein no image signal processing circuitry is disposed between the hardware sensor array and the hardware scanning window array.