Portable handheld device with multi-core microcoded image processor

• US 9,432,529 B2
• Filed: 09/15/2012
• Issued: 08/30/2016
• Est. Priority Date: 07/15/1997
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Silverbrook, Kia
• Primary Examiner(s): Menberu, Beniyam
• Application Number:
  US13/620,861
• Publication Number:
  US 20130013893A1
• Time in Patent Office: 1,445 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B41J 15/044 Cassettes or cartridges containing continuous copy material, tape, for setting into printing devices
• CPC Class Codes: B41J 2/14 Structure thereof only for on-demand ink jet heads
• CPC Class Codes: B41J 2/14427 Structure of ink jet print heads with thermal bend detached actuators
• CPC Class Codes: B41J 2/16 Production of nozzles
• CPC Class Codes: B41J 2/1601 Production of bubble jet print heads B41J2/1606, B41J2/162 take precedence
• CPC Class Codes: B41J 2/1623 bonding and adhesion
• CPC Class Codes: B41J 2/1626 etching
• CPC Class Codes: B41J 2/1628 dry etching
• CPC Class Codes: B41J 2/1629 wet etching
• CPC Class Codes: B41J 2/1631 photolithography
• CPC Class Codes: B41J 2/1632 machining
• CPC Class Codes: B41J 2/1635 dividing the wafer into individual chips
• CPC Class Codes: B41J 2/1637 molding
• CPC Class Codes: B41J 2/1639 sacrificial molding
• CPC Class Codes: B41J 2/1642 thin film formation by CVD [chemical vapor deposition]
• CPC Class Codes: B41J 2/1643 thin film formation by plating
• CPC Class Codes: B41J 2/1645 thin film formation by spincoating
• CPC Class Codes: B41J 2/1646 thin film formation by sputtering
• CPC Class Codes: B41J 2/1648 Production of print heads with thermal bend detached actuators
• CPC Class Codes: B41J 2/16585 for paper-width or non-reciprocating print heads
• CPC Class Codes: B41J 2/175 : Ink supply systems ; Circuit parts therefor
• CPC Class Codes: B41J 2/17503 : Ink cartridges
• CPC Class Codes: B41J 2/17513 : Inner structure
• CPC Class Codes: B41J 2/17546 : electronically
• CPC Class Codes: B41J 2/17596 : Ink pumps, ink valves
• CPC Class Codes: B41J 2002/041 : Electromagnetic transducer
• CPC Class Codes: B41J 2002/14491 : Electrical connection
• CPC Class Codes: B41J 2202/21 : Line printing
• CPC Class Codes: B41J 29/02 : Framework
• CPC Class Codes: B42D 15/02 : Postcards; Greeting, menu, business or like cards; Letter cards or letter-sheets B42D25/00 takes precedence; envelopes B65D27/00
• CPC Class Codes: B42D 2035/08 : Alphanumerical data
• CPC Class Codes: B42D 2035/34 : Markings visible under particular conditions or containing coded information
• CPC Class Codes: B42D 2035/50 : Corresponding markings appearing in different shapes or at different places
• CPC Class Codes: B42D 25/00 : Information-bearing cards or sheet-like structures characterised by identification or security features; Manufacture thereof printing processes to produce identification or security features B41M3/14
• CPC Class Codes: B42D 25/20 : characterised by a particular use or purpose
• CPC Class Codes: B42D 25/23 : Identity cards
• CPC Class Codes: B82Y 30/00 : Nanotechnology for materials or surface science, e.g. nanocomposites
• CPC Class Codes: G03B 17/02 : Bodies
• CPC Class Codes: G03B 27/00 : Photographic printing apparatus
• CPC Class Codes: G03B 27/02 : Exposure apparatus for contact printing
• CPC Class Codes: G06F 1/1626 : with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 12/0866 : for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 21/79 : in semiconductor storage media, e.g. directly-addressable memories
• CPC Class Codes: G06F 21/86 : Secure or tamper-resistant housings
• CPC Class Codes: G06F 2212/2022 : Flash memory
• CPC Class Codes: G06F 2221/2129 : Authenticate client device independently of the user
• CPC Class Codes: G06F 3/12 : Digital output to print unit , e.g. line printer, chain printer
• CPC Class Codes: G06K 1/121 : by printing code marks applying code marks to labels B65C9/46; marking or coding completed packages B65B61/26
• CPC Class Codes: G06K 19/06037 : multi-dimensional coding
• CPC Class Codes: G06K 19/073 : Special arrangements for circuits, e.g. for protecting identification code in memory protection against unauthorised use of computer memory G06F12/14
• CPC Class Codes: G06K 7/10722 : Photodetector array or CCD scanning
• CPC Class Codes: G06K 7/14 : using light without selection of wavelength, e.g. sensing reflected white light G06K7/10831 - G06K7/1097 take precedence
• CPC Class Codes: G06K 7/1417 : 2D bar codes
• CPC Class Codes: G06T 1/20 : Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 3/02 : Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G07F 7/08 : by coded identity card or credit card or other personal identification means without personal verification means G07F7/02
• CPC Class Codes: G07F 7/086 : by passive credit-cards adapted therefor, e.g. constructive particularities to avoid counterfeiting, e.g. by inclusion of a physical or chemical security-layer for security documents G07D7/00; for the reading of record-carriers in general G06K7/00; for the design of coded credit-cards G06K19/10
• CPC Class Codes: G07F 7/12 : Card verification
• CPC Class Codes: G09G 2300/026 : Video wall, i.e. juxtaposition of a plurality of screens to create a display screen of bigger dimensions
• CPC Class Codes: G09G 2310/0281 : Arrangement of scan or data electrode driver circuits at the periphery of a panel not inherent to a split matrix structure
• CPC Class Codes: G11C 11/56 : using storage elements with more than two stable states represented by steps, e.g. of voltage, current, phase, frequency
• CPC Class Codes: G11C 16/22 : Safety or protection circuits preventing unauthorised or accidental access to memory cells
• CPC Class Codes: H01L 23/576 : using active circuits
• CPC Class Codes: H01L 2924/00 : Indexing scheme for arrangements or methods for connecting or disconnecting semiconductor or solid-state bodies as covered by H01L24/00
• CPC Class Codes: H01L 2924/0002 : Not covered by any one of groups H01L24/00, H01L24/00 and H01L2224/00
• CPC Class Codes: H04N 1/00127 : Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00278 : with a printing apparatus, e.g. a laser beam printer
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/00355 : Mark-sheet input sensing record carriers G06K7/00; character recognition G06V30/10; document-oriented image-based pattern recognition G06V30/40
• CPC Class Codes: H04N 1/0044 : for image preview or review, e.g. to help the user position a sheet
• CPC Class Codes: H04N 1/00965 : using a plug-in memory module, e.g. memory card, memory stick
• CPC Class Codes: H04N 1/00968 : by scanning marks on a sheet mark-sheet input means H04N1/00355; sensing record carriers G06K7/00
• CPC Class Codes: H04N 1/04 : Scanning arrangements , i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versaH04N1/387 takes precedence
• CPC Class Codes: H04N 1/2112 : using still video cameras
• CPC Class Codes: H04N 1/2154 : the still video camera incorporating a hardcopy reproducing device, e.g. a printer
• CPC Class Codes: H04N 1/2307 : Circuits or arrangements for the control thereof, e.g. using a programmed control device, according to a measured quantity H04N1/27, H04N1/29 take precedence
• CPC Class Codes: H04N 1/32122 : in a separate device, e.g. in a memory or on a display separate from image data
• CPC Class Codes: H04N 1/32133 : on the same paper sheet, e.g. a facsimile page header
• CPC Class Codes: H04N 2101/00 : Still video cameras
• CPC Class Codes: H04N 2201/0008 : Connection or combination of a still picture apparatus with another apparatus
• CPC Class Codes: H04N 2201/0084 : Digital still camera
• CPC Class Codes: H04N 2201/3222 : of processing required or performed, e.g. forwarding, urgent or confidential handling
• CPC Class Codes: H04N 2201/3242 : of processing required or performed, e.g. for reproduction or before recording H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3261 : of multimedia information, e.g. a sound signal
• CPC Class Codes: H04N 2201/3264 : of sound signals
• CPC Class Codes: H04N 2201/3269 : of machine readable codes or marks, e.g. bar codes or glyphs
• CPC Class Codes: H04N 2201/3276 : of a customised additional information profile, e.g. a profile specific to a user ID
• CPC Class Codes: H04N 2201/328 : Processing of the additional information
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/843 : Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H05K 1/14 : Structural association of two or more printed circuits providing electric connection to or between printed circuits H05K1/11, H01R12/00
• CPC Class Codes: H05K 1/189 : characterised by the use of a flexible or folded printed circuit H05K3/326 takes precedence
• CPC Class Codes: Y10S 358/9091 : Electronic still camera or scene reproducer

Abstract

A portable handheld device includes a CPU for processing a script; a multi-core processor for processing images; and a flash memory connected to the CPU, the flash memory storing therein a table of micro-codes. The multi-core processor includes a plurality of micro-coded processing units. The CPU is configured to read one or more micro-codes from the flash memory and load the one or more micro-codes into the processing unit to execute the script being processed thereby.

Claims

1. A portable device comprising:

an image sensor configured for sensing an image and generating image data;

a multi-core processor including a plurality of processing units, the plurality of processing units being configured to process data;

a flash memory storing therein a table of instructions;

an input buffer common to the plurality of processing units of the multi-core processor and configured to write data to the multi-core processor; and

a CPU communicatively coupled to the flash memory and the input buffer and configured to read one or more instructions from the flash memory;

wherein the input buffer is separate from the CPU,

wherein the CPU is configured to cause the one or more instructions to load into the plurality of processing units,

wherein the image sensor is configured for providing the image data to the input buffer for processing by the multi-core processor,

wherein the CPU and the multi-core processor are part of a single chip, wherein the single chip includes an authentication interface that includes a plurality of ports to connect to an on-device authentication chip and to another authentication chip used with another device.

2. The portable device according to claim 1, wherein the plurality of processing units are connected in parallel by a crossbar switch.

3. The portable device according to claim 1, wherein each processing unit of the plurality of processing units includes a RAM for storing therein the one or more instructions.

4. The portable device according to claim 1, wherein each processing unit of the plurality of processing units includes a local input buffer and a local output buffer, both the local input buffer and the local output buffer being connected to a data bus coupled to a data cache.

5. The portable device according to claim 1, wherein the CPU, the multi-core processor, and the flash memory are provided as part of a system-on-chip integrated device.

6. The portable device according to claim 5, further comprising

an image sensor interface provided on a chip of the system-on-chip integrated device for receiving the image data from the image sensor, wherein the image sensor interface communicates the image data to the input buffer, wherein the image sensor is directly connected to the image sensor interface and the image sensor interface is directly connected to the input buffer.

7. The portable device according to claim 1, wherein the CPU processes a script, the script includes processor instructions that are executable by the multi-core processor.

8. The portable device according to claim 7, wherein the script includes processor instructions that, when executed by the plurality of processing units, cause the multi-core processor to perform at least one scene transformation on the data.

9. The portable device according to claim 1, wherein the one or more instructions are executed by the plurality of processing units and cause the plurality of processing units of the multi-core processor to perform scene transformations including one or more of the following:

image scaling, image warping, image convolving, image brushing, compositing, and image filtering.

10. The portable device according to claim 1, wherein the one or more instructions are executed by the plurality of processing units and provide the plurality of processing units with the functionality to perform the following:

histogram collecting, color transforming, and image conversion.

11. The portable device according to claim 1, wherein the plurality of processing units of the micro-core processor are synchronized and work in parallel, and wherein each processing unit of the micro-core processor includes a respective rewritable memory or a respective RAM.

12. The portable device according to claim 8, wherein the CPU and the multi-core processor are part of a single chip, wherein the multi-core processor processes the data according to the at least one scene transformation specified in the script.

13. The portable device according to claim 1, wherein the CPU and the multi-core processor are part of a single chip, wherein the single chip includes a sensor interface, wherein the sensor interface is operatively coupled to the image sensor, wherein the image sensor includes one or more of the following:

a CMOS sensor, an active pixel sensor and a charge couple device.

14. The portable device according to claim 1, further comprising an output buffer communicatively coupled to the CPU, the output buffer interrupts the CPU when data is placed into the output buffer.

20. The portable device according to claim 5, further comprising a memory separate from the system-on-chip integrated device, wherein the multi-core processor reads the image data from the input buffer and writes the image data to the memory.

21. The portable device according to claim 20, wherein the system-on-chip integrated device further comprises a data cache, wherein the multi-core processor writes the image data to the data cache prior to the image data being written to the memory.

15. A portable handheld device comprising:

a CPU for processing a script;

a multi-core processor for processing images; and

a flash memory connected to the CPU, the flash memory storing therein a table of micro-codes,

wherein the multi-core processor includes a plurality of micro-coded processing units,

wherein the CPU is configured to read one or more micro-codes from the flash memory and load the one or more micro-codes into the micro-coded processing units to execute the script being processed thereby,

wherein the CPU and the multi-core processor are part of a single chip, wherein the single chip includes an authentication interface, and

wherein the authentication interface of the single chip includes a plurality of ports to connect to an on-device authentication chip and to another authentication chip used with another device.

16. The portable device according to claim 15, wherein the plurality of micro-coded processing units are connected in parallel by a crossbar switch.

17. The portable device according to claim 15, wherein each micro-coded processing unit includes a micro-code RAM for storing therein the micro-codes loaded by the CPU.

18. The portable device according to claim 15, wherein each micro-coded processing unit includes an input buffer and an output buffer, both the input buffer and the output buffer being connected to a data bus to effect connection with a data cache.

19. The portable device according to claim 15, wherein the CPU, the multi-core processor, and the flash memory are provided as part of a system-on-chip integrated device comprising:

an image sensor for sensing an image; and

an image sensor interface provided on a chip of the system-on-chip integrated device for receiving image data from the image sensor, wherein the image sensor interface communicates the image data to the multi-core processor for processing thereby in accordance with the script.