SYSTEMS AND METHODS FOR PROCESSING IMAGES

• US 20140267652A1
• Filed: 12/20/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: OrCam Technologies Ltd
• Original Assignee: OrCam Technologies Ltd
• Inventors: Wexler, Yonatan, Shashua, Amnon
• Granted Patent: US 9,542,613 B2
• Time in Patent Office: Days
• US Class Current: 348/63
• CPC Class Codes: A61F 9/08 Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/144 using a slot moved over the image; using discrete sensing elements at predetermined points; using automatic curve following means
• CPC Class Codes: G06V 30/1478 of characters or characters lines
• CPC Class Codes: G06V 30/226 of cursive writing
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: G06V 30/418 : Document matching, e.g. of document images
• CPC Class Codes: G06V 30/43 : Editing text-bitmaps, e.g. alignment, spacing; Semantic analysis of bitmaps of text without OCR
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G08B 3/10 : using electric transmission; using electromagnetic transmission
• CPC Class Codes: G08B 6/00 : Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G09B 21/00 : Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 21/001 : Teaching or communicating with blind persons G09B21/02 - G09B21/06 take precedence
• CPC Class Codes: G09B 21/003 : using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 : using audible presentation of the information
• CPC Class Codes: G10L 13/00 : Speech synthesis; Text to speech systems
• CPC Class Codes: H04N 23/50 : Constructional details
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/61 : based on recognised objects
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 : by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A apparatus and method are provided for processing images. In one embodiment, the apparatus includes an image sensor configured to capture real time images from an environment of a user. The apparatus also includes a mobile power source, and at least one processor device configured to process, at an initial resolution, images to determine existence of a trigger, and access rules associating image context with image capture resolution to enable images of a first context to be processed at a lower capture resolution than images of a second context. The processor device analyzes at least one first image, selects a first image capture resolution rule, and applies the first image capture resolution rule to a subsequent captured image. The processor device analyzes at least one second image, selects a second image capture resolution rule, and applies the second image capture resolution rule to a second subsequent captured image.

Claims

1. An apparatus for processing images, the apparatus comprising:

an image sensor configured to capture real time images at a plurality of resolutions from an environment of a user;

a mobile power source;

at least one processor device configured to;

process, at an initial resolution, a plurality of images from the image sensor to determine existence of a trigger;

access rules associating image context with image capture resolution to enable images of a first context to be processed at a lower capture resolution than images of a second context;

analyze at least one first image to determine a first image context;

select a first image capture resolution rule based on the determined first image context, wherein the first rule defines a first resolution;

apply the first image capture resolution rule to at least one subsequent captured image;

analyze at least one second image to determine a second image context;

select a second image capture resolution rule based on the determined second image context, wherein the second rule defining a second resolution; and

apply the second image capture resolution rule to at least one second subsequent captured image, wherein the second resolution is greater than the first resolution.

2. The apparatus of claim 1, wherein the image sensor is further configured to be connected to glasses worn by the user, and is thereby configured to be movable with a head of the user.

3. The apparatus of claim 1, wherein the mobile power source is rechargeable and contained within a housing that holds the at least one processor device.

4. The apparatus of claim 1, wherein the trigger is associated with an object in the environment of the user.

5. The apparatus of claim 4, wherein the at least one processor device is further configured to determine the context using at least one of:

an identity of the object, a background of the object, a location of the trigger in the images, and information indicative of a location of the user.

6. The apparatus of claim 1, wherein the first image capture resolution is greater than the initial resolution.

7. The apparatus of claim 1, wherein the first image capture resolution is substantially equal to the initial resolution.

8. The apparatus of claim 1, wherein the initial resolution has resolution lower than 5 megapixels.

9. The apparatus of claim 1, wherein the initial resolution has resolution lower than 80,000 pixels.

10. The apparatus of claim 1, wherein the first image capture resolution is higher than 300,000 pixels.

11. The apparatus of claim 1, wherein the first image capture resolution is higher than 10 megapixels.

12. The apparatus of claim 1, wherein the first image capture resolution rule further defines a plurality of image portions, each image portion being associated with a differing resolution to be captured in the at least one subsequent captured image.

13. The apparatus of claim 1, wherein the first image capture resolution rule defines a plurality of image portions, and at least one image portion is not to be captured in the at least one subsequent captured image.

14. The apparatus of claim 1, wherein the first image capture resolution rule includes zoom information.

15. The apparatus of claim 1, wherein the at least one processor device is further configured to execute an action from multiple context-based actions, wherein the multiple context-based actions include at least one action associated with the first image capture resolution and at least one additional action associated with the second image capture resolution.

16. The apparatus of claim 1, wherein the at least one processor device is further configured to identify objects in images captured in the second image capture resolution.

17. The apparatus of claim 1, wherein the at least one processor device is further configured to execute optical character recognition on images captured in the second image capture resolution.

18. An apparatus for processing images, the apparatus comprising:

an image sensor configured to capture real time images from an environment of a user;

a mobile power source;

at least one processor device configured to;

process, at an initial frame rate, a plurality of images from the image sensor to determine existence of a trigger;

access rules associating image context with image capture frame rare to enable images of a first context to be processed at a lower frame rate than images of a second context;

analyze at least one first image to determine a first image context;

select a first image capture frame rate rule based on the determined image context, the first rule defining a first frame rate;

apply the first frame rare to at least one subsequent captured image;

analyze at least one second image to determine a second image context;

select a second image capture frame rare rule based on the determined second image context, the second rule defining a second frame rate; and

apply the second frame rate to at least one second subsequent captured image, wherein the second frame rate is higher than the first frame rate.

19. The apparatus of claim 18, wherein the image sensor is further configured to be connected to glasses worn by the user, and is thereby configured to be movable with a head of the user.

20. The apparatus of claim 18, wherein the initial frame rate is associated with an image resolution, and the second frame rate is associated with a differing image resolution.

21. The apparatus of claim 18, wherein the first frame rate is higher than the initial frame rate.

22. The apparatus of claim 18, wherein the first frame rate is substantially equal to the initial frame rate.

23. The apparatus of claim 18, wherein the initial frame rate is less than 10 frames-per-second.

24. The apparatus of claim 18, wherein the initial frame rate is less than 1 frame-per-second.

25. The apparatus of claim 18, wherein the first frame rate is higher than 9 frames-per-second.

26. The apparatus of claim 18, wherein the first frame rate is higher than 29 frames-per-second.

27. A method for processing images, the method comprising:

capturing a plurality of images at an initial resolution from an environment of a user;

identifying existence of a trigger in at least one image;

accessing rules associating image context with image capture resolution to enable images of a first context to be processed at a lower capture resolution than images of a second context;

analyzing at least one first image to determine a first image context;

selecting a first image capture resolution rule based on the determined first image context, wherein the first rule defining a first resolution;

applying the first image capture resolution rule to at least one subsequent captured image;

analyzing at least one second image to determine a second image context;

selecting a second image capture resolution rule based on the determined second image context, wherein the second rule defining a second resolution; and

applying the second image capture resolution rule to at least one second subsequent captured image, wherein the second resolution is greater than the first resolution.

28. The method of claim 27, wherein the first resolution is substantially equal to the initial resolution.

29. The method of claim 27, wherein the first resolution is greater than the initial resolution.

30. A software product stored on a non-transitory computer readable medium and comprising data and computer implementable instructions for carrying out the method of claim 27.