Method and apparatus for a compact and high resolution mind-view communicator

• US 10,064,552 B1
• Filed: 01/06/2017
• Issued: 09/04/2018
• Est. Priority Date: 06/04/2009
• Status: Active Grant

• Current Assignee: Optics Innovation LLC
• Original Assignee: IPAL Incorporated
• Inventors: Vaziri, Masoud
• Primary Examiner(s): HUANG, FRANK F
• Application Number:
  US15/400,399
• Time in Patent Office: 606 Days
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/1107 Measuring contraction of parts of the body, e.g. organ, muscle apparatus for measuring work or force in general G01L
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/168 Evaluating attention deficit, hyperactivity
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61N 1/3606 adapted for a particular treatment
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/775 : between a recording apparatus and a television receiver
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/22 : Adaptations for optical transmission
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal

Abstract

An embodiment of a mind-view communication apparatus includes a first portable unit and a second portable unit. The first portable unit includes an eyeglass frame, at least one first optical unit disposed on the eyeglass frame for capturing at least one scene image corresponding to a field of view of a user, and at least one second optical unit disposed on the eyeglass frame for capturing at least one eye image corresponding to at least a portion of at least one eye of the user. The second portable unit is in communication with the first portable unit and includes at least one processor configured for receiving the at least one scene image and the at least one eye image, determining a direction within the field of view to which the at least one eye is directed based upon the at least one eye image, and generating a subset of the at least one scene image based on the determined direction.

Claims

1. A method for a multi-camera device, comprising the steps of:

capturing a main scene image with a main camera, the main camera having a first field of view, and the main scene image having a first image resolution and a plurality of image pixels,

capturing an auxiliary scene image with an auxiliary camera, the auxiliary camera having a second field of view, and the auxiliary scene image having a second image resolution and a plurality of image pixels, wherein the second image resolution being higher than the first image resolution, and the auxiliary scene image and the main scene image having an overlap area, wherein the overlap area corresponding to a subset of the scene that is within the field of view of the main camera and the field of view of the auxiliary camera,

receiving the main scene image and the auxiliary scene image by at least one processor,

selecting, based on a predetermined criterion, at least one subset of the main scene image having no image pixel in the overlap area, wherein the selected at least one subset of the main scene image corresponding to a subset of the scene that is not within the field of view of the auxiliary camera,

utilizing the auxiliary scene image and the main scene image to increase the image resolution of the selected at least one subset of the main scene image having no image pixel in the overlap area, and

transmitting the increased resolution image of the at least one subset of the main scene image having no image pixel in the overlap area.

2. The method of claim 1 wherein the field of view of the auxiliary camera is a subset of the field of view of the main camera.

3. The method of claim 1 wherein the at least one processor further configured to extract the depth information of a subset of the scene based on the main scene image and the auxiliary scene image.

4. The method of claim 2 wherein the main and the auxiliary cameras are color cameras and generate color images.

5. The method of claim 1 further including the step of utilizing an image registration technique to relate a subset of the main scene image to a subset of the auxiliary scene image.

6. The method of claim 1 further including the step of utilizing a Fast Fourier Transform technique to increase the image resolution of the at least one subset of the main scene image having no image pixel in the overlap area.

7. An imaging apparatus comprising:

a main camera having a first field of view for capturing at least one main scene image, wherein the at least one main scene image having a first image resolution and a plurality of image pixels;

at least one auxiliary camera having a second field of view for capturing at least one auxiliary scene image, wherein the at least one auxiliary scene image having a second image resolution and a plurality of image pixels, wherein the second image resolution being higher than the first image resolution, and wherein the field of view of the at least one auxiliary camera and the field of view of the main camera at least partially overlapping and the at least one auxiliary scene image and the at least one main scene image having an overlap area, wherein the overlap area corresponding to a subset of the scene that is within the field of view of the main camera and the field of view of the at least one auxiliary camera;

at least one processor, in communication with the main camera and the at least one auxiliary camera, the at least one processor configured for;

receiving the at least one main scene image from the main camera and the at least one auxiliary scene image from the at least one auxiliary camera,

selecting, based on at least one predetermined criterion, at least one subset of the at least one main scene image having no image pixel in the overlap area, wherein the selected at least one subset of the least one main scene image corresponding to a subset of the scene that is not within the field of view of the at least one auxiliary camera,

utilizing the at least one auxiliary scene image and the at least one main scene image to increase the image resolution of the selected at least one subset of the at least one main scene image having no image pixel in the overlap area, and

transmitting the increased resolution image of the at least one subset of the at least one main scene image having no image pixel in the overlap area.

8. The imaging apparatus of claim 7 wherein the main camera is a color camera and includes a color filter array.

9. The imaging apparatus of claim 8 wherein the at least one auxiliary camera has no color filter array and generates gray scale images.

10. The imaging apparatus of claim 8 wherein the at least one auxiliary camera has a color filter array to generate color images.

11. The imaging apparatus of claim 10 wherein the at least one auxiliary camera and the main camera have two different types of color filter arrays.

12. The imaging apparatus of claim 7 wherein the field of view of the at least one auxiliary camera is a subset of the field of view of the main camera.

13. The imaging apparatus of claim 7 wherein the at least one processor further configured for utilizing an image registration technique to relate a subset of the at least one main scene image to a subset of the at least one auxiliary scene image.

14. The imaging apparatus of claim 7 wherein the at least one processor further configured for utilizing a Fast Fourier Transform technique to increase the image resolution of the at least one subset of the at least one main scene image having no image pixel in the overlap area.

15. The imaging apparatus of claim 7 wherein the apparatus is disposed on a pair of eyeglass frame to be worn by a user.

16. The imaging apparatus of claim 15, further including at least one eye tracking camera disposed on the eyeglass frame, directed towards one eye of the user, for capturing at least a plurality of eye images corresponding to at least a portion of one eye of the user and transmitting the plurality of eye images to the at least one processor.

17. The imaging apparatus of claim 16, wherein the at least one processor determining a gaze direction of the user based on at least one eye image captured by the at least one eye tracking camera and utilizing the determined gaze direction to select a subset of the at least one main scene image.

18. A method for a multi-camera device, comprising the steps of:

capturing a main scene image with a main camera, the main camera having a first field of view, and the main scene image having a first image resolution and a plurality of image pixels,

capturing an auxiliary scene image with an auxiliary camera, the auxiliary camera having a second field of view, and the auxiliary scene image having a second image resolution and a plurality of image pixels, wherein the second image resolution being higher than the first image resolution, and the auxiliary scene image and the main scene image having an overlap area, wherein the overlap area corresponding to a subset of the scene that is within the field of view of the main camera and the field of view of the auxiliary camera,

receiving the main scene image and the auxiliary scene image by at least one processor,

selecting, based on a predetermined criterion, at least one subset of the main scene image having a plurality of pixels inside and outside the overlap area,

utilizing the auxiliary scene image and the main scene image to increase the image resolution of the selected at least one subset of the main scene image, and

transmitting the increased resolution image of the at least one subset of the main scene image.

19. The method of claim 18 wherein the field of view of the auxiliary camera is a subset of the field of view of the main camera.

20. The method of claim 18 wherein the auxiliary camera has no color filter array and generates gray scale images.