Non-interference field-of-view support apparatus for a panoramic facial sensor

• US 9,344,612 B2
• Filed: 11/11/2011
• Issued: 05/17/2016
• Est. Priority Date: 02/15/2006
• Status: Active Grant

• Current Assignee: Virtual Video Reality By Ritchey Limited Liability Corporation
• Original Assignee: Kenneth Ira Ritchey, Kurtis John Ritchey
• Inventors: Ritchey, Kenneth Ira, Ritchey, Kurtis John
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): PONTIUS, JAMES M
• Application Number:
  US13/294,986
• Publication Number:
  US 20120113209A1
• Time in Patent Office: 1,649 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0156 with optionally usable elements
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/142 : Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 : Conference systems

Abstract

A support apparatus with adjustable mechanisms hold a spherical field-of-regard image and audio sensor assembly located at its distal end in front of a users face such that the user'"'"'s fine focus field-of-view of the foreground is not blocked by the mast or the sensor assembly. Automated mechanisms rotate and extend the armature and sensor assembly into position for face-to-face panoramic hands-free video teleconferencing, gaming, or logging. A portable wired or wireless host electronic device with a personal assistant application and sensor correlation system interactively communicates with support apparatus circuitry and servos, imagery, audio, eye and ROI tracking, neural, and user input and feedback systems to orchestrate responses to a local or remote user. A sensor assembly includes a VLSIC multi-ROI processing system with integrated camera and display that hides the assembly and provides information to a user or onlooker.

Claims

1. A head-mounted non-interference field-of-view apparatus comprising:

a support housing for supporting the apparatus on a head of a user;

an eye tracking system configured to determine a field of view of the user;

a field-of-view sensor assembly including;

a sensor housing;

a camera system mounted within the sensor housing, the camera system having a plurality of objective lenses facing outward and configured to provide continuous field-of-view coverage of a foreground scene, the camera system being in visual line of sight to at least one eye of the user; and

a display device facing towards the user;

a support armature integrated with the sensor assembly and connected to the sensor assembly at a distal end of the support armature and the support housing at a proximal end of the support armature;

circuitry configured to communicatively connect the camera system to a portable digital device;

at least one actuator attached to the support armature, the at least one actuator being configured to shift the sensor assembly between a deployed position forward of the user'"'"'s face and an alternate position, the at least one actuator being responsive to command and control signals from the portable digital device and the sensor assembly such that the camera system is moved forward of the user'"'"'s face when a hands-free panoramic video telephone conversation is initiated; and

an image processing system for operating on field-of-view imagery provided by the camera system for determining a location of the sensor assembly and an offset between the user and the sensor assembly, the image processing system being configured to modify the field-of-view imagery according to a shape of the field-of-view sensor assembly, the display device being configured to display the modified imagery so as to mask the support armature and field-of-view sensor assembly between the user'"'"'s at least one eye and the foreground scene such that the support armature and field-of-view sensor assembly are hidden and do not block the user'"'"'s fine focus field-of-view.

2. The apparatus according to claim 1, wherein the portable digital device includes a personal digital assistant being configured to:

learn preferences of the user over time and personalize results according to the preferences;

use natural language processing to answer questions, make recommendations, and perform actions by delegating requests to web services;

provide panoramic video teleconferencing capabilities; and

provide a touch screen graphic user interface to facilitate defining system parameters for manual and automated command and control of the support housing, support armature, sensor assembly, and related peripheral devices that assist in commanding and controlling the apparatus.

3. The apparatus according to claim 1, wherein the sensor assembly and portable digital device are simultaneously responsive to read out imagery for transmission and display of a region of interest of the user derived by the eye-tracking system and sensor assembly as the eye-tracking system tracks the at least one eye of the user.

4. The apparatus according to claim 1, wherein the further comprises processing means for panning and zooming around the foreground scene, displaying a graphical user interface window via the display device, or overlaying secondary imagery over the modified imagery via the display device.

5. The apparatus according to claim 1, wherein the sensor assembly further comprises at least one microphone mounted within the sensor housing and facing outward for recording audio signatures.

6. The apparatus according to claim 1, wherein the image processing system is integrated into an electronic device of at least one of a tablet, laptop, PC, head supported structure, head mounted device, eye mounted device, panoramic theater, or telecommunications system and network.

7. The apparatus according to claim 1, wherein the camera system is stereoscopic.

8. The apparatus according to claim 1, wherein the camera system comprises a spherically shaped very large scale integrated circuit (VLSIC).

9. The apparatus according to claim 1, wherein the camera system includes multiple region-of-interest (ROI) processing, tracking, and image output capabilities.

10. The apparatus according to claim 1, wherein at least one of the support armature and display includes an integrated image capture and display system.

11. The apparatus according to claim 1, further comprising a see-through electronic paper display including near-field-of-view optics, the electronic paper display being mounted below the sensor assembly and being configured to present images derived from the sensor assembly and the personal digital assistant to the user, the at least one actuator being configured to move the sensor assembly and support armature outside of a mid and distant range fine focus field of view of the user such that the display is positioned just below the fine focus field of view and within a reading distance of the user without blocking the fine focus field of view for allowing the user to look over the armature and sensor assembly to view the foreground scene, glance down to read the electronic paper display, and selectively position the support armature for looking through the electronic paper display for augmented reality applications.

12. A head-mounted non-interference field-of-view apparatus comprising:

a support housing including at least one fastener for securing the support housing to a head of a user;

an eye tracking system configured to determine the user'"'"'s field of view;

a spherical field-of-view sensor assembly including;

a sensor housing;

a camera system mounted within the sensor housing, the camera system having a plurality of objective lenses facing outward and configured to provide continuous spherical field-of-view coverage of a foreground scene, the camera system being in visual line of sight to at least one eye of the user; and

a display device facing towards the user;

a support armature integrated with the sensor assembly and connected to the sensor assembly at a distal end of the support armature and the support housing at a proximal end of the support armature;

a flexible cable and connector extending through the support armature;

circuitry configured to communicatively connect the camera system to a portable digital device via the flexible cable and connector;

at least one actuator attached to the support armature, the at least one actuator being configured to shift the sensor assembly between a deployed position forward of the user'"'"'s face and an alternate position, the at least one actuator being responsive to command and control signals from the portable digital device and the sensor assembly such that the camera system is moved forward of the user'"'"'s face when a hands-free panoramic video telephone conversation is initiated; and

an image processing system for operating on field-of-view imagery provided by the camera system for determining a location of the sensor assembly and an offset between the user and the sensor assembly, the image processing system being configured to modify the field-of-view imagery according to a shape of the field-of-view sensor assembly, the display device being configured to display the modified imagery so as to mask the support armature and field-of-view sensor assembly between the user'"'"'s at least one eye and the foreground scene such that the support armature and field-of-view sensor assembly are hidden and do not block the user'"'"'s fine focus field of view.