Mobile user borne brain activity data and surrounding environment data correlation system

• US 9,101,279 B2
• Filed: 06/11/2012
• Issued: 08/11/2015
• Est. Priority Date: 02/15/2006
• Status: Active Grant

• Current Assignee: Virtual Video Reality By Ritchey Limited Liability Corporation
• Original Assignee: Virtual Video Reality By Ritchey LLC
• Inventors: Ritchey, Kenneth Ira, Ritchey, Kurtis John
• Primary Examiner(s): Harrell, Robert B
• Application Number:
  US13/507,190
• Publication Number:
  US 20130063550A1
• Time in Patent Office: 1,156 Days
• Field of Search: 348/36, 600/544
• US Class Current: 1/1
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0024 for multiple sensor units attached to the patient, e.g. using a body or personal area network
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/245 specially adapted for magnetoencephalographic [MEG] signals
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/37 Intracranial electroencephalography [IC-EEG], e.g. electrocorticography [ECoG]
• CPC Class Codes: A61B 5/372 Analysis of electroencephalograms
• CPC Class Codes: A61B 5/384 Recording apparatus or displays specially adapted therefor
• CPC Class Codes: A61B 5/4064 Evaluating the brain A61B5/031, A61B5/369, A61B5/14553 take precedence
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/686 Permanently implanted devices, e.g. pacemakers, other stimulators, biochips A61B5/6861 takes precedence
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/745 using a holographic display
• CPC Class Codes: A61B 6/501 for diagnosis of the head, e.g. neuroimaging or craniography
• CPC Class Codes: G01R 33/4806 Functional imaging of brain activation
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 15/10 : by adding a part, e.g. close-up attachment
• CPC Class Codes: G02B 2027/0138 : comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 : comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 : slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 : Head mounted
• CPC Class Codes: G03B 37/00 : Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• CPC Class Codes: G03H 1/2294 : Addressing the hologram to an active spatial light modulator
• CPC Class Codes: G05D 1/0016 : characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 : by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G06F 1/1626 : with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 : the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 16/90 : Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 16/90335 : Query processing
• CPC Class Codes: G06F 2203/0381 : Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 : Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 : Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/147 : using display panels
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2370/16 : Use of wireless transmission of display information
• CPC Class Codes: G09G 2380/08 : Biomedical applications
• CPC Class Codes: G09G 3/003 : to produce spatial visual effects
• CPC Class Codes: G09G 3/03 : specially adapted for displays having non-planar surfaces, e.g. curved displays
• CPC Class Codes: G09G 3/035 : for flexible display surfaces
• CPC Class Codes: G09G 5/026 : Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/14 : Display of multiple viewports
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: H04N 23/58 : Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control

Abstract

A mobile user borne brain activity data and surrounding environment data correlation system comprising a brain activity sensing subsystem, a recording subsystem, a measurement computer subsystem, a user sensing subsystem, a surrounding environment sensing subsystem, a correlation subsystem, a user portable electronic device, a non-transitory computer readable medium, and a computer processing device. The mobile user borne system collects and records brain activity data and surrounding environment data and statistically correlates and processes the data for communicating the data into a recipient biological, mechanical, or bio-mechanical system.

Claims

1. A mobile user borne system comprising:

a brain activity sensing subsystem configured to collect data corresponding to brain activity of a user;

a measurement computer subsystem configured to quantify perceptions of the user;

a user sensing subsystem configured to collect data corresponding to user events;

a surrounding environment sensing subsystem configured to collect data corresponding to the user'"'"'s surrounding environment;

a recording subsystem configured to record said data;

a correlation subsystem configured to create relationships between said data corresponding to said brain activity of said user and said data corresponding to said user events and surrounding environment;

a user portable electronic device including an interactive graphic user interface and configured to operate as a host computer processing subsystem for command, control, and processing of signals to and from said brain activity sensing, user sensing, surrounding environment sensing, and correlation subsystems of said user borne system,

said brain activity sensing, recording, and measurement computer subsystems being portable and in a communicating relationship to said user portable electronic device;

said brain activity sensing subsystem being commanded by said user portable electronic device to transmit brain activity and pattern data to said correlation subsystem;

said user sensing, surrounding environment sensing, recording, and measurement computer subsystems being portable and in a communicating relationship to said user portable electronic device;

said user sensing and surrounding environment sensing subsystems being commanded by said user portable electronic device to transmit processed sensor data to said correlation subsystem; and

said correlation subsystem being configured to receive and perform correlation processing operations to determine an extent of neural relationships between data received from said user portable electronic device and said brain activity sensing, user sensing, and surrounding environment sensing subsystems to derive neural correlates of consciousness of conscious precepts of the user;

a non-transitory computer readable medium configured to store data from said subsystems for performing queries on real-time and near real-time data received from said subsystems for determining whether to keep or disregard said data based on pre-established rule-sets and user interactive command and control from said user portable electronic device; and

a computer processing device configured to process and communicate at least a portion of said data logged and derived by said subsystems into at least one of a recipient biological, mechanical, or bio-mechanical system.

2. The system according to claim 1, wherein said mobile user borne system is communicatively connected to at least one remote computer system for storing and processing data derived by said user portable electronic device, brain activity sensing, user sensing, surrounding environment sensing, or correlation subsystem via a wireless network.

3. The system according to claim 1, wherein the user portable electronic device further comprises headgear having audio earbuds, a microphone, and an integrated augmented reality processing and interactive audio-visual display device in communication with said subsystems to provide real-time or near real-time user feedback through said interactive visual display, audio earbuds, and microphone based on the user'"'"'s thoughts and actions and circumstances in the surrounding environment determined by the computer processing device that learns and organizes said data based on input from said mobile user borne system.

4. The system according to claim 1, further comprising at least one eye-tracking device for aiding in determining the conscious precept of said user.

5. The system according to claim 1, further comprising headgear including said brain activity sensing, user sensing, and surrounding environment sensing subsystems therein, wherein said surrounding environment sensing subsystem includes a photo diode display, a photo sensor imager, and a charged micro-mirror having at least one micro-lens, said micro-mirror being oriented at 45 degrees and positioned to reflect an image from said photo diode display through said at least one micro-lens via a linear optical path;

said photo sensor imager being positioned on an off-axis optical path, said charged micro-mirror being positioned to block an image from being recorded by said photo sensor imager through said at least one micro-lens, said photo diode being transparent to allow imagery to be transmitted through said linear optical path when not charged.

6. The system according to claim 1, further comprising headgear including said brain activity sensing, user sensing, and surrounding environment sensing subsystems, wherein said surrounding environment sensing subsystem includes an objective lens system and at least one three-dimensional Very Large Scale Integrated Circuit (VLSIC) comprising a panoramic sensor and an OLED display including at least one unexcited transparent portion and a remaining active portion and at least one Region-of-Interest (ROI) image sensor configured to receive at least one image transmitted through said at least one unexcited transparent portion of said OLED display through said objective lens system to said light sensitive surface of the ROI image, wherein said OLED is located along an outer surface of said panoramic sensor, and wherein said remaining active portion of said OLED displays an image and blocks a portion of a panoramic scene from reaching the ROI sensor.

7. The system according to claim 1, wherein at least some portion of said data derived and logged from said mobile user borne system is operated upon by a natural language user interface comprising a computer-assisted translation program that incorporates computational linguistics to process machine language into textual or synthesized human speech to facilitate communication with at least one of said recipient biological, mechanical, or bio-mechanical systems.

8. The system according to claim 1, wherein said mobile user borne system includes a computerized sub-vocalization processing subsystem.

9. The system according to claim 1, wherein said portable electronic device further comprises at least one internet search engine application for searching data and signatures representing conscious precepts derived from said sensing, surrounding environment sensing, recording, and measurement computer subsystems.

10. The system according to claim 1, wherein said portable electronic device further comprises at least one social network engine application for searching data and signatures representing conscious precepts derived from said brain activity sensing, recording, and measurement computer subsystems.

11. The system according to claim 1, wherein the mobile user borne system is configured to operate as a life logging and digital assistant to said user.

12. The system according to claim 1, wherein the correlation computer subsystem is configured to derive a historical relational database by operating on data transmitted from said portable electronic device and said brain activity sensing, recording, measurement computer, sensing, surrounding environment sensing and measurement computer subsystems, said historical relational database defining computer language relationships between objects in the surrounding environment and brain activity specific to said user'"'"'s experience, point-of-view, and perception of the world.

13. The system according to claim 1, wherein said mobile user borne system is configured to provide data derived from at least one of said user portable electronic device and peripheral sensing, surrounding environment sensing, brain activity sensing, and correlation subsystems into a computer simulation system.

14. The system according to claim 1, wherein said mobile user borne system is configured to input at least some portion of said data logged or derived by said mobile user borne system into a robotic device.

15. The system according to claim 1, wherein at least some portion of said data derived and logged by said subsystems is introduced to the user to initiate neural cell formation in a target being, and stimulate existing neural cells in the target being.

16. The system according to claim 1, further configured to derive and a relational database into said non-transitory computer readable medium so as to record a substantially spherical audio-visual representation of the user'"'"'s environment as sensed by said sensing, surrounding environment sensing, recording, and measurement computer subsystems and geo-spatial information sensed by said user portable electronic device corresponding to place cells identified by said brain activity sensing subsystem.

17. The system according to claim 1, wherein said mobile user borne system is configured to provide at least some portion of said data into a recipient so as to produce a surrogate biological, mechanical, or bio-mechanical system with similar attributes to that of said user.

18. The system according to claim 1, wherein said mobile user borne system is configured to provide at least some portion of said data into a recipient being, said recipient being a clone of said user and having similar looks and consciousness as said user.

19. The system according to claim 1, wherein said mobile user borne system is configured to process at least some portion of said data for input into a recipient biological or bio-mechanical system via deoxyribonucleic acid (DNA) or ribonucleic acid (RNA) steganography data embedding techniques such that at least some portion of said embedded data may be passed on through a reproduction process of the biological or bio-mechanical system.

20. The system according to claim 1, wherein said mobile user borne system is configured to process at least some portion of said data for input into a recipient biological or bio-mechanical system via a nanobot such that at least some portion of said data is transferred into said recipient in order to produce a biological or bio-mechanical system with similar attributes to that of the user.

21. A method of logging and correlating experience data of a user and communicating said data to a recipient biological, mechanical, or bio-mechanical system, the method comprising the steps of:

providing a mobile user borne system comprising brain activity sensing, measurement computer, surrounding environment sensing, recording, and correlation subsystems;

providing a user portable electronic device including an interactive graphical user interface and configured to operate as a host computer processing subsystem for command, control, and processing of signals to and from said brain activity sensing, measurement computer, surrounding environment sensing, recording, and correlation subsystems;

collecting data corresponding to brain activity of the user via a brain activity sensing subsystem;

quantifying perceptions of the user via a measurement computer subsystem;

collecting data corresponding to sense user events via a user sensing subsystem;

collecting data corresponding to the user'"'"'s surrounding environment via a surrounding environment sensing subsystem;

recording said data via a recording subsystem;

transmitting said data to said correlation subsystem;

creating relationships between said data corresponding to said brain activity of said user and said data corresponding to said user events and surrounding environment via a correlation subsystem;

determining an extent of neural relationships between data received from said user portable electronic device and said brain activity sensing, measurement computer, surrounding environment sensing, recording, and correlation subsystems;

determining whether to keep or disregard said data based on pre-established rule-sets and user interactive command and control from said user portable electronic device; and

communicating at least a portion of said data into at least one of said recipient biological, mechanical, or bio-mechanical systems.

22. A mobile user borne system comprising:

a brain activity sensing subsystem configured to collect data corresponding to brain activity of a user;

a measurement computer subsystem configured to quantify perceptions of the user;

an eye mounted sensing subsystem configured to collect data corresponding to information the user observes visually;

a recording subsystem configured to record said data;

a correlation subsystem configured to create relationships between said data corresponding to said brain activity of said user and said data corresponding to said user'"'"'s eye mounted sensing system; and

a user portable electronic device including an interactive graphic user interface and configured to operate a host computer processing subsystem for command, control, and processing of signals to and from said brain activity sensing, eye mounted sensing, and correlation subsystems of said user borne system,

said brain activity sensing, recording, and measurement computer subsystems being portable and in a communicating relationship to said user portable electronic device,

said eye mounted device sensing, recording, and measurement computer subsystems being portable and in communicating relationship to said user portable electronic device.