Augmented reality personal assistant apparatus

• US 9,823,737 B2
• Filed: 07/01/2014
• Issued: 11/21/2017
• Est. Priority Date: 04/07/2008
• Status: Active Grant

• Current Assignee: Mohammad A. Mazed, Sayeeda Mazed
• Original Assignee: Mohammad A. Mazed, Sayeeda Mazed
• Inventors: Mazed, Sayeeda, Mazed, Mohammad A
• Primary Examiner(s): VILLECCO, JOHN M
• Application Number:
  US14/120,835
• Publication Number:
  US 20160004298A1
• Time in Patent Office: 1,239 Days
• Field of Search: 345632, 345633
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G09G 2380/02 Flexible displays
• CPC Class Codes: G09G 2380/08 Biomedical applications
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP
• CPC Class Codes: G09G 3/03 specially adapted for displays having non-planar surfaces, e.g. curved displays
• CPC Class Codes: G09G 3/346 based on modulation of the reflection angle, e.g. micromirrors micromirrors devices per se G02B26/0833
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G16C 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: H01L 2224/48091 Arched
• CPC Class Codes: H01L 2924/00014 the subject-matter covered by the group, the symbol of which is combined with the symbol of this group, being disclosed without further technical details
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/21 from near infrared [NIR] radiation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Various embodiments of an intelligent augmented reality personal assistant apparatus integrated (or co-packaged) with an eye motion sensor, a microprocessor or an intelligent microprocessor and an intelligent rendering algorithm are disclosed. Such an augmented reality personal assistant apparatus can interpret/analyze/learn activities, communication or contextual information of a user and recommend relevant and useful information to the user.

Claims

1. A subsystem, as an augmented reality personal assistant comprising:

(a) a first sensor, wherein the first sensor is an eye motion sensor;

(b) a camera sensor;

(c) a decoder;

(d) a display and a projector, wherein one or more pixels or three-dimensional (3-D) pixels of the display are integrated with a second sensor; and

wherein the first sensor is electrically coupled with the camera sensor,

wherein the camera sensor is electrically coupled with the decoder,

wherein the decoder is electrically coupled with the display or the projector,

wherein the first sensor is determining an item or a person that a user is looking at within the user'"'"'s field of view,

wherein the decoder is converting the camera sensor'"'"'s reading of the item or the person targeted by the first sensor in the user'"'"'s field of view into a text or a picture image or a video image,

wherein the text or the picture image or the video image, converted by the decoder is either viewed on the display or projected by the projector on a surface,

(e) a first microprocessor, wherein the first microprocessor comprises memristors, wherein the first microprocessor further electrically couples with a first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant storing a system operating algorithm,

wherein the memristors are arranged in two-dimension (2-D) or in three-dimension (3-D),

wherein the first microprocessor is electrically coupled with the first sensor, the camera sensor, the decoder, the display and the projector,

wherein the subsystem, as the augmented reality personal assistant and a portable Internet appliance are electrically or wirelessly coupled with a cloud based server or a cloud based data storage media,

wherein the portable internet appliance comprises;

a second microprocessor or a neural microprocessor,

wherein the second microprocessor or the neural processor electrically couples with a second non-transitory local data storage media of the portable internet appliance storing an intelligent rendering algorithm.

2. The subsystem, as the augmented reality personal assistant according to claim 1, further comprises:

a contact lens or a contact lens integrated with a nanoscaled optical structure.

3. The subsystem, the augmented reality personal assistant according to claim 1, further electrically or wirelessly couples with or comprises:

a third sensor selected from the group consisting of;

a gesture sensor and a touch sensor.

4. A subsystem, as an augmented reality personal assistant for social graph or personal analytics comprising:

(a) a first sensor, wherein the first sensor is an eye motion sensor;

(b) a camera sensor;

(c) a decoder;

(d) a display and a projector, wherein one or more pixels or three-dimensional (3-D) pixels of the display are integrated with a second sensor;

(e) a microphone;

(f) a location determination module;

(g) a first microprocessor;

(h) an electrical powering component; and

wherein the first sensor is electrically coupled with the camera sensor,

wherein the camera sensor is electrically coupled with the decoder,

wherein the decoder is electrically coupled with the display or the projector,

wherein the first sensor is determining an item or a person that a user is looking at within the user'"'"'s field of view,

wherein the decoder is converting the camera sensor'"'"'s reading of the item or the person targeted by the first sensor in the user'"'"'s field of view into a text or a picture image or a video image,

wherein the text or the picture image or the video image, converted by the decoder is either viewed on the display or projected by the projector of on a surface,

wherein the first microprocessor is electrically coupled with the first sensor, the camera sensor, the decoder, the display, the projector, the microphone, the location determination module and the electrical powering component,

wherein the first microprocessor further comprises memristors, wherein the first microprocessor further electrically couples with a first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant storing a system operating algorithm,

wherein the memristors are arranged in two-dimension (2-D) or in three-dimension (3-D),

wherein the first microprocessor further electrically couples with or comprises;

a set of computer implementable instructions to interpret or analyze or learn activities or contextual information of the user,

wherein the said set of computer implementable instructions is stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in a cloud based data storage media or a cloud based server,

wherein the first microprocessor further electrically couples with or comprises;

a set of computer implementable instructions to interpret or analyze or learn a location of the user,

wherein the said set of computer implementable instructions is stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server,

wherein the subsystem, as the augmented reality personal assistant and a portable internet appliance are electrically or wirelessly coupled with the cloud based data storage media or the cloud based server,

wherein the portable internet appliance comprises;

a second microprocessor or a neural microprocessor,

wherein the second microprocessor or the neural processor electrically couples with a second non-transitory local data storage media of the portable internet appliance storing an intelligent rendering algorithm.

5. The subsystem, as the augmented reality personal assistant according to claim 4, further comprises:

a contact lens or a contact lens integrated with a nanoscaled optical structure.

6. The subsystem, as the augmented reality personal assistant according to claim 4, further electrically or wirelessly couples with or comprises:

a third sensor selected from the group consisting of;

a gesture sensor and a touch sensor.

7. The subsystem, as the augmented reality personal assistant according to claim 4, further comprises:

a component selected from the group consisting of;

a radio frequency identification device (RFID) and a near field communication (NFC) transceiver.

8. The subsystem, as the augmented reality personal assistant according to claim 4, further comprises:

a transceiver module selected from the group consisting of;

a radio transceiver, a millimeter wave transceiver and a terahertz band transceiver.

9. The subsystem, as the augmented reality personal assistant according to claim 4, further comprises:

coupling with an algorithm selected from the group consisting of;

a face recognition algorithm, a gesture recognition algorithm, a sound recognition algorithm, a voice recognition algorithm and a voice-to-text conversion algorithm stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server.

10. The subsystem, as the augmented reality personal assistant according to claim 4, further comprises:

coupling with a pattern recognition algorithm or a data mining algorithm stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server.

11. The subsystem, as the augmented reality personal assistant according to claim 4, further comprises:

coupling with a predictive algorithm stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server.

12. A subsystem, as an augmented reality personal assistant for recommending a user comprising:

(a) a first sensor, wherein the first sensor is an eye motion sensor;

(b) a camera sensor;

(c) a decoder;

(d) a display and a projector, wherein one or more pixels or three-dimensional (3-D) pixels of the display are integrated with a second sensor;

(e) a microphone;

(f) a location determination module;

(g) a first microprocessor;

(h) an electrical powering component; and

wherein the first sensor is electrically coupled with the camera sensor,

wherein the camera sensor is electrically coupled with the decoder,

wherein the decoder is electrically coupled with the display or the projector,

wherein the first sensor is determining an item or a person that a user is looking at within the user'"'"'s field of view,

wherein the decoder is converting the camera sensor'"'"'s reading of the item or the person targeted by the first sensor in the user'"'"'s field of view into a text or a picture image or a video image,

wherein the text or the picture image or the video image, converted by the decoder is either viewed on the display or projected by the projector of on a surface,

wherein the first microprocessor is electrically coupled with the first sensor, the camera sensor, the decoder, the display, the projector, the microphone, the location determination module and the electrical powering component,

wherein the first microprocessor comprises memristors, wherein the first microprocessor further electrically couples with a first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant storing a system operating algorithm,

wherein the memristors are arranged in two-dimension (2-D) or in three-dimension (3-D),

wherein the first microprocessor further electrically couples with or comprises;

a set of computer implementable instructions to interpret or analyze or learn activities or contextual information of the user,

wherein the said set of computer implementable instructions is stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in a cloud based data storage media or a cloud based server,

wherein the first microprocessor further electrically couples with or comprises;

a set of computer implementable instructions to interpret or analyze or learn personal communication of the user in a natural language,

wherein the said set of computer implementable instructions is stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server,

wherein the personal communication of the user is selected from the group consisting of;

voice communication, text communication, e-mail communication and video communication,

wherein the subsystem, as the augmented reality personal assistant and a portable internet appliance are electrically or wirelessly coupled with the cloud based data storage media or the cloud based server,

wherein the portable internet appliance comprises;

a second microprocessor or a neural microprocessor,

wherein the second microprocessor or the neural processor electrically couples with a second non-transitory local data storage media of the portable internet appliance storing an intelligent rendering algorithm.

13. The subsystem, as the augmented reality personal assistant according to claim 12, further comprises:

a contact lens or a contact lens integrated with a nanoscaled optical structure.

14. The subsystem, as the augmented reality personal assistant according to claim 12, further electrically or wirelessly couples with or comprises:

a third sensor selected from the group consisting of;

a gesture sensor and a touch sensor.

15. The subsystem, as the augmented reality personal assistant according to claim 12, further comprises:

a component selected from the group consisting of;

a radio frequency identification device (RFID) and a near field communication (NFC) transceiver.

16. The subsystem, as the augmented reality personal assistant according to claim 12, further comprises:

a transceiver module selected from the group consisting of;

a radio transceiver, a millimeter wave transceiver and a terahertz band transceiver.

17. The subsystem, as the augmented reality personal assistant according to claim 12, further comprises:

coupling with an algorithm selected from the group consisting of;

a face recognition algorithm, a gesture recognition algorithm, a sound recognition algorithm, a voice recognition algorithm and a voice-to-text conversion algorithm stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server.

18. The subsystem, as the augmented reality personal assistant according to claim 12, further comprises:

coupling with a pattern recognition algorithm or a data mining algorithm stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server.

19. The subsystem, as the augmented reality personal assistant according to claim 12, further comprises:

coupling with a predictive algorithm stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server.

20. The subsystem, as the augmented reality personal assistant according to claim 12, further comprises:

coupling with a set of computer implementable instructions for recommending or advising the user stored in the first non-transitory local data storage media of the subsystem, as the augmented reality personal assistant or in the cloud based data storage media or the cloud based server.