System and method for large data augmented reality applications on smartglasses

• US 10,459,534 B2
• Filed: 07/14/2017
• Issued: 10/29/2019
• Est. Priority Date: 07/14/2017
• Status: Active Grant

• Current Assignee: ThirdEye Gen, Inc.
• Original Assignee: ThirdEye Gen, Inc.
• Inventors: Cherukuri, Nikhil
• Primary Examiner(s): Woo, Stella L.
• Application Number:
  US15/649,782
• Publication Number:
  US 20190018505A1
• Time in Patent Office: 837 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/001 Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

An augmented reality system for communication between a plurality of user devices. The system includes a wearable device with a headset, controller, display, camera, earphone and microphone. The controller has a local memory with a maximum video file size. The camera records video data for storage in the local memory. A server is configured for data communication with the plurality user devices, the server has a server memory and plurality of user accounts. The server is configured to establish data connection between the wearable device and at least one other user device. A video management located is configured to determine whether the video data stored in the local memory equals the maximum video file size and in this event, the video management engine is configured to transfer the video data to the server for storage in the server memory and delete the video data from the local memory.

Claims

1. An augmented reality system for communication between a plurality of user devices including wearable devices and non-wearable user stations, the system comprising

a wearable device including a headset and a controller, the headset having a display, camera, earphone and microphone, the controller being configured with a local memory having a maximum video file size, the camera being configured to record video data and store the video data in the local memory;

a server configured for data communication with the plurality user devices, the server having a server memory and plurality of user accounts, each user account being associated with at least one of the plurality of user devices, the server being configured to establish data connection between the wearable device and at least one other user device;

a video management engine located on the controller, the video management engine being configured to determine whether the video data stored in the local memory equals the maximum video file size and in this event, the video management engine being configured to transfer the video data to the server for storage in the server memory and then delete the video data from the local memory; and

an image management engine located on the controller, the server being configured to transfer image data to the wearable device, the wearable device controller having a maximum image size, the image management engine being configured to determine if the image data exceeds the maximum image size and in this event, resize the image to less than the maximum image size.

2. The system of claim 1 further comprising an authentication server configured to authenticate users.

3. The system of claim 1 wherein the user station has a separate computing platform including at least one of a personal digital assistant (PDA), smart phone, tablet, laptop or desktop computer.

4. The system of claim 1 wherein the wearable device includes at least one sensor configured to generate sensor data based on movement of the headset.

5. The system of claim 4 further comprising a sensor management engine located on the controller, the sensor management engine being configured to manipulate an image on the display based on the sensor data.

6. The system of claim 1 wherein the image management engine is configured to convert the image to base64 encoding.

7. The system of claim 6 wherein the image management engine is configured to convert the base64 encoded image to a binary format with 8 bits per byte.

8. The system of claim 7 wherein the image management engine is configured to convert the binary image into tiles.

9. The system of claim 1 wherein the image data includes annotations.

10. The system of claim 1 wherein the server is configured to support one-way communication between user devices.

11. The system of claim 1 wherein the server is configured to support two-way communication between user devices.

12. The system of claim 1 wherein the server includes a database associated with each user for storage of the video data and image data.

13. A method for augmented reality communication between a plurality of user devices including wearable devices and non-wearable user stations, the method comprising

providing a wearable device including a headset and a controller, the headset having a display, camera, earphone and microphone, the controller being configured with a local memory having a maximum video file size, the camera being configured to record video data and store the video data in the local memory;

providing a server configured for data communication with the plurality user devices, the server having a server memory and plurality of user accounts, each user account being associated with at least one of the plurality of user devices, the server being configured to establish data connection between the wearable device and at least one other user device;

providing a video management engine located on the controller, the video management engine being configured to determine whether the video data stored in the local memory equals the maximum video file size and in this event, the video management engine being configured to transfer the video data to the server for storage in the server memory and then delete the video data from the local memory; and

providing an image management engine located on the controller, the server being configured to transfer image data to the wearable device, the wearable device controller having a maximum image size, the image management engine being configured to determine if the image data exceeds the maximum image size and in this event, resize the image to less than the maximum image size.

14. The method of claim 13 further comprising providing an authentication server configured to authenticate users.

15. The method of claim 13 wherein the user station has a separate computing platform including at least one of a personal digital assistant (PDA), smart phone, tablet, laptop or desktop computer.

16. The method of claim 13 wherein the wearable device includes at least one sensor configured to generate sensor data based on movement of the headset.

17. The method of claim 16 further comprising a sensor management engine located on the controller, the sensor management engine being configured to manipulate an image on the display based on the sensor data.

18. The method of claim 13 wherein the image management engine is configured to convert the image to base64 encoding.

19. The method of claim 18 wherein the image management engine is configured to convert the base64 encoded image to a binary format with 8 bits per byte.

20. The method of claim 19 wherein the image management engine is configured to convert the binary image into tiles.

21. The method of claim 13 wherein the image data includes annotations.

22. The method of claim 13 wherein the server is configured to support one-way communication between user devices.

23. The method of claim 13 wherein the server is configured to support two-way communication between user devices.

24. The method of claim 13 wherein the server includes a database associated with each user for storage of the video data and image data.