Systems and methods for adaptive transmission of data

• US 8,184,069 B1
• Filed: 06/20/2011
• Issued: 05/22/2012
• Est. Priority Date: 06/20/2011
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Rhodes, Charles C.
• Primary Examiner(s): Liang, Regina
• Application Number:
  US13/164,445
• Time in Patent Office: 337 Days
• Field of Search: 345 7- 9, 348/14.01, 348/14.03, 348/14.05, 348/14.12, 348/115, 359/630
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597

Abstract

The present disclosure describes systems and methods for transmitting, receiving, and displaying data. The systems and methods may be directed to providing a constant or substantially constant data transmission rate (e.g., frame rate per second) to a device and controlling bandwidth by presenting information directed to an area of interest to a user. Bandwidth can be lowered, for example by presenting high resolution data directed to the area of interest to the user (e.g., an area to which the user is looking or “gazing” using a heads-up display), and lower resolution data directed to other areas. Data can be transmitted and received at a constant frame rate or substantially constant frame rate, and gaze direction and progressive compression/decompression techniques can be used to transmit data focused on areas directed to an area of interest to the user.

Claims

1. A method for transmitting data, comprising:

receiving information indicating an image of interest associated with a head-mounted display device, wherein the image of interest is within a field of view of the head-mounted display device;

receiving prioritization instructions indicating a priority of contents of the image of interest;

based on the information indicating the image of interest and the prioritization instructions, determining an order of transmission of data associated with the image of interest;

sending data associated with the image of interest in the determined order such that data that pertains to the image of interest having a first priority is sent in a first time frame slot followed by data that pertains to the image of interest having a second priority in a second time frame slot, wherein the first priority is higher than the second priority;

progressively sending in the determined order additional data that pertains to the image of interest in subsequent time frame slots until completion of transmission of data associated with the image of interest or until receipt of another request for data from the head-mounted display device; and

maintaining a substantially constant data transmission rate by controlling time allocated to encoding data for each time frame slot.

2. The method of claim 1, wherein sending data comprises wirelessly sending data to the head-mounted display device.

3. The method of claim 1, wherein the head-mounted display device includes a heads-up display.

4. The method of claim 1, further comprising:

receiving information indicating one or more of a location of the head-mounted display device and a gaze direction associated with the head-mounted display device; and

based on one or more of the location of the head-mounted display device and the gaze direction associated with the head-mounted display device, determining the order of transmission of data associated with the image of interest.

5. The method of claim 1, wherein the head-mounted display device includes a head-mounted display, and wherein data associated with the image of interest is determined based on a head orientation of a user of the head-mounted display.

6. The method of claim 1, further comprising sending data associated with the image of interest in the determined order based additionally on one or more information received from the head-mounted display device indicating a time to decode received data and a percentage of received data successfully decoded.

7. The method of claim 1, further comprising determining the order of transmission of data associated with the image of interest based also on one or more information received from the head-mounted display device including a time to decode received data and a percentage of received data successfully decoded.

8. The method of claim 1, wherein sending data associated with the image of interest in the determined order comprises sending prioritized subsets of data of the image of interest, and wherein the method further comprises determining how to encode data for transmission to the head-mounted display device based on one or more of the prioritized subsets of data, a time to decode received data at the head-mounted display device, and a percentage of received data at the head-mounted display device successfully decoded.

9. The method of claim 1, further comprising upon receiving another request for data from the head-mounted display device, encoding and sending data associated with information of the another request even if encoding and transmission of data associated with information of a data frame currently being encoded and transmitted is not completely encoded and transmitted.

10. The method of claim 1, wherein the prioritization instructions indicating the priority of contents of the image of interest include one or more of a location associated with audio captured by the head-mounted display device, and a direction associated with motion of the head-mounted display device.

11. The method of claim 1, wherein the prioritization instructions indicating the priority of contents of the image of interest are based on a weighting function taking into account one or more of a gaze direction associated with a user of the head-mounted display device, a location associated with audio captured by the head-mounted display device, and a direction associated with motion of the head-mounted display device.

12. The method of claim 1, wherein the information indicating the image of interest indicates one or more of a location of the head-mounted display device and a direction that the head-mounted display device is pointing, and wherein determining the order of transmission of data associated with the image of interest comprises:

identifying contents of the image of interest that are associated with the prioritization instructions; and

prioritizing the identified contents ahead of other contents of the image of interest.

13. The method of claim 1, further comprising:

receiving the prioritization instructions indicating the priority of contents of the image of interest after sending at least a portion of data associated with the image of interest in the determined order;

based on the prioritization instructions, determining a new order of transmission of data associated with the image of interest; and

sending data associated with the image of interest in the new order.

14. An article of manufacture including a tangible computer-readable memory having computer-readable instructions encoded thereon, the instructions comprising:

instructions for receiving information indicating an image of interest associated with a head-mounted display device, wherein the image of interest is within a field of view of the head-mounted display device;

instructions for receiving prioritization instructions indicating a priority of contents of the image of interest;

based on the information indicating the image of interest and the prioritization instructions, instructions for determining an order of transmission of data associated with the image of interest;

instructions for sending data associated with the image of interest in the determined order such that data that pertains to the image of interest having a first priority is sent in a first time frame slot followed by data that pertains to the image of interest having a second priority in a second time frame slot, wherein the first priority is higher than the second priority;

instructions for progressively sending in the determined order additional data that pertains to the image of interest in subsequent time frame slots until completion of transmission of data associated with the image of interest or until receipt of another request for data from the head-mounted display device; and

instructions for maintaining a substantially constant data transmission rate by controlling time allocated to encoding data for each time frame slot.

15. The article of manufacture of claim 14, further comprising instructions for:

receiving information indicating one or more of a location of the head-mounted display device and a gaze direction associated with the head-mounted display device; and

based on one or more of the location of the head-mounted display device and the gaze direction associated with the head-mounted display device, determining the order of transmission of data associated with the image of interest.

16. The article of manufacture of claim 14, further comprising instructions for:

receiving the prioritization instructions indicating the priority of contents of the image of interest after sending at least a portion of data associated with the image of interest in the determined order;

based on the prioritization instructions, identifying contents of the image of interest that are associated with the prioritization instructions;

prioritizing the identified contents ahead of other contents of the image of interest;

based on the prioritization, determining a new order of transmission of data associated with the image of interest; and

sending data associated with the image of interest in the new order.

17. The article of manufacture of claim 14, further comprising instructions for sending data associated with the image of interest in the determined order based additionally on one or more information received from the head-mounted display device indicating a time to decode received data and a percentage of received data successfully decoded.

18. The article of manufacture of claim 14, further comprising instructions for determining the order of transmission of data associated with the image of interest based also on one or more information received from the head-mounted display device including a time to decode received data and a percentage of received data successfully decoded.

19. A system comprising:

means for receiving information indicating an image of interest associated with a head-mounted display device, wherein the image of interest is within a field of view of the head-mounted display device;

means for receiving prioritization instructions indicating a priority of contents of the image of interest;

based on the information indicating the image of interest and the prioritization instructions, means for determining an order of transmission of data associated with the image of interest;

means for sending data associated with the image of interest in the determined order such that data that pertains to the image of interest having a first priority is sent in a first time frame slot followed by data that pertains to the image of interest having a second priority in a second time frame slot, wherein the first priority is higher than the second priority;

means for progressively sending in the determined order additional data that pertains to the image of interest in subsequent time frame slots until completion of transmission of data associated with the image of interest or until receipt of another request for data from the head-mounted display device; and

means for maintaining a substantially constant data transmission rate by controlling time allocated to encoding data for each time frame slot.

20. The system of claim 19, further comprising:

means for receiving information indicating one or more of a location of the head-mounted display device and a gaze direction associated with the head-mounted display device; and

based on one or more of the location of the head-mounted display device and the gaze direction associated with the head-mounted display device, means for determining the order of transmission of data associated with the image of interest.