Video communications methods using network packet segmentation and unequal protection protocols, and wireless devices and vehicles that utilize such methods

• US 10,430,665 B2
• Filed: 09/07/2017
• Issued: 10/01/2019
• Est. Priority Date: 09/07/2017
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Yu, Xin, Bai, Fan, Zhang, Wende, Sergakis, John
• Primary Examiner(s): Owens, Tsion B
• Application Number:
  US15/698,282
• Publication Number:
  US 20190073539A1
• Time in Patent Office: 754 Days
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field

Abstract

A video communications method includes segmenting an image frame or an image frame portion into first and second source network packet blocks. The first source network packet block includes a first number of source network packets and the second network packet block includes a second number of source network packets. The method further includes encoding the first source network packet block to produce a first encoded network packet block and encoding the second source network packet block to produce a second encoded network packet block. The first encoded network packet block includes a first number of encoded network packets and the second encoded network packet block includes a second number of encoded network packets. Still further, the method includes transmitting the first and second encoded network packet blocks over a wireless network. A wireless device and a vehicle may utilize the video communications method.

Claims

1. A video communications method comprising:

segmenting an image frame or an image frame portion into a first source network packet block and a second source network packet block, wherein the first source network packet block comprises a first number of source network packets and the second network packet block comprises a second number of source network packets;

encoding the first source network packet block to produce a first encoded network packet block and encoding the second source network packet block to produce a second encoded network packet block, wherein the first encoded network packet block comprises a first number of encoded network packets and the second encoded network packet block comprises a second number of encoded network packets, and wherein the first number of encoded network packets is greater than or equal to the first number of source network packets and the second number of encoded network packets is greater than or equal to the second number of source network packets; and

transmitting the first encoded network packet block and the second encoded network packet block over a wireless network.

2. The method of claim 1, wherein the first number of encoded network packets is greater than the first number of source network packets or the second number of encoded network packets is greater than the second number of source network packets.

3. The method of claim 2, wherein the first number of encoded network packets and the second number of encoded network packets are determined by:

determining a maximum number of encoded packets (Q_max) on the basis of an anticipated decoding rate of a receiving device that is configured to receive the transmitted first and second encoded network packet blocks;

determining a set of first probabilities, wherein each first probability of the set of first probabilities represents the probability that the first encoded network packet block having M₁ encoded network packets will be decodable at the receiving device, wherein each first probability of the set of first probabilities represents a different value of M₁ less than Q_max;

determining a set of second probabilities, wherein each second probability of the set of second probabilities represents the probability that the second encoded network packet block having M₂ encoded network packets will be decodable at the receiving device, wherein each second probability of the set of second probabilities represents a different value of M₂ less than Q_max;

selecting one first probability of the set of first probabilities and selecting one second probability of the set of second probabilities such that the one selected first probability multiplied by the one selected second probability has a value greater than or equal to a value of any first probability of the set of first probabilities multiplied by any second probability of the set of second probabilities; and

setting the first number of encoded network packets equal to the value M₁ represented by the one selected first probability and setting the second number of encoded network packets equal to the value M₂ represented by the one selected second probability.

4. The method of claim 2, wherein the first number of encoded network packets and the second number of encoded network packets are determined by:

(1) determining a maximum number of encoded packets (Q_max) on the basis of an anticipated decoding rate of a receiving device that is configured to receive the transmitted first and second encoded network packet blocks;

(2) determining a first probability that the first encoded network packet block having M₁ encoded network packets will be decodable at the receiving device, wherein M₁ has a value equal to the first number of source network packets;

(3) determining a second probability that the second encoded network packet block having M₂ encoded network packets will be decodable at the receiving device, wherein M₂ has a value equal to the second number of source network packets;

(4) determining a third probability that the first encoded network packet block having M₁+1 encoded network packets will be decodable at the receiving device and determining a fourth probability that the second encoded network packet block having M₂+1 encoded network packets will be decodable at the receiving device;

(5) determining a first incremental gain in probability of the first encoded network packet block on the basis of the third probability divided by the first probability and determining a second incremental gain in probability of the second encoded network packet block on the basis of the fourth probability divided by the second probability;

(6) if the first incremental gain in probability is greater than the second incremental gain in probability, resetting the value of M₁ to M₁+1, alternatively, if the second incremental gain in probability is greater than the first incremental gain in probability, resetting the value of M₂ to M₂+1;

repeating steps (2) through (6) until a sum of M₁ and M₂ equals Q_max; and

setting the first number of encoded network packets equal to the value M₁ when the sum of M₁ and M₂ equals Q_max and setting the second number of encoded network packets equal to the value M₂ when the sum of M₁ and M₂ equals Q_max.

5. The method of claim 1, further comprising, prior to segmenting the image frame or the image frame portion, selecting the image frame or the image frame portion for transmission over the wireless network.

6. The method of claim 5, wherein selecting the image frame for transmission over the wireless network comprises selecting the image frame from an image frame group that comprises at least one image frame of relatively lower importance and at least one image frame of relatively higher importance, wherein relative importance is defined on the basis of a relative ability of an image frame to be decoded into an image at a receiving device that is configured to receive the transmitted first and second encoded network packet blocks, and wherein selecting the image frame from the image frame group comprises:

assigning a different unequal protection weighting value to each of the at least one image frame of relatively lower importance and the at least one image frame of relatively higher importance;

determining a packet drop rate of encoded network packets transmitted over the wireless network;

determining a relative received signal strength (RSSI) at the receiving device; and

selecting the image frame from the image frame group based on the unequal protection weighting values, the packet drop rate, and the RSSI.

7. The method of claim 6, wherein assigning the different unequal protection weighting values comprises assigning a relatively higher unequal protection weighting value to the at least one image frame of relatively higher importance and assigning a relatively lower unequal protection weighting value to the at least one image frame of relatively lower importance.

8. The method of claim 7, wherein the image frame group comprises an I-frame, a P-frame, and a B-frame, wherein the I-frame is assigned a relatively higher unequal protection weighting value as compared with the P-frame, and wherein the P-frame is assigned a relatively higher unequal protection weighting value as compared with the B-frame.

9. The method of claim 5, wherein selecting the image frame portion for transmission over the wireless network comprises selecting the image frame portion from an image frame portion group that comprises at least one image frame portion comprising an object of relatively lower importance and at least one image frame portion comprising an object of relatively higher importance, wherein relative importance is defined on the basis of a relatedness of an object within an image frame portion to the functioning of an electronic application of a receiving device that is configured to receive the transmitted first and second encoded network packet blocks, and wherein selecting the image frame portion from the image frame portion group comprises:

assigning a different unequal protection weighting value to each of the at least one image frame portion comprising an object of relatively lower importance and the at least one image frame portion comprising an object of relatively higher importance;

determining a packet drop rate of encoded network packets transmitted over the wireless network;

determining a relative received signal strength (RSSI) at the receiving device; and

selecting the image frame portion from the image frame portion group based on the unequal protection weighting values, the packet drop rate, and the RSSI.

10. The method of claim 9, wherein assigning the different unequal protection weighting values comprises assigning a relatively higher unequal protection weighting value to the at least one image frame portion comprising an object of relatively higher importance and assigning a relatively lower unequal protection weighting value to the at least one image frame portion comprising an object of relatively lower importance.

11. The method of claim 10, wherein the image frame portion group comprises an image frame portion comprising a pedestrian object and an image frame portion comprising a bicyclist or vehicle object, wherein the image frame portion comprising the pedestrian object is assigned a relatively higher unequal protection weighting value as compared with the image frame portion comprising the bicyclist or vehicle object.

12. The method of claim 5, wherein selecting the image frame or the image frame portion for transmission over the wireless network is performed using a probabilistic adaptive selection protocol that operates on the basis of an unequal protection weighting value assigned to the image frame or the image frame portion, a packet drop rate of encoded network packets transmitted over the wireless network, and a relative received signal strength (RSSI) at a receiving device that is configured to receive the transmitted first and second encoded network packet blocks.

13. The method of claim 1, wherein transmitting the first encoded network packet block and the second encoded network packet block over the wireless network is performed such that the first encoded network packet block is transmitted prior to the second encoded network packet block or the second encoded network packet block is transmitted prior to the first encoded network packet block, wherein the method further comprises determining an order of transmitting the first and second encoded network packet blocks, wherein determining the order of transmission comprises:

determining a relative importance of the first encoded network packet block and a relative importance of the second encoded network packet block, wherein relative importance is defined on the basis of a relative ability of an encoded network packet block to be decoded into an image at a receiving device that is configured to receive the transmitted first and second encoded network packet blocks;

assigning a different unequal protection weighting value to each of the first encoded network packet block and the second encoded network packet block based on their respective determined relative importance;

determining a packet drop rate of encoded network packets transmitted over the wireless network;

determining a relative received signal strength (RSSI) at the receiving device; and

determining the order of transmission of the first and second encoded network packet blocks based on the unequal protection weighting values, the packet drop rate, and the RSSI.

14. The method of claim 13, wherein the first encoded network block comprises a wavelet function data block representing the image frame or the image frame portion at a relatively lower resolution and the second encoded network block comprises a wavelet function data block representing a high-frequency component, and wherein assigning the different unequal protection weighting values comprises assigning a relatively higher unequal protection weighting value to the first encoded network block and assigning a relatively lower unequal protection weighting value to the second encoded network block.

15. A wireless device comprising an electronic processing device and a digital memory device, wherein the digital memory device comprises a resident application comprising computer-readable instructions configured to cause the electronic processing device to:

segment an image frame or an image frame portion into a first source network packet block and a second source network packet block, wherein the first source network packet block comprises a first number of source network packets and the second network packet block comprises a second number of source network packets;

encode the first source network packet block to produce a first encoded network packet block and encode the second source network packet block to produce a second encoded network packet block, wherein the first encoded network packet block comprises a first number of encoded network packets and the second encoded network packet block comprises a second number of encoded network packets, and wherein the first number of encoded network packets is greater than or equal to the first number of source network packets and the second number of encoded network packets is greater than or equal to the second number of source network packets; and

transmit the first encoded network packet block and the second encoded network packet block over a wireless network.

16. The wireless device of claim 15, wherein the wireless device is configured as a smart phone, a tablet computer, an image scanner, a music player, a camera, or a drone.

17. The wireless device of claim 15, wherein the computer-readable instructions are further configured to cause the electronic processing device to transmit the first encoded network packet block and the second encoded network packet block over the wireless network to a vehicle, wherein the vehicle comprises a telematics unit and a visual display, wherein the telematics unit is configured to receive the first encoded network packet block and the second encoded network packet block over the wireless network as transmitted by the wireless device, wherein the telematics unit is further configured to decode the first encoded network packet block and the second encoded network packet block to generate a decoded image, and wherein the telematics unit is further configured to cause the visual display to display the decoded image.