Video transmission and sharing over ultra-low bitrate wireless communication channel

• US 10,375,534 B2
• Filed: 12/22/2011
• Issued: 08/06/2019
• Est. Priority Date: 12/22/2010
• Status: Active Grant

• Current Assignee: Behrooz Rezvani
• Original Assignee: Seyyer, Inc.
• Inventors: Rezvani, Behrooz
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Hasan, Syed Y
• Application Number:
  US13/334,726
• Publication Number:
  US 20120327243A1
• Time in Patent Office: 2,784 Days
• Field of Search: 348158, 348 78, 348723, 348725, 348 86, 345473, 37524018, 382115, 382128, 382181, 382224, 382243, 382116, 382190, 382218, 455466, 455557, 709206, 370239
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 19/0018 Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 5/38 Transmitter circuitry for the transmission of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 7/12 Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal H04N7/24 takes precedence
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control
• CPC Class Codes: H04W 4/14 Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals

Abstract

Techniques for transmitting and sharing a video sequence over an ultra-low bandwidth channel, such as a short message service (SMS) channel, are disclosed herein. A video is segmented into regions of various interest levels. A set of parameters is developed from a video region of a high interest, wherein the parameters represent a mapping function of a database to model the video region. The set of parameters is transmitted over the ultra-low bandwidth channel to a remote device, wherein the remote device also has access to an instance of the database. The remote device synthesizes the video by using the mapping function of the database, which is represented by the transmitted set of parameters.

Claims

1. A method comprising:

capturing or retrieving a video sequence at a communication device that includes a plurality of databases, each of the plurality of databases comprising a plurality of images, the plurality of databases including a first database and a second database;

segmenting a first frame of the video sequence into at least a first region and a second region;

using the first database, determining a first mapping function for the first region;

synthesizing a first image of the first region using the first mapping function and the first database;

determining a first synthesis fidelity of the first image of the first region;

using the second database, determining a second mapping function for the first region;

synthesizing a second image of the first region using the second mapping function and the second database;

determining a second synthesis fidelity of the second image of the first region;

selecting the first image based on a selection criterion, the selection criterion resulting in the second synthesis fidelity being less optimal than the first synthesis fidelity; and

sending the first mapping function and information identifying the first database from the communication device to at least one remote device over a wireless communication link in an ultra low bit rate channel.

2. The method of claim 1, wherein the ultra low bit rate channel is a short message service (SMS) channel.

3. The method recited in claim 1, wherein the first mapping function represents a linear combination of a plurality of images from the first database.

4. The method recited in claim 3, wherein the first mapping function comprises a set of coefficients.

5. The method recited in claim 1, wherein the first mapping function comprises a plurality of eigenvectors.

6. The method recited in claim 1, further comprising:

selecting the first and second databases based on an identity of an object in the video sequence.

7. The method recited in claim 6, wherein the object in the video sequence is a human being.

8. The method recited in claim 6, wherein the object in the video sequence is a particular human being, and wherein the first database comprises images of the particular human being.

9. The method recited in claim 6, further comprising:

optimizing the first database based on a characteristic of the particular human being.

10. The method recited in claim 9, wherein the characteristic is an age, a gender, or an ethnicity.

11. The method recited in claim 1, wherein the first and second databases include different views of an object.

12. The method recited in claim 11, wherein the object is a face, and wherein the first database includes a plurality of front views and the second database includes a plurality of partial or full side views.

13. The method recited in claim 1, wherein the selection criterion is a first selection criterion, and further comprising:

using a third database of the plurality of databases, determining a third mapping function for the first region;

synthesizing a third image of the first region using the third mapping function and the third database;

determining a third synthesis fidelity of the third image of the first region; and

based on a second selection criterion, determining that the third synthesis fidelity is less optimal than the first synthesis fidelity.

14. The method recited in claim 13, wherein the third mapping function represents a linear combination of a plurality of images from the third database.

15. The method recited in claim 14, wherein the third mapping function comprises a set of coefficients.

16. The method recited in claim 13, wherein the third mapping function comprises a plurality of eigenvectors.

17. The method recited in claim 13, wherein (a) the first selection criterion is a gender and the second selection criterion is an age range, or vice versa, or (b) the first selection criterion is an age range and the second selection criterion is an ethnicity, or vice versa, or (c) the first selection criterion is a gender and the second selection criterion is an ethnicity, or vice versa.

18. The method recited in claim 1, further comprising:

segmenting a second frame of the video sequence into at least a third region and a fourth region, wherein the third region corresponds to the first region;

using the first database, determining a third mapping function for the third region; and

sending, to the at least one remote device, the third mapping function.

19. The method recited in claim 18, wherein the third mapping function represents a linear combination of a plurality of images from the first database.

20. The method recited in claim 19, wherein the third mapping function comprises a set of coefficients.

21. The method recited in claim 18, wherein a boundary of the third region differs from a boundary of the first region.

22. The method recited in claim 18, wherein segmenting the second frame of the video sequence into at least the third region and the fourth region comprises:

identifying the third region; and

identifying the fourth region based on a relationship between the fourth region and the third region.

23. The method recited in claim 18, further comprising:

synthesizing a third image of the third region using the third mapping function and the first database;

determining a third synthesis fidelity of the third image of the third region;

using a third database of the plurality of databases, determining a fourth mapping function for the third region;

synthesizing a fourth image of the third region using the fourth mapping function and the third database;

determining a fourth synthesis fidelity of the fourth image of the third region;

selecting the third image based on the selection criterion, the selection criterion resulting in the fourth synthesis fidelity being less optimal than the third synthesis fidelity.

24. The method recited in claim 1, wherein the selection criterion is that the first synthesis fidelity is more optimal than the second synthesis fidelity.

25. The method recited in claim 1, wherein the selection criterion accounts for a gender, an age range, or an ethnicity.

26. The method recited in claim 1, wherein the first synthesis fidelity is based on a difference between the first image and the first region, and the second synthesis fidelity is based on a difference between the second image and the first region.

27. The method recited in claim 1, wherein the first synthesis fidelity is a sum of pixel value differences between the first region and the first image, and the second synthesis fidelity is a sum of pixel value differences between the first region and the second image, and wherein the selection criterion is to select the lower of the first synthesis fidelity and the second synthesis fidelity.

28. The method recited in claim 1, wherein a bit rate of the ultra low bit rate channel is no greater than 6,500 bits per second.

29. The method recited in claim 1, further comprising:

sending information identifying a version of the first database.

30. The method recited in claim 1, further comprising:

updating the first database based on the first frame of the video sequence.

31. The method recited in claim 1, further comprising:

identifying a change in the second region relative to a previously-processed frame of the video sequence; and

sending, to the at least one remote device, information characterizing the change in the second region.

32. The method recited in claim 31, wherein the previously-processed frame of the video sequence is two or more frames before the first frame.