Model-Based Error Resilience in Data Communication

• US 20100014768A1
• Filed: 07/16/2008
• Published: 01/21/2010
• Est. Priority Date: 07/16/2008
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Bhattacharjya, Anoop K., Xiao, Jing
• Granted Patent: US 8,180,167 B2
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder

Abstract

Disclosed are embodiments of systems and methods to use a model-based technique for image error recovery in data communication. A low-dimensional representation is constructed of an image that contains errors. A manifold comprising image representations and a statistical model of the manifold are used to correct the errors in the image.

Claims

1. A method for correcting an error in an image comprising pixels, the method comprising:

constructing a measurement vector of the image based upon at least some characteristics of the image;

computing a low dimensional vector from the measurement vector;

computing a mapped low dimensional vector by mapping the low dimensional vector onto a manifold comprising a plurality of low dimensional vectors; and

using the mapped low dimensional vector to generate a display image.

2. The method of claim 1 wherein mapping the low dimensional vector comprises computing a projection of the low dimensional vector onto the manifold.

3. The method of claim 2 wherein the step of computing a projection of the low dimensional vector onto the manifold comprises:

computing a modeled manifold by selecting a set of low dimensional vectors from the manifold that are within a threshold distance of the low dimensional vector; and

computing a projection of the low dimensional vector onto the modeled manifold.

4. The method of claim 1 wherein mapping the low dimensional vector onto a manifold comprises:

computing a path traversed on the manifold using a set of low dimensional vectors that were previously mapped onto the manifold; and

using the path to predict a projection of the low dimensional vector onto the manifold.

5. The method of claim 1 wherein the step of using the mapped low dimensional vector to generate a display image comprises generating a display image wherein an adjustment to a non-error pixel value in the image is limited to be no more than a threshold value.

6. The method of claim 1, wherein the step of computing a mapped low dimensional vector comprises:

responsive to the manifold comprising multiple models, using projection onto convex sets to map the low dimensional vector.

7. The method of claim 1 further comprising updating the manifold by:

adding a low dimensional vector to the manifold; and

responsive to satisfaction of at least one update criterion, updating a model of the manifold.

8. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform at least the steps of the method of claim 1.

9. A method for correcting an error in an image comprising pixels, the method comprising:

computing a measurement vector of the image using warping and three-dimensional pose information from the image;

computing a low dimensional vector from the measurement vector;

computing a mapped low dimensional vector by mapping the low dimensional vector onto a manifold comprising a plurality of low dimensional vectors; and

using the mapped low dimensional vector to generate a display image.

10. The method of claim 9 wherein the step of computing a measurement vector of the image using warping and three-dimensional pose information from the image comprises:

obtaining warped image data from the image by warping at least a portion of the image into a fix view;

obtaining three-dimensional image data from the image by determining a three-dimensional pose of at least a portion of the image; and

forming a vector for the image using the warped image data and the three-dimensional image data.

11. The method of claim 9 wherein mapping the low dimensional vector onto a manifold comprises computing a projection of the low dimensional vector onto the manifold.

12. The method of claim 11 wherein the step of computing a projection of the low dimensional vector onto the manifold comprises:

computing a modeled manifold by selecting a set of low dimensional vectors from the manifold that are within a threshold distance of the low dimensional vector; and

computing a projection of the low dimensional vector onto the modeled manifold.

13. The method of claim 9 wherein mapping the low dimensional vector onto a manifold comprises:

computing a path traversed on the manifold using a set of low dimensional vectors that were previously mapped onto the manifold; and

using the path to predict a projection of the low dimensional vector onto the manifold.

14. The method of claim 13 wherein the path is a time-parameterized path.

15. The method of claim 9 further comprising updating the manifold by:

adding a low dimensional vector to the manifold; and

responsive to satisfaction of at least one update criterion, updating a model of the manifold.

16. The method of claim 9 wherein the step of using the mapped low dimensional vector to generate a display image comprises generating a display image wherein an adjustment to a non-error pixel value in the image is limited to be no more than a threshold value.

17. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform at least the steps of the method of claim 9.

18. A system for correcting an error in an image comprising pixels, the system comprising:

a measurement vector generator, coupled to receive an image, that constructs a measurement vector of the image based upon at least some characteristics of the image;

an image error corrector, coupled to receive the measurement vector, that computes a mapped low dimensional vector from the measurement vector by computing a low dimensional vector from the measurement vector and mapping the low dimensional vector onto a manifold comprising a plurality of low dimensional vectors from a plurality of images; and

a display image generator, coupled to receive a mapped low dimensional vector, that uses the mapped low dimensional vector to generate a display image.

19. The system of claim 18 further comprising a manifold builder, coupled to receive a low dimensional vector, that updates the manifold by:

adding a low dimensional vector to the manifold; and

responsive to satisfaction of at least one update criterion, updating a model of the manifold.

20. The system of claim 18 wherein the display image generator adjusts the display image by using a noise-determined threshold to adjust characteristics of pixels within the image that were not corrected in the display image.