Business method for selectable semantic codec pairs for very low data-rate video transmission

• US 8,126,276 B2
• Filed: 02/21/2001
• Issued: 02/28/2012
• Est. Priority Date: 02/21/2001
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bolle, Rudolf M., Connell, Jonathan
• Primary Examiner(s): Borissov, Igor
• Application Number:
  US09/789,957
• Publication Number:
  US 20020114525A1
• Time in Patent Office: 4,024 Days
• Field of Search: 705/1, 705/2, 705/7, 705/11, 705/13, 705/26, 705/27, 705/500, 725/9, 725/10, 725/12, 725/21, 725/105, 725/110, 340/539.18, 340/539.17, 340/539.16, 340/539.25, 340/539.22, 348/143, 348/169, 370/260, 382/232, 382/233, 375/E7.137, 375/E7.132, 375/E7.126, 375/E7.026, 375/E7.152, 375/E7.172, 375/E7.083, 375/E7.076
• US Class Current: 382/232
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals

Abstract

A business method for using computer image processing for selectable task-based digital video compression is described. The method is intended to reduce travel of experts and let these experts direct field agents in performing tasks remotely. The tasks to be performed in the remote field can be monitoring, manipulating, and navigating. A field agent performs the manipulation and navigation operations, this agent can be a human being or a robot. The task-based compression algorithms use computer vision techniques to extract the bare minimum amount of information from the remote field scenery to allow the task to be performed. High frame rate photo-realistic reconstruction of the remote scene is not generally necessary.

Claims

1. A business method for providing advice to a remote location, the method comprising the steps of:

inputting one or more video streams at a field location for transmission of image data, obtained from a sensor, to an office location, the image data including a pictorial representation of at least one object in an image viewed by the sensor;

selecting, at an office location, a selected compression process out of a set of different compression processes for compressing video of said one or more video streams, the selecting being based on a task, each of said different compression processes serving to modify a video representation of said at least one object by implementation of a symbolic representation of the object, wherein the symbolic representation of the object has less data points to be transmitted than the number of data points in the pictorial representation of the object,

compressing one or more of the video streams at the field location using the selected compression process to obtain a compressed video;

transmitting the compressed video from the field location to the office location, wherein the office location uses the compressed video to provide advice to the field location for performing the task;

transmitting, to the field location, an identification of at least one object related to the task;

automatically detecting and tracking said at least one object based on attributes of the object, wherein during said tracking, said selecting step is repeated for use of different compression processes to enable transmission of images of reduced detail among images of higher resolution in said transmitting of the compressed video;

transmitting attributes of said at least one object to the office location; and

controlling manipulation of the object by a field agent at the field location to accomplish the task, wherein the controlling is accomplished remotely with the aid of communication from the office location to the field location;

wherein, in said selecting of the compression process, a viewer of images received at the office location from the field location adjusts the compressions of images from said sensor to be transmitted in said transmitting of the compressed video to provide for reduced compression wherein a relatively high resolution image is required and to provide for increased compression wherein a relatively low resolution image is acceptable, thereby enabling said transmitting of the compressed video to provide more high resolution imaging over a communication link of reduced bandwidth.

2. A business method, as in claim 1, further comprising the steps of:

sending advice from the office location to the field location after viewing the compressed video at the office location; and

wherein, in the step of compressing one or more of the video streams, the compression provides for a semantic compression of the one or more of the video streams for reducing data of an image to be transmitted in the step of transmitting the compressed video from the field location to the office location.

3. A business method, as in claim 1, where the office location causes the field location to change the selected compression process one or more times during the task.

4. A business method, as in claim 1, where the set of compression processes provides semantic compression processes, and the set includes one or more of the following compression processes:

selective edge retention, line finding, motion segmentation, patch tracking, template matching, object tracking, spatially varying resolution, MPEG-1, MPEG-2, MPEG-4, M-JPEG, M-JBIG, H.261, H.323, and region segmentation.

5. A business method, as in claim 1, where a user in the office location has greater expertise in the task than a user in the field location.

6. A business method, as in claim 1, where the task includes any one or more of the following:

diagnostics with an expert at a remote site, an observation, a scene inspection, a scene surveillance, a navigation, an objection localization, a control, a maintenance procedure, and a diagnostic procedure.

7. A business method, as in claim 1, where a user in the office location can not personally access the field location during the performance of the task.

8. A business method, as in claim 7, where the user in the office location can not personally access the field location for one or more of the following reasons:

a weather condition, a hazard, a geographic inaccessibility, a time constraint, a political barrier, a personal constraint, a physical constraint, a medical constraint, and an economic constraint.

9. The business method of claim 1 further comprising the selected compression process having a compression ratio greater than 100:

1.

10. The business method of claim 1 wherein the set of compression processes comprises at least one of selective edge retention, line finding, motion segmentation, patch tracking, template matching, object tracking, spatially varying resolution and region segmentation.

11. The business method of claim 1 wherein using the selected compression process to obtain a compressed video further comprises representing only parts of a scene at the field location.

12. The business method of claim 1 further comprising the selected compression process transmitting only a portion of a scene information that is relevant to the task.

13. The business method of claim 1 further comprising selecting another compression process out of the set of compression processes that will return a higher resolution image of a selected portion of a scene at the field location from the compressed video and using the another compression process to obtain a second compressed video with the higher resolution image of the scene.

20. The method of claim 1 further comprising transmitting tracking information of the at least one object.

21. The method of claim 1 further comprising specifying a class of objects for which a detection system is available.

22. The method of claim 21 wherein the class of objects is flesh tone.

23. The business method of claim 1, wherein the set of compression processes includes a compression process providing a compression of the at least one object in the image without a compression of all objects in the image.

24. The business method of claim 1, wherein the set of compression processes includes a compression process providing a deletion of a portion of the image without a deletion of another portion of the image.

25. The business method of claim 1, wherein the set of compression processes includes a compression process providing a depiction of an edge with deletion of texture of a high-textured region of the at least one object in the image.

26. The business method of claim 1, wherein the set of compression processes includes a compression process providing a representation of a boundary with deletion of other aspects of the at least one object in the image.

31. The business method of claim 1, wherein the set of compression processes includes a compression process providing a sampling grid with varying pixel sizes for viewing the at least one object in the image, the viewing via the sampling grid providing for a relatively high resolution at said object and for a relatively low resolution in a region of the image distant from said object.

14. A method of using task-based compression in tele-operations comprising:

selecting, from an office site, at least one compression process from a set of different compression processes operative for compression of video to be transmitted in a video stream, the selected compression process being adapted to transmit only that portion of a scene information from a field site that is relevant to a particular task, the scene information including a pictorial representation of at least one object in a scene at the field site, each of said different compression processes serving to modify a video representation of the at least one object by implementation of a symbolic representation of the object, wherein the symbolic representation of the object has less data points to be transmitted than the number of data points in the pictorial representation of the object;

compressing a signal comprising scene information obtained from a sensor by using the at least one selected compression process at the field site;

transmitting the compressed signal to the office site;

decompressing the compressed signal at the office site;

identifying at least one object to be tracked;

detecting said at least one object;

tracking said at least one object, wherein during said tracking, said selecting is repeated for use of different compression processes to enable transmission of images of reduced detail among images of higher resolution in said transmitting of the compressed signal;

transmitting detection attributes of said at least one tracked object to the office site; and

controlling manipulation of said at least one object by a field agent at the field site to accomplish the task, wherein the controlling is accomplished remotely with the aid of communication from the office site to the field site;

wherein, in said selecting of the compression process, a viewer of images received at the office site from the field site adjusts the compressions of images from said sensor to be transmitted in said transmitting of the compressed signal to provide for reduced compression wherein a relatively high resolution image is required and to provide for increased compression wherein a relatively low resolution image is acceptable, thereby enabling said transmitting of the compressed signal to provide more high resolution imaging over a communication link of reduced bandwidth.

15. The method of claim 14 further comprising:

displaying the decompressed signal at the office site;

selecting another portion of the scene from the displayed decompressed signal that is desired to be viewed;

selecting, at the office site, at least one other compression process that will return a higher resolution image of the another portion of the scene desired to be viewed;

transmitting the at least one other compression process to the field sight;

acquiring, at the field site, another signal of the portion of the scene desired to be viewed;

compressing the another signal using the at least one other selected compression process;

transmitting the another compressed signal to the office site; and

decompressing the another compressed signal at the office site.

16. The method of claim 14 further comprising selecting two or more compression processes from the set of compression process to be used to acquire and compress the signal with the scene information from the field site, and displaying different compressed signals at the office site on at least one output device.

17. The method of claim 14 wherein the set of compression processes includes semantic compression processes, and the set comprises at least one of selective edge retention, line finding, motion segmentation, patch tracking, template matching, object tracking, spatially varying resolution and region segmentation.

18. The method of claim 14 further comprising the selected compression process having a compression ratio greater than 100:

1.

19. The method of claim 14 wherein using the at least one selected compression process to compress the signal further comprises representing only parts of a scene at the field location in the compressed signal.

27. The method of claim 14, wherein the set of compression processes includes a compression process providing a compression of the at least one object in an image of the scene at the field site without a compression of all objects in said image.

28. The method of claim 14, wherein the set of compression processes includes a compression process providing a deletion of a portion of an image of the scene at the field site without a deletion of another portion of said image.

29. The method of claim 14, wherein the set of compression processes includes a compression process providing a depiction of an edge with deletion of a high-textured region of the at least one object in an image of the scene at the field site.

30. The method of claim 14, wherein the set of compression processes includes a compression process providing a representation of a boundary with deletion of other aspects of the at least one object in an image of the scene at the field site.

32. The method of claim 14, wherein the set of compression processes includes a compression process providing a sampling grid with varying pixel sizes for viewing the at least one object in an image of the scene at the field site, the viewing via the sampling grid providing for a relatively high resolution at said object and for a relatively low resolution in a region of the image distant from said object.