Image coding apparatus, image decoding apparatus and image processing system

• US 7,734,104 B2
• Filed: 09/01/2006
• Issued: 06/08/2010
• Est. Priority Date: 11/30/2005
• Status: Active Grant

• Current Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Yamane, Kenji, Kohno, Michinari
• Primary Examiner(s): Couso; Jose L
• Application Number:
  US11/515,367
• Publication Number:
  US 20070121720A1
• Time in Patent Office: 1,376 Days
• Field of Search: 382/190, 382/195, 382/224, 382232-233, 382/236, 382/239, 382/305, 348552-553, 375/240.01, 375/240.25, 725/143, 725/153
• US Class Current: 382/232
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

An importance level determination unit receives: input from a camera of frames of moving images taken; input from a switch connected to a server which is operated manually by hand; and voice input from a microphone. Based on these it determines the importance level of each image frame. An encoder compresses and encodes the image frames obtained by the camera and generates moving picture streams. A packetizer packetizes the moving picture streams and gives them to a tag attachment unit. The tag attachment unit attaches the importance level set for the respective image frames to packet headers of the moving picture streams as a priority tag. A transmission unit transmits packets with a priority tag to a network which includes a router with a packet priority control function.

Claims

1. An image coding apparatus comprising:

an importance level determination unit which determines, by frame, an importance level of a moving image of a person taken with a camera based on an amount of a characteristic obtained from a facial image of the person;

an encoder which encodes frames of the moving image;

a packetizer which transforms the frames, which have been encoded, into packets;

a tag attachment unit which attaches a priority tag reflecting the importance level to the packet header; and

a transmission unit which transmits packets with the priority tag to a network which includes a router with a packet priority control function.

2. The image coding apparatus according to claim 1, wherein

the importance level determination unit, by also judging the person'"'"'s voice level as an amount of a characteristic, determines, by frame, an importance level of a moving image of the person.

3. The image coding apparatus according to claim 1, wherein

the importance level determination unit, when the person manually conducts a specified operation, increases the importance level of the frames captured while the operation is being conducted.

4. The image coding apparatus according to claim 2, wherein

the importance level determination unit, when the person manually conducts a specified operation, increases the importance level of the frames captured while the operation is being conducted.

5. An image decoding apparatus comprising:

a receiving unit which receives multiple moving picture streams that have been divided into packets with a priority tag attached to each header from a network which includes a router with a packet priority control function;

a priority determination unit which obtains the priority tag from each header of multiple packets received during a specified period and determines the moving picture stream whose packets have the highest priority tag received during the specified period to be a priority moving picture stream;

a decoder which decodes each of the multiple moving picture streams and regenerates multiple moving images; and

a synthesis unit which synthesizes the multiple moving images so that a moving image regenerated from the priority moving picture stream is highlighted when it is displayed.

6. The image decoding apparatus according to claim 5, wherein the decoder includes multiple threads which are each an executable unit for doing parallel decoding processing of each of the multiple moving picture streams;

the multiple threads are given priority scheduling and receive computing resources; and

a moving picture stream, whose packets have a high-priority tag, is assigned to a high-priority thread and undergoes decoding processing.

7. The image decoding apparatus according to claim 6, wherein

in the event decoding processing of each moving image to be regenerated from each moving picture stream cannot be completed in time for playback, due to a shortage of the computing resources which need to be allocated for decoding processing of the multiple moving picture streams, scheduling is done so that the thread assigned to decoding processing of a moving picture stream, whose packets have a low-priority tag, is suspended and computing resources allocated to the thread are released.

8. An image processing system in which an image decoding apparatus is connected to multiple image coding apparatuses via a network, wherein each of the multiple image coding apparatuses comprises:

an importance level determination unit which determines, by frame, an importance level of a moving image taken with a camera;

an encoder which encodes frames of the moving image;

a packetizer which transforms the frames, which have been encoded, into packets;

a tag attachment unit which attaches a priority tag reflecting the importance level to the packet header; and

a transmission unit, which transmits packets with the priority tag to a network which includes a router with a packet priority control function, and

the image decoding apparatus comprises;

a receiving unit which receives moving picture streams that have been divided into packets with the priority tag attached to each header from each of the multiple image coding apparatuses via the network;

a priority determination unit which obtains the priority tag from each header of multiple packets received during a specified period and determines the moving picture stream whose packets have the highest priority tag received during the specified period to be a priority moving picture stream;

a decoder which decodes each of the moving picture streams from the multiple image coding apparatuses and regenerates multiple moving images; and

a synthesis unit which synthesizes the multiple moving images so that a moving image regenerated from the priority moving picture stream is highlighted when it is displayed.

9. The image processing system according to claim 8, wherein

the importance level determination unit which determines, by frame, an importance level of a moving image of a person taken with a camera based on an amount of a characteristic obtained from a facial image of the person.

10. The image processing system according to claim 9, wherein

the importance level determination unit, by also judging the person'"'"'s voice level as an amount of characteristic, determines, by frame, an importance level of a moving image of the person.

11. The image processing system according to claim 9, wherein

the importance level determination unit, when the person manually conducts a specified operation, increases the importance level of the frames captured while the operation is being conducted.

12. The image processing system according to claim 8, wherein

the decoder includes multiple threads which are each an executable unit for doing parallel decoding processing of each of the moving picture streams from the multiple image coding apparatuses;

the multiple threads are given priority scheduling and receive computing resources; and

a moving picture stream, whose packets have a high-priority tag, is assigned to a high-priority thread and undergoes decoding processing.

13. The image processing system according to claim 12, wherein

in the event decoding processing of each moving image to be regenerated from each moving picture stream cannot be completed in time for playback, due to a shortage of the computing resources which need to be allocated for decoding processing of the multiple moving picture streams from the multiple image coding apparatuses, scheduling is done so that the thread assigned to decoding processing of a moving picture stream whose packets have a low-priority tag is suspended and computing resources allocated to the thread are released.

14. A computer-readable recording medium having stored therein a set of instructions that when executed, cause a computer to perform:

receiving multiple moving picture streams that have been divided into packets with a priority tag attached to each header from a network which includes a router with a packet priority control function;

obtaining the priority tag from each header of multiple packets received during a specified period and determining the moving picture stream whose packets have the highest priority tag received during the specified period to be a priority moving picture stream;

decoding each of the multiple moving picture streams and regenerating multiple moving images; and

synthesizing the multiple moving images so that a moving image regenerated from the priority moving picture stream is highlighted when it is displayed.