Image encoding apparatus, image encoding method, and recording medium

• US 9,986,239 B2
• Filed: 01/27/2014
• Issued: 05/29/2018
• Est. Priority Date: 03/27/2013
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Takada, Jun
• Primary Examiner(s): Vo, Tung
• Assistant Examiner(s): Becker, Joseph
• Application Number:
  US14/780,114
• Publication Number:
  US 20160044311A1
• Time in Patent Office: 1,583 Days
• Field of Search: None
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks

Abstract

Provided is an image encoding apparatus capable of encoding a region important to a user with high quality. An update interval index calculation unit calculates an update interval index representing an index having a long update interval in an entire update tile overlapping the update region for each update region. An importance region specification unit specifies an important region, from each update region by using the update interval index. An quality control unit allocates a code amount, which is larger than a result obtained by dividing the total target code amount per one frame proportionally according to a ratio of an area of an entire important region with respect to an area of an entire update region, as a target code amount of the entire important region and specifies a quality value of the entire important region on the basis of the target code amount.

Claims

1. An image encoding apparatus comprising circuitry configured to:

store an update time of a tile which is a region obtained by dividing a frame of an image into a predetermined size;

detect an update tile, which is a tile in which update occurs between a latest frame and a frame before the latest frame, and an update region, which is a region where update occurs, when the latest frame of the image has been provided;

calculate an update interval index representing an index having a long update interval in an entire update tile overlapping the update region, for each update region;

specify an important region from each update region by using the update interval index;

calculate a proportional division result by dividing a total target code amount, per one frame, proportionally according to a ratio defined as an area of an entirety of the important region to an area of an entirety of the update region;

allocate a code amount, as an amount larger than the proportional division result, to be a target code amount of the entire important region, and specify a quality value of the entire important region on the basis of the target code amount; and

encode the entire important region with the quality value.

2. The image encoding apparatus according to claim 1, wherein the circuitry is further configured to:

update and hold a quality value of a tile when a quality value for at least a part of regions in the tile has been decided;

select a low quality tile that is a tile having a quality value equal to or less than a threshold value among tiles other than the update tile,

in the specifying an important region, calculate the importance of each update region by using the update interval index, specify an important region from each update region on the basis of the importance, and calculate the importance of each tile, other than the update tile, by using a passage time from an update time,

in the specifying a quality value, decide a quality value of an entire unimportant region when encoding the entire unimportant region, which is a region not related to an important region of the update regions, as an initial value of the quality value of the entire unimportant region by using a code amount obtained by excluding a code amount when encoding the entire important region from a total target code amount per one frame, sequentially reducing a quality value of the entire unimportant region from the initial value,

calculate a number of low quality tiles to be able to be subject to intra update, and

specify the quality value of the entire unimportant region when quality of an unimportant region and an entire low quality tile to be able to be subject to intra update is estimated to be maximum;

encode the entire unimportant region with the quality value; and

encode low quality tiles to be able to be subject to the intra update with a quality value for the intra update in an importance order.

3. The image encoding apparatus according to claim 2, wherein the circuitry is further configured to:

calculate character likelihood representing likelihood related to a display region of a character for each tile and decides character likelihood of each update region, and

wherein, in the specifying an important region, calculate the importance of each update region by using the update interval index and the character likelihood and calculates the importance of the tile by using the passage time from the update time and the character likelihood.

4. The image encoding apparatus according to claim 2, wherein the circuitry is further configured to:

calculate a pointer proximity, which is a reciprocal of a distance between a central coordinate of a tile and a predetermined coordinate at a time of update when the update has occurred in the tile within a constant time from an operation from an exterior to the predetermined coordinate, and

wherein, in the specifying an important region, calculate the importance of the tile by using the passage time from the update time and the pointer proximity.

5. The image encoding apparatus according to claim 3, wherein the circuitry is further configured to:

calculate a pointer proximity, which is a reciprocal of a distance between a central coordinate of a tile and a predetermined coordinate at a time of update when the update has occurred in the tile within a constant time from an operation from an exterior to the predetermined coordinate, and

wherein, in the specifying an important region, calculate the importance of the tile by using the passage time from the update time and the pointer proximity.

6. The image encoding apparatus according to claim 1, wherein, when the ratio of the area of the entire important region with respect to the area of the entire update region is equal to or more than a predetermined value, in the specifying an important region, the circuitry is further configured to decide the entire update region as an unimportant region.

7. The image encoding apparatus according to claim 2, wherein, when the ratio of the area of the entire important region with respect to the area of the entire update region is equal to or more than a predetermined value, in the specifying an important region, the circuitry is further configured to decide the entire update region as an unimportant region.

8. The image encoding apparatus according to claim 3, wherein, when the ratio of the area of the entire important region with respect to the area of the entire update region is equal to or more than a predetermined value, in the specifying an important region, the circuitry is further configured to decide the entire update region as an unimportant region.

9. The image encoding apparatus according to claim 4, wherein, when the ratio of the area of the entire important region with respect to the area of the entire update region is equal to or more than a predetermined value, in the specifying an important region, the circuitry is further configured to decide the entire update region as an unimportant region.

10. The image encoding apparatus according to claim 1, wherein, for each update region, in the calculating an update interval index, the circuitry is further configured to calculate, as the update interval index, a long update interval tile occupation rate, which is a rate of a number of tiles having an update interval equal to or more than a threshold value, among a number of update tiles overlapping the update region.

11. The image encoding apparatus according to claim 2, wherein, for each update region, in the calculating an update interval index, the circuitry is further configured to calculate as the update interval index, a long update interval tile occupation rate, which is a rate of a number of tiles having an update interval equal to or more than a threshold value, among a number of update tiles overlapping the update region.

12. The image encoding apparatus according to claim 3, wherein, for each update region, in the calculating an update interval index, the circuitry is further configured to calculate as the update interval index, a long update interval tile occupation rate, which is a rate of a number of tiles having an update interval equal to or more than a threshold value, among a number of update tiles overlapping the update region.

13. The image encoding apparatus according to claim 4, wherein, for each update region, in the calculating an update interval index, the circuitry is further configured to calculate as the update interval index, a long update interval tile occupation rate, which is a rate of a number of tiles having an update interval equal to or more than a threshold value, among a number of update tiles overlapping the update region.

14. The image encoding apparatus according to claim 5, wherein, for each update region, in the calculating an update interval index, the circuitry is further configured to calculate as the update interval index, a long update interval tile occupation rate, which is a rate of a number of tiles having an update interval equal to or more than a threshold value, among a number of update tiles overlapping the update region.

15. The image encoding apparatus according to claim 1, wherein, for each update region, in the calculating an update interval index, the circuitry is further configured to calculate as the update interval index, an average value of update intervals of each update tile overlapping the update region.

16. The image encoding apparatus according to claim 2, wherein, for each update region, in the calculating an update interval index, the circuitry is further configured to calculate as the update interval index, an average value of update intervals of each update tile overlapping the update region.

17. An image encoding method comprising the processing of:

detecting an update tile, which is a tile in which update occurs between a latest frame and a frame before the latest frame, and an update region, which is a region where update occurs, when the latest frame of an image has been provided;

storing an update time of an update tile at a time of detection of the update tile and storing an update time for each tile;

calculating an update interval index representing an index having a long update interval in an entire update tile overlapping the update region, for each update region;

specifying an important region from each update region by using the update interval index;

calculating a proportional division result by dividing a total target code amount, per one frame, proportionally according to a ratio defined as an area of an entirety of the important region to an area of an entirety of the update region;

allocating a code amount, as an amount larger than the proportional division result, to be a target code amount of the entire important region, and specifying a quality value of the entire important region on the basis of the target code amount; and

encoding the entire important region with the quality value.

18. The image encoding method according to claim 17, wherein the method further comprises:

calculating importance of each update region by using the update interval index and specifying an important region from each update region on the basis of the importance;

updating and holding a quality value of a tile when a quality value for at least a part of regions in the tile has been decided;

selecting a low quality tile that is a tile having a quality value equal to or less than a threshold value among tiles other than the update tile;

calculating importance of each tile, other than the update tile, by using a passage time from an update time;

deciding a quality value of an entire unimportant region when encoding the entire unimportant region, which is a region not related to an important region of the update regions, as an initial value of the quality value of the entire unimportant region by using a code amount obtained by excluding a code amount when encoding the entire important region from a total target code amount per one frame, sequentially reducing a quality value of the entire unimportant region from the initial value, calculating a number of low quality tiles to be able to be subject to intra update, and specifying the quality value of the entire unimportant region when quality of an unimportant region and an entire low quality tile to be able to be subject to intra update is estimated to be maximum;

encoding the entire unimportant region with the quality value; and

encoding low quality tiles to be able to be subject to the intra update with a quality value for the intra update in an importance order.

19. A non-transitory computer readable medium that stores therein an image encoding program, which is applied to a computer including a tile information storage means which stores an update time of a tile which is a region obtained by dividing a frame of an image into a predetermined size, causing the computer to execute processing of:

detecting an update tile, which is a tile in which update occurs between a latest frame and a frame before the latest frame, and an update region, which is a region where update occurs, when the latest frame of the image has been provided;

calculating an update interval index representing an index having a long update interval in an entire update tile overlapping the update region, for each update region;

specifying an important region from each update region by using the update interval index;

calculating a proportional division result by dividing a total target code amount, per one frame, proportionally according to a ratio defined as an area of an entirety of the important region to an area of an entirety of the update region;

allocating a code amount, as an amount larger than the proportional division result, to be a target code amount of the entire important region, and specifying a quality value of the entire important region on the basis of the target code amount; and

encoding the entire important region with the quality value.

20. The image encoding program according to claim 19, wherein the processing further comprises:

updating and holding a quality value of a tile when a quality value for at least a part of regions in the tile has been decided;

selecting a low quality tile that is a tile having a quality value equal to or less than a threshold value among tiles other than the update tile, the importance region specification process including calculating importance of each update region by using the update interval index, specifying an important region from each update region on the basis of the importance, and calculating importance of each tile, other than the update tile, by using a passage time from an update time, the quality control process including deciding a quality value of an entire unimportant region when encoding the entire unimportant region, which is a region not related to an important region of the update regions, as an initial value of the quality value of the entire unimportant region by using a code amount obtained by excluding a code amount when encoding the entire important region from a total target code amount per one frame, sequentially reducing a quality value of the entire unimportant region from the initial value, calculating a number of low quality tiles to be able to be subject to intra update, and specifying the quality value of the entire unimportant region when quality of an unimportant region and an entire low quality tile to be able to be subject to intra update is estimated to be maximum;

encoding the entire unimportant region with the quality value; and

encoding low quality tiles to be able to be subject to the intra update with a quality value for the intra update in an importance order.