Image processing method, image display system, and computer program

• US 9,123,132 B2
• Filed: 11/19/2008
• Issued: 09/01/2015
• Est. Priority Date: 11/22/2007
• Status: Expired due to Fees

• Current Assignee: Semiconductor Energy Laboratory Co., Ltd.
• Original Assignee: Semiconductor Energy Laboratory Co., Ltd.
• Inventors: Tamura, Hikaru
• Primary Examiner(s): Chu, Randolph I
• Assistant Examiner(s): BLOOM, NATHAN J
• Application Number:
  US12/273,735
• Publication Number:
  US 20090136158A1
• Time in Patent Office: 2,477 Days
• Field of Search: 382/100, 382/103, 382/107, 382/153, 382173-174, 382/181, 382/190, 382254-258, 382260-266, 382/269, 382274-275, 382/276, 382282-284, 348/207.99, 34820899-2086, 34820812-20815, 348/222.1, 345/418, 345/581, 345611-612, 345/617, 345/629, 345/634, 345/647
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/16 Determination of a pixel data signal depending on the signal applied in the previous frame
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 3/2092 Details of a display terminals using a flat panel, the details relating to the control arrangement of the display terminal and to the interfaces thereto suitable for both CRT and flat panel G09G5/003; specific for a CRT G09G1/165
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

A moving object display region where a moving object is displayed and a background display region are extracted based on a difference image of frame images. Blurring processing such as averaging processing is performed on an image in the background display region. Processing for further sharpening an image, such as contrast enhancement, is performed on an image in the moving object display region.

Claims

1. An image processing method for processing a moving image including a plurality of frame images, comprising the steps of:

extracting a moving object display region where a moving object is displayed from image at least one frame image;

determining whether the moving object display region accounts for more than a value of proportion of a target frame image; and

performing a first kind of image processing on a pixel included in the moving object display region and performing a second kind of image processing on a pixel which is not included in the moving object display region in the frame image to generate a correction frame image when the moving object display region accounts for less than a value of proportion of a frame image,

displaying the target frame image on an image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image,

wherein the first kind of image processing is different from the second kind of image processing,

wherein the second image processing comprises blurring processing, and

wherein at least one kind of image processing among contrast conversion processing for reducing contrast and mosaic processing is performed as the blurring processing.

3. The image processing method according to claim 1, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing performed on the pixel included in the moving object display region.

22. The image processing method according to claim 1, wherein the value of proportion of the target frame is in a range of 50%-70%.

2. An image processing method for processing a moving image including a plurality of frame images, comprising the steps of:

generating a first binary difference image based on a difference image between a target frame image to be subjected to correction and a frame image which is k frame or k frames before the target frame image where k is one of 1 and 2;

generating a second binary difference image based on a difference image between the target frame image and a frame image which is k frame or k frames after the target frame image;

extracting a moving object display region from a logical product of the first binary difference image and the second binary difference image to generate a mask image formed from the moving object display region and a background display region;

determining whether the moving object display region accounts for more than a value of proportion of the target frame image; and

performing a first kind of image processing on a pixel in the same position as a pixel included in the moving object display region and performing a second kind of image processing a pixel in the same position as a pixel included in the background display region in the target frame image to generate a correction frame image when the moving object display region accounts for less than the value of proportion of the frame image,

displaying the target frame image on an image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image,

wherein the first kind of image processing is different from the second kind of image processing,

wherein the second image processing includes blurring processing, and

wherein at least one kind of image processing among contrast conversion processing for reducing contrast and mosaic processing is performed as the blurring processing.

4. The image processing method according to claim 2, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing performed on the pixel in the same position as the pixel included in the moving object display region.

5. The image processing method according to claim 2 or claim 4, wherein the second kind of image processing comprises at least one of averaging processing, contrast conversion processing for reducing contrast, noise removal processing, and mosaic processing performed as the blurring processing.

23. The image processing method according to claim 2, wherein the value of proportion of the target frame is in a range of 50%-70%.

6. An image display system comprising:

an image display portion for displaying a moving image including a plurality of frame images;

a moving object region extraction portion for extracting a moving object display region where a moving object is displayed from frame image at least one frame image;

a processing portion for determining whether the moving object display region accounts for more than a value of proportion of a target frame image; and

a correction frame image generation portion for generating a correction frame image by performing a first kind of image processing on a pixel included in the moving object display region and performing a second kind of image processing on a pixel which is not included in the moving object display region in the frame image,

wherein the first kind of image processing is different from the second kind of image processing,

wherein the second image processing comprises blurring processing,

wherein the correction frame image is displayed on the image display portion when the moving object display region accounts for less than the value of proportion of the frame image, and

wherein the target frame image is displayed on the image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image.

8. The image display system according to claim 6, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing performed on the pixel included in the moving object display region.

10. The image display system according to any one of claims 6 to 9, wherein the second kind of image processing comprises at least one of averaging processing, contrast conversion processing for reducing contrast, noise removal processing, and mosaic processing performed as the blurring processing.

21. An electronic device comprising the image display system according to claim 6.

24. The image display system according to claim 6, wherein the value of proportion of the target frame is in a range of 50%-70%.

7. An image display system comprising:

an image display portion for displaying a moving image including a plurality of frame images;

a binary difference image generation portion for generating a first binary difference image based on a difference image between a target frame image to be subjected to correction and a frame image which is k frame or k frames before the target frame image and for generating a second binary difference image based on a difference image between the target frame image and a frame image which is k frame or k frames after the target frame image where k is one of 1 and 2;

a mask image generation portion for extracting a moving object display region from a logical product of the first binary difference image and the second binary difference image to generate a mask image formed from the moving object display region and a background display region;

a processing portion for determining whether the moving object display region accounts for more than a value of proportion of the target frame image; and

a correction frame image generation portion for performing a first kind of image processing on a pixel in the same position as a pixel included in the moving object display region and performing a second kind of image processing a pixel in the same position as a pixel included in the background display region in the target frame image to generate a correction frame image,

wherein the first kind of image processing is different from the second kind of image processing,

wherein the second kind of image processing comprises blurring processing,

wherein the correction frame image is displayed on the image display portion when the moving object display region accounts for less than the value of proportion of the frame image, and

wherein the target frame image is displayed on the image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image.

9. The image display system according to claim 7, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing performed on the pixel in the same position as the pixel included in the moving object display region.

25. The image display system according to claim 7, wherein the value of proportion of the target frame is in a range of 50%-70%.

11. An image processing device for a moving image, comprising:

a data storage portion for storing a computer program,

a moving object region extraction portion for extracting a moving object display region where a moving object is displayed from a frame image forming a moving image;

a processing portion for determining whether the moving object display region accounts for more than a value of proportion of a target frame image; and

a correction frame image generation portion for performing a first kind of image processing on a pixel included in the moving object display region and performing a second kind of image processing on a pixel which is not included in the moving object display region in the frame image to generate a correction frame image when the moving object display region accounts for less than the value of proportion of the frame image,

wherein the first kind of image processing is different from the second kind of image processing,

wherein the second kind of image processing comprises blurring processing,

wherein at least one kind of image processing among contrast conversion processing for reducing contrast and mosaic processing is performed as the blurring processing, and

wherein the target frame image is displayed on an image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image.

13. The image processing device according to claim 11, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing performed on the pixel included in the moving object display region.

26. The image processing device according to claim 11, wherein the value of proportion of the target frame is in a range of 50%-70%.

12. An image processing device for a moving image, comprising:

a data storage portion for storing a computer program,

a binary difference image generation portion for generating a first binary difference image based on a difference image between a target frame image to be subjected to correction among a plurality of target frame images forming a moving image and a frame image which is k frame or k frames before the target frame image where k is one of 1 and 2, and for generating a second binary difference image based on a difference image between the target frame image and a frame image which is k frame or k frames after the target frame image;

a mask image generation portion for extracting a moving object display region from a logical product of the first binary difference image and the second binary difference image to generate a mask image formed from the moving object display region and a background display region;

a processing portion for determining whether the moving object display region accounts for more than a value of proportion of the target frame image; and

a correction frame image generation portion for performing a first kind of image processing on a pixel in the same position as a pixel included in the moving object display region and performing a second kind of image processing on a pixel in the same position as a pixel included in the background display region in the target frame image to generate a correction frame image when the moving object display region accounts for less than the value of proportion of the frame image,

wherein the first kind of image processing is different from the second kind of image processing,

wherein the second kind of image processing comprises blurring processing,

wherein at least one kind of image processing among contrast conversion processing for reducing contrast and mosaic processing is performed as the blurring processing, and

wherein the target frame image is displayed on an image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image.

14. The image processing device according to claim 12, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing performed on the pixel in the same position as the pixel included in the moving object display region.

27. The image processing device according to claim 12, wherein the value of proportion of the target frame is in a range of 50%-70%.

15. A computer-readable non-transitory memory device storing a computer program for implementation by an image processing device for processing a moving image, the computer program comprising code segments configured to cause the image processing device to:

extract from a frame image forming a moving image a moving object display region in which a moving object is displayed;

determine whether the moving object display region accounts for more than a value of proportion of a target frame image; and

perform a first kind of image processing on a pixel included in the moving object display region and perform a second kind of image processing on a pixel which is not included in the moving object display region to generate a correction frame image when the moving object display region accounts for less than the value of proportion of the frame image,

displaying the target frame image on an image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image,

wherein the second kind of image processing comprises blurring processing.

17. The computer readable non-transitory memory device according to claim 15, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing.

19. The computer readable non-transitory memory device according to any one of claims 15 to 18, wherein the blurring processing comprises at least one of averaging processing, contrast conversion processing for reducing contrast, noise removal processing, and mosaic processing.

28. The computer readable non-transitory memory device according to claim 15, wherein the value of proportion of the target frame is in a range of 50%-70%.

16. A computer-readable non-transitory memory device storing a computer program for implementation by an image processing device processing for a moving image, the computer program comprising code segments configured to cause the image processing device to:

generate a first binary difference image based on a difference image between a target frame image to be subjected to correction among a plurality of target frame images forming a moving image and a frame image which is k frame(s) before the target frame image, wherein k is one of 1 and 2;

generate a second binary difference image based on a difference image between the target frame image and a frame image which is k frame(s) after the target frame image;

extract a moving object display region from a logical product of the first binary difference image and the second binary difference image to generate a mask image formed from the moving object display region and a background display region;

determine whether the moving object display region accounts for more than a value of proportion of the target frame image; and

perform a first kind of image processing on a pixel in the same position as a pixel included in the moving object display region and performing a second kind of image processing on a pixel in the same position as a pixel included in the background display region in the target frame image to generate a correction frame image when the moving object display region accounts for less than the value of proportion of the frame image,

displaying the target frame image on an image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image,

wherein the second kind of image processing comprises blurring processing.

18. The computer readable non-transitory memory device according to claim 16, wherein the first kind of image processing comprises at least one of sharpening processing, contrast enhancement processing, and edge enhancement processing.

29. The computer readable non-transitory memory device according to claim 16, wherein the value of proportion of the target frame is in a range of 50%-70%.

20. An image processing method for processing a moving image including a plurality of frame images, comprising the steps of:

extracting a moving object display region where a moving object is displayed from at least one frame image to generate a mask image formed from the moving object display region and a background display region;

determining whether the moving object display region accounts for more than a value of proportion of a target frame image; and

performing a first kind of image processing on a pixel included in the moving object display region and performing a second kind of image processing a pixel which is not included in the moving object display region in the frame image to generate a correction frame image when the moving object display region accounts for less than a value of proportion of a frame image,

displaying the target frame image on an image display portion without forming the correction frame image when the moving object display region accounts for more than the value of proportion of the target frame image,

wherein the second kind of image processing comprises blurring processing,

wherein at least one kind of image processing among contrast conversion processing for reducing contrast, and mosaic processing is performed as the blurring processing.

30. The image processing method according to claim 20, wherein the value of proportion of the target frame is in a range of 50%-70%.