Dynamic image digest automatic editing system and dynamic image digest automatic editing method

• US 7,880,936 B2
• Filed: 03/27/2006
• Issued: 02/01/2011
• Est. Priority Date: 02/15/1999
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Shiiyama, Hirotaka
• Primary Examiner(s): Zimmerman; Mark K
• Assistant Examiner(s): Tran; Dung D
• Application Number:
  US11/389,202
• Publication Number:
  US 20060164702A1
• Time in Patent Office: 1,772 Days
• Field of Search: 358 11- 113, 358/537, 358/452, 382/232, 348 141- 1416, 348/135, 348/218.1, 348/700
• US Class Current: 358/452
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612

Abstract

An image processing section 10 calculates degree of similarity among a plurality of image frames of dynamic image data, determines scene-change frames based on the calculated degree of similarity, and performs automatic editing and preparation of a digest dynamic image of the dynamic image data by merging a specified duration of frames, giving priority to scenes whose scene-change frame has a low degree of similarity to an immediately preceding frame or some preceding frames, or by merging, from scenes delimited by a scene change, a specified duration of frames having a degree of similarity in keeping with the user'"'"'s indication of whether or not a “little change digest” or a “much change digest” is to be prepared.

Claims

1. An image processing system comprising:

a calculating unit configured to calculate a degree of similarity from among a plurality of image frames of image data;

a determining unit configured to determine scene-change frames based on the degree of similarity calculated by said calculating unit;

a digest image preparation unit configured to perform automatic editing and preparation of a digest image of the image data, by merging a specified duration of frames having a low degree of similarity with an immediately preceding frame or some preceding frames;

a reception unit configured to receive, from a user, an instruction of an entire time of the digest image to be prepared by merging the duration of frames, and

a time calculation unit configured to calculate a total time of the durations of frames to be merged before completion of merging the frames;

wherein said digest image preparing unit is further configured to preferentially merge the duration of frames having a lower degree of similarity and to adjust the number of the duration of frames to be merged, in order to automatically prepare the digest image having the entire time specified in the instruction received by said reception unit by merging the durations of frames in the order of the degree of similarity from the duration having the lowest degree of similarity, until the total time calculated for the merging result becomes substantially the same as the entire time.

2. The image processing system according to claim 1, further comprising:

a detecting unit configured to detect blank scenes; and

an exception processing unit configured to perform exception processing in which an initial image frame after exclusion of a blank scene detected at the beginning of the digest image by said detecting unit is deemed a scene-change frame.

3. The image processing system according to claim 2, wherein said exception processing unit also performs exception processing in which a final image frame after exclusion of a blank scene detected at the end of the digest image by said detecting unit is deemed a scene-change frame.

4. The image processing system according to claim 3, wherein when a time duration for each scene-change frame of two scenes in close proximity is less than the specified duration, frame information from a scene-change frame of the first scene and frame information through a frame from the scene-change frame of the second scene are treated as the result of merging of the scene-change frames of the two scenes into one scene whose duration is equal to the specified duration.

5. The image processing system according to claim 4, wherein when the duration of the scene-change frame of the second scene to be merged falls within a specified duration of the scene-change frame of the first scene, all the frame information of the scene-change frame of the second scene are merged with the scene-change frame of the first scene.

6. The image processing system according to claim 1, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

7. The image processing system according to claim 2, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

8. The image processing system according to claim 3, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

9. The image processing system according to claim 4, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

10. The image processing system according to claim 5, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

11. The image processing system according to claim 1, wherein said image preparation unit is further configured to perform automatic editing and preparation of a digest image of the image data by merging a specified duration of frames having a high degree of similarity with an immediately preceding frame or some preceding frames, on receipt of instructions to prepare a second type of digest image.

12. An image processing system comprising:

a calculating unit configured to calculate a degree of similarity from among a plurality of image frames of image data;

a determining unit configured to determine scene-change frames based on the degree of similarity calculated by said calculating unit;

a digest image preparation unit configured to perform automatic editing and preparation of a digest image of the image data, by merging a specified duration of frames having a high degree of similarity with an immediately preceding frame or some preceding frames;

a reception unit configured to receive, from a user, an instruction of an entire time of the digest image to be prepared by merging the duration of frames, and

time calculation unit configured to calculate a total time of the durations of frames to be merged before completion of merging the frames;

wherein said digest image preparing unit is further configured to preferentially merge the duration of frames having a higher degree of similarity and to adjust the number of the duration of frames to be merged, in order to automatically prepare the digest image having the entire time specified in the instruction by said reception unit, by merging the durations of frames in the order of the degree of similarity from the duration having the highest degree of similarity, until the total time calculated for the merging result becomes substantially the same as the entire time.

13. The image processing system according to claim 12, further comprising:

a detecting unit configured to detect blank scenes; and

an exception processing unit configured to perform exception processing in which an initial image frame after exclusion of a blank scene detected at the beginning of the digest image by said detecting unit is deemed a scene-change frame.

14. The image processing system according to claim 13, wherein said exception processing unit also performs exception processing in which a final image frame after exclusion of a blank scene detected at the end of the digest image by said detecting unit is deemed a scene-change frame.

15. The image processing system according to claim 14, wherein when a time duration for each scene-change frame of two scenes in close proximity is less than the specified duration, frame information from a scene-change frame of the first scene and frame information through a frame from the scene-change frame of the second scene are treated as the result of merging of the scene-change frames of the two scenes into one scene whose duration is equal to the specified duration.

16. The image processing system according to claim 15, wherein when the duration of the scene-change frame of the second scene to be merged falls within a specified duration of the scene-change frame of the first scene, all the frame information of the scene-change frame of the second scene are merged with the scene change frame of the first scene.

17. The image processing system according to claim 12, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

18. The image processing system according to claim 13, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

19. The image processing system according to claim 14, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

20. The image processing system according to claim 15, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

21. The image processing system according to claim 16, wherein a user is able to select and specify whether to save a digest as a file, or replay the digest and discard the data.

22. An image processing method comprising the steps of:

calculating a degree of similarity from among a plurality of image frames of image data;

determining scene-change frames based on the calculated degree of similarity;

performing automatic editing and preparation of a digest image of the image data, by merging from each scene delimited by a scene change a specified duration of frames having a low degree of similarity with an immediately preceding frame or some preceding frames;

receiving, from a user, an instruction of an entire time of the digest image to be prepared by merging the duration of frames, and

calculating a total time of the durations of frames to be merged before completion of merging the frames;

wherein, in said performing step, the duration of frames having a lower degree of similarity is preferentially merged and the number of the duration of frames to be merged is adjusted in order to automatically prepare the digest image having the entire time specified in the instruction received in said receiving step, by merging the durations of frames in the order of the degree of similarity from the duration having the lowest degree of similarity, until the total time calculated for the merging result becomes substantially the same as the entire time.

23. The image processing method according to claim 22, further comprising the steps of:

detecting blank scenes; and

performing exception processing in which an initial image frame after exclusion of a blank scene detected at the beginning of the digest image in said detecting step is deemed a scene-change frame.

24. The image processing method according to claim 23, further comprising the step of:

performing exception processing in which a final image frame after exclusion of a blank scene detected at the end of the digest image in said detecting step is deemed a scene-change frame.

25. The image processing method according to claim 24, wherein when a time duration for each scene-change frame of two scenes in close proximity is less than the specified duration, frame information from a scene-change frame of the first scene and frame information through a frame from the scene-change frame of the second scene are treated as the result of merging of the scene-change frames of the two scenes into one scene whose duration is equal to the specified duration.

26. The image processing method according to claim 25, wherein when the duration of the scene-change frame of the second scene to be merged falls within a specified duration of the scene-change frame of the first scene, all the frame information of the scene-change frame of the second scene are merged with the scene-change frame of the first scene.

27. The image processing method according to claim 22, wherein, in said performing step, automatic editing and preparation of a digest image of the image data are performed by merging a specified duration of frames having a high degree of similarity with an immediately preceding frame or some preceding frames, on receipt of instructions to prepare a second type of digest image.

28. An image processing method comprising the steps of:

calculating a degree of similarity from among a plurality of image frames of image data;

determining scene-change frames based on the calculated degree of similarity;

performing automatic editing and preparation of a digest image of the image data, by merging from each scene delimited by a scene change a specified duration of frames having a high degree of similarity with an immediately preceding frame or some preceding frames

receiving, from a user, an instruction of an entire time for the digest image to be prepared by merging the duration of frames; and

calculating a total time of the durations of frames to be merged before completion of merging the frames;

wherein, in said performing step, the duration of frames having a higher degree of similarity is preferentially merged and the number of the duration of frames to be merged is adjusted in order to automatically prepare the digest image having the entire time specified in the instruction received by said reception step, by merging the durations of frames in the order of the degree of similarity from the duration having the highest degree of similarity, until the total time calculated for the merging result becomes substantially the same as the entire time.

29. The image processing method according to claim 28, further comprising the steps of:

detecting blank scenes; and

performing exception processing in which an initial image frame after exclusion of a blank scene detected at the beginning of the digest image in said detecting step is deemed a scene-change frame.

30. The image processing method according to claim 29, further comprising the step of:

performing exception processing in which a final image frame after exclusion of a blank scene detected at the end of the digest image in said detecting step is deemed a scene-change frame.

31. The image processing method according to claim 30, wherein when a duration for each between scene-change frame of two scenes in close proximity is less than the specified duration, frame information from a scene-change frame of the first scene and frame information through a frame from the scene-change frame of the second scene are treated as the result of merging of the scene-change frames of the two scenes into one scene whose duration is equal to the specified duration.

32. The image processing method according to claim 31, wherein when the duration of the scene-change frame of the second scene to be merged falls within a specified duration of the scene-change frame of the first scene, all the frame information of the scene-change frame of the second scene are merged with the scene-change frame of the first scene.

33. A non-transitory computer-readable recording medium recording program code of an image processing method comprising the steps of:

calculating a degree of similarity from among a plurality of image frames of image data;

determining scene-change frames based on the calculated degree of similarity;

performing automatic editing and preparation of a digest image of the image data, by merging from each scene delimited by a scene change a specified duration of frames having a low degree of similarity with an immediately preceding frame or some preceding frames;

receiving, from a user, an instruction of an entire time of the digest image to be prepared by merging the duration of frames, and

calculating a total time of the durations of frames to be merged before completion of merging the frames;

wherein, in said performing step, the duration of frames having a lower degree of similarity is preferentially merged and the number of the duration of frames to be merged is adjusted in order to automatically prepare the digest image having the entire time specified in the instruction received in said receiving step, by merging the durations of frames in the order of the degree of similarity from the duration having the lowest degree of similarity, until the total time calculated for the merging result becomes substantially the same as the entire time.

34. The computer-readable recording medium according to claim 33, wherein, in said performing step, automatic editing and preparation of a digest image of the image data are performed by merging a specified duration of frames having a high degree of similarity with an immediately preceding frame or some preceding frames, on receipt of instructions to prepare a second type of digest image.

35. A non-transitory computer-readable recording medium recording program code of an image processing method comprising the steps of:

calculating a degree of similarity from among a plurality of image frames of image data;

determining scene-change frames based on the calculated degree of similarity;

performing automatic editing and preparation of a digest image of the image data, by merging from each scene delimited by a scene change a specified duration of frames having a high degree of similarity with an immediately preceding frame or some preceding frames;

receiving, from a user, an instruction of an entire time of the digest image to be prepared by merging the duration of frames, and

calculating a total time of the durations of frames to be merged before completion of merging the frames;

wherein, in said performing step, the duration of frames having a higher degree of similarity is preferentially merged and the number of the duration of frames to be merged is adjusted in order to automatically prepare the digest image having the entire time specified in the instruction received in said receiving step by merging the durations of frames in the order of the degree of similarity from the duration having the highest degree of similarity, until the total time calculated for the merging result becomes substantially the same as the entire time.