IMAGE PROCESSING DEVICE AND IMAGE PROCESSING METHOD

• US 20100158471A1
• Filed: 04/23/2007
• Published: 06/24/2010
• Est. Priority Date: 04/24/2006
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ogikubo, Junichi
• Granted Patent: US 8,780,756 B2
• Time in Patent Office: Days
• US Class Current: 386/278
• CPC Class Codes: G06F 16/743 a collection of video files or sequences
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G11B 27/032 on tapes G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/323 Time code signal, e.g. on a cue track as SMPTE- or EBU-time code
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 1/00442 Simultaneous viewing of a plurality of images, e.g. using a mosaic display arrangement of thumbnails
• CPC Class Codes: H04N 1/00453 arranged in a two dimensional array
• CPC Class Codes: H04N 1/32112 in a separate computer file, document page or paper sheet, e.g. a fax cover sheet H04N1/32122 takes precedence
• CPC Class Codes: H04N 1/32122 in a separate device, e.g. in a memory or on a display separate from image data
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4438 Window management, e.g. event handling following interaction with the user interface
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 7/0122 the input and the output signals having different aspect ratios
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8063 : using time division multiplex of the PCM audio and PCM video signals
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal

Abstract

The present invention has been made to facilitate search or management of video data. The present invention acquires feature parameters representing the feature of video data or audio data or associated parameters linked to the video data or audio data together with the video data and generates metadata from the acquired feature parameters or associated parameters with use of coordinate parameters converted from feature values designated by the feature parameters or associated values designated by the associated parameters so as to display the video data through a display window on a display space defined by display axes determined by the acquired feature parameters or associated parameters.

Claims

1. An image processing apparatus characterized by comprising:

acquisition means for acquiring feature parameters representing the feature of video data or audio data or associated parameters linked to the video data or audio data together with the video data; and

metadata generation means for generating metadata from the feature parameters or associated parameters acquired from the acquisition means with use of coordinate parameters converted from feature values designated by the feature parameters or associated values designated by the associated parameters so as to display the video data through a display window on a display space defined by display axes determined by the feature parameters or associated parameters acquired by the acquisition means.

2. The image processing apparatus according to claim 1, characterized in that

the acquisition means acquires the feature parameters from the video data in units of a picture or a GOP.

3. The image processing apparatus according to claim 1, characterized in that

the acquisition means acquires the feature parameter from the audio data at a predetermined time interval.

4. The image processing apparatus according to claim 1, characterized by further comprising a video encoder that encodes the video data to generate a video stream.

5. The image processing apparatus according to claim 4, characterized by further comprising recording means for recording the metadata generated by the metadata generation means in a recording medium together with the video stream generated by the encoder.

6. The image processing apparatus according to claim 4, characterized in that

the video encoder generates, from the video data, a first video data having a first resolution and a second video data having a resolution lower than the first resolution and encodes the generated first and second video data respectively to generate a first video stream and a second video stream.

7. The image processing apparatus according to claim 4, characterized in that

the video encoder encodes the video data using a first format to generate a first video stream and encodes the video data using a second format having a data amount smaller than that of the first format to generate a second video stream.

8. The image processing apparatus according to claim 6, characterized in that

the recording means records the first video stream and second video stream in different recording media.

9. The image processing apparatus according to claim 7, characterized in that

the recording means records the first video stream and second video stream in different recording media.

10. The image processing apparatus according to claim 5, characterized by further comprising transmission means for transmitting the metadata and video stream recoded in the recording medium to a playback unit that plays back the video stream.

11. The image processing apparatus according to claim 1, further comprising selection means for selecting, as display parameters, the feature parameters or associated parameters used for changing the display mode of the display window through which the video data is displayed from among a plurality of the feature parameters or associated parameters, characterized in that

the metadata generation means generates metadata including the display parameters so as to change a display mode of the display window through which the video data is displayed in accordance with the values of the display parameters selected by the selection means.

12. The image processing apparatus according to claim 11, characterized in that

the display mode includes the size of the display window, contrast inside the display window, transmittance inside the display window, or highlight of the frame of the display window.

13. The image processing apparatus according to claim 1, further comprising selection means for selecting, as playback mode parameters, the feature parameters or associated parameters used for selecting whether a high resolution video data having a first resolution or low resolution video data having a resolution lower than the first resolution is to be displayed from among a plurality of the feature parameters or associated parameters, characterized in that

the metadata generation means generates metadata including the display parameters so as to selectively display the high resolution video data or low resolution video data in accordance with the values of the playback mode parameters selected by the selection means.

14. The image processing apparatus according to claim 13, characterized by further comprising transmission means for selecting a high resolution video stream obtained by encoding the high resolution video data or low resolution video stream obtained by encoding the low resolution video data in accordance with the values of the display parameters selected by the selection means and transmitting the selected video stream.

15. The image processing apparatus according to claim 1, further comprising selection means for selecting, as playback mode parameters, the feature parameters or associated parameters used for selecting whether video data based on a first format or video data based on a second format having a data amount smaller than that of the first format is to be displayed from among a plurality of the feature parameters or associated parameters, characterized in that

the metadata generation means generates metadata including the playback mode parameters so as to selectively display the video data based on the first format or video data based on the second format in accordance with the values of the playback mode parameters selected by the selection means.

16. The image processing apparatus according to claim 15, characterized by further comprising transmission means for selecting a first video stream obtained by encoding the video data based on the first format or second video stream obtained by encoding the video data based on the second format in accordance with the values of the display parameters selected by the selection means and transmitting the selected video stream.

17. The image processing apparatus according to claim 1, characterized in that

in the case where the distance between a plurality of video data on the display space falls within a predetermined value, the metadata generation means generates, as the metadata, similarity information representing that the images of the video data are similar to each other.

18. The image processing apparatus according to claim 1, characterized in that

the display space includes a three-dimensional display space defined by the display axes determined by the three different feature parameters or associated parameters.

19. The image processing apparatus according to claim 1, characterized in that

the feature parameters include at least one of the motion amount of the video data, luminance thereof, hue thereof, red component thereof, blue component thereof, and green component thereof.

20. The image processing apparatus according to claim 3, characterized in that

the feature parameters include the signal level of the audio.

21. The image processing apparatus according to claim 1, characterized in that

the associated parameters include a time code of the video data.

22. The image processing apparatus according to claim 1, characterized in that

the associated parameters include time information representing the time at which the video data is shot or recorded.

23. The image processing apparatus according to claim 1, characterized in that

the associated parameters include positional information representing the location at which the video data is shot or recorded.

24. The image processing apparatus according to claim 1, characterized in that

the associated parameters include identification information for identifying an owner or creator of the video data.

25. The image processing apparatus according to claim 1, characterized in that

the associated parameters include identification information for identifying a user that has accessed the video data.

26. The image processing apparatus according to claim 1, characterized in that

the associated parameters include the number of times the video data is accessed.

27. The image processing apparatus according to claim 1, characterized in that

the associated parameters include group identification information for identifying the group of the video data.

28. The image processing apparatus according to claim 1, characterized in that

the associated parameters include face identification information for identifying the faces of persons included in the video data.

29. The image processing apparatus according to claim 1, characterized in that

the associated parameters include a title, keyword, or subtitle information included in the video data.

30. An image processing method characterized by comprising:

an acquisition step of acquiring feature parameters representing the feature of video data or audio data or associated parameters linked to the video data or audio data together with the video data; and

a metadata generation step of generating metadata from the feature parameters or associated parameters acquired in the acquisition step with use of coordinate parameters converted from feature values designated by the feature parameters or associated values designated by the associated parameters so as to display the video data through a display window on a display space defined by display axes determined by the feature parameters or associated parameters acquired in the acquisition step.