Image processing device and image processing method

• US 8,780,756 B2
• Filed: 04/23/2007
• Issued: 07/15/2014
• Est. Priority Date: 04/24/2006
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ogikubo, Junichi
• Primary Examiner(s): Mesfin, Yemane
• Assistant Examiner(s): DAVENPORT, MON CHERI S
• Application Number:
  US11/993,764
• Publication Number:
  US 20100158471A1
• Time in Patent Office: 2,640 Days
• Field of Search: 375/240.01, 375/240.08, 375/295, 375/316, 370/252, 370/254, 715700-711, 715851-853, 715716-723, 715730-734, 715/810, 715/838, 715780-784, 725/38, 725/25
• US Class Current: 370/254
• CPC Class Codes: G06F 16/743 a collection of video files or sequences
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G11B 27/032 on tapes G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/323 Time code signal, e.g. on a cue track as SMPTE- or EBU-time code
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 1/00442 Simultaneous viewing of a plurality of images, e.g. using a mosaic display arrangement of thumbnails
• CPC Class Codes: H04N 1/00453 arranged in a two dimensional array
• CPC Class Codes: H04N 1/32112 in a separate computer file, document page or paper sheet, e.g. a fax cover sheet H04N1/32122 takes precedence
• CPC Class Codes: H04N 1/32122 in a separate device, e.g. in a memory or on a display separate from image data
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4438 Window management, e.g. event handling following interaction with the user interface
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 7/0122 the input and the output signals having different aspect ratios
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8063 : using time division multiplex of the PCM audio and PCM video signals
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal

Abstract

The present invention has been made to facilitate search or management of video data. The present invention acquires feature parameters representing the feature of video data or audio data or associated parameters linked to the video data or audio data together with the video data and generates metadata from the acquired feature parameters or associated parameters with use of coordinate parameters converted from feature values designated by the feature parameters or associated values designated by the associated parameters so as to display the video data through a display window on a display space defined by display axes determined by the acquired feature parameters or associated parameters.

Claims

1. An image processing apparatus comprising:

circuitry configured to

acquire feature parameters representing one or more features of each of a plurality of portions of video data or audio data, or associated parameters linked to each of the plurality of portions of the video data or audio data;

generate metadata from the acquired feature parameters or associated parameters;

sequentially display, in accordance with a passage of time, images associated with the plurality of portions of the video data or audio data in a display window corresponding to the video data or audio data on a display space defined by display axes determined by the acquired feature parameters or associated parameters, and

update the display position of the display window for each of the plurality of portions of the video data or audio data based on the feature parameters or associated parameters of the respective portion of the video data or audio data.

2. The image processing apparatus according to claim 1, wherein

the circuitry is configured to acquire the feature parameters from the video data in units of a picture or a group of pictures.

3. The image processing apparatus according to claim 1, wherein

the circuitry is configured to acquire the feature parameters from the audio data at a predetermined time interval.

4. The image processing apparatus according to claim 1, wherein

the circuitry is further configured to encode the video data to generate a video stream.

5. The image processing apparatus according to claim 4, wherein

the circuitry is further configured to record the generated metadata in a recording medium together with the generated video stream.

6. The image processing apparatus according to claim 4, wherein

the circuitry is configured to generate, from the video data, a first video data having a first resolution and a second video data having a resolution lower than the first resolution and to encode the generated first and second video data respectively to generate a first video stream and a second video stream.

7. The image processing apparatus according to claim 4, wherein

the circuitry is configured to encode the video data using a first format to generate a first video stream and to encode the video data using a second format having a data amount smaller than that of the first format to generate a second video stream.

8. The image processing apparatus according to claim 6, wherein

the circuitry is configured to record the first video stream and second video stream in different recording media.

9. The image processing apparatus according to claim 7, wherein

the circuitry is configured to record the first video stream and second video stream in different recording media.

10. The image processing apparatus according to claim 5, wherein

the circuitry is further configured to transmit the metadata and video stream recorded in the recording medium to a playback circuitry that plays back the video stream.

11. The image processing apparatus according to claim 1, wherein the circuitry is configured to

select, as display parameters, the feature parameters or associated parameters used for changing the display mode of the display window through which the video data is displayed from among a plurality of the feature parameters or associated parameters, and

generate metadata including the display parameters so as to change the display mode of the display window through which the video data is displayed in accordance with the values of the selected display parameters.

12. The image processing apparatus according to claim 11, wherein

the display mode includes the size of the display window, contrast inside the display window, transmittance inside the display window, or highlight of the frame of the display window.

13. The image processing apparatus according to claim 1, wherein the circuitry is configured to

select, as playback mode parameters, the feature parameters or associated parameters used for selecting whether a high resolution video data having a first resolution or low resolution video data having a resolution lower than the first resolution is to be displayed from among a plurality of the feature parameters or associated parameters, and

generate metadata including the playback mode parameters so as to selectively display the high resolution video data or low resolution video data in accordance with the values of the selected playback mode parameters.

14. The image processing apparatus according to claim 13, wherein

the circuitry is further configured to select a high resolution video stream obtained by encoding the high resolution video data or low resolution video stream obtained by encoding the low resolution video data in accordance with the values of the selected playback mode parameters and transmit the selected video stream.

15. The image processing apparatus according to claim 1, wherein the circuitry is configured to

select, as playback mode parameters, the feature parameters or associated parameters used for selecting whether video data based on a first format or video data based on a second format having a data amount smaller than that of the first format is to be displayed from among a plurality of the feature parameters or associated parameters, and

generate metadata including the playback mode parameters so as to selectively display the video data based on the first format or video data based on the second format in accordance with the values of the selected playback mode parameters.

16. The image processing apparatus according to claim 15, wherein the circuitry is configured to

select a first video stream obtained by encoding the video data based on the first format or second video stream obtained by encoding the video data based on the second format in accordance with the values of the selected playback mode parameters and transmit the selected video stream.

17. The image processing apparatus according to claim 1, wherein

in the case where the distance between a plurality of video data on the display space falls within a predetermined value, the circuitry generates, as the metadata, similarity information representing that the images of the video data are similar to each other.

18. The image processing apparatus according to claim 1, wherein

the coordinate positions correspond to three display axes which are determined by three different feature types.

19. The image processing apparatus according to claim 1, wherein

the feature parameters include at least one of the motion amount of the video data, luminance thereof, hue thereof, red component thereof, blue component thereof, and green component thereof.

20. The image processing apparatus according to claim 3, wherein

the feature parameters include the signal level of the audio.

21. An image processing method of an information processing apparatus, the method comprising:

acquiring feature parameters representing a feature of each of a plurality of portions of video data or audio data, or associated parameters linked to each of the plurality of portions of the video data or audio data; and

generating, by a processor of the information processing apparatus, metadata from the feature parameters or associated parameters acquired in the acquiring step so as to sequentially display, in accordance with a passage of time, images associated with the plurality of portions of the video data or audio data in a display window corresponding to the video data or the audio data on a display space defined by display axes determined by the feature parameters or associated parameters acquired in the acquiring step, and to update the display position of the display window for each of the plurality of portions of the video data or audio data based on the feature parameters or associated parameters of the respective portion of the video data or audio data.

22. An image processing apparatus comprising:

an extraction unit configured to extract feature parameters representing a feature of each of a plurality of portions of video data or audio data, or associated parameters linked to each of the plurality of portions of the video data or audio data; and

a processor configured

to generate metadata from the feature parameters or associated parameters extracted by the extraction unit,

to sequentially display, in accordance with a passage of time, images associated with the plurality of portions of the video data or audio data in a display window corresponding to the video data or audio data on a display space defined by display axes determined by the feature parameters or associated parameters extracted by the extraction unit, and

to update the display position of the display window for each of the plurality of portions of the video data or audio data based on the feature parameters or associated parameters of the respective portion of the video data or audio data.

23. The image processing apparatus according to claim 22, wherein

the processor is configured to update each of the coordinate positions on the display space at which the display window is displayed for each image of the video data that is displayed in the display window.