System and method for providing object-based video service

• US 20030197720A1
• Filed: 04/01/2003
• Published: 10/23/2003
• Est. Priority Date: 04/17/2002
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Ji-Yeun, Kim, Chang-Yeong, Moon, Young-Su
• Application Number:
  US10/402,998
• Publication Number:
  US 20030197720A1
• Time in Patent Office: Days
• US Class Current: 715/716
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

A system and method for providing object-based video services are provided. The system for providing object-based video services comprises an interactive video authoring unit which tracks an object of interest for each frame of input moving pictures with using a predetermined object tracking algorithm, generates an object tracking scenario, and links each object to additional information; a service server which stores the moving pictures, the object tracking scenario, and additional information linked to each object, and if a service is requested through a communications network, provides the object tracking algorithm, the stored moving pictures, the object tracking scenario, and the additional information; and a client unit which receives the object tracking algorithm from the service server through the communications network and installs the algorithm, and if the moving pictures, the object tracking scenario, and the additional information are provided, generates an object label image corresponding to each frame of the moving pictures according to the object tracking scenario by using the installed object tracking algorithm, extracts additional information corresponding to an object selected by a user based on the generated object label image, and displays the additional information. Accordingly, since the service server stores only tracking scenario data taking up relatively less memory than object label images, memory use in the service server can be greatly reduced. Also, since the service server transmits only tracking scenario data whose amount is very small, transmission efficiency can increase while transmission error and data distortion can be minimized.

Claims

1. An object-based video service system comprising:

an interactive video authoring unit which tracks an object of interest for each frame of input moving pictures with using a predetermined object tracking algorithm, generates an object tracking scenario, and links each object to additional information;

a service server which stores the moving pictures, the object tracking scenario, and the additional information linked to each object, and if a service is requested through a communications network, provides the object tracking algorithm, the stored moving pictures, the object tracking scenario, and the additional information; and

a client unit which receives the object tracking algorithm from the service server through the communications network and installs the object tracking algorithm, and if the moving pictures, the object tracking scenario, and the additional information are provided, generates an object label image corresponding to each frame of the moving pictures according to the object tracking scenario by using the installed object tracking algorithm, extracts additional information corresponding to an object selected by a user based on the generated object label image, and displays the extracted additional information.

2. The object-based video service system of claim 1, wherein the interactive video authoring unit comprises:

an object tracking unit which performs object tracking for each object by using the object tracking algorithm, and stores initial object label images, in which the areas of interest objects are expressed in different gray values in a frame where an event occurs, parameters used in object tracking, and the result of object tracking;

a tracking scenario generating unit generates the object tracking scenario based on the initial object label images, the parameter values used in object tracking, and the result of object tracking; and

an additional information link unit which links related additional information to each object.

3. The object-based video service system of claim 1, wherein the client unit comprises:

a first memory unit which receives moving pictures from the service server and stores the moving pictures;

an object tracking unit which receives the object tracking scenario and the object tracking algorithm from the service server, and using the object tracking algorithm, generates in real time an object label image corresponding to each frame of the moving pictures, according to the object tracking scenario;

a second memory unit which stores the object label image for each frame generated in the object tracking unit;

a third memory unit which receives additional information linked to each object from the service server, and stores the received additional information; and

an interactive video reproducing unit which reproduces moving pictures stored in the first memory unit, referring to the object label image for each frame, recognizes an object selected by the user, extracts additional information corresponding to the recognized object from the third memory unit, and provides the extracted additional information.

4. An interactive video apparatus comprising:

a memory unit which receives moving pictures and additional information linked to each object that are transmitted through a communications network, and stores the moving pictures and the additional information;

an object tracking unit which if an object tracking scenario is transmitted through the communications network, drives a predetermined object tracking algorithm, generates in real time an object label image for each frame according to the object tracking scenario, and stores the generated object label image for each frame in the memory unit; and

an interactive video reproducing unit which reproduces the moving pictures stored in the memory unit, and referring to the object label image for each frame, recognizes an object selected by a user, extracts additional information on the recognized object from the memory unit, and provides the extracted additional information.

5. The interactive video apparatus of claim 4, wherein the object tracking unit receives the predetermined object tracking algorithm from an external server through the communications network.

6. The interactive video apparatus of claim 4, wherein the interactive video reproducing unit comprises:

a display unit which reproduces the moving pictures stored in the memory and provides interface with the user;

an object recognizing unit which referring to the object label image for each frame stored in the memory unit, recognizes the label of the object which is mapped with a pixel selected by the user;

an information extracting unit which referring to the memory unit, extracts additional information corresponding to the object label recognized in the object recognizing unit; and

a reproduction driving unit which performs control so that the additional information extracted in the information extracting unit is displayed on the display unit.

7. An object-based video service method in which object-based interactive moving pictures are provided to a client unit belonging to a user, the method comprising:

(a) tracking an object of interest in each frame of the moving pictures according to a predetermined object tracking algorithm, while generating and storing an object tracking scenario;

(b) linking additional information to be provided for each object, and storing the additional information;

(c) if a request of providing the moving picture from the client unit is received, providing object-based video data, including the moving pictures, the object tracking scenario, and the additional information;

(d) the client unit generating in real time an object label image corresponding to each frame of the moving pictures according to the object tracking scenario with using the object tracking algorithm; and

(e) based on the generated object label image, displaying additional information corresponding to an object selected by the user.

8. The object-based video service method of claim 7, wherein if the client unit requests to provide the object tracking algorithm, when the object-based video data are provided, the object tracking algorithm is provided together.

9. The object-based video service method of claim 7, wherein the step (a) comprises:

(a1) assigning an area for one or more object of interest in the first frame of the moving pictures;

(a2) performing object tracking for each object from the first frame, by applying an object tracking parameter for each interest object for tracking the interest object in the first frame to the object tracking algorithm;

(a3) if a predetermined event on an object occurs in a predetermined frame in the object tracking, applying a new object tracking parameter for tracking of the object to the object tracking algorithm, and performing again the tracking of the object from the frame in which the event occurred;

(a4) generating object label images corresponding to the frames in which events occur; and

(a5) generating an object tracking process as the object tracking scenario based on the object label images, the object tracking parameters for respective objects, and the result of object tracking.

10. The object-based interactive video service method of claim 9, wherein the steps (a3) and (a4) comprise:

(a31) generating and storing an object label image in which the areas of interest objects in the first frame are expressed in different gray values;

(a32) in a frame in which tracking of a predetermined object fails, after resetting an object area for the corresponding object, changing the corresponding object tracking parameter, and applying the changed object tracking parameter from the frame in which the tracking of the object failed, tracking the object;

(a33) generating an object label image corresponding to the reset object area;

(a34) in a frame in which a new object appears, after setting an object area for the corresponding object and a parameter for tracking the corresponding object, and applying the set parameter to the object tracking algorithm, tracking the new object from the frame in which the new object appeared; and

(a35) generating an object label image corresponding to the object area set in the step (a34).

11. A computer readable medium having embodied thereon a computer program for the object-based interactive video service method of claim 7.

12. An object-based video reproducing method in which object-based moving pictures are displayed on a display unit interfacing with a user, the method comprising:

(a) receiving moving pictures, an object tracking scenario on an object tracking process, and additional information linked to each object, from the outside, and storing the data in a memory;

(b) generating in real time an object label image corresponding to each frame of the moving pictures according to the object tracking scenario, by using an object tracking algorithm, and storing the object label images;

(c) reproducing the stored moving pictures, and recognizing an object selected by the user by referring to an object label image corresponding to each frame; and

(d) extracting additional information linked to the recognized object from the memory and providing the extracted additional information.

13. The object-based reproducing method of claim 12, wherein the object tracking algorithm is provided by an external server when there is a request.

14. The object-based reproducing method of claim 12, wherein the steps (c) and (d) comprises:

(c1) reproducing the stored moving pictures through the display unit and recognizing a pixel on a frame which is selected by the user to select an object;

(c2) recognizing the label of an object which is mapped with the pixel recognized in step (c1) by referring to the object label image for each frame; and

(c3) extracting additional information corresponding to the object label recognized in step (c2) from the memory and displaying the extracted additional information.

15. A computer readable medium having embodied thereon a computer program for the object-based interactive video reproducing method of claim 12