VIDEO PLAYER FOR EXHIBITING CONTENT OF VIDEO SIGNALS WITH CONTENT LINKING TO INFORMATION SOURCES

• US 20090024617A1
• Filed: 07/22/2008
• Published: 01/22/2009
• Est. Priority Date: 07/22/2007
• Status: Active Grant

• Current Assignee: Overlay TV Holdings Limited
• Original Assignee: Overlay Tv Inc.
• Inventors: COPE, Tyler Andrew
• Granted Patent: US 8,001,116 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/75 Clustering; Classification
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/237 Communication with additional data server
• CPC Class Codes: H04N 21/2542 for selling goods, e.g. TV shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/47815 Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8583 : by creating hot-spots
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal

Abstract

A method and apparatus for retrieving information relevant to tracked objects appearing in a display of a video signal is disclosed. The method is performed by a viewing computer having stored thereon an augmented display tool. In response to a user requesting the video signal, a content directory storing content information relevant to the tracked objects is acquired from a video-overlay server. The augmented display tool causes the viewing computer to acquire and display the video signal and record a time measurement and spatial coordinates of each point selected by a viewer using a pointing device. The augmented display tool uses the content directory to find an object identifier corresponding to each selected point and extracts relevant information from a global object directory maintained at the video-overlay server.

Claims

1. A process performed by a computing device for displaying a video signal in response to a request from a viewer of said computing device, the process comprising:

communicating to a video-overlay server an identifier of said video signal, said identifier provided by said viewer;

receiving from said video-overlay server a network address of a video-streaming source providing said video signal and a content directory of said video signal;

requesting and receiving said video signal from said video-streaming source;

producing a current display of said video signal; and

in response to said viewer pointing to a specific object on said current display;

recording a pointing time at which said computing device detects said pointing; and

accessing said content directory to retrieve information specific to said pointing time.

2. The process of claim 1, wherein said step of accessing comprises further steps of:

determining a spatial position corresponding to said pointing time;

selecting a candidate object from among a set of tracked objects within contents of said video signal, said tracked objects identified in said content directory;

extracting two appearance times of said candidate object surrounding said pointing time and corresponding two contours from tracking records stored in said content directory during a prior display of said video signal;

ascertaining inclusion of said spatial position within a selected contour from said two contours; and

where said inclusion is determined, assigning a specific identifier of said specific object as an identifier of said candidate object.

3. The process of claim 2, further comprising a step of:

where said spatial position is outside said two contours, computing an interpolated contour corresponding to said pointing time; and

where said spatial position is included within said interpolated contour, assigning a specific identifier of said specific object as an identifier of said candidate object.

4. The process of claim 2 further comprising:

displaying a description corresponding to said specific identifier, said description extracted from a list of descriptions of said tracked objects included in said content directory;

extracting, from said content directory, a subset of network addresses corresponding to said specific identifier; and

establishing a link to at least one network address listed in said subset of network addresses.

5. The process of claim 2 further comprising:

storing, in a storage medium of said computing device, a description corresponding to said specific identifier, said description extracted from a list of descriptions of said tracked objects included in said content directory;

extracting, from said content directory, a subset of network addresses corresponding to said specific identifier; and

storing, in said storage medium, said subset of network addresses.

6. The process of claim 2 wherein the step of ascertaining inclusion of said spatial position within said selected contour further comprises steps of:

approximating selected contour by a rectangular shape defined by a diagonal connecting two points (a₁, b₁) and (a₂, b₂); and

determining that said spatial position, defined by a point (x, y), is within said selected contour subject to the condition;

a₁≦

x≦

a₂ and b₁≦

y≦

b₂.

7. The process of claim 3 further comprising:

approximating each of said two contours by a rectangular shape defined by a respective diagonal of positive slope so that one of said two contours is represented by a diagonal connecting point (a₁, b₁) to point (a₂, b₂), and the other is represented by diagonal connecting point (u₁, v₁) to point (u₂, v₂); and

determining said interpolated contour as a rectangular shape defined by a diagonal connecting a first point of spatial coordinates (x₁, y₁) to a second point of spatial coordinates (x₂, y₂), where x₁, y₁, x₂, and y₂ are determined as;

x₁=α

u₁+(1−

α

)a₁ and y₁=α

v₁+(1−

α

)b₁; and

x₂=α

u₂+(1−

α

)a₂ and y₂=α

v₂+(1−

α

)b₂;

where α

=(t−

t₁)/(t₂−

t₁), t being said pointing time, and t₁ and t₂ are display times, determined from said prior display, of points (a₁, b₁) and (u₁, v₁), respectively.

8. The process of claim 7 further comprising a step of determining that said spatial position, defined by a point (x, y), is within said interpolated constellation subject to the condition:

x₁≦

x≦

x₂ and y₁≦

y≦

y₂.

9. The process of claim 1 further comprising:

interrupting said current display during a time interval following detection of said pointing;

adding said time interval to a summation of interruption time, said summation initialized to zero at a starting instant of said current display; and

determining said pointing time as a time indication derived from a clock of said computing device minus said starting instant of said current display minus said summation.

10. A computer readable medium having stored thereon a computer-executable program operable in a host computer to cause the host computer to:

receive a request from a viewer of said host computer to display a video signal;

forward said request to a video-overlay server;

receive from said video-overlay server a network address of a video-streaming source providing said video signal and a content directory of said video signal, said content directory comprising presence times of selected contents of said video signal and network addresses of sources of information relevant to said selected contents;

request and receive said video signal from said video-streaming source;

display said video signal on a display device controlled by said host computer;

detect pointing times at which said viewer points to a particular displayed object;

compare said pointing times to said presence times to find a counterpart content, in said selected contents, of said particular displayed object; and

where said counterpart is found, extract a subset of said network addresses, said subset corresponding to said counterpart content.

11. The computer readable medium of claim 10 further storing computer-executable instructions which causes said host computer to:

sort said presence times according to contents and group presence times of each content into sets of presence times, each set of presence times contained within a video-frame period;

determine spatial contours corresponding to said sets of presence times;

determine a respective spatial position for each of said pointing times;

determine inclusion of said respective spatial position within one of said contours; and

determine an interpolated spatial contour, between two spatial counters, corresponding to a pointing time

12. A method performed by a computer for retrieving information relevant to objects appearing in a display of a video signal on a screen of said computer in response to a user of said computer requesting said video signal, the method comprising:

acquiring a content directory of said video signal from a video-overlay server, said content directory storing content information relevant to a set of tracked objects;

acquiring said video signal from a video source;

starting said display of said video signal;

recording a time measurement and spatial coordinates of each selected point of a number of selected points on said display identified by a pointing device controlled by said user; and

upon termination of said display;

finding an object identifier in said content directory of an object, from among said set of tracked objects, corresponding to each said selected point using said time measurement and spatial coordinates; and

where said object identifier is found, storing a record comprising said object identifier and said time measurement in a cache memory of said computer.

13. The method of claim 12 further comprising steps of:

grouping records in said cache memory into record groups each record group corresponding to a common object; and

retrieving further information relevant to said common object from a global object directory maintained at said video-overlay server.

14. The method of claim 13 further comprising steps of:

sorting said record groups according to a number of records in each group; and

displaying said further information in an order determined by said number of records.

15. The method of claim 13 further comprising steps of:

sorting said record groups according to smallest time measurement; and

displaying said further information in an order determined by said smallest time measurement.

16. The method of claim 13 further comprising a step of acquiring said further information from network sites identified in said content directories.

17. The method of claim 12 wherein said finding comprises steps of:

selecting, from a tracking record stored in said content directory, nearest two tracking time instants, corresponding to a common object, surrounding said time measurement of a specific selected point;

determining coordinates of an interpolated point corresponding to said time measurement;

determining proximity of said spatial coordinates of said specific selected point to said interpolated point; and

where said proximity is within a predefine threshold, determining that said common object corresponds to said specific selected point.

18. The method of claim 17 wherein said spatial coordinates are Cartesian coordinates and said step of determining said proximity comprises a further step of determining the greater of:

deviation of a first coordinate of said spatial coordinates from a respective coordinate of said interpolated point; and

deviation of a second coordinate of said spatial coordinates from a respective coordinate of said interpolated point.

19. The method of claim 13 further comprising steps of:

extracting time measurements corresponding to a selected object from a respective record group to produce a list of extracted time measurements;

identifying, for each extracted time measurement, a corresponding frame of said video signal; and

displaying said corresponding frame and an icon of said selected object, said icon retrieved from said global object directory.

20. The method of claim 12 further comprising step of:

formulating an array of object identifiers of objects corresponding to said selected points;

determining a count of selected points for each object identifier;

associate said count with a respective entry in said array; and

report said array and associated count to said video-overlay server.