Method and apparatus for providing additional information of video using visible light communication

• US 10,271,009 B2
• Filed: 01/16/2015
• Issued: 04/23/2019
• Est. Priority Date: 01/07/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Ann, Jong-Hoon, Won, Eun-Tae, Son, Jae-Seung
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Mesa, Jose M
• Application Number:
  US14/598,579
• Publication Number:
  US 20150124173A1
• Time in Patent Office: 1,558 Days
• Field of Search: 386248
• CPC Class Codes: H04B 10/116 Visible light communication
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8545 for generating interactive applications
• CPC Class Codes: H04N 21/8583 by creating hot-spots
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 7/025 Systems for the transmission of digital non-picture data, e.g. of text during the active part of a television frame transmission of digital non-picture data during the vertical blanking interval only H04N7/088
• CPC Class Codes: H04N 7/08 Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band , e.g. by time division H04N7/007 takes precedence

Abstract

A method and apparatus for providing additional information included in a video displayed on a display device using visible light communication (VLC). A data packet including video data and additional information for an object included in the video data is received. The video data is extracted from the data packet, and the video data is decoded. The additional information from the data packet is extracted, and the additional information decoded. The decoded video data is output through the display device, and at the same time, the additional information is transmitted for a particular object included in a video based on a VLC protocol using a light emitting device prepared in the display device. The additional information providing apparatus includes an image sensor module, a display module, a visible light receiving module, an additional information manager and a controller.

Claims

1. A method in an electronic device, comprising:

receiving a data packet including a video frame of a video and a plurality of additional information associated with a plurality of objects in the video frame;

obtaining, from the data packet, the video frame and the plurality of additional information including first additional information for a first object and a second additional information for a second object in the video frame, and decoding the video frame and the plurality of additional information;

generating a plurality of visible light signals from a plurality of LEDs of a display by encoding the obtained the plurality of additional information into a displayable format using Visible Light Communication (VLC); and

transmitting first visible light signals corresponding the first additional information and second visible light signals corresponding the second additional information to an external device using VLC,

wherein at least one part of the plurality of visible light signals include the first visible light signals and the second visible light signals,

wherein the first additional information is transmitted by VLC from first LEDs of the plurality of LEDs disposed in a first portion of the display where the first object is displayed, and the second additional information is transmitted by VLC from second LEDs of the plurality of LEDs disposed in a second portion of the display where the second object is displayed, and

wherein the external device receives an input selecting an entry from an additional information list displayed on the external device, and, in response to the input, identifies one of the first additional information and the second additional information from the plurality of visible light signals.

2. The method of claim 1, wherein each of the plurality of additional information comprises an identifier indicating a video frame where an object is included, information on a location where the object is provided within the video frame, and an identifier of the object, a name of the object, and detailed information of the object, and

wherein the plurality of visible light signals is captured by an image sensor of the external device to identify the plurality of additional information for display on a display of the external device from the plurality of visible light signals.

3. The method of claim 1, wherein each of the plurality of additional information is decoded and transmitted in response to receiving an input request via an input device.

4. The method of claim 1, wherein the first additional information comprising a first data packet represents associated with the first object on the first portion, and the second additional information comprising a second data packet associated with the second object on the second portion.

5. The method of claim 4, wherein the first additional information and the second additional information are transmitted via the respective first and second data packets regardless of interaction between the display and the external device.

6. The method according to claim 1, wherein transmitting the first additional information using the first portion comprises transmitting the additional information using at least a portion of the light emitting devices utilized to display the first object.

7. The method of claim 1, further comprising:

extracting audio data from the data packet and decoding the audio data.

8. The method of claim 1, further comprising:

storing the first or second additional information at the further device in a memory.

9. The method of claim 1, further comprises:

requesting and acquiring the plurality of additional information selectively.

10. A method in a mobile terminal, comprising:

capturing, by an image sensor, a video displayed on an external device including a plurality of visible light signals;

in response to capturing the video, obtaining a plurality of visible light signals transmitted from the external device using Visible Light Communication (VLC), wherein the plurality of visible light signals include first visible light signals corresponding to first additional information and second visible light signals corresponding to second additional information;

displaying an additional information list listing all available additional information indicated within the plurality of visible light signals on a display of the mobile terminal;

in response to detecting an input to the mobile terminal selecting the first additional information from the additional information list, identifying the first additional information associated with a first object of the video, and displaying the identified first additional information on the display; and

in response to detecting an input to the mobile terminal selecting the second additional information from the additional information list, identifying the second additional information associated with a second object of the video and displaying the identified second additional information on the display,

wherein the first visible light signals are transmitted from first LEDs disposed in a first portion of a display of the external device where the first object is displayed, and

wherein the second visible light signals are transmitted from second LEDs disposed in a second portion of the display of the external device where the second object is displayed.

11. The method of claim 10, further comprising:

storing the first additional information and second additional information in a memory.

12. An apparatus, comprising:

a receiving module which receives a data packet including video frame of a video and a plurality of additional information associated with a plurality of objects;

a display;

at least one processor; and

a memory storing programming instructions executable by the at least one processor to cause the apparatus to;

obtain, from the data packet, the video frame and the plurality of additional information including first additional information for a first object and a second additional information for a second object displayed in the video frame, and decodes the video frame and the plurality of additional information, and

generate a plurality of visible light signals from a plurality of LEDs of the display by encoding the plurality of additional information into a displayable format using Visible Light Communication (VLC); and

transmit first visible light signals corresponding the first additional information and second visible light signals corresponding the second additional information to an external device using VLC,

wherein at least one part of the plurality of visible light signals include the first visible light signals and the second visible light signals,

wherein the first additional information is transmitted by VLC from first LEDs of the plurality of LEDs disposed in a first portion of the display where the first object is displayed, and the second additional information is transmitted by VLC from second LEDs of the plurality of LEDs disposed in a second portion of the display where the second object is displayed, and

wherein the external device receives an input selecting an entry from an additional information list displayed on the external device, and, in response to the input, identifies one of the first additional information and the second additional information from the plurality of visible light signals.

13. The apparatus of claim 12 wherein the programming instructions are further executable by the at least one processor to cause the apparatus to:

control for extracting audio data from the data packet, restoring an audio signal by decoding audio data, and outputting the restored audio signal, and

wherein the video including the plurality of visible light signals is captured by an image sensor of the external device identify the plurality of additional information for display on a display of the external device from the plurality of visible light signals.

14. A mobile terminal, comprising:

an image sensor;

a local display;

at least one processor; and

a memory storing programming instructions executable by the at least one processor to cause the mobile terminal to;

capture, by the image sensor, a video displayed on an external device including a plurality of visible light signals;

in response to capturing the video, obtain a plurality of visible light signals transmitted from the external device using a Visible Light Communication (VLC), wherein the plurality of visible light signals include first visible light signals corresponding to first additional information and second visible light signals corresponding to second additional information,

display an additional information list listing all available additional information indicated within the plurality of visible light signals on the local display;

in response to detecting an input to the mobile terminal selecting the first additional information from the additional information list, identify the first additional information associated with a first object of the video and display the identified first additional information on the local display; and

in response to detecting an input to the mobile terminal selecting the second additional information from the additional information list, identify the second additional information associated with a second object of the video and display the identified second additional information on the local display,

wherein the first visible light signals are transmitted from first LEDs disposed in a first portion of a display of the external device where the first object is displayed, and

wherein the second visible light signals are transmitted from second LEDs disposed in a second portion of the display of the external device where the second object is displayed.

15. The mobile terminal of claim 14, wherein the first visible light signals and the second visible signals are modulated by a predetermined communication portocol.

16. The mobile terminal of claim 15, further comprising:

a memory for storing the first additional information and second additional information.