Apparatus, systems and methods for accessing information based on an image presented on a display

• US 9,749,685 B2
• Filed: 07/23/2015
• Issued: 08/29/2017
• Est. Priority Date: 07/23/2015
• Status: Active Grant

• Current Assignee: Dish Technologies LLC (Dish Network Corporation)
• Original Assignee: Echostar Technologies LLC (Echostar Corporation)
• Inventors: Tilaye, Swapnil, Garcia Navarro, Carlos
• Primary Examiner(s): SHEPARD, JUSTIN E
• Application Number:
  US14/807,178
• Publication Number:
  US 20170026709A1
• Time in Patent Office: 768 Days
• Field of Search: None
• CPC Class Codes: H04N 21/40 Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
• CPC Class Codes: H04N 21/41 Structure of client; Structure of client peripherals
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/414 Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/422 Input-only peripherals , i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] input devices also receiving signals from specially adapted client devices H04N21/4104
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43 Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware real-time communication protocols in data switching networks H04L65/00
• CPC Class Codes: H04N 21/43079 of additional data with content streams on multiple devices
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4882 for displaying messages, e.g. warnings, reminders arrangements for providing short real-time information to substation in data switching networks H04L12/1895
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

Systems and methods are operable to control operation of a portable media device based on machine readable information of a graphical artifact shown on a display concurrently with presentation of a video portion of a media content event. A portable media device, using a portable media device provisioned with an image capture device, captures at least one image that includes the display that is presenting the graphical artifact and the video portion of the media content event, identifies the graphical artifact in the captured at least one image, determines the machine readable data based on the identified graphical artifact, and operates the portable media device to perform at least one operation based on the determined machine readable data.

Claims

1. A method, comprising:

presenting a graphical artifact on a portion of a display concurrently with presentation of a video portion of a media content event, wherein the graphical artifact has machine readable data encoded thereon using a plurality of data marks;

identifying, in a pre-capture image captured by a portable media device provisioned with an image capture device, the display that is shown in a field of view of the image capture device;

automatically performing a zooming operation with the image capture device;

capturing, using the portable media device provisioned with the image capture device, at least one image that includes the display that is presenting the graphical artifact wherein at least a size of the display is enlarged in the image captured by the portable media device as a result of the automatic zoom operation;

identifying the graphical artifact in the captured at least one image;

determining the machine readable data based on the identified graphical artifact; and

operating the portable media device to perform at least one operation based on the determined machine readable data.

2. The method of claim 1, wherein the machine readable data of the graphical artifact includes textual information pertaining to the presented media content event, and wherein the at least one operation performed by the portable media device comprises:

displaying the textual information on a display of the portable media device.

3. The method of claim 1, wherein the machine readable data of the graphical artifact includes information to communicatively couple the portable media device with a remote information system via a communication network, and wherein the at least one operation performed by the portable media device comprises:

receiving, at the portable media device, supplemental information from the remote information system; and

presenting the received supplemental information using the portable media device.

4. The method of claim 3, wherein presenting the supplemental information using the portable media device comprises:

displaying a stream of closed captioning text on the display of the portable media device, wherein a language of the presented closed captioning text is different from a language used in dialogue of the presented media content event.

5. The method of claim 3, wherein presenting the supplemental information using the portable media device comprises:

presenting an audible audio stream with at least dialogue using the portable media device, wherein a language of the presented audible audio stream dialogue is different from a language used in audible dialogue of the presented media content event.

6. The method of claim 1, further comprising:

separately receiving the graphical artifact and the media content event at a media device that is controlling presentation of the media content event on the display;

wherein presenting the graphical artifact on the portion of the display currently with presentation of the video portion of the media content event comprises;

determining, at the media device, a region of the display that is not being used to present the video portion of the media content event; and

defining, at the media device, a presentation location for the graphical artifact in the identified region of the display that is not being used for display of the media content event,

wherein the graphical artifact is presented in the defined presentation location.

7. The method of claim 1, wherein presenting the graphical artifact on the portion of the display currently with presentation of the video portion of the media content event comprises:

presenting the graphical artifact in a predefined region of the display,

wherein the presented graphical artifact completely obscures a portion of the media content event that corresponds to the predefined region that is used for presentation of the graphical artifact.

8. The method of claim 1, wherein the video portion of the media content event comprises a series of serially presented video frames, and wherein presenting the graphical artifact comprises:

presenting the graphical artifact on at least three adjacent video frames, wherein a duration of the presentation of the graphical artifact is less than a duration of a human visual perception threshold such that a viewer is not able to perceive presentation of the graphical artifact; and

subsequently presenting the next ones of the video frames without presentation of the graphical artifact,

wherein at least one still image captured by the portable media device includes at least one of the at least three video frames having the presented graphical artifact, and

wherein the machine readable data is determinable from at least one of the at least three adjacent video frames having the presented graphical artifact.

9. The method of claim 8, wherein the at least one of the at least three video frames having the presented graphical artifact is a first video frame, wherein determined machine readable data determined from the first video frame is first machine readable data, the method further comprising:

identifying the graphical artifact in an adjacent second video frame that also presented the graphical artifact;

determining second machine readable data from the graphical artifact presented in the second video frame; and

comparing the first machine readable data and the second machine readable data,

wherein the at least one operation is performed by the portable media device when the first machine readable data is the same as the second machine readable data.

10. The method of claim 1, wherein prior to presenting the graphical artifact on the portion of the display, the method further comprising:

receiving the media content event in a media content stream at a media device that controls presentation of the media content event on the display;

communicating the video portion of the media content event from the media device to the display for presentation;

receiving, at the media device, information that defines the at least one operation that is to be performed by the portable media device;

generating, at the media device, the graphical artifact; and

communicating the graphical artifact from the media device to the display for presentation on the display.

11. A method, comprising:

presenting a graphical artifact on a portion of a display concurrently with presentation of a video portion of a media content event, wherein the graphical artifact has machine readable data encoded thereon using a plurality of data marks, wherein the video portion of the media content event comprises a series of serially presented video frames, and wherein presenting the graphical artifact comprises;

presenting the graphical artifact in a first one of the series of serially presented video frames, wherein the graphical artifact is presented at a first presentation location on the display; and

presenting the graphical artifact in a second one of the series of serially presented video frames, wherein the graphical artifact is presented at a second presentation location on the display, wherein the second presentation location is different from the first presentation location;

capturing, using a portable media device provisioned with an image capture device, at least one image that includes the display that is presenting the graphical artifact, wherein the portable media device captures at least one still image that has the graphical artifact presented at the first presentation location or at the second presentation location;

identifying the graphical artifact in the captured at least one image;

determining the machine readable data based on the identified graphical artifact; and

operating the portable media device to perform at least one operation based on the determined machine readable data.

12. A method, comprising:

presenting a graphical artifact on a portion of a display concurrently with presentation of a video portion of a media content event, wherein the graphical artifact has machine readable data encoded thereon using a plurality of data marks, wherein the video portion of the media content event comprises a series of serially presented video frames, and wherein presenting the graphical artifact comprises;

presenting the graphical artifact in a first group of adjacent serially presented video frames,

wherein the graphical artifact is presented at a first presentation location on the display in each of the first group of adjacent serially presented video frames,

and wherein a first total presentation duration of the graphical artifact at the first presentation location is less than a duration of a human visual perception threshold such that a user of the portable media device is not able to perceive the presentation of the graphical artifact presented at the first presentation location; and

subsequently presenting the graphical artifact in a second group of adjacent serially presented video frames,

wherein the graphical artifact is presented at a second presentation location on the display in each of the second group of adjacent serially presented video frames,

wherein a second total presentation duration of the graphical artifact at the second presentation location is less than the duration of the human visual perception threshold such that the user of the portable media device is not able to perceive the presentation of the graphical artifact presented at the second presentation location, and

wherein the second presentation location is different from the first presentation location,

capturing, using a portable media device provisioned with an image capture device, at least one image that includes the display that is presenting the graphical artifact, wherein the portable media device captures at least one still image that has the graphical artifact presented at the first presentation location or at the second presentation location, and

identifying the graphical artifact in the captured at least one image;

determining the machine readable data based on the identified graphical artifact; and

operating the portable media device to perform at least one operation based on the determined machine readable data.

13. A method, comprising:

presenting a graphical artifact on a portion of a display concurrently with presentation of a video portion of a media content event, wherein the graphical artifact has machine readable data encoded thereon using a plurality of data marks;

identifying, in a pre-capture image captured by a portable media device provisioned with an image capture device, the display that is shown in a field of view of the image capture device,

wherein an orientation operation is performed to re-orient the display into a center of a captured at least one image;

capturing, using the portable media device, at least one image that includes the display that is presenting the graphical artifact;

identifying the graphical artifact in the captured at least one image;

determining the machine readable data based on the identified graphical artifact; and

operating the portable media device to perform at least one operation based on the determined machine readable data.

14. The method of claim 13, wherein the pre-capture image is a first pre-capture image, the method further comprising:

identifying, in a second pre-capture image, the graphical artifact that is shown in the field of view of the image capture device,

performing an automatic zooming operation with the image capture device,

wherein a size of the graphical artifact is enlarged in the image captured by the portable media device.

15. The method of claim 13, further comprising:

performing an automatic focusing operation with the image capture device to focus the graphical artifact in the image captured by the portable media device.

16. A portable media device comprising:

an image capture device;

a display;

a transceiver; and

a processor system communicatively coupled to the image capture device, the display, and the transceiver, wherein the processor system is configured to;

identify, in a pre-capture image captured by the portable media device provisioned with the image capture device, the display that is shown in a field of view of the image capture device;

re-orient the display into a center of a next captured at least one image that is to be captured by the image capture device, wherein the next captured at least one image includes at least one graphical artifact that is concurrently presented with a video portion of a media content event on the display, wherein the graphical artifact has machine readable data encoded thereon using a plurality of data marks that is configured to control at least one operation of the portable media device;

automatically perform a zooming operation with the image capture device, wherein at least a size of the display is enlarged in the next captured at least one image that is captured by the portable media device; and

determine the machine readable data from the captured image with the graphical artifact;

where in response to determining the machine readable data, the processor system is further configured to;

operate the portable media device in accordance with the machine readable data.

17. The portable media device of claim 16, wherein the machine readable data determined from the graphical artifact comprises textual information, and wherein the processor system is further configured to:

present the textual information on the display of the portable media device.

18. The portable media device of claim 16, wherein the machine readable data determined from the graphical artifact comprises information to establish a communication link to a remote information system and to request supplemental information accessible from the remote information system, and wherein the processor system is further configured to:

establish the communication link to the remote information system in response to determining the machine readable data; and

receive the supplemental information from the remote information system, wherein the supplemental information is closed captioning text in a language that is different from the language of audible dialogue of the presented media content event; and

present the closed captioning text in synchronism with the video portion of the media content event being presented on the display.