Interactive generating query related to telestrator data designating at least a portion of the still image frame and data identifying a user is generated from the user designating a selected region on the display screen, transmitting the query to the remote information system

• US 7,240,075 B1
• Filed: 09/24/2003
• Issued: 07/03/2007
• Est. Priority Date: 09/24/2002
• Status: Expired due to Fees

• Current Assignee: expHand, Inc.
• Original Assignee: expHand, Inc.
• Inventors: Lincoln, Larry A., Nemirofsky, Frank Robert
• Primary Examiner(s): CHANNAVAJJALA, SRIRAMA T
• Application Number:
  US10/670,998
• Time in Patent Office: 1,378 Days
• Field of Search: 707 1- 57, 707100-1041, 707/200, 705 1- 2, 705/14, 705 26- 27, 345/2.1, 345/357.07, 345419-420, 345/357.03, 345/629, 345/630, 434/252, 725/116, 725/131, 725/86, 709210-218, 396/435, 348 21- 22, 348/14.03, 348/141, 348/584, 348/586, 382/276, 34235706-35709
• US Class Current: 1/1
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

Methods, apparatuses, and systems enabling an interactive information retrieval system that includes a user interface allowing users to graphically generate information queries for objects of interest found in various media, such as television, video, and print. The invention can allow users to designate objects of interest within a television or video display and receive information related to the objects of interest. The invention can also allow users to generate queries designating objects of interest within a graphical image using a GUI.

Claims

1. A method for graphically generating a search query for transmission to a remote information system comprising:

transmitting a broadcast to a display device;

receiving a command to pause the broadcast from a remote device wherein the remote device has a display screen;

transmitting a still image frame to the display device and the remote device upon receiving the pause command wherein the still image frame comprises a predefined image frame that corresponds to an image frame of the broadcast that was displayed at substantially the moment when the pause command was received;

receiving telestrator data designating at least a portion of the still image frame and data identifying a user wherein the telestrator data is generated from the user designating a selected region on the display screen of the remote device;

generating the search query comprising the telestrator data, the data identifying the user, and data identifying the still image frame; and

transmitting the search query to the remote information system.

2. The method of claim 1, further comprising displaying the telestrator data on the display device.

3. The method of claim 2, wherein the telestrator data is overlaid onto the still image frame displayed on the display device.

4. The method of claim 1, wherein the broadcast comprises a satellite broadcast or a cable broadcast.

5. The method of claim 1, wherein the broadcast comprises a previously recorded broadcast stored on a storage device.

6. The method of claim 1, wherein the still image frame comprises an image frame of the broadcast that was displayed at substantially the moment when the pause command was received.

7. The method of claim 1, wherein the telestrator data designates an object of interest in the still image frame.

8. The method of claim 7, wherein the telestrator data comprises lines and/or curves enclosing the object of interest within the still image frame.

9. The method of claim 7, wherein the telestrator data comprises one or more pixels placed directly atop the object of interest within the still image frame.

10. The method of claim 7, wherein the telestrator data comprises a scribble placed directly atop the object of interest within the still image frame.

11. The method of claim 7, wherein the object of interest is an object shown in the still image frame for which the user desires further information.

12. The method of claim 1, wherein transmitting the query to a remote information system is carried out over a telephone network.

13. The method of claim 1, wherein transmitting the query to a remote information system is carried out over a computer network.

14. The method of claim 13, wherein the computer network comprises the Internet.

15. The method of claim 1, wherein transmitting the query to a remote information system is carried out over a wireless network.

16. The method of claim 1, wherein the commands and the data are received in the form of wireless signals.

17. The method of claim 16, wherein the wireless signals comprise infrared signals.

18. The method of claim 16, wherein the wireless signals comprise Bluetooth signals.

19. The method of claim 16, wherein the wireless signals comprise 802.11 signals.

20. The method of claim 1, wherein the data identifying the still image frame comprises a frame number.

21. The method of claim 1, wherein the data identifying the still image frame comprises a time value.

22. The method of claim 1, wherein the data identifying the still image frame is found in the vertical blanking interval.

23. The method of claim 1, wherein the data identifying the still image frame is found in the broadcast signal.

24. The method of claim 1, wherein the data identifying the still image frame includes a program identifier.

25. The method of claim 1, wherein the display device comprises a television.

26. An apparatus for graphically generating a search query for transmission to a remote information system comprising:

a processor;

a memory;

a wireless communications system;

a software application, physically stored in the memory, for generating a query, comprising instructions operable to cause the processor and the apparatus to;

transmit a broadcast to a display device;

receive a command to pause the broadcast from a remote device wherein the remote device has a display screen;

transmit a still image frame to the display device and the remote device upon receiving the pause command wherein the still image frame comprises a predefined image frame that corresponds to an image frame of the broadcast that was displayed at substantially the moment when the pause command was received;

receive telestrator data designating at least a portion of the still image frame and data identifying a user wherein the telestrator data is generated from the user designating a selected region on the display screen of the remote device;

generate the search query comprising the telestrator data, the data identifying the user, and data identifying the still image frame; and

transmit the search query to the remote information system.

27. The apparatus of claim 26, wherein the wireless communications system comprises an infrared communications system.

28. The apparatus of claim 26, wherein the wireless communications system comprises a Bluetooth communications system.

29. The apparatus of claim 26, wherein the wireless communications system comprises an 802.11 communications system.

30. The apparatus of claim 26, wherein the software application further comprises instructions operable to cause the processor and the apparatus to display the telestrator data on the display device.

31. The apparatus of claim 30, wherein the telestrator data is overlaid onto the still image frame displayed on the display device.

32. The apparatus of claim 26, wherein the still image frame comprises an image frame of the broadcast that was displayed at substantially the moment when the pause command was received.

33. The apparatus of claim 26, wherein the telestrator data designates an object of interest in the still image frame.

34. The apparatus of claim 26, wherein transmitting the query to a remote information system is carried out over a telephone network, a computer network, or a wireless network.

35. The apparatus of claim 26, wherein the commands and the data are received in the form of wireless signals.

36. The apparatus of claim 35, wherein the wireless signals comprise infrared signals, Bluetooth signals, or 802.11 signals.

37. The apparatus of claim 26, wherein the data identifying the still image frame includes a program identifier.

38. A computer program product, physically stored on a machine-readable medium, for graphically generating a search query for transmission to a remote information system, comprising instructions operable to cause a programmable processor to:

transmit a broadcast to a display device;

receive a command to pause the broadcast from a remote device wherein the remote device has a display screen;

transmit a still image frame to the display device and the remote device upon receiving the pause command wherein the still image frame comprises a predefined image frame that corresponds to an image frame of the broadcast that was displayed at substantially the moment when the pause command was received;

receive telestrator data designating at least a portion of the still image frame and data identifying a user wherein the telestrator data is generated from the user designating a selected region on a selected region of a display screen of the remote device;

generate the search query comprising the telestrator data, the data identifying the user, and data identifying the still image frame; and

transmit the search query to the remote information system.

39. A data processing system for graphically generating a search query for transmission to a remote information system, comprising:

means for transmitting a broadcast to a display device;

means for receiving a command to pause the broadcast from a remote device wherein the remote device has a display screen;

means for transmitting a still image frame to the display device and the remote device upon receiving the pause command wherein the still image frame comprises a predefined image frame that corresponds to an image frame of the broadcast that was displayed at substantially the moment when the cause command was received;

means for receiving telestrator data designating at least a portion of the still image frame and data identifying a user wherein the telestrator data is generated from the user designating a selected region on the display screen of the remote device;

means for generating the search query comprising the telestrator data, the data identifying the user, and data identifying the still image frame; and

means for transmitting the search query to the remote information system.

40. A method for graphically generating a search query for transmission to a remote information system comprising:

transmitting a pause command to a receiver wherein the pause command results in a still image frame on a display device;

receiving still image data describing the still image frame wherein the still image frame comprises an image frame of a broadcast that was displayed by the receiver at substantially the moment when the pause command was transmitted;

displaying a representation of the still image frame on a display screen based on the transmitted still image data;

receiving user input comprising telestrator data designating a portion of the representation wherein the telestrator data is generated from a user designating a selected region on the display screen and wherein the telestrator data contains search parameter information to be used as a basis for the search query;

transmitting the telestrator data to the receiver; and

transmitting data identifying the user to the receiver.

41. The method of claim 40, wherein the still image frame comprises a box having an aspect ratio that corresponds to an aspect ratio of a broadcast displayed by the receiver.

42. The method of claim 40, wherein the telestrator data designates an object of interest in the still image frame.

43. The method of claim 42, wherein the telestrator data comprises lines and/or curves enclosing the object of interest within the still image frame.

44. The method of claim 42, wherein the telestrator data comprises one or more pixels placed directly atop the object of interest within the still image frame.

45. The method of claim 42, wherein the telestrator data comprises a scribble placed directly atop the object of interest within the still image frame.

46. The method of claim 42, wherein the object of interest is an object shown in the still image frame for which the user desires further information.

47. The method of claim 40, wherein the commands and the data are transmitted in the form of wireless signals.

48. The method of claim 47, wherein the wireless signals comprise infrared signals.

49. The method of claim 47, wherein the wireless signals comprise Bluetooth signals.

50. The method of claim 47, wherein the wireless signals comprise 802.11 signals.

51. An apparatus for graphically generating a search query for transmission to a remote information system comprising:

a display screen;

a user interface;

a wireless communications system;

a processor;

a memory; and

a client application, physically stored in the memory, for generating a query, comprising instructions operable to cause the processor and the apparatus to;

transmit a pause command to a receiver wherein the pause command results in a still image frame on a display device;

receive still image data describing the still image frame wherein the still image frame comprises an image frame of a broadcast that was displayed by the receiver at substantially the moment when the pause command was transmitted;

display a representation of the still image frame on the display screen;

receive user input comprising telestrator data designating a portion of the representation wherein the telestrator data is generated from a user designating a selected region on the display screen and wherein the telestrator data contains search parameter information to be used as a basis for the search query;

transmit the telestrator data to the receiver; and

transmit data identifying the user to the receiver.

52. The apparatus of claim 51, wherein the still image frame comprises a box having an aspect ratio that corresponds to an aspect ratio of a broadcast displayed by the receiver.

53. The apparatus of claim 51, wherein the telestrator data designates an object of interest in the still image frame.

54. The apparatus of claim 51, wherein the object of interest is an object shown in the still image frame for which the user desires further information.

55. The apparatus of claim 51, wherein the commands and the data are transmitted in the form of wireless signals.

56. The apparatus of claim 55, wherein the wireless signals comprise infrared signals.

57. The apparatus of claim 55, wherein the wireless signals comprise Bluetooth signals.

58. The apparatus of claim 55, wherein the wireless signals comprise 802.11 signals.

59. A computer program product, physically stored on a machine-readable medium, for graphically generating a search query for transmission to remote information system, comprising instructions operable to cause a programmable processor to:

transmit a pause command to a receiver wherein the pause command results in a still image frame on a first display device;

receive still image data describing the still image frame wherein the still image frame comprises an image frame of a broadcast that was displayed by the receiver at substantially the moment when the cause command was transmitted;

display a representation of the still image frame on a display screen;

receive user input comprising telestrator data designating a portion of the representation wherein the telestrator data is generated from a user designating a selected region on the display screen and wherein the telestrator data contains search parameter information to be used as a basis for the search query;

transmit the telestrator data to the receiver; and

transmit data identifying the user to the receiver.

60. A data processing system for graphically generating a search query for transmission to a remote information system, comprising:

means for transmitting a pause command to a receiver wherein the pause command results in a still image frame on a display device;

means for receiving still image data describing the still image frame wherein the still image frame comprises an image frame of a broadcast that was displayed by the receiver at substantially the moment when the pause command was transmitted;

means for displaying a representation of the still image on a display screen of the data processing system;

means for receiving user input comprising telestrator data designating a portion of the representation wherein the telestrator data is generated from a user designating a selected region on the display screen and wherein the telestrator data contains search parameter information to be used as a basis for the search query;

means for transmitting the telestrator data to the receiver; and

means for transmitting data identifying the user to the receiver.

61. A method for retrieving information, about an object, utilizing a graphically generated search query transmitted from a remote location comprising:

receiving the search query comprising telestrator data, data identifying a user, data identifying a still image frame, and a program identifier wherein the telestrator data is generated from the user designating a selected region on a first display screen of a first device that displays a still image frame wherein the still image frame is also displayed on a second display screen of a second device, wherein the still image frame comprises an image frame of a broadcast that was displayed by a receiver at substantially the moment when a pause command was transmitted;

identifying an object locator table associated with the program identifier;

retrieving a portion of the object locator table based on the data identifying the still image frame;

identifying objects of interest in the retrieved portion of the object locator table using the telestrator data; and

providing information associated with the identified objects of interest to the user.

62. The method of claim 61, wherein the program identifier is station identification information, channel identification information, or vertical blanking interval data.

63. The method of claim 61, wherein the data identifying the still image frame is a frame number or a time value.

64. The method of claim 61, wherein the object locator table comprises a relational database.

65. The method of claim 61, wherein the telestrator data comprises x, y positional parameters on the still image frame or x, y composite ratios relative to the image aspect of the still image frame.

66. The method of claim 61, wherein providing information to the user comprises sending an e-mail to the user, providing the information on an Internet website for the user, or transmitting the information to a client device of the user.

67. An apparatus for retrieving information, about an object, utilizing a graphically generated search query transmitted from a remote location to the apparatus comprising:

a processor;

a memory;

a communications system;

a software application, physically stored in the memory, for generating a query, comprising instructions operable to cause the processor and the apparatus to;

receive the search query comprising telestrator data, data identifying a user, data identifying a still image frame, and a program identifier wherein the telestrator data is generated from the user designating a selected region on a display screen of a first device that displays a still image frame wherein the still image frame is also displayed on a second display screen of a second device, wherein the still image frame comprises an image frame of a broadcast that was displayed by a receiver at substantially the moment when a pause command was transmitted;

identify an object locator table associated with the program identifier;

retrieve a portion of the object locator table based on the data identifying the still image frame;

identify objects of interest in the retrieved portion of the object locator table using the telestrator data; and

provide information associated with the identified objects of interest to the user.

68. A computer program product, physically stored on a machine-readable medium, for retrieving information, about an object, utilizing a graphically generated search query transmitted from a remote location, comprising instructions operable to cause a programmable processor to:

receive the search query comprising telestrator data, data identifying a user, data identifying a still image frame, and a program identifier wherein the telestrator data is generated from the user designating a selected region on a display screen of a first device that displays a still image frame wherein the still image frame is also displayed on a second display screen of a second device, wherein the still image frame comprises an image frame of a broadcast that was displayed by a receiver at substantially the moment when a pause command was transmitted;

identify an object locator table associated with the program identifier;

retrieve a portion of the object locator table based on the data identifying the still image frame;

identify objects of interest in the retrieved portion of the object locator table using the telestrator data; and

provide information associated with the identified objects of interest to the user.

69. A data processing system for retrieving information, about an object, utilizing a graphically generated search query transmitted from a remote location comprising:

means for receiving the search query comprising telestrator data, data identifying a user, data identifying a still image frame, and a program identifier wherein the telestrator data is generated from the user designating a selected region on a display screen of a first device that displays a still image frame wherein the still image frame is also displayed on a second display screen of a second device, wherein the still image frame comprises an image frame of a broadcast that was displayed by a receiver at substantially the moment when a pause command was transmitted;

means for identifying an object locator table associated with the program identifier;

means for retrieving a portion of the object locator table based on the data identifying the still image frame;

means for identifying objects of interest in the retrieved portion of the object locator table using the telestrator data; and

means for providing information associated with the identified objects of interest to the user.