ADAPTIVE PATTERN RECOGNITION BASED CONTROLLER APPARATUS AND METHOD AND HUMAN-FACTORED INTERFACE THEREFORE

• US 20070061022A1
• Filed: 08/29/2006
• Published: 03/15/2007
• Est. Priority Date: 12/23/1991
• Status: Active Grant

• Current Assignee: Hoffberg Family Trust 2, Blanding Hovenweep LLC (Intellectual Ventures LLC), STEVEN M. HOFFBERG 2004-1 GRAT
• Original Assignee: Hoffberg Family Trust 2
• Inventors: Hoffberg, Steven, Hoffberg-Borghesani, Linda
• Granted Patent: US 9,727,042 B2
• Time in Patent Office: Days
• US Class Current: 700/83
• CPC Class Codes: G05B 19/0426 Programming the control sequence
• CPC Class Codes: G05B 2219/23026 Recognise user input pattern and present possible intended program
• CPC Class Codes: G05B 2219/24162 Biometric sensor, fingerprint as user access password
• CPC Class Codes: G05B 2219/2613 Household appliance in general
• CPC Class Codes: G05B 2219/2615 Audio, video, tv, consumer electronics device
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42221 Transmission circuitry, e.g. infrared [IR] or radio frequency [RF]
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4263 involving specific tuning arrangements, e.g. two tuners
• CPC Class Codes: H04N 21/4332 by placing content in organized collections, e.g. local EPG data repository interfaces, Database management systems or updating for information retrieval G06F16/23; details of retrieval of video data and associated meta data in video database G06F16/739
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/4415 using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/44224 : Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/4532 : involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4622 : Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/466 : Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/4667 : Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/4668 : for recommending content, e.g. movies
• CPC Class Codes: H04N 21/47 : End-user applications
• CPC Class Codes: H04N 21/4753 : for user identification, e.g. by entering a PIN or password cryptographic authentication protocols H04L9/32; networks authentication protocols H04L63/08
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/4826 : using recommendation lists, e.g. of programs or channels sorted out according to their score
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 5/765 : Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/782 : on tape
• CPC Class Codes: H04N 5/84 : using optical recording H04N5/80, H04N5/89, H04N5/91 take precedence
• CPC Class Codes: H04N 9/8042 : involving data reduction

Abstract

The need for a more readily usable interface for programmable devices is widely recognized. The present invention relates to programmable sequencing devices, or, more particularly, the remote controls for consumer electronic devices. The present invention provides an enhanced interface for facilitating human input of a desired control sequence in a programmable device by employing specialized visual feedback. The present invention also relates to a new interface and method of interfacing with a programmable device, which is usable as an interface for a programmable video cassette recorder.

Claims

19. A method, comprising the steps of:

storing a history of usage, determining a user'"'"'s context, determining a set of actions available for selection, and presenting to the user one or more of the set of actions in dependence on at least one of a history of use and a status of use, appropriate for the user'"'"'s context.

20. The method according to claim 19, wherein the action is suggested dependent on at least a portion of a history of user actions.

21. The method according to claim 19, wherein the action is suggested dependent on a system status.

22. A method of searching available media content for content of interest to a user, comprising:

receiving content-identifying parameters from a user;

defining a logical sequence of content identifiers corresponding to the content-identifying parameters and a user preference profile, to provide a personalized result for the user; and

presenting the personalized result set to the user for a selection of a content identifier;

automatically accessing the content associated with the content identifier.

23. A programmable system, comprising:

(a) a communication interface with an on-line database, for communicating information from the programmable system comprising a request for information from the on-line database, and for communicating information from the on-line database to the programmable system comprising a response to the request for information;

(b) a user interface, comprising a plurality of predefined schemas for interaction with the user, each schema having a first portion comprising a set of predetermined functional elements which are consistently presented in said plurality of predefined schemas, and a second portion comprising a set of non-predetermined functional elements dependent on a user input and a response from the on-line database; and

(c) a user data entry portion of at least one of said schemas, wherein the programmable system at least one of presents to the user a prediction of a user-desired entry, and communicates to the user if a user entry differs from a predicted user-desired entry.

24. An apparatus, comprising:

a first database for storing data relating to media previously presented to the user;

a second database for storing information describing available media;

a processor, for analyzing information in the first and second databases and formulating a list representing a plurality of predicted preferred media, the list excluding at least one media redundancy with another media on the list; and

an input for receiving a selection of media by a user from the list.

25. The apparatus according to claim 24, further comprising an electronic communication interface for selectively receiving and presenting to the user the selected media.

26. The apparatus according to claim 24, wherein he second database comprises an editorially derived description of media.

27. The apparatus according to claim 24, wherein he second database comprises an automatically derived analysis of media.

28. The apparatus according to claim 24, wherein said processor is remote to a user, with respect to the input.

29. The apparatus according to claim 24, wherein said processor is local to a user, with respect to the input.

30. An apparatus, comprising:

a first database for storing data relating to media previously presented to the user;

a second database for storing information describing available media;

a processor, for analyzing information in the first and second databases and formulating a list representing a plurality of predicted preferred media, the list being biased against media previously presented to the user; and

an input for receiving a selection of media by a user from the list.

31. A user interface system, comprising:

an input for receiving a user input;

a first memory location for persistently storing at least a portion of a history of user interaction with the user interface;

an input for receiving context information defining an environment of operation of the user interface system;

a processor for predicting a desired user action based on the user input, the stored portion of the history of user interaction with the user interface, the context of use, and a set of predetermined or adaptive rules.

32. A data storage medium storing a media program having a plurality of segments, at least a portion of the segments being time coded, wherein during reproduction of the media program, a portion of the segments are replaced in dependence on a staleness of a respective time code of a respective segment.

1. A method for providing a user interface for a media processing device, comprising:

(a) providing a media recommender to recommend an entertainment media object based at least in part on a media usage history analysis to implicitly derive user preferences;

(b) conducting an interactive dialog with a user; and

(c) selectively processing at least one media object based on the interactive dialog, and the media recommendation.

2. The method according to claim 1, further comprising the step of providing a speech information processing component.

3. The method according to claim 1, further comprising the step of providing an image information processing component.

4. The method according to claim 1, wherein a user identification is determined by a biometric analysis.

5. The method according to claim 4, wherein the biometric analysis comprises speech analysis.

6. The method according to claim 4, wherein the biometric analysis comprises image analysis.

7. The method according to claim 6, wherein the biometric analysis comprises facial analysis.

8. The method according to claim 1, wherein the recommendation is group-aware.

9. The method according to claim 1, wherein there are a plurality of users, each user having a respective user preference, wherein the program recommendation is based on the respective user preferences from the plurality of users.

10. The method according to claim 1, wherein said selectively processing step comprises recording a recommended program.

11. The method according to claim 1, wherein said selectively processing step comprises presenting a list of recommended programs.

12. The method according to claim 1, wherein the media processing device comprises a video tuner.

13. The method according to claim 1, wherein the media processing device comprises a packet data network interface.

14. The method according to claim 1, wherein the interactive dialog is conducted through a speech interface.

15. The method according to claim 1, wherein the interactive dialog is conducted through a graphic user interface.

16. The method according to claim 1, wherein the recommender semantically analyzes a media object.

17. The method according to claim 1, wherein the recommender classifies a genre of a media object.

18. The method according to claim 1, wherein the recommender determines a composite preference for a plurality of users.

33. The data storage medium according to claim 32, wherein the media program segments comprise at least one commercial advertisement.

34. The data storage medium according to claim 33, wherein at least a portion of the commercial advertisements are replaced with alternate commercial advertisements.

35. The data storage medium according to claim 33, wherein the commercial advertisements are demographically targeted.

35-1. The data storage medium according to claim 33, wherein the commercial advertisements are targeted based on a user profile.

36. A method for controlling a media system, comprising:

adaptively and implicitly determining a viewer preference based on a user input;

automatically characterizing the program material based on its content;

correlating the characterized content of the program material with the determined viewer preference; and

selectively processing the program material based on a result of the correlating step, and the user input.

37. A system comprising:

(a) a mass storage access device, storing entertainment media content;

(b) a control, controlling access to the mass storage access device;

(c) a data interface, communicating data packets defining a user-desired instruction of the control, at least one communication path of the data interface communicating wirelessly; and

(d) a user interface, permitting a human to interact with the control, to define an instruction of the control, the user interface having a menu structure and a graphic user interface, the user interface being remote with respect to the data interface from the mass storage access device.

38. A system comprising:

(a) a mass storage access device, storing entertainment media content;

(b) a control, controlling access to the mass storage access device;

(c) a data interface, communicating data packets defining a user-desired instruction of the control;

(d) a user interface, permitting a human to interact with the control, to define an instruction of the control, the user interface having a menu structure and a graphic user interface, the user interface being remote with respect to the data interface from the mass storage access device, the user interface presenting a plurality of choices for the user to select, based on a plurality of selection criteria, and permitting a selection of at least one of the choices for processing by the control.

39. A system comprising:

(a) a mass storage access device, storing entertainment media content;

(b) a communication interface for receiving electronic program guide information from an on-line database;

(c) a telephony system interface; and

(d) a direct manipulation-type graphic user interface for presenting a selected portion of the electronic program guide information, and being adapted for receiving a user selection of at least one program, and for controlling the telephony system interface.

40. A system comprising:

(a) a mass storage access controller, adapted for storing a stream of media content; and

(b) a menu-driven graphic user interface for controlling the mass storage access controller, receiving a human user instruction through a direct manipulation-type user input, at least one available menu selection of the menu-driven graphic user interface being selectively presented dependent on an implicitly derived user preference.

41. A programmable system, comprising:

(a) a communication interface for communicating with an on-line database indexing available content; and

(b) a hypertext graphic user interface, comprising a user input screen, having a first set of predetermined functional elements having consistent function and placement, and a second set of elements comprising index information from the on-line database generated in response to a user input,

wherein a selection of one of the second set of elements by a user triggers a storage in local memory of corresponding content separate from the on-line database.

42. A user interface system, comprising:

an input for receiving a user input;

an output for presenting to the user a set of options, a complexity of presented options being adaptively determined based on an estimated user sophistication, based on at least the user input; and

an output, for implementing a selected option by the user through the user input.

43. A user interface system, comprising:

an input for receiving a user input;

an output for presenting to the user a set of options, the set of options being dependent at least in part on a prior efficiency of user inputs in achieving a result; and

an output, for implementing a selected option by the user through the user input.

44. A user interface system, comprising:

an input for receiving a user input;

an output for presenting to the user a set of options, the set of options being dependent at least in part a context and a predicted set of likely desired functions; and

an output, for implementing a selected option by the user through the user input.

45. A user interface system, comprising:

an input for receiving a user input;

an output for presenting to the user a set of options, a complexity of presented options being adaptively determined based on an estimated user sophistication, based on at least the user input; and

an output, for implementing a selected option by the user through the user input.

46. A user interface system, comprising:

an input for receiving a user input;

an output for presenting to the user a set of options, the set of options being dependent at least in part on a prior efficiency of user inputs in achieving a result; and

an output, for implementing a selected option by the user through the user input.

47. A user interface system, comprising:

an input for receiving a user input;

an output for presenting to the user a set of options, the set of options being dependent at least in part a context and a predicted set of likely desired functions; and

an output, for implementing a selected option by the user through the user input.

48. A set top box comprising:

a video-conferencing system for bidirectionally communicating digitally compressed audio and video information;

a tuner for selecting a channel from a plurality of available video channels;

a memory for storing an electronic program guide;

a direct manipulation style user interface for selecting a program from the program guide, controlling the tuner, and controlling the video-conferencing system; and

an output adapted for presenting a video conference and a channel selected by the tuner.

49. A control system, comprising:

a data input;

a controller, processing the data and producing an output according to a predetermined algorithm; and

a user input for programming the controller,

wherein the predetermined algorithm is adaptive to both the data input and the user input.

50. A media search system, comprising:

a database having a plurality of records, each record having an identifier for a media stream and a set of automatically acquired text relating to a content of a respective media stream; and

a user interface, receiving a user request, defining a list of available media streams corresponding to the user request, and receiving a user input for selecting one of the available media streams.

51. A media processing system, comprising:

(a) a digital memory for storing an electronic program guide providing characteristics of a plurality of media programs;

(b) a digital media buffer, for storing at least a portion of a media program;

(c) a speech input for receiving spoken commands from a user; and

(d) a digital processor, for receiving the spoken commands through the speech input, and controlling a transfer of at least a portion of at least one media program with respect to the buffer in dependence on the spoken commands.

52. An environmental security system, comprising:

a least one input for receiving a signal from a environmental security sensor;

a computer network interface;

an image processor for communicating digitally compressed images through the computer network interface; and

a processor, for receiving the at least one input, controlling the image processor, and communicating through the computer network interface.

53. The system according to claim 52, further comprising a local area network interface, wherein at least one of the data input and the user input is communicated through the local area network interface.

54. A system comprising:

(a) a compressed voice telecommunications interface; and

(b) a direct manipulation-type graphic user interface for controlling the telecommunications interface, for controlling at least one function selected from the group consisting of a call diversion function, an automated teledictation function, a conferencing function, caller identification-database interaction function, and a videotel function.

55. A system comprising:

(a) a mass storage access device;

(b) a digital image and voice telecommunications interface; and

(c) a direct manipulation-type graphic user interface for controlling the mass storage access device, and voice telecommunications interface.

56. A system comprising:

(a) a control, producing control signals for controlling a system and operating in accordance with a predetermined program, a set of user-defined instructions;

(b) a data interface, communicating the user-defined instructions to the control;

(c) an adaptive graphic user interface, providing feedback on a state of the control to the user, and presenting a subset of available options for human interaction with the control, to define an instruction of the control, the subset being dependent on at least one adaptation parameter which varies in dependence on a past history of user input.

57. A method for controlling a climate of an environment, comprising:

receiving a sensor input representing a climate environment parameter to be controlled;

determining a status of a portal between the environment and an external space; and

controlling a climate of the environment to reduce a climate loss through the portal in dependence on the climate environmental parameter and the portal status.

58. A system comprising:

(a) a mass storage access device, storing entertainment media content;

(b) a control, controlling access to the mass storage access device;

(c) a data interface, communicating data packets defining a user-desired instruction of the control, at least one communication path of the data interface communicating wirelessly;

(d) a user interface, permitting a human to interact with the control, to define an instruction of the control, the user interface having a menu structure and a graphic user interface, the user interface being remote with respect to the data interface from the mass storage access device.

59. A system comprising:

(a) a mass storage access device, storing entertainment media content;

(b) a control, controlling access to the mass storage access device;

(c) a data interface, communicating data packets defining a user-desired instruction of the control;

(d) a user interface, permitting a human to interact with the control, to define an instruction of the control, the user interface having a menu structure and a graphic user interface, the user interface being remote with respect to the data interface from the mass storage access device, the user interface presenting a plurality of choices for the user to select, based on a plurality of selection criteria, and permitting a selection of at least one of the choices for processing by the control.

60. A method for processing an image, comprising:

providing at least two images representing time-sequential or relative spatial displacement images of at least one object;

extracting from the at least two images spatial object information;

populating a three-dimensional model of the at least one object from the images and spatial object information; and

classifying the object based on the three dimensional model.

61. The method according to claim 60, wherein said classifying step comprises determining a relation of the three dimensional model with at least one object exemplar.

62. The method according to claim 61, wherein an identification of the object is proposed based on a closeness of a relation between the three dimensional object and the at least one object exemplar.

63. A system for analyzing an electronic representation of a physical scene, having dimensions of space and time, comprising:

an input for receiving the electronic representation;

processing the electronic representation to extract objects therein;

determining morphological degrees of freedom of the extracted objects;

selecting a model corresponding to an extracted object,

encoding the extracted objects as a set of temporally changing states of the degrees of freedom of the corresponding model; and

outputting the identification of the corresponding model and set of temporally changing states.

64. A system for analyzing an electronic representation of a three dimensional object in a time-sequence of images, comprising:

processing the electronic representation to extract a time-varying foreground image;

representing at least a portion of the extracted time-varying foreground image as an object based on a correspondence of the extract a time-varying foreground image with a model of the object; and

outputting an identification of the model.

65. A system for compressing a video sequence, comprising:

processing the electronic representation to identify objects within the video sequence;

characterizing the identified objects;

coding the identified objects using compression parameters in dependence on the characterization; and

outputting the encoded objects as a representation of at least a portion of the contents of the video sequence.

66. A system for analyzing an electronic representation of a physical scene, comprising:

an input for receiving the electronic representation comprising a series of two-dimensional images;

processing the electronic representation to estimate a representation having a physical dimensionality greater than two;

correlating the estimated representation with a set of physical object exemplars, to identify physical objects projected in the two-dimensional images; and

encoding the identified objects.

67. An image processing method, comprising:

(a) processing a sequence of images to extract a three dimensional object represented as a sequence of two dimensional projections; and

(b) representing the three dimensional object as a set of parameters defining the object, defining a translation of the object in the image, and defining a change in projection of the object

68. An image processing method, comprising:

(a) processing a sequence of images to extract at least one three dimensional object represented as a sequence of two dimensional projections;

(b) comparing the at least one extracted object to a set of exemplars to determine a correspondence with at least one exemplar, having at least one degree of freedom;

(c) representing the at least one object as a set of parameters representing an identification of the at least one exemplar, and defining the at least one degree of freedom; and

(d) optionally expressing an error vector relating to a difference between an image and a representation of the image defined by the sets of parameters for the at least one object.

69. The image processing method according to claim 68, further comprising the step of normalizing the extracted at least one three dimensional object before comparing it to the set of exemplars.

70. The image processing method according to claim 69, wherein a set of normalizing parameters are determined during said normalizing, said normalizing parameters being used to define said degrees of freedom.

71. The image processing method according to claim 68, wherein an exemplar is selected based on a ranked degree of correlation between the object and a respective exemplar.

72. The image processing method according to claim 68, wherein at least one exemplar is adaptively defined based on the sequence of images.

73. The image processing method according to claim 68, wherein a plurality of exemplars are jointly used to define a single object.

74. The image processing method according to claim 68, wherein an object is defined with respect to a hybrid exemplar resulting from a combination of a plurality of exemplars.

75. The image processing method according to claim 68, wherein an association between an object and at least one exemplar is temporally persistent through a sequence of images, wherein once a set of corresponding exemplars is determined, the association persists in preference to a reassociation with a different set of exemplars.

76. The image processing method according to claim 75, wherein a decision to reassociate the object with a different set of exemplars is dependent on a complexity of the error vector.

77. The image processing method according to claim 68, further comprising the step of outputting a semantic description of the sequence of images based on the corresponding at least one exemplar.

78. The image processing method according to claim 77, further comprising the step of outputting a semantic description of the sequence of images at least based on the determined degrees of freedom.

79. The image processing method according to claim 68, wherein the identification of the at least one exemplar is appended to the sequence of images as descriptive metadata.

80. A medical support system, comprising:

(a) a user interface, adapted for receiving a caregiver identification, and caregiver input with respect to treatment of a patient;

(b) a medical data interface for receiving patient-specific data;

(c) a processor receiving a caregiver-specific profile, and analyzing at least the patient-specific data and producing an output through the user interface in dependence on the caregiver profile, adapted for highlighting for significant medical events, and a prediction of an appropriate action by the caregiver in response.