Control system and user interface for network of input devices

• US 7,690,017 B2
• Filed: 05/02/2002
• Issued: 03/30/2010
• Est. Priority Date: 05/03/2001
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric US Incorporated (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Digital Electronics America Incorporated (Mitsubishi Electric Corporation)
• Inventors: Maxon, Brian, Houda, Pavel, Palmer, George E., Stecyk, Polly, Pu, Shenta T., Perry, Robert A., Zanfino, Martin, Peterson, Brian
• Primary Examiner(s): Salce; Jason P
• Application Number:
  US10/138,690
• Publication Number:
  US 20020174270A1
• Time in Patent Office: 2,889 Days
• Field of Search: 725 38- 39, 725 47- 48, 725/52, 725/59, 725/78, 725 80- 82, 725/85, 348/734, 348552-554, 348705-706, 348/460, 348/731, 715716-718, 715/714, 700/17, 340/825.69, 340/825.22, 340/825.24, 340/825.25
• US Class Current: 725/80
• CPC Class Codes: H04L 12/2805 Home Audio Video Interoperability [HAVI] networks
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 2012/2849 Audio/video appliances
• CPC Class Codes: H04L 2012/285 Generic home appliances, e.g. refrigerators

Abstract

Apparatus, methods, and systems for centrally and uniformly controlling the operation of a variety of devices, such as communication, consumer electronic, audio-video, analog, digital, 1394, and the like, over a variety of protocols within a network system and, more particularly, a control system and uniform user interface for centrally controlling these devices in a manner that appears seamless and transparent to the user. In a preferred embodiment, a command center or hub of a network system includes a context and connection permutation sensitive control system that enables centralized and seamless integrated control of all types of input devices. The control system preferably includes a versatile icon based graphical user interface that provides a uniform, on-screen centralized control system for the network system. The user interface, which includes a visual recognition system, enables the user to transparently control multiple input devices over a variety of protocols while operating on a single control layer of an input command device. In an alternative embodiment, the control system also enables gated signal pass-through control while avoiding signal jamming.

Claims

1. A method of controlling a network system comprising a primary video display and audio output device (PDD) and a plurality of input audio-video devices, wherein the plurality of input audio-video devices include one or more analog audio-video devices and one or more digital audio-video devices, comprising the steps of

generating a non-device specific graphical user interface (GUI) from a central control unit (CCU) of a network system and displaying the GUI on the PDD, wherein the GUI includes an input device selection menu and a device transport menu, and wherein the input device selection menu includes an icon corresponding to each of the one or more analog audio-video devices and the one or more digital audio-video devices,

receiving CCU specific user input commands corresponding to selection and operation of a plurality of input audio-video devices and network system functions at the CCU, wherein the network system includes a plurality of input audio-video devices comprising one or more analog audio-video devices and one or more digital audio-video devices,

translating within the CCU the CCU specific user input commands received by the CCU to device specific output messages to operate the plurality of input audio-video devices,

transmitting the output messages from the CCU to the plurality of input audio-video devices, wherein the output messages are transmitted to the one or more analog audio-video devices by directing an IR signal incorporating the output message from the CCU to the analog audio-video device, wherein the output messages are transmitted to the one or more digital audio-video devices over a bidirectional bus, and wherein the one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus, and

directing an audio and/or video signal from a first source device of the plurality of input audio-video devices to a sink device of the plurality of input audio-video devices via device specific output messages transmitted from the CCU to the plurality of input audio-video devices while directing a signal from a second source device of the plurality of input audio-video devices to the PDD, wherein the first and second source devices and the sink device include one or more analog audio-video devices selected from the plurality of input audio-video devices of the network system.

2. The method of claim 1 further comprising the step of maintaining within memory in the CCU a list of all input devices including the plurality of input audio-video devices, all input device connections, and all I/O ports within the network system.

3. The method of claim 1 wherein the receiving step further comprises the step of detecting input command signals from a remote control device.

4. The method of claim 3 further comprising the step of transmitting input command signals from a single control layer of the remote control device corresponding to the CCU for control of all input audio-video devices.

5. The method of claim 4 wherein the transmitting input command signals step further comprises the steps of transmitting input commands for a first input audio-video device from a first control layer of the remote control device, and transmitting input commands for a second input audio-video device without switching to a second control layer of the remote control device, wherein the first input audio-video device is an analog device and the second input audio-video device is a digital device.

6. The method of claim 1 further comprising the step of inputting commands from an input device, wherein identical input commands are used to drive both analog and digital input audio video devices.

7. The method of claim 1 wherein the step of translating further comprises the step of transforming user input commands for analog devices into device appropriate messages comprising device specific IR code.

8. The method of claim 1 wherein the step of translating further comprises the step of transforming user input commands for digital devices into device appropriate messages formulated by device control modules corresponding to the digital devices.

9. The method of claim 1 wherein the translating step further comprises the step of determining if the input command is a direct device action event or a menu related event.

10. The method of claim 9 further comprising the step of determining whether the input command is a menu display event, a menu navigation event, or a menu device action event.

11. The method of claim 10 further comprising the step of displaying a device selection menu on a display screen coupled to the central control unit.

12. The method of claim 11 further comprising the step of visually indicating navigation location within the device selection menu.

13. The method of claim 11 further comprising the step of representing devices within the system as graphical icons displayed within the device selection menu.

14. The method of claim 13 further comprising the step of displaying a device selection window, a video selection window, and an audio sink window as part of the device selection menu.

15. The method of claim 14 further comprising the step of displaying a connection window.

16. The method of claim 13 further comprising the step of displaying a plurality of visual indicators corresponding to a disabled device, current source or sink device, and menu location.

17. The method of claim 13 further comprising the step of displaying a plurality of visual indicators providing visual recognition of a disabled device, current source or sink device/current menu location, not current source or sink device/current menu location, current source or sink device/not current menu location, and not current source or sink device/not current menu location.

18. The method of claim 11 wherein the device selection menu includes screen representations corresponding to the plurality of audio-video input devices comprising a mix of one or more analog and one or more digital devices within a network system, and

selecting a screen representation of a first audio-video input device to select the input device as a source device.

19. The method of claim 18 further comprising the step of selecting a function button from a device control menu displayed on the screen to drive the source device according to the selected function.

20. The method of claim 18 further comprising the step of navigating within the device selection menu to reach a screen representation of the first input device.

21. The method of claim 19 further comprising the step of selecting a second audio-video input device as a new input source device including the steps of

inputting an input command to display the device selection menu on the screen, and

selecting a screen representation of a second audio-video input device to select the second input device as a new source device.

22. The method of claim 21 further comprising the step of selecting a function button from the device control menu displayed on the screen to drive the new source device according to the selected function.

23. The method of claim 21 further comprising the step of navigating within the device selection menu to reach a screen representation of the second input device.

24. The method of claim 1 further comprising the steps

creating a device list comprising a list of all audio-video input devices coupled to the central control unit, a connection port list comprising a list of all connection ports within the network system, and a device connection list comprising a list of all connections between input devices and the control unit within the network system, and

maintaining the device, connection port and device connection lists in memory within the central control unit.

25. The method of claim 24 wherein the control unit is an AV system.

26. The method of claim 25 wherein the AV system is a television.

27. The method of claim 26 wherein the television is a digital television.

28. The method of claim 24 further comprising the step of creating a device object for each device in the device list.

29. The method of claim 28 further comprising the step of storing the device object in a device object list.

30. The method of claim 28 wherein the device objects includes device identification, device type, device I/O capabilities and device control information.

31. The method of claim 30 wherein the control unit includes a device control module, the device objects further include device control module interfacing information.

32. The method of claim 29 wherein the step of creating a device object for each device in the device list further comprises the step of creating unit level device objects to represent a device as a whole and sub-unit level device objects to represent sub-units of a device.

33. The method of claim 32 further comprising the step of connecting the sub-unit level device objects to the corresponding unit level device objects.

34. The method of claim 28 wherein the step of creating a device object comprising the step of discovering a device coupled to the control unit and creating a device object.

35. The method of claim 32 wherein the step of creating unit and sub-level device objects comprises the steps of discovering a device coupled to the control unit and scanning the device object list for a unit level device object for the discovered device.

36. The method of claim 35 further comprising the step of creating unit and sub-unit level device objects for the discovered device when no unit level device object is present in the device object list.

37. The method of claim 35 further comprising the step of creating sub-unit level device objects for the discovered device corresponding to the unit level device object present in the device object list.

38. The method of claim 28 further comprising the step of creating a device interconnect object for each connection port in the connection port list.

39. The method of claim 38 further comprising the step of storing the device interconnect objects in a device interconnect list.

40. The method of claim 39 wherein the information stored in the device interconnect object includes a port number, content, direction, and type.

41. The method of claim 40 further comprising the step of

determining current system state of the network system, and wherein the receiving step includes receiving uniform user input commands at the central control unit, and wherein the translating step includes

translating the user input commands, based on stored device and device interconnectivity data, to device appropriate output messages to drive the plurality of peripheral devices.

42. The method of claim 41 further comprising the step of transmitting the output messages to the plurality of audio-video input devices.

43. The method of claim 41 wherein the receiving step further comprising the step of detecting input command signals from a remote control device.

44. The method of claim 43 further comprising the step of transmitting input command signals from a single control layer of the remote control device for control of the plurality of audio-video input devices.

45. The method of claim 44 wherein the transmitting step further comprises the steps of transmitting input commands for a first device of the plurality of audio-video input devices from a first control layer of the remote control device, and transmitting input commands for a second device of the plurality of audio-video input devices without switching to a second control layer of the remote control device.

46. The method of claim 41 further comprising the step of inputting uniform commands from an input devices for the plurality of audio-video input devices.

47. The method of claim 41 further comprising the step of directing a signal from a first source audio-video input device to a sink device.

48. The method of claim 47 further comprising the steps of directing a signal from a second audio-video input device to the sink device, and performing all necessary device connecting and input switching without further user input.

49. The method of claim 48 wherein the translating step further comprises the step of determining if the input command is a direct device action event or a menu related event.

50. The method of claim 49 further comprising the step of determining whether the input command is a menu display event, a menu navigation event, or a menu device action event.

51. The method of claim 50 further comprising the step of displaying a device selection menu on a display screen coupled to the central control unit.

52. The method of claim 41 further comprising the steps of

inputting a command to display a device selection menu on a screen coupled to the central control unit, wherein the device selection menu includes screen representations corresponding to a plurality of audio-video input devices, and

selecting a screen representation of a first audio-video input device to select the audio-video device as a source device.

53. The method of claim 52 further comprising the step of selecting a function button from a device control menu displayed on the screen to drive the source device according to the selected function.

54. The method of claim 52 further comprising the step of navigating within the device selection menu to reach a screen representation of the first audio-video input device.

55. The method of claim 53 further comprising the step of selecting a second audio-video device as a new input source device including the steps of

inputting an input command to display the device selection menu on the screen, and

selecting a screen representation of a second peripheral device to select the second peripheral device as a new source device.

56. The method of claim 55 further comprising the step of selecting a function button from the device control menu displayed on the screen to drive the new source device according to the selected function.

57. The method of claim 55 further comprising the step of navigating within the device selection menu to reach a screen representation of the second audio-video device.

58. A method of controlling a network of devices comprising analog and digital input audio-video devices, comprising the steps of

receiving a plurality of uniform user input commands at a central control unit of a network system comprising a primary audio-video display unit and a plurality of input audio-video devices interconnected to the central control unit and controlled from the central control unit over a plurality of communication interfaces, wherein the plurality of input audio-video devices includes one or more analog and one or more digital input devices, wherein the input commands are central control unit specific input commands,

determining current system state of the network system,

based on stored device and device interconnectivity data, translating within the central control unit the user input commands to device specific output messages to drive the plurality of input audio-video devices via messages communicated from the central control unit to the one or more digital audio-video devices over a bidirectional bus and via messages communicated from the central control unit to the one or more analog audio-video devices via IR signals, wherein one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus, and

directing a signal from a first source input audio-video device to a sink input audio-video device, via device appropriate output messages transmitted from the central control unit to the input audio-video devices while directing a signal from a second source input audio-video device to the primary video display and audio output device, wherein the first and second source and the sink input audio-video devices include one or more analog devices.

59. The method of claim 58 further comprising the step of maintaining within memory in the central control unit a list of all of the plurality of input audio-video devices, all device connections within the network system, and all I/O ports within the network system.

60. The method of claim 58 further comprising the step of transmitting the output messages to the plurality of input audio-video devices.

61. The method of claim 58 wherein the receiving step further comprises the step of detecting input command signals from a remote control device.

62. The method of claim 61 further comprising the step of transmitting input command signals from a single control layer of the remote control device for control of the plurality of devices.

63. The method of claim 62 wherein the transmitting step further comprises the steps of transmitting input commands for a first device of the plurality of input audio-video devices that is controlled over a first communication interface from a first control layer of the remote control device, and transmitting input commands for a second device of the plurality of input audio-video devices that is controlled over a second communication interface without switching to a second control layer of the remote control device.

64. The method of claim 58 further comprising the step of inputting uniform commands from an input device for the plurality of input audio-video devices, wherein identical type input commands are used to drive input audio-video devices controlled over a first communication interface and input audio-video devices controlled over a second communication interface.

65. The method of claim 58 wherein the translating step further comprises the step of determining if the input command is a direct device action event or a menu related event.

66. The method of claim 65 further comprising the step of determining whether the input command is a menu display event, a menu navigation event, or a menu device action event.

67. A method of controlling a network of devices including analog and digital input devices, comprising the steps of

determining current system state of the network system comprising a plurality of peripheral devices interconnected to a central control unit, wherein the plurality of peripheral devices include one or more analog and one or more digital input audio-video devices,

visually indicating current system state by the display of non-input signal visual indicators including a current video sink indicator and a current audio sink device indicator on a display screen coupled to the central control unit,

receiving a plurality of uniform user input commands at the central control unit, wherein the input commands are central control unit specific commands, and

based on stored device and device interconnectivity data, translating the user input commands within the central control unit to device specific output messages to drive the plurality of peripheral devices via messages communicated from the central control unit to the one or more digital audio-video devices over a bidirectional bus and via messages communicated from the central control unit to the one or more analog audio-video devices via IR signals, wherein one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus.

68. The method of claim 67 further comprising the step of displaying a device selection menu on a display screen coupled to a central control unit.

69. The method of claim 68 wherein the device selection menu includes screen representations corresponding to a plurality of peripheral devices within the network system, and

selecting a screen representation of a first peripheral device to select the peripheral device as a source device.

70. The method of claim 69 further comprising the step of selecting a function button from a device control menu displayed on the screen to drive the source device according to the selected function.

71. The method of claim 69 further comprising the step of navigating within the device selection menu to reach a screen representation of the first peripheral device.

72. The method of claim 70 further comprising the step of selecting a second peripheral device as a new input source device including the steps of

inputting an input command to display the device selection menu on the screen, and

selecting a screen representation of a second peripheral device to select the second peripheral device as a new source device.

73. The method of claim 72 further comprising the step of selecting a function button from the device control menu displayed on the screen to drive the new source device according to the selected function.

74. The method of claim 72 further comprising the step of navigating within the device selection menu to reach a screen representation of the second peripheral device.

75. The method of claim 67 further comprising the steps of

inputting a first command through a uniform user interface to a context sensitive control system to select a source device, and

inputting a second command through the uniform user interface to the control system to drive the source device.

76. The method of claim 75 further comprising the step of selecting a second input device as a new source device including the steps of

inputting a third command through user interface to select a new source device;

and

inputting a fourth command through the user interface to drive the new source device.

77. The method of claim 67 further comprising the step of visually indicating input source device options from the plurality of peripheral devices.

78. The method of claim 77 further comprising the step of visually indicating sink device options from the plurality of peripheral devices.

79. The method of claim 78 further comprising the step of visually indicating which network system device is the current sink device.

80. The method of claim 79 further comprising the step of visually indicating which peripheral device is the current source device.

81. The method of claim 80 further comprising the step of visually indicating the existence of a peer-to-peer connection between peripheral devices.

82. The method of claim 80 further comprising the step of visually indicating a peripheral device is unavailable as a source device or sink device.

83. The method of claim 68 further comprising the step of visually indicating navigation location within the device selection menu.

84. The method of claim 83 further comprising the step of representing devices within the system as graphical icons displayed within the device selection menu.

85. The method of claim 84 further comprising the step of displaying a plurality of menu windows.

86. The method of claim 84 further comprising the step of displaying a plurality of visual indicators corresponding to a disabled device, current source or sink device, and menu navigation location.

87. The method of claim 84 further comprising the step of displaying a plurality of visual indicators providing visual recognition of a disabled device, current source or sink device/current menu location, not current source or sink device/current menu location, current source or sink device/not current menu location, and not current source or sink device/not current menu location.

88. A method of controlling a network system comprising a primary video display and audio output device (PDD) and a plurality of input audio-video devices, wherein the plurality of input audio-video devices include one or more analog and one or more digital devices, comprising the steps of

receiving user input commands at a central control unit (CCU) of a network system, wherein the input commands are CCU specific commands, input audio-video devices comprising mix of one or more analog and one or more digital audio-video devices,

translating within the CCU the user input commands received by the CCU to device specific output messages to drive the plurality of input audio-video devices via messages communicated from the central control unit to the one or more digital audio-video devices over a bidirectional bus and via messages communicated from the central control unit to the one or more analog audio-video devices via IR signals, wherein one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus, and

directing an audio and/or video signal from a first source device of the plurality of input audio-video devices to a sink device of the plurality of input audio-video devices, via device specific output messages transmitted from the CCU to the plurality of input audio-video devices while directing a signal from a second source device to the PDD, wherein the first and second source devices and the sink device include one or more analog audio-video devices selected from the plurality of input audio-video devices of the network system.

89. A method of controlling a network system comprising a primary video display and audio output device a plurality of audio-video input devices, wherein the a plurality of audio-video input devices includes a mix of one or more analog and one or more digital devices, comprising the steps of

outputting the audio and video components of a first signal from a first audio-video input device on a primary video display and audio output device (PDD) of a network system, wherein the network system includes a plurality of audio-video input devices comprising a mix of one or more analog and one or more digital devices,

receiving at the PDD a PDD specific user input command to output a second signal from a second audio-video input device to a second video display and audio output device or to one of the one or more analog or one or more digital devices,

translating within the PDD the PDD specific user input command to device specific output messages to drive the first and second audio-video input devices via messages communicated from the central control unit to the one or more digital audio-video devices over a bidirectional bus and via messages communicated from the central control unit to the one or more analog audio-video devices via IR signals, wherein one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus, and

performing all necessary device connecting and input switching without further user input to cease outputting the audio and video components of the first signal from the first audio-video input device to the PDD, switch video input connection and output the video component of the second signal on the PDD and direct the audio component of the second signal from the second audio-video input device to a third audio-video input device and output it therefrom, wherein the first, second and third audio-video input devices include one or more analog and one or more digital devices selected from the plurality of audio-video input devices of the network system.

90. A method of controlling a network system comprising a primary video display and audio output device and a plurality of input audio-video devices, wherein the plurality of input audio-video devices include one or more analog and one or more digital devices, comprising the steps of

visually indicating current system state by the display of graphical indicators including a current source device indicator, a current video sink device indicator and a current audio sink device indicator on a primary video display screen coupled to a central control unit of a network system, wherein the network system includes a plurality of input audio-video devices comprising a mix of one or more analog and one or more digital devices, and wherein the source and sink devices include one or more analog devices selected from the plurality of input audio-video devices of the network system,

receiving central control unit specific user input commands at the central control unit, and

translating within the central control unit the user input commands to device specific output messages to drive the plurality of input audio-video devices via messages communicated from the central control unit to the one or more digital audio-video devices over a bidirectional bus and via messages communicated from the central control unit to the one or more analog audio-video devices via IR signals, wherein one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus.

91. The method of claim 90 further comprising the step of visually indicating input source device options from the plurality of plurality of input audio-video devices.

92. The method of claim 91 further comprising the step of visually indicating audio sink device options from the plurality of input audio-video devices.

93. The method of claim 92 further comprising the step of visually indicating which network system device is the current video sink device.

94. The method of claim 93 further comprising the step of visually indicating which input audio-video device is the current input source device and which network system device is the current audio sink.

95. The method of claim 94 further comprising the step of visually indicating the existence of a peer-to-peer connection between input audio-video devices.

96. The method of claim 94 further comprising the step of visually indicating an input audio-video device is unavailable as an input source device or audio sink device.

97. A method of controlling a network of input audio-video devices comprising analog and digital devices, comprising the steps of

receiving a plurality of uniform user input commands at a central control unit of a network system comprising a plurality of input audio-video devices interconnected to the central control unit, wherein the plurality of input audio-video devices include one or more analog and one or more digital devices, and wherein the user input commands are central control unit specific commands,

determining current system state of the network system,

based on stored device and device interconnectivity data, translating the user input commands within the central control unit to device specific output messages to drive the plurality of input audio-video devices via messages communicated from the central control unit to the one or more digital audio-video devices over a bidirectional bus and via messages communicated from the central control unit to the one or more analog audio-video devices via IR signals, wherein one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus, and

directing a signal from a first source input audio-video device to a sink input audio-video device, via device appropriate output messages transmitted from the central control unit to the input audio-video devices while directing a signal from a second source input audio-video device to the primary video display and audio output device, wherein the first and second source and the sink peripheral devices include one or more analog devices of the plurality of input audio-video devices.

98. A method of controlling a network of devices including analog and digital input devices, comprising the steps of

outputting the audio and video components of a first signal from a first input audio-video device on a primary video display and audio output device of a network system including a plurality of input audio-video devices, wherein the plurality of input audio-video devices include one or more analog and one or more digital input devices,

receiving a non-device specific user input command to output a second signal from a second input audio-video device of the plurality of input audio-video devices,

determining current system state of the network system,

based on stored device and device interconnectivity data, translating the user input command to device specific output messages to drive the plurality of input audio-video devices via messages communicated from the central control unit to the one or more digital audio-video devices over a bidirectional bus and via messages communicated from the central control unit to the one or more analog audio-video devices via IR signals, wherein one or more analog audio-video devices being incapable of receiving or sending messages over a bidirectional bus, and

performing all necessary device connecting and input switching without further user input to cease outputting the audio and video components of the first signal from the first input audio-video device to the primary video display and audio output unit, switch video input connection and output the video component of the second signal on the primary display unit, and direct the audio component of the second signal from the second input audio-video device to a third input audio-video device of the plurality of input audio-video devices and output it therefrom, wherein one or more of the first, second and third input audio-video devices is an analog input device.