REMOTE CONTROL INTERFACE PROVIDING HEAD-UP OPERATION AND VISUAL FEEDBACK WHEN INTERACTING WITH AN ON SCREEN DISPLAY

• US 20120185801A1
• Filed: 01/17/2012
• Published: 07/19/2012
• Est. Priority Date: 01/18/2011
• Status: Abandoned Application

• Current Assignee: Savant Systems, LLC
• Original Assignee: Savant Systems, LLC
• Inventors: Madonna, Robert P., Cipollo, Nicholas J.
• Application Number:
  US13/351,848
• Publication Number:
  US 20120185801A1
• Time in Patent Office: Days
• US Class Current: 715/834
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04N 21/42208 Display device provided on the remote control
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control

Abstract

In one embodiment, a remote control interface is provided that allows a user to interact with a programmable multimedia controller from a mobile device having a touch screen display, in a largely “head-up” manner, while providing visual feedback on the mobile device to confirm touch input. The user may enter touch input, including taps, holds swipes and pans, on the touch screen display. Such touch input may be processed and communicated to the programmable multimedia controller, which displays an on-screen display menu system on a display device coupled to the programmable multimedia controller. The user may direct the majority of his or her attention to the on-screen display menu system on the display device. However, some visual feedback may also be displayed on the touch screen display of the mobile device that is specific to the type of touch input received on the touch screen display.

Claims

1. A system comprising:

a programmable multimedia controller coupled to, and configured to control a plurality of different types of electronic devices including one or more audio source devices, one or more video source devices, one or more audio destination devices and one or more display devices, the programmable multimedia controller configured to display an on-screen display menu system on at least one of the display devices, the on-screen display menu system including a plurality of selectable options that are rotatable in the on-screen display menu system and selectable from the on-screen display menu system in response to control commands;

mobile device separate from the programmable multimedia controller and from the at least one display device, the mobile device configured to operate as a remote control for the programmable multimedia controller through which control commands for interacting with the on-screen display may be entered, the mobile device including a wireless interface that enables wireless communication with the programmable multimedia controller, a touch screen display, a processor, and a memory configured to store at least a remote control interface client application that when executed by the processor is operable to;

display an input interface on the touch screen display, the input interface having a gesture field,

detect touch input from a user in the gesture field on the mobile device,

determine a type of the received touch input from among a plurality of types of touch input,

in response to the touch input, send one or more control commands to the programmable multimedia controller to cause the programmable multimedia controller to rotate or select one of the options in the on-screen display menu system displayed on the at least one display device, and

in response to the touch input, display one or more indicators in the gesture field to provide visual feedback to the user on the mobile device that is specific to the type of the touch input, the provided visual feedback to differ for different types of the plurality of types of touch input.

2. The system of claim 1, wherein the input interface further includes a plurality of virtual buttons separate from the gesture field, and the remote control interface client application when executed is further operable to:

detect additional touch input from the user having a location that coincides with a location of a virtual button,

in response to the additional touch input, send one or more control commands to the programmable multimedia controller to cause the programmable multimedia controller to perform an action corresponding to the virtual button, and

in response to the additional touch input, display a visual indication on the touch screen of the mobile device to provide visual feedback to the user that the virtual button was pressed.

3. The system of claim 1, wherein the type of the touch input is a tap and the indicator is a circular animation about the location of the tap.

4. The system of claim 1, wherein the type of the touch input is a hold and the indicator is a circular animation about the location of the hold that remains visible for a duration of the hold.

5. The system of claim 1, wherein the type of the touch input is a potential pan and the indicator is a one or more directional indictors that indicate a direction of the potential pan.

6. The system of claim 5, wherein a number of the one or more directional indicators is based on a distance moved in the detected input interface, such that greater distance of movement causes a greater number of directional indicators to be displayed in the gesture field.

7. The system of claim 1, wherein the type of the touch input is an ongoing pan and the indicator is a plurality of directional indictors that indicate a direction of the ongoing pan and that remain visible for a duration of the ongoing pan.

8. The system of claim 1, wherein the type of the touch input is a swipe and the indicator is a plurality of directional indictors that indicate a direction of the swipe.

9. The system of claim 1, wherein the mobile device is a smartphone and the display device is a television.

10. The system of claim 1, wherein the mobile device is a tablet computer and the display device is a television.

11. The system of claim 1, wherein the programmable multimedia controller is also coupled to, and configured to control, at least one electronic device selected from the group consisting of:

a closed-circuit television (CCTV) control system, a heating, ventilation and air conditioning (HVAC) control system, an energy management system, a security system, an electronic lighting controller, and a motor operated device controller.

12. A method comprising:

cause an on-screen display menu system to be displayed on a television, the menu system including a plurality of selectable options that may be manipulated in the on-screen display menu system and may be selected from the on-screen display menu system in response to control commands;

displaying an input interface on a touch screen display of a mobile device that is separate from the television, the input interface having a gesture field;

detecting touch input from a user in the gesture field on the mobile device;

determining a type of the touch input from among a plurality of types of touch input;

in response to the touch input, sending one or more control commands to manipulate or select one of the options in the on-screen display menu system displayed on the television; and

in response to the touch input, displaying one or more indicators in the gesture field to provide visual feedback to the user on the mobile device that is specific to the type of the touch input, the provided visual feedback to indicate for at least some types of touch input a direction corresponding to the touch input to differentiate between touch input of a same type but of different directions.

13. The method of claim 12, wherein the input interface further includes a plurality of virtual buttons separate from the gesture field, and the method further comprises:

detecting additional touch input from the user having a location that coincides with a location of a virtual button;

in response to the additional touch input, sending one or more control commands to perform an action corresponding to the virtual button; and

in response to the touch input, displaying a visual indication on the touch screen of the mobile device to provide visual feedback to the user that the virtual button was pressed.

14. The method of claim 12, wherein the type of the received input is a potential pan.

15. The method of claim 12, wherein the type of the received input is an ongoing pan

16. The method of claim 12, wherein the type of the received input is a swipe.

17. The method of claim 12, wherein the mobile device is a smartphone.

18. The method of claim 12, wherein the mobile device is a tablet computer.

19. The method of claim 12, wherein the displaying an on-screen display menu system on the television is performed by a programmable multimedia controller coupled to the television, the programmable multimedia controller configured to control a plurality of different types of electronic devices including one or more audio source devices, one or more video source devices, one or more audio destination devices and one or more display devices other than the television, wherein the mobile device is in wireless communication with the programmable multimedia controller.

20. A non-transitory computer readable media storing executable instructions that when executed by a processor are operable to:

cause an on-screen display menu system to be displayed on a display device, the on-screen display menu system including a plurality of options that may be selected from the on-screen display menu system;

display an input interface on a touch screen display of a mobile device, the input interface having one or more virtual buttons and a gesture field;

detect touch input from the user in the input interface having a location that coincides with a location of a virtual button;

in response to the touch input, cause an action corresponding to the virtual button to be performed by a programmable multimedia controller coupled to the display device;

in response to the touch input, display a visual indication on the touch screen of the mobile device to provide visual feedback to the user that the virtual button was pressed;

detect additional touch input from a user in the gesture field on the mobile device;

determine a type of the additional touch input from among a plurality of types of touch input;

in response to the additional touch input, cause one of the options in the on-screen display menu system that is displayed on the display device to be selected from the on-screen display menu system and an action corresponding to the selected option performed by the programmable multimedia controller; and

in response to the additional touch input, display one or more indicators in the gesture field to provide visual feedback to the user on the mobile device that is specific to the type of the received touch input, the provided visual feedback to differ for different types of the plurality of types of touch input.