MOBILE DEVICE WHICH AUTOMATICALLY DETERMINES OPERATING MODE

• US 20100317332A1
• Filed: 06/12/2009
• Published: 12/16/2010
• Est. Priority Date: 06/12/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bathiche, Steven N., Estrada, Julio, Abel, Miller T., Westerinen, William J., Migos, Charles J.
• Granted Patent: US 9,014,685 B2
• Time in Patent Office: Days
• US Class Current: 455/418
• CPC Class Codes: G08C 2201/20 Binding and programming of remote control devices
• CPC Class Codes: H04B 1/202 by remote control
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 2012/2841 Wireless
• CPC Class Codes: H04L 2012/2849 Audio/video appliances
• CPC Class Codes: H04M 2201/38 Displays
• CPC Class Codes: H04M 2203/1016 Telecontrol
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42208 Display device provided on the remote control
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/42227 the keys being reprogrammable, e.g. soft keys
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/8186 specially adapted to be executed by a peripheral of the client device, e.g. by a reprogrammable remote control

Abstract

A mobile device such as a cell phone is used to remotely control an electronic appliance such as a television or personal computer. In a setup phase, the mobile device captures an image of the electronic appliance and identifies and stores scale-invariant features of the image. A user interface configuration such as a virtual keypad configuration, and a communication protocol, can be associated with the stored data. Subsequently, in an implementation phase, another image of the electronic appliance is captured and compared to the stored features in a library to identify a match. In response, the associated user interface configuration and communication protocol are implemented to control the electronic appliance. In a polling and reply process, the mobile device captures a picture of a display of the electronic device and compares it to image data which is transmitted by the electronic appliance.

Claims

1. A hand-held mobile device which communicates by wireless signals, comprising:

an image sensor;

a user interface;

a memory which stores processor-readable code; and

at least one processor which executes the processor-readable code, the at least one processor;

captures at least a first image of the electronic appliance using the image sensor at a first time, stores first data in the memory based on the at least a first image, captures at least a second image of the electronic appliance using the image sensor at a second time after the first time, provides second data based on the at least a second image, determines if the first data matches the second data, and if the first data matches the second data, configures the user interface for use with the electronic appliance.

2. The hand-held mobile device of claim 1, wherein:

the first data identifies scale-invariant features of the at least a first image, the second data identifies scale-invariant features of the at least a second image, and the at least one processor determines whether a correlation between the scale-invariant features of the at least a first image and the scale-invariant features of the at least a second image is greater than a predefined threshold.

3. The hand-held mobile device of claim 1, wherein:

to configure the user interface the at least one processor accesses data in the memory which associates a user interface configuration with the at least a first image.

4. The hand-held mobile device of claim 1, wherein:

the at least one processor receives a user command via the user interface which identifies a type of the electronic appliance, and configures the user interface based on the identified type of the electronic appliance.

5. The hand-held mobile device of claim 4, wherein:

the identified type is at least one of a television and a personal computer.

6. The hand-held mobile device of claim 1, wherein:

the image sensor is a camera which detects visible light.

7. The hand-held mobile device of claim 1, wherein:

to configure the user interface, the at least one processor provides a touch screen which display buttons which allow a user to control the electronic appliance.

8. The hand-held mobile device of claim 1, wherein after the user interface is configured for use with the electronic appliance, the at least one processor activates the mobile device to transmit a wireless signal to the electronic appliance with information to be displayed on a display device of the electronic appliance.

9. The hand-held mobile device of claim 8, wherein:

the at least one processor the mobile device to transmit the wireless signal is responsive to an incoming telephone call received by the mobile device, and the information to be displayed on the display device relates to the incoming telephone call.

10. The hand-held mobile device of claim 8, wherein:

the information to be displayed on the display device mirrors information displayed on the user interface.

11. The hand-held mobile device of claim 8, further comprising:

at least one sensor, the at least one sensor indicates when the mobile device is in a picture-taking orientation, and the at least one processor activates the mobile device to transmit the wireless signal responsive to the at least one sensor.

12. A processor-implemented method for configuring a mobile device which communicates by wireless signals, comprising:

activating the mobile device to transmit a wireless signal, the wireless signal is received by a first electronic appliance associated with a first display device, the first electronic appliance transmits a wireless signal in response to receipt of the wireless signal from the mobile device, the wireless signal transmitted by the first electronic appliance comprises first image data corresponding to at least one image displayed on the first display device and information for controlling the first electronic appliance;

receiving the wireless signal transmitted by the first electronic appliance at the mobile device;

capturing the at least one image displayed on the first display device using an image sensor of the mobile device;

providing reference image data based on the at least one image displayed on the first display device;

determining if the first image data matches the reference image data; and

based on whether the first image data matches the reference image data, deciding whether to configure a user interface of the mobile device based on the information for controlling the first electronic appliance.

13. The processor-implemented method of claim 12, wherein:

the at least one image displayed on the first display device is displayed responsive to receipt of the wireless signal from the mobile device by the at least a first electronic appliance.

14. The processor-implemented method of claim 12, wherein:

the information for controlling the first electronic appliance includes an identifier of a protocol for controlling the first electronic appliance, and the mobile device accesses information for implementing the protocol based on the identifier.

15. The processor-implemented method of claim 12, wherein:

the first electronic appliance comprises a personal computer, and the information for controlling the first electronic appliance includes an identifier of an application running on the personal computer.

16. The processor-implemented method of claim 12, wherein:

the wireless signal from the mobile device is received by a second electronic appliance having associated with a second display device, the second electronic appliance transmits a wireless signal in response to receipt of the wireless signal from the mobile device, the wireless signal transmitted by the second electronic appliance comprises second image data corresponding to at least one image displayed on the second display device, and information for controlling the second electronic appliance, the reference image data includes the at least one image displayed on the second display device;

receiving the wireless signal transmitted by the second electronic appliance at the mobile device;

determining if the second image data matches the reference image data; and

when the first image data matches the reference image data, and the second image data matches the reference image data, deciding whether to configure the user interface based on the information for controlling the first electronic appliance or based on the information for controlling the second electronic appliance by determining whether the first image data or second image data is a better match with the reference image data.

17. The processor-implemented method of claim 16, wherein:

the determining whether the first image data or second image data is a better match with the reference image data comprises determining whether the first or second electronic appliance is more centrally located in an image which represented by the reference image data.

18. A processor-implemented method for configuring a mobile device which communicates by wireless signals, comprising:

activating the mobile device to transmit a wireless signal, the wireless signal is received by an electronic appliance associated with a display device, the electronic appliance provides at least one synchronization image on the display device in response to receipt of the wireless signal from the mobile device;

sensing the at least one synchronization image using a light sensor of the mobile device;

identifying a user interface configuration based on the at least one sensed synchronization image; and

configuring a user interface of the mobile device with the identified user interface configuration to control the electronic appliance.

19. The processor-implemented method of claim 18, wherein:

the at least one synchronization image indicates whether the first electronic appliance is a television or personal computer.

20. The processor-implemented method of claim 18, wherein:

the identifying the user interface configuration based on the at least one sensed synchronization image comprises selecting the user interface configuration from a library of different user interface configurations based on the at least one sensed synchronization image, the different user interface configurations are associated with different types of electronic appliances.