Virtual remote functionality

• US 9,654,563 B2
• Filed: 05/01/2015
• Issued: 05/16/2017
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Biscotti Incorporated
• Original Assignee: Biscotti Incorporated
• Inventors: Shoemake, Matthew B., Ahmed, Syed Nadeem
• Primary Examiner(s): Dubasky, Gigi L
• Application Number:
  US14/702,390
• Publication Number:
  US 20150244807A1
• Time in Patent Office: 746 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/1093 Some peer nodes performing special functions
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/141 Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4542 Blocking scenes or portions of the received content, e.g. censoring scenes
• CPC Class Codes: H04N 21/4755 for defining user preferences, e.g. favourite actors or genre retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306
• CPC Class Codes: H04N 5/76 Television signal recording

Abstract

Novel tools and techniques are described for providing remote control of consumer electronics devices, and, more particularly, to tools and techniques for providing virtual remote control of consumer electronics devices that do not have dedicated remote controllers.

Claims

1. A method, comprising:

broadcasting, with a first user device, a notification indicating that a peer-to-peer wireless connection may be established with the first user device;

determining, with the first user device, whether a peer-to-peer connection with a second user device is available;

establishing, with the first user device, a first peer-to-peer wireless connection with the second user device and communicating with the second user device over the first peer-to-peer wireless connection, based on a determination that the peer-to-peer connection is available and on a request to establish peer-to-peer wireless connection received from the second user device, the request being sent by the second user device after the second user device receives the broadcasted notification;

based on a determination that no peer-to-peer connection is available, determining whether the second user device is available on a local network to which the first user device is connected;

based on a determination that the second user device is available on the local network, communicating with the second user device over the local network; and

based on a determination that the second user device is not available on the local network, communicating with the second user device over the Internet;

generating, with the first user device, a first pairing code for pairing with the second user device;

sending, with the first user device, the first pairing code to a first display device for display on the first display device;

receiving, with the first user device and from the second user device, first user input including a second pairing code;

determining, with the first user device, whether the second pairing code matches the first pairing code;

establishing, with the first user device, the second user device as a first remote controller for the first user device based on a determination that the second pairing code matches the first pairing code.

2. The method of claim 1, further comprising:

establishing, with the second user device, a network connection between the first user device and a network; and

receiving, with the first user device, remote control instructions from the second user device over the network connection.

3. The method of claim 2, wherein the network is at least one of an Ethernet network, a local area network (“

LAN”

), a wide area network (“

WAN”

), a wireless wide area network (“

WWAN”

), a virtual private network (“

VPN”

), an intranet, an extranet, the Internet, a public switched telephone network (“

PSTN”

), an infra-red network, or a radio frequency (“

rf”

) network.

4. The method of claim 2, wherein receiving, with the first user device, remote control instructions from the second user device over the network connection comprises receiving, with the first user device, remote control instructions from the second user device via a user interface of a webpage.

5. The method of claim 1, wherein the first user device is one of a video communication device, a video calling device, an image capture device, a presence detection device, a video recording device, a video playback device, an audio recording device, an audio playback device, a tablet computer, a laptop computer, a desktop computer, a toy vehicle, a toy aircraft, a drone, or a consumer electronic device sold without a dedicated remote controller.

6. The method of claim 1, wherein the first user device is a gaming console.

7. The method of claim 1, wherein the second user device is one of a smart phone, a mobile phone, a tablet computer, a desktop computer, a laptop computer, a portable gaming device, or a remote control device.

8. The method of claim 1, further comprising:

establishing, with the first user device, a second peer-to-peer wireless connection with a third user device;

generating, with the first user device, a third pairing code for pairing with the third user device;

sending, with the first user device, the third pairing code to a second display device for display on the second display device;

receiving, with the first user device and from the third user device over the second peer-to-peer wireless connection, second user input including a fourth pairing code;

determining, with the first user device, whether the fourth pairing code matches the third pairing code;

establishing, with the first user device, the third user device as a second remote controller for the first user device based on a determination that the fourth pairing code matches the third pairing code.

9. The method of claim 8, further comprising:

detecting, with the first user device, that the second user device is unavailable;

wherein establishing, with the first user device, a second peer-to-peer wireless connection with a third user device comprises establishing the second peer-to-peer wireless connection based on detecting that the second user device is unavailable.

10. The method of claim 8, further comprising:

receiving user input, the user input instructing the first user device to initiate pairing;

wherein establishing, with the first user device, a second peer-to-peer wireless connection with a third user device comprises establishing the second peer-to-peer wireless connection based on the user input.

11. The method of claim 10, wherein the user input is received over a network.

12. The method of claim 10, wherein the user input is received via a physical button on the first user device.

13. The method of claim 8, wherein the first display device and the second display device are the same display device.

14. The method of claim 1, wherein the first display device is external to, and communicatively coupled with, the first user device.

15. The method of claim 1, wherein the first pairing code is at least one of a personal identification number (“

PIN”

), a numeric combination, an alphanumeric combination, a plain text code, an image-based pairing code, a binary code, a hexadecimal code, a bar code, a matrix bar code, or a quick response (“

QR”

) code, and wherein the second pairing code is at least one of a personal identification number (“

PIN”

), a numeric combination, an alphanumeric combination, a plain text code, an image-based pairing code, a binary code, or a hexadecimal code.

16. The method of claim 15, wherein the second user device comprises a camera that is used to capture an image of the first pairing code that is displayed on the first display device, wherein the second pairing code comprises the image-captured first pairing code.

17. The method of claim 15, wherein the image-based pairing code of the second pairing code and the image-based pairing code of the first pairing code may be determined to match even if they differ, so long as the difference is within a predetermined threshold.

18. The method of claim 1, wherein the first peer-to-peer wireless connection is one of a Bluetooth connection, a WiFi connection, or a near field communication (“

NFC”

) connection.

19. The method of claim 1, wherein broadcasting the notification indicating that a peer-to-peer wireless connection may be established with the first user device comprises continuously broadcasting, with a first user device, a notification indicating that a peer-to-peer wireless connection may be established with the first user device.

20. The method of claim 1, wherein broadcasting the notification indicating that a peer-to-peer wireless connection may be established with the first user device comprises broadcasting, with a first user device, a notification indicating that a peer-to-peer wireless connection may be established with the first user device, only in response to receiving a probe message from the second user device requesting that the first user device identify itself.

21. The method of claim 1, wherein the notification indicating that a peer-to-peer wireless connection may be established with the first user device comprises information regarding the first user device that comprises at least one of a user-selected name for the first user device, a default name for the first user device, a model name for the first user device, a model number for the first user device, or a serial number of the first user device.

22. The method of claim 1, further comprising:

receiving, with the first user device and from a media content source, media content comprising at least one of image content or video content;

sending, with the first user device, the at least one of image content or video content to the first display device for display on the first display device;

wherein sending the first pairing code to the first display device for display on the first display device comprises;

combining, with the first user device, the at least one of image content or video content with the first pairing code into a combined content, such that the first pairing code is superimposed over a portion of the at least one of image content or video content, when both are simultaneously displayed on the first display device;

sending, with the first user device, the combined content for display on the first display device.

23. A first user device, comprising:

at least one processor;

a video output interface to provide video output to a first display device;

a peer-to-peer communication interface device;

a storage medium in communication with the at least one processor, the storage medium having encoded thereon a set of instructions executable by the at least one processor to control operation of the first user device, the set of instructions comprising;

instructions to broadcast a notification indicating that a peer-to-peer wireless connection may be established with the first user device via the peer-to-peer communication interface device;

instructions to determine whether a peer-to-peer wireless connection with a second user device is available;

instructions to establish the peer-to-peer wireless connection with the second user device and to communicate with the second user device over the first peer-to-peer wireless connection, based on a determination that the first peer-to-peer wireless connection is available and on a request to establish peer-to-peer wireless connection received from the second user device, the request being sent by the second user device after the second user device receives the broadcasted notification;

instructions to determine whether the second user device is available on a local network to which the first user device is connected, based on a determination that no peer-to-peer wireless connection is available;

instructions to communicate with the second user device over the local network, based on a determination that the second user device is available on the local network; and

instructions to communicate with the second user device over the Internet, based on a determination that the second user device is not available on the local network;

instructions to generate a first pairing code for pairing with the second user device;

instructions to send the first pairing code to the first display device for display on the first display device;

instructions to receive first user input including a second pairing code from the second user device;

instructions to determine whether the second pairing code matches the first pairing code; and

instructions to establish the second user device as a first remote controller for the first user device based on a determination that the second pairing code matches the first pairing code.

24. The device of claim 23, further comprising:

a network interface device,

wherein the set of instructions further comprises;

instructions to establish a network connection, via the network interface device, between the first user device and a network;

instructions to receive remote control instructions from the second user device over the network connection.

25. The device of claim 24, wherein the network is at least one of an Ethernet network, a local area network (“

LAN”

), a wide area network (“

WAN”

), a wireless wide area network (“

WWAN”

), a virtual private network (“

VPN”

), an intranet, an extranet, the Internet, a public switched telephone network (“

PSTN”

), an infra-red network, or a radio frequency (“

rf”

) network.

26. The device of claim 24, wherein the instructions to receive remote control instructions from the second user device over the network connection comprise instructions to receive remote control instructions from the second user device via a user interface of a webpage.

27. The device of claim 23, wherein the first user device is one of a video communication device, a video calling device, an image capture device, a presence detection device, a video recording device, a video playback device, an audio recording device, an audio playback device, a tablet computer, a laptop computer, a desktop computer, a toy vehicle, a toy aircraft, a drone, or a consumer electronic device sold without a dedicated remote controller.

28. The device of claim 23, wherein the first user device is a gaming console.

29. The device of claim 23, wherein the second user device is one of a smart phone, a mobile phone, a tablet computer, a desktop computer, a laptop computer, a portable gaming device, or a remote control device.

30. The device of claim 23, wherein the set of instructions further comprises:

instructions to establish a second peer-to-peer wireless connection with a third user device via the peer-to-peer communication interface device;

instructions to generate a third pairing code for pairing with the third user device;

instructions to send the third pairing code to the second display device for display on the second display device;

instructions to receive second user input including a fourth pairing code from the third user device over the second peer-to-peer wireless connection;

instructions to determine whether the fourth pairing code matches the third pairing code;

instructions to establish the third user device as a second remote controller for the first user device based on a determination that the fourth pairing code matches the third pairing code.

31. The device of claim 23, wherein the first pairing code is at least one of a personal identification number (“

PIN”

), a numeric combination, an alphanumeric combination, a plain text code, an image-based pairing code, a binary code, a hexadecimal code, a bar code, a matrix bar code, or a quick response (“

QR”

) code, and wherein the second pairing code is at least one of a personal identification number (“

PIN”

), a numeric combination, an alphanumeric combination, a plain text code, an image-based pairing code, a binary code, or a hexadecimal code.

32. The device of claim 23, wherein the first peer-to-peer wireless connection is one of a Bluetooth connection, a WiFi connection, or a near field communication (“

NFC”

) connection.

33. The device of claim 23, further comprising:

a video capture device; and

an audio capture device,

wherein the set of instructions further comprises;

instructions to capture, using the video capture device, video data comprising physical gestures of a user;

instructions to capture, using the audio capture device, audio data comprising voice commands of the user;

instructions to analyze at least one of the captured video data or the captured audio data to determine one or more control commands of a plurality of predetermined control commands that matches at least one of the physical gestures of the user or the voice commands of the user; and

instructions to control operation of the first user device in accordance with the one or more control commands.

34. The device of claim 33, wherein the one or more control commands comprise one or more of control commands for controlling the first user device to establish the first peer-to-peer wireless connection with the second user device, control commands for controlling the first user device to establish the second user device as a first remote controller for the first user device based on a determination that the second pairing code matches the first pairing code, or control commands for causing the first user device to actuate built-in functionalities.

35. A method, comprising:

activating, with a first user device comprising a video capture device and an audio capture device, the video capture device and the audio capture device, in response to being connected to a power source;

capturing, using the video capture device, video data comprising physical gestures of a user;

capturing, using the audio capture device, audio data comprising voice commands of the user;

analyzing, with the first user device, at least one of the captured video data or the captured audio data to determine one or more control commands of a plurality of predetermined control commands that match at least one of the physical gestures of the user or the voice commands of the user, wherein the one or more control commands comprises control commands to establish a network connection with a server over a network and control commands to establish a second user device as a remote controller for the first user device;

establishing, with the first user device, the network connection with the server over the network, in accordance with the one or more control commands that comprise at least one of gesture commands or voice commands;

establishing, with the server, the second user device as a first remote controller for the first user device, based at least in part on the one or more control commands that comprise at least one of gesture commands or voice commands;

determining, with the first user device, whether a peer-to-peer connection with a second user device is available;

based on a determination that the peer-to-peer connection is available, communicating with the second user device over the peer-to-peer connection;

based on a determination that no peer-to-peer connection is available, determining whether the second user device is available on a local network to which the first user device is connected;

based on a determination that the second user device is available on the local network, communicating with the second user device over the local network; and

based on a determination that the second user device is not available on the local network, communicating with the second user device over the Internet.