Mobile device and control method

• US 10,602,046 B2
• Filed: 05/15/2018
• Issued: 03/24/2020
• Est. Priority Date: 07/11/2017
• Status: Active Grant

• Current Assignee: HTC Corporation
• Original Assignee: HTC Corporation
• Inventors: Pan, Sheng-Yang
• Primary Examiner(s): Chen, Chiawei
• Application Number:
  US15/980,731
• Publication Number:
  US 20190020809A1
• Time in Patent Office: 679 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 2012/2841 Wireless
• CPC Class Codes: H04L 2101/622 Layer-2 addresses, e.g. medium access control [MAC] addresses
• CPC Class Codes: H04L 61/4541 Directories for service discovery
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

A control method suitable for a mobile device comprising a camera includes operations as follows: obtaining a description of a first wireless device adjacent to the mobile device over a first wireless communication; capturing a first image of a physical environment by the camera; recognizing a first candidate object within the first image; matching the first candidate object with the description of the first wireless device; and in response to that the first candidate object matches with the description of the first wireless device and a first predetermined instruction is received by the camera, generating a first command according to the description of the first wireless device, wherein the first command is to be transmitted to the first wireless device over a second wireless communication for manipulating the first wireless device.

Claims

1. A control method, suitable for a mobile device comprising a camera, the control method comprising:

obtaining a description of a first wireless device adjacent to the mobile device through a plurality of broadcast packets;

capturing a first image of a physical environment by the camera;

recognizing a first candidate object within the first image;

matching the first candidate object with the description of the first wireless device;

in response to that the first candidate object matches with the description of the first wireless device and a first predetermined instruction is received by the camera, generating a first command according to the description of the first wireless device, wherein the first command is to be transmitted to the first wireless device over a second wireless connection for manipulating the first wireless device;

in response to that the first candidate object matches with the description of the first wireless device, capturing a series of images of the physical environment by the camera, wherein the series of images comprises a second image and a third image captured after the second image for a predetermined time period;

recognizing the first candidate object within the second image; and

in response to that the first candidate object is absent within the third image, disconnecting the second wireless connection.

2. The control method of claim 1, wherein the first predetermined instruction comprises user hand motions.

3. The control method of claim 1, wherein the operation of matching the first candidate object with the description of the first wireless device further comprises:

displaying the first candidate object;

emphasizing an outline of the first candidate object;

displaying the description of the wireless device;

matching or pairing the first candidate object with the description of the first wireless device based on user hand motions; and

in response to that the first candidate object matches with the description of the first wireless device, establishing the second wireless connection between the first wireless device and the mobile device according to the description of the first wireless device.

4. The control method of claim 3, further comprising:

periodically obtaining information from the first wireless device over the second wireless connection; and

displaying a virtual user interface according to the location of the first candidate object, wherein the virtual user interface is generated according to the obtained information.

5. The control method of claim 4, wherein the mobile device displays the virtual user interface only if the first candidate object overlaps with a region locating at the center of the first image, or only if the first candidate object is stared by a user.

6. The control method of claim 4, wherein at least a portion of an outline of the virtual user interface connects with at least a portion of the outline of the first candidate object.

7. The control method of claim 1, further comprising:

obtaining a description of a second wireless device adjacent to the mobile device over a third wireless connection;

recognizing a second candidate object within the first image;

matching the first candidate object and the second candidate object with the description of the first wireless device and the description of the second wireless device, respectively; and

in response to that the second candidate object matches with the description of the second wireless device and a second predetermined instruction is received by the camera, generating a second command according to the description of the second wireless device, wherein the second command is to be transmitted to the second wireless device over a fourth wireless connection for manipulating the second wireless device.

8. The control method of claim 7, wherein the operation of matching the first candidate object and the second candidate object with the description of the first wireless device and the description of the second wireless device, respectively, further comprises:

establishing the second wireless connection between the first wireless device and the mobile device according to the description of the first wireless device;

establishing the fourth wireless connection between the second wireless device and the mobile device according to the description of the second wireless device;

transmitting a first verification command to the first wireless device over the second wireless connection, so as to trigger the first wireless device to generate a first notification;

determining whether the first notification is detected by the camera or not;

transmitting a second verification command to the second wireless device over the fourth wireless connection, so as to trigger the second wireless device to generate a second notification;

determining whether the second notification is detected by the camera or not;

in response to that the first notification is detected by the camera, determining the first candidate object is matched with the description of the first wireless device; and

in response to that the second notification is detected by the camera, determining the second candidate object is matched with the description of the second wireless device.

9. The control method of claim 8, wherein the mobile device transmits the first verification command and the second verification command sequentially.

10. The control method of claim 8, wherein the mobile device matches or pairs the first candidate object with the first wireless device only if the first notification locates within a region surrounded by an outline of the first candidate object, and matching or pairing the second candidate object with the second wireless device only if the second notification locates within a region surrounded by an outline of the second candidate object.

11. The control method of claim 1, wherein the description of the first wireless device comprises media access control (MAC) address and a list of operable commands,

wherein the mobile device establishes the second wireless connection according to the MAC address of the first wireless device, and sends the first command according to the list of operable commands of the first wireless device.

12. A mobile device comprising:

a camera, configured to capture a first image of a physical environment;

a wireless communication circuit, configured to obtain a description of a first wireless device adjacent to the mobile device through a plurality of broadcast packets;

a processing circuit, coupled with the camera and the wireless communication circuit, and being adapted to;

recognize a first candidate object within the first image;

match the first candidate object with the description of the first wireless device; and

in response to that the first candidate object matches with the description of the first wireless device and a first predetermined instruction is received by the camera, generate a first command according to the description of the first wireless device, wherein the first command is to be transmitted to the first wireless device over a second wireless connection for manipulating the first wireless device,

wherein in response to that the first candidate object matches with the description of the first wireless device, the processing circuit instructs the camera to capture a series of images of the physical environment, the series of images comprises a second image and a third image captured after the second image for a predetermined time period, and the processing circuit recognizes the first candidate object within the second image,

wherein in response to that the first candidate object is absent within the third image, the processing circuit instructs the wireless communication circuit to disconnect the second wireless connection.

13. The mobile device of claim 12, wherein the first predetermined instruction comprises user hand motions.

14. The mobile device of claim 12, further comprising:

a display circuit, coupled with the processing circuit;

wherein the processing circuit is further adapted to;

instruct the display circuit to display the first image comprising the first candidate object;

instruct the display circuit to emphasize an outline of the first candidate object;

instruct the display circuit to display the description of the first wireless device;

match or pair the candidate object with the description of the first wireless device based on user hand motions; and

in response to that the first candidate object matches with the description of the first wireless device, establish the second wireless connection between the first wireless device and the wireless communication circuit according to the description of the first wireless device.

15. The mobile device of claim 14, wherein the processing circuit instructs the wireless communication circuit to periodically obtain information from the first wireless device over the second wireless connection, and instructs the display circuit to display a virtual user interface according to the location of the first candidate object,

wherein the virtual user interface is generated according to the obtained information.

16. The mobile device of claim 15, wherein the display circuit displays the virtual user interface only if the first candidate object overlaps with a region locating at the center of the first image, or only if the first candidate object is stared by a user.

17. The mobile device of claim 15, wherein at least a portion of an outline of the user interface overlaps with at least a portion of the outline of the first candidate object.

18. The mobile device of claim 12, wherein the wireless communication circuit are further configured to obtain a description of a second wireless device adjacent to the mobile device over a third wireless connection, and

wherein the processing circuit are further adapted to:

recognize a second candidate object within the first image;

match the first candidate object and the second candidate object with the description of the first wireless device and the description of the second wireless device, respectively; and

in response to that the second candidate object matches with the description of the second wireless device and a second predetermined instruction is received by the camera, generate a second command according to the description of the second wireless device, wherein the second command is to be transmitted to the second wireless device over a fourth wireless connection for manipulating the second wireless device.

19. The mobile device of claim 18, wherein when the processing circuit matches the first candidate object and the second candidate object with the description of the first wireless device and the description of the second wireless device, respectively, the processing circuit is further adapted to:

establish the second wireless connection between the first wireless device and the wireless communication circuit according to the description of the first wireless device;

establish the fourth wireless connection between the second wireless device and the wireless communication circuit according to the description of the second wireless device;

instruct the wireless communication circuit to transmit a first verification command to the first wireless device over the second wireless connection, so as to trigger the first wireless device to generate a first notification;

determine whether the first notification is detected by the camera or not;

instruct the wireless communication circuit to transmit a second verification command to the second wireless device over the fourth wireless connection, so as to trigger the second wireless device to generate a second notification;

determine whether the second notification is detected by the camera or not;

in response to that the first notification is detected by the camera, determine the first candidate object is matched with the description of the first wireless device; and

in response to that the second notification is detected by the camera, determine the second candidate object is matched with the description of the second wireless device.

20. The mobile device of claim 19, wherein the processing circuit instructs the wireless communication circuit to transmit the first verification command and the second verification command sequentially.

21. The mobile device of claim 19, wherein the processing circuit matches or pairs the first candidate object with the first wireless device only if the first notification locates within a region surrounded by an outline of the first candidate object, and matching or pairing the second candidate object with the second wireless device only if the second notification locates within a region surrounded by an outline of the second candidate object.

22. The mobile device of claim 12, wherein the description of the first wireless device comprises MAC address and a list of operable commands,

wherein the processing circuit establishes the second wireless connection according to the MAC address of the first wireless device, and sends the first command according to the list of operable commands of the first wireless device.