Device control by speech commands with microphone and camera to acquire line-of-sight information
First Claim
1. A device control method of a controller connected to a first device, a second device, a camera, and a microphone,the first device being installed at a first location in a first space,the second device being installed at a second location in a second space which is not overlapped with the first space,the camera detecting a line-of-sight direction of a user,the microphone detecting speech of the user,the first space being visible to the user, andthe second space not being visible to the user,the device control method comprising:
- acquiring sound data indicating the detected speech of the user from the microphone;
determining whether the detected speech of the user includes a phrase indicating the first device or the second device;
acquiring, from the camera, line-of-sight information indicating the line-of-sight direction of the user when it is determined that the speech does not include the phrase;
determining whether the line-of-sight direction of the user indicates a third location other than the first location in the first space, based on the line-of-sight information;
identifying, in a case where the line-of-sight direction of the user is determined to indicate the third location, the second device as an object of control;
generating a control command corresponding to the sound data; and
transmitting the control command to the object of control.
1 Assignment
0 Petitions
Accused Products
Abstract
A first device is installed at a first location in a first space visible to a user. A second device is installed at a second location in a second space not visible to the user. A device control method acquires line-of-sight information indicating a line-of-sight direction of the user from a camera. The line-of-sight direction of the user is determined based on the line-of-sight information. In a case where the line-of-sight direction indicates a third location other than the first location, the second device is identified as an object of control. Sound data indicating speech of the user is acquired from a microphone, a control command corresponding to the sound data is generated, and the control command is transmitted to the object of control.
22 Citations
19 Claims
-
1. A device control method of a controller connected to a first device, a second device, a camera, and a microphone,
the first device being installed at a first location in a first space, the second device being installed at a second location in a second space which is not overlapped with the first space, the camera detecting a line-of-sight direction of a user, the microphone detecting speech of the user, the first space being visible to the user, and the second space not being visible to the user, the device control method comprising: -
acquiring sound data indicating the detected speech of the user from the microphone; determining whether the detected speech of the user includes a phrase indicating the first device or the second device; acquiring, from the camera, line-of-sight information indicating the line-of-sight direction of the user when it is determined that the speech does not include the phrase; determining whether the line-of-sight direction of the user indicates a third location other than the first location in the first space, based on the line-of-sight information; identifying, in a case where the line-of-sight direction of the user is determined to indicate the third location, the second device as an object of control; generating a control command corresponding to the sound data; and transmitting the control command to the object of control. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A controller connected to a first device, a second device, a camera, and a microphone,
wherein the first device is installed at a first location in a first space, wherein the second device is installed at a second location in a second space which is not overlapped with the first space, wherein the camera detects a line-of-sight direction of a user, wherein the microphone detects speech of the user, wherein the first space is visible to the user, wherein the second space is not visible to the user, and wherein the controller acquires sound data indicating the detected speech of the user from the microphone, determines whether the detected speech of the user includes a phrase indicating the first device or the second device, acquires, from the camera, line-of-sight information indicating the line-of-sight direction of the user when it is determined that the speech does not include the phrase, determines whether the line-of-sight direction of the user indicates a third location other than the first location in the first space, based on the line-of-sight information, identifies, in a case where the line-of-sight direction of the user is determined to indicate the third location, the second device as an object of control, generates a control command corresponding to the sound data, and transmits the control command to the object of control.
Specification
-
- Resources
-
Current AssigneePanasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
-
Original AssigneePanasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
-
InventorsOkubo, Masafumi, Tanaka, Keiichi, Nakamura, Tomonori
-
Primary Examiner(s)Lerner, Martin
-
Application NumberUS15/174,933Publication NumberTime in Patent Office533 DaysField of Search704231, 704235, 704236, 704270, 704275, 7042701US Class CurrentCPC Class CodesG06F 3/013 Eye tracking input arrangem...G06F 3/167 Audio in a user interface, ...G10L 15/22 Procedures used during a sp...G10L 15/24 Speech recognition using no...G10L 2015/223 Execution procedure of a sp...G10L 2015/228 of application contextH04L 12/282 based on user interaction w...H04L 12/2825 Reporting to a device locat...H04L 12/2829 involving user profiles acc...H04L 2012/285 Generic home appliances, e....
