Input commands via visual cues
First Claim
Patent Images
1. A method, comprising:
- receiving, from a client device, at a management computing system remote from the client device, a request for a web page of a website hosted by a web server of the management computing system, the request received via an application executing on the client device;
transmitting, to the client device, the web page associated with the request;
requesting, by the management computing system from the client device, permission for the application executing on the client device to access a camera associated with the client device;
receiving, from the client device, permission for the application to access the camera while a user of the client device operates the application executing on the client device;
receiving, from a camera associated with the client device, at the management computing system, a video stream of a user interacting with the web page of the website via the application executing on the client device, the video stream comprising a time series of images of the user;
parsing, by the management computing system, the video stream to identify one or more visual cues of the user;
identifying a current page of the website accessed by the user via the application by parsing a uniform resource locator associated with the request for the web page;
mapping the identified one or more visual cues to an operation available on the current page of the website; and
executing the mapped operation.
1 Assignment
0 Petitions
Accused Products
Abstract
Embodiments disclosed herein generally relate to a method and system of generating text input via facial recognition. A computing system receives a video stream of a user operating an application on a client device. The video stream includes a time series of images of the user. The computing system parses the video stream to identify one or more visual cues of the user. The computing system identifies a current page of the application. The computing system maps the identified on or more visual cues to an operation available on the current page of the application. The computing system executes the mapped operation.
-
Citations
20 Claims
-
1. A method, comprising:
-
receiving, from a client device, at a management computing system remote from the client device, a request for a web page of a website hosted by a web server of the management computing system, the request received via an application executing on the client device; transmitting, to the client device, the web page associated with the request; requesting, by the management computing system from the client device, permission for the application executing on the client device to access a camera associated with the client device; receiving, from the client device, permission for the application to access the camera while a user of the client device operates the application executing on the client device; receiving, from a camera associated with the client device, at the management computing system, a video stream of a user interacting with the web page of the website via the application executing on the client device, the video stream comprising a time series of images of the user; parsing, by the management computing system, the video stream to identify one or more visual cues of the user; identifying a current page of the website accessed by the user via the application by parsing a uniform resource locator associated with the request for the web page; mapping the identified one or more visual cues to an operation available on the current page of the website; and executing the mapped operation. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A system, comprising:
-
a processor; and memory storing program code, which, when executed by the processor, performs an operation, comprising; receiving, from a remote client device, a request for a web page of a website hosted by a web server of the system, the request received via an application executing on the client device; transmitting, to the client device, the web page associated with the request; requesting, from the client device, permission for the application executing on the client device to access a camera associated with the client device; receiving, from the client device, permission for the application to access the camera while a user of the client device operates the application executing on the client device; receiving, from the client device, a video stream of a user accessing the web page hosted by the web server via the application executing on the client device; parsing a time series of images in the video stream to identify a command visually communicated by the user; identifying a current page of the website accessed by the user via the application by parsing a uniform resource identifier associated with the request for the web page; mapping the identified visual command of the user to an operation available on the current page of the website; and executing the mapped operation. - View Dependent Claims (9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A method, comprising:
-
requesting, by a client device, a web page of a website hosted by a remote server, the request sent via an application executing on the client device; receiving, by the client device from the remote server, the web page associated with the request; receiving, by the client device from the remote server, a request for the application executing on the client device to access a camera associated with the client device; granting, by the client device to the application, access to the camera associated with the client device while a user of the client device is operating the application executing on the client device; capturing, by the camera associated with the client device, a video stream of a user while the user is operating the application on the client device; parsing, by the client device, the video stream to identify one or more visual cues to identify a command visually communicated by the user; mapping, by the client device, the one or more visual cues performed by the user to one or more stored commands; sending, by the client device, the mapped command to the remote server to identify an operation the stored command maps to on a current page of the website by identifying a uniform resource locator associated with the request for the web page; receiving, by the client device, the operation from the remote server; and updating, by the client device, an interface of the client device to prompt the user with a text-based description of the operation. - View Dependent Claims (18, 19, 20)
-
Specification