Automating user input from onscreen content
First Claim
1. A method comprising:
- receiving a selection of a first image representing a screen captured on a mobile device associated with a user, the first image having a first timestamp;
determining, using a data store of images of previously captured screens of the mobile device, a reference image from the data store that has a prior timestamp that is prior to the first timestamp;
identifying, based on the first image and the reference image, a plurality of user input actions occurring between the prior timestamp and the first timestamp, each of the user input actions having a respective timestamp;
identifying, based on the reference image, a software application installed on the mobile device, the reference image representing a screen of the software application;
launching, on the mobile device, the software application to a state tailored to the reference image; and
causing, based on the respective timestamps of the user input actions, the software application of the mobile device to replay the user input actions,wherein replaying the user input actions causes the software application to transition from the state that is tailored to the reference image to a state that corresponds to the first image.
2 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods are provided for automating user input using onscreen content. For example, a method includes receiving a selection of a first screen capture image representing a screen captured on a mobile device associated with a user, the first image having a first timestamp. The method also includes determining, using a data store of images of previously captured screens of the mobile device, a reference image from the data store that has a timestamp prior to the first timestamp, identifying a plurality of images in the data store that have respective timestamps between the timestamp for the reference image and the first timestamp, and providing the reference image, the plurality of images, and the first image to the mobile device.
-
Citations
20 Claims
-
1. A method comprising:
-
receiving a selection of a first image representing a screen captured on a mobile device associated with a user, the first image having a first timestamp; determining, using a data store of images of previously captured screens of the mobile device, a reference image from the data store that has a prior timestamp that is prior to the first timestamp; identifying, based on the first image and the reference image, a plurality of user input actions occurring between the prior timestamp and the first timestamp, each of the user input actions having a respective timestamp; identifying, based on the reference image, a software application installed on the mobile device, the reference image representing a screen of the software application; launching, on the mobile device, the software application to a state tailored to the reference image; and causing, based on the respective timestamps of the user input actions, the software application of the mobile device to replay the user input actions, wherein replaying the user input actions causes the software application to transition from the state that is tailored to the reference image to a state that corresponds to the first image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A computing device, comprising:
-
at least one processor; and at least one memory device storing instructions that, when executed, cause the at least one processor to; receive a selection of a first image representing a screen captured on a computing device associated with a user, the first image having a first timestamp; determine, using a data store of images of previously captured screens of the computing device, a reference image from the data store that has a prior timestamp that is prior to the first timestamp; identify, based on the first image and the reference image, a plurality of user input actions occurring between the prior timestamp and the first timestamp, each of the user input actions having a respective timestamp; identify, based on the reference image, a software application installed on the computing device, the reference image representing a screen of the software application; launch, on the computing device, the software application to a state that is tailored to the reference image; and cause, based on the respective timestamps of the user input actions, the software application of the computing device to replay the user input actions, wherein the instructions that cause the software application to replay each of the user input actions further include instructions that cause the software application to transition from the state that is tailored to the reference image to a state that corresponds to the first image. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
-
17. A mobile device, comprising:
-
at least one processor; and at least one memory device storing instructions that, when executed, cause the at least one processor to; receive, at the mobile device and from a second mobile device, a first image representing a screen captured on the second mobile device, wherein the mobile device is associated with a user, wherein the second mobile device is associated with the user or an additional user, and wherein the first image has a first timestamp; receive, at the mobile device, a selection of the first image; determine, using a data store of images of previously captured screenshots, a reference image from the data store that has a prior timestamp that is prior to the first timestamp; identify, based on the first image and the reference image, a plurality of user input actions occurring between the prior timestamp and the first timestamp, each of the user input actions having a respective timestamp; identify, based on the reference image, a software application installed on the mobile device, the reference image representing a screen of the software application; launch, on the mobile device, the software application to a state that is tailored to the reference image; and cause, based on the respective timestamp for the user input actions, the software application of the mobile device to replay the user input actions, wherein the instructions that cause the software application to replay the user input actions include instructions that cause the software application to transition from the state that is tailored to the reference image to a state that corresponds to the first image. - View Dependent Claims (18, 19, 20)
-
Specification