×

Automating user input from onscreen content

  • US 10,592,261 B1
  • Filed: 04/01/2019
  • Issued: 03/17/2020
  • Est. Priority Date: 07/11/2014
  • Status: Active Grant
First Claim
Patent Images

1. A method comprising:

  • receiving a selection of a first image representing a screen captured on a mobile device associated with a user, the first image having a first timestamp;

    determining, using a data store of images of previously captured screens of the mobile device, a reference image from the data store that has a prior timestamp that is prior to the first timestamp;

    identifying, based on the first image and the reference image, a plurality of user input actions occurring between the prior timestamp and the first timestamp, each of the user input actions having a respective timestamp;

    identifying, based on the reference image, a software application installed on the mobile device, the reference image representing a screen of the software application;

    launching, on the mobile device, the software application to a state tailored to the reference image; and

    causing, based on the respective timestamps of the user input actions, the software application of the mobile device to replay the user input actions,wherein replaying the user input actions causes the software application to transition from the state that is tailored to the reference image to a state that corresponds to the first image.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×