DRAG-AND-DROP ON A MOBILE DEVICE

• US 20160117072A1
• Filed: 10/24/2014
• Published: 04/28/2016
• Est. Priority Date: 10/24/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Petrou, David
• Granted Patent: US 9,606,716 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/543 User-generated data transfer, e.g. clipboards, dynamic data exchange [DDE], object linking and embedding [OLE]

Abstract

Implementations provide an improved drag-and-drop operation on a mobile device. For example, a method includes identifying a drag area in a user interface of a first mobile application in response to a drag command, identifying an entity from a data store based on recognition performed on content in the drag area, receiving a drop location associated with a second mobile application, determining an action to perform in the second mobile application based on the drop location, and performing the action in the second mobile action using the entity. Another method may include receiving a selection of a smart copy control for a text input control in a first mobile application, receiving a selected area of a display generated by a second mobile application, identifying an entity in the selected area, automatically navigating back to the text input control, and pasting a description of the entity in the text input control.

Claims

1. A method comprising:

identifying, on a touchscreen display, a drag area in a user interface of a first mobile application in response to a drag command from a user of a mobile device;

identifying an entity from a graph-based data store based on recognition performed on content in the drag area;

receiving a drop location associated with a second mobile application, the second mobile application differing from the first mobile application;

determining an action to be performed in the second mobile application based on the drop location; and

performing the action in the second mobile action using the entity.

2. The method of claim 1, wherein the action includes sharing the entity via a new message when the second mobile application includes a communications application.

3. The method of claim 1, wherein the action includes initiating a search using a description of the entity when the second mobile application has a search user interface.

4. The method of claim 1, further comprising identifying the drop in response to a voice command that identifies a user interface of the second mobile application.

5. The method of claim 1, wherein the drop location includes an icon that launches the second mobile application.

6. The method of claim 5, wherein the action includes launching a particular user interface in the second mobile application based on a type of the entity.

7. The method of claim 1, further comprising:

determining that the drop location includes a text input control,

wherein performing the action includes pasting a text description for the entity from the graph in the text input control.

8. The method of claim 1, further comprising:

displaying an indication of the drag area from the first mobile application on the display while the user navigates to the drop location in the second mobile application.

9. The method of claim 1, wherein the entity is a first entity and the method further comprises:

storing the first entity and the second mobile application in a data store;

identifying at least a second entity from the graph in a second drag area in response to a second drag command from the user;

determining that an entity type for the second entity matches an entity type for the first entity; and

based on the determination, providing the second mobile application as a suggested drop location.

10. The method of claim 1, further comprising:

sending the drag area to a server, the server identifying the at least one entity.

11. The method of claim 1, wherein the drag area includes an image and the entity is recognized in the image.

12. A mobile device comprising:

a display;

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the mobile device to;

identify a drag area in a user interface of a first mobile application in response to a drag command from a user of the mobile device;

determine, based on recognition performed on content in the drag area, that the drag area includes at least one entity from a graph-based data store storing a graph having entities as nodes of the graph;

receive a drop location;

determine the drop location is associated with a contact entity; and

automatically initiate a communication to the contact entity, the communication including information about the entity from the graph.

13. The mobile device of claim 12, wherein the drop location is identified in a voice command that identifies the contact entity.

14. The mobile device of claim 12, wherein the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:

determine that the drop location occurs on an email address,

wherein the communication is an email message.

15. The mobile device of claim 14, wherein the mobile device includes a camera that generates a picture preview user interface and wherein the email address appears in the picture preview user interface.

16. The mobile device of claim 12, wherein the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:

determine that a drop operation occurs on a social media identifier,

wherein the communication includes sharing the entity with the social media identifier via a social media application.

17. The mobile device of claim 12, wherein the mobile device includes a camera that generates a picture preview user interface and the drag area appears in the picture preview user interface.

18. The mobile device of claim 12, wherein the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:

display an indication of the drag area on the display while the user navigates to the drop location.

19. The mobile device of claim 12, wherein the drag area is image-based and the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:

search for a public URL that matches the drag area; and

provide the public URL in the communication.

20. The mobile device of claim 12, wherein the drag area is image-based and the memory further stores instructions that, when executed by the at least one processor, cause the mobile device to:

upload the drag area to a server;

receive a URL from the server, the URL being a location of the drag area on the server; and

provide the URL in the communication.

21. The mobile device of claim 12, wherein determining that the drag area includes the at least one entity includes:

sending the drag area to a server, the server storing the graph-based data store; and

receiving information about the entity from the server.

22. A method comprising:

receiving a selection of a smart copy control for a text input control on a mobile device, the text input control being in a first mobile application;

receiving a selected area of a display generated by a second mobile application;

identifying at least one entity of a graph-based data store in the selected area based on recognition performed on content of the selected area;

automatically navigating back to the text input control in the first mobile application; and

pasting a text description of the entity from the graph in the text input control.

23. The method of claim 22, further comprising:

providing at least a second entity as a suggestion, an entity type of the second entity being compatible with the first mobile application.

24. The method of claim 22, further comprising:

providing a third mobile application as a suggestion, the third mobile application selected based on prior smart copy operations.

25. The method of claim 22, wherein the entity is identified from an image in the selected area.