Capturing objects in editable format using gestures

• US 10,331,330 B2
• Filed: 09/25/2012
• Issued: 06/25/2019
• Est. Priority Date: 09/25/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Petrov, Nikolay Andreevich
• Primary Examiner(s): Barrett, Ryan
• Application Number:
  US13/993,639
• Publication Number:
  US 20150177944A1
• Time in Patent Office: 2,464 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Embodiments of methods, systems, and storage medium associated with capturing, with a user device, at least a portion of an object based on a user gesture indicating a command are disclosed herein. In one instance, the method may include identifying a gesture associated with an object of interest external to the user device; capturing an image of at least a portion of the object of interest based on a result of the identifying the gesture; and providing the portion of the object in an editable format based on the image. Other embodiments may be described and/or claimed.

Claims

1. At least one non-transitory computing device-readable storage medium having instructions stored thereon that, in response to execution by a computing device, cause the computing device to:

identify a gesture associated with a displayed first image of an object of interest external to the computing device, the gesture indicating a combination of commands comprising a first command to select and capture at least a portion of the object of interest from the displayed first image, the selected portion including one or more moving non-textual elements, a second command to perform an operation on the selected portion of the object of interest, and a third command to expose a set of inferred dynamic properties associated with the one or more moving non-textual elements;

based on a result of identifying the gesture, select the at least a portion of the object from the displayed first image, calculate the set of inferred dynamic properties, and capture a second image of the at least a portion of the object of interest and the exposed inferred dynamic properties from the displayed first image of the object, in response to the first and third commands provided by the identified gesture;

wherein the computing device comprises a mobile device, and wherein to capture the second image includes to determine and provide a suggestion of a particular positioning of the computing device relative to the displayed first image of the object to capture the second image of the at least a portion of the object of interest from the displayed first image of the object;

perform the indicated operation on the captured second image of the portion of the object, according to the second command indicated by the identified gesture; and

process the second image, after performance of the operations indicated by the first and second commands, to generate a third image, the third image to be provided in an editable format for use in one or more further operations in accordance with one or more further commands.

2. The non-transitory computing device-readable storage medium of claim 1, wherein the instructions, in response to execution by the computing device, further cause the computing device to capture one or more properties associated with the portion of the object of interest, according to the combination of commands indicated by the identified gesture.

3. The non-transitory computing device-readable storage medium of claim 2, wherein the one or more properties are selected from at least one of dynamic characteristics associated with the portion of the object of interest or static parameters associated with the portion of the object of interest.

4. The non-transitory computing device-readable storage medium of claim 1, wherein the instructions, in response to execution by the computing device, further cause the computing device to:

receive a command indicating a mode of operation of the computing device, wherein the mode of operation includes at least one of a text mode or an image mode; and

change the mode of operation in accordance with the mode indicating command, wherein the instructions cause the computing device to operate with text or portions of text in the text mode, and to operate with images or portions of images in the image mode.

5. The non-transitory computing device-readable storage medium of claim 4, wherein the mode indicating command is selected from at least one of a gesture command or a voice command.

6. The non-transitory computing device-readable storage medium of claim 1, wherein the object is associated with at least one of a real surface or a virtual surface.

7. The non-transitory computing device-readable storage medium of claim 1, wherein the instructions, in response to execution by the computing device, further cause the computing device to copy the selected portion of the object of interest to a clipboard, according to the combination of commands.

8. The non-transitory computing device-readable storage medium of claim 7, wherein the copied portion of the object of interest is rendered for display on the computing device.

9. The non-transitory computing device-readable storage medium of claim 7, wherein the object comprises a text, wherein to identify a gesture includes to determine a placement of two fingers of a user relative to a portion of the text to be selected and copied, wherein one finger is to indicate an upper limit of the portion of the text to be selected and copied, and another finger is to indicate a lower limit of the portion of the text to be selected and copied, wherein to determine a placement includes to determine whether the two fingers are placed to a left or to a right from the portion of the text to be selected and copied.

10. The non-transitory computing device-readable storage medium of claim 7, wherein the object comprises a text, wherein to identify a gesture includes to determine that two fingers of a user are wrapped around of a portion of the text to be selected and copied, wherein the portion of the text comprises a sentence or part of the sentence.

11. The non-transitory computing device-readable storage medium of claim 1, wherein the operation associated with the portion of the object of interest includes at least one of:

editing the portion of the object of interest or incorporating the portion of the object of interest into a document.

12. The non-transitory computing device-readable storage medium of claim 1, wherein the portion of the object of interest is a selected one of text or image.

13. The non-transitory computing device-readable storage medium of claim 1, wherein the inferred dynamic properties include at least one of a speed, a velocity or a rotational speed of at least one of the moving non-textual elements.

14. The non-transitory computing device-readable storage medium of claim 13, wherein calculate the set of inferred dynamic properties includes to calculate in real or near-real time at least one of the speed, velocity or rotational speed.

15. A computing device comprising:

a processor; and

an object capture application operated by the processor to;

identify a gesture associated with a displayed first image of an object of interest external to the computing device, the gesture indicating a combination of commands comprising a first command to select and capture at least a portion of the object of interest from the displayed first image, the selected portion including one or more moving non-textual elements, a second command to perform an operation on the selected portion of the object of interest, and a third command to expose a set of inferred dynamic properties associated with the one or more moving non-textual elements;

based on a result of identifying the gesture, select the at least a portion of the object from the displayed first image, calculate the set of inferred dynamic properties, and capture a second image of the at least a portion of the object of interest and the exposed inferred dynamic properties from the displayed first image of the object, in response to the first and third commands provided by the identified gesture;

wherein the computing device comprises a mobile device, and wherein to capture the second image includes to determine and provide a suggestion of a particular positioning of the computing device relative to the displayed first image of the object to capture the second image of the at least a portion of the object of interest from the displayed first image of the object;

perform the indicated operation on the captured second image of the portion of the object, in response to the second command; and

process the second image, after performance of the operation indicated by the second command, to generate a third image, or cause another computing device to process the second image, after performance of the operations indicated by the first and second commands, to generate the third image, the third image to be provided in an editable format for use in one or more further operations in accordance with one or more further commands.

16. The computing device of claim 15, wherein the object capture application is further configured to capture, in response to a request, one or more properties associated with the portion of the object of interest.

17. The computing device of claim 16, wherein the one or more properties are selected from at least one of dynamic characteristics associated with the portion of the object of interest or static parameters associated with the portion of the object of interest.

18. The computing device of claim 17, wherein the dynamic characteristics include one or more parameters associated with a motion of the portion of the object, and wherein the static parameters include at least one of size, color, or language associated with the portion of the object.

19. The computing device of claim 15, wherein the inferred dynamic properties include at least one of a speed, a velocity or a rotational speed of at least one of the moving non-textual elements.

20. The computing device of claim 19, wherein calculate the set of inferred dynamic properties includes to calculate in real or near-real time at least one of the speed, velocity or rotational speed.

21. A computer-implemented method comprising:

identifying, by a computing device, a gesture associated with a displayed first image of an object of interest external to the computing device, the gesture indicating a combination of commands comprising a first command to select and capture at least a portion of the object of interest from the displayed first image, the selected portion including one or more moving non-textual elements, a second command to perform an operation on the selected portion of the object of interest, and a third command to expose a set of inferred dynamic properties associated with the one or more moving non-textual elements;

selecting, by the computing device, the at least a portion of the object of interest from the displayed first image of the object in response to the first command, according to the identified gesture;

calculating the set of inferred dynamic properties;

capturing, by the computing device, a second image of the selected portion of the object of interest and the exposed inferred dynamic properties from the displayed first image, in response to the first and third commands;

wherein the computing device comprises a mobile device, and wherein capturing the second image includes determining and providing a suggestion of a particular positioning of the computing device relative to the displayed first image of the object to capture the second image of the at least a portion of the object of interest from the displayed first image of the object;

performing, by the computing device, the indicated operation on the captured second image of the portion of the object according to the second command indicated by the gesture; and

processing the second image, after performance of the operations indicated by the first and second commands, to generate a third image, the third image to be provided in an editable format for use in one or more further operations in accordance with one or more further commands.

22. The computer-implemented method of claim 21, wherein the editable format includes a digital form of representation of the portion of the object.

23. The computer-implemented method of claim 21, further comprising capturing, by the computing device, one or more properties associated with the portion of the object of interest.

24. The computer-implemented method of claim 21, wherein identifying a gesture associated with an object of interest external to the computing device includes:

identifying, by the computing device, the gesture based on a library of gestures.

25. The computer-implemented method of claim 24, wherein the library of gestures is stored in a data store and is accessible by the computing device.

26. The computer-implemented method of claim 21, wherein the portion of the object of interest is a selected one of text or image.

27. The computer-implemented method of claim 21, wherein the inferred dynamic properties include at least one of a speed, a velocity or a rotational speed of at least one of the moving non-textual elements.

28. The computer-implemented method of claim 27, wherein calculating the set of inferred dynamic properties further comprises calculating in real or near-real time at least one of the speed, velocity or rotational speed.