Systems and methods for content-aware selection

• US 9,785,240 B2
• Filed: 03/18/2013
• Issued: 10/10/2017
• Est. Priority Date: 03/18/2013
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Denoue, Laurent, Carter, Scott
• Primary Examiner(s): Lo, Weilun
• Assistant Examiner(s): Lobo, Carl
• Application Number:
  US13/846,840
• Publication Number:
  US 20140282242A1
• Time in Patent Office: 1,667 Days
• Field of Search: 715810
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

Systems and methods detect simple user gestures to enable selection of portions of segmented content, such as text, displayed on a display. Gestures may include finger (such as thumb) flicks or swipes as well as flicks of the handheld device itself. The used finger does not occlude the selected text, allowing users to easily see what the selection is at any time during the content selection process. In addition, the swipe or flick gestures can be performed by a non-dominant finger such as a thumb, allowing users to hold the device and make the selection using only one hand. After making the initial selection of a target portion of the content, to extend the selection, for example to the right, the user simply swipes or flicks the finger over the touchscreen to the right. The user could also flick the entire device in a move gesture with one hand.

Claims

1. A computer-implemented method performed in a computerized system comprising a central processing unit, a display device and a memory, the computer-implemented method comprising:

a. displaying a segmented content on the display device, wherein the content is segmented using a plurality of content segmentation boundaries comprising word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content;

b. detecting an initial selection of a portion of the displayed segmented content by a user and marking a boundary of the selected portion of the displayed segmented content on the display device;

c. while continuing to mark on the display device the boundary of the selected portion, detecting a swipe or flick gesture of the user outside of the marked boundary of the selected portion of the displayed segmented content, wherein a starting point of the detected user gesture is outside of a start portion of the boundary and an end portion of the boundary of the selected portion;

d. responsive to the detecting of the gesture, using the central processing unit to reposition at least a portion of the boundary of the initially selected portion of the displayed segmented content based on at least one characteristic of the gesture and the segmentation of the content, wherein a location of the detected gesture in relation to the selected portion of the displayed content determines whether the start portion of the boundary of the selected portion of the displayed content or the end portion of the boundary of the selected portion of the displayed content is repositioned, wherein when the gesture is detected above the selected portion of the displayed segmented content, the start portion of the boundary of the selected portion of the displayed content is repositioned and when the gesture is detected below the selected portion of the displayed segmented content, the end portion of the boundary of the selected portion of the displayed content is repositioned and wherein the portion of the boundary of the selected portion of the displayed content is repositioned based on the word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content; and

e. marking the repositioned boundary of the selected portion of the segmented content on the display device.

2. The computer-implemented method of claim 1, further comprising repeating c. through e. for at least one additional gesture of the user.

3. The computer-implemented method of claim 1, wherein the segmented content comprises a textual content segmented based on at least one of:

word boundaries, expression boundaries, sentence boundaries or punctuations.

4. The computer-implemented method of claim 1, wherein the segmented content comprises a graphical content comprising a plurality of graphical elements clustered into a plurality of segments based of spatial, temporal or contextual proximity.

5. The computer-implemented method of claim 1, wherein the extent of the repositioning of the portion of the boundary of the selected portion of the displayed content is based at least on a type of the gesture.

6. The computer-implemented method of claim 1, wherein a starting point of the detected user gesture is outside of a close proximity of the start portion of the boundary and the end portion of the boundary of the selected portion.

7. The computer-implemented method of claim 1, wherein the content is segmented using a plurality of content segmentation boundaries located within the content.

8. The computer-implemented method of claim 7, wherein the portion of the boundary of the selected portion of the displayed content is repositioned to a location of one of the plurality of content segmentation boundaries.

9. The computer-implemented method of claim 7, wherein the plurality of content segmentation boundaries comprises at least word boundaries and sentence boundaries.

10. The computer-implemented method of claim 7, wherein the plurality of content segmentation boundaries are defined by at least white spaces and punctuations.

11. The computer-implemented method of claim 7, wherein the repositioning of the at least a portion of the boundary of the selected portion of the displayed content is slowed when the repositioned portion of the boundary reaches one of the plurality of content segmentation boundaries.

12. The computer-implemented method of claim 1, wherein the portion of the boundary of the selected portion of the displayed content is repositioned in a same direction as a direction of the gesture.

13. The computer-implemented method of claim 1, wherein the gesture is detected independently of a keyboard of the computerized system.

14. The computer-implemented method of claim 1, wherein the extent of the repositioning of the at least a portion of the boundary of the selected portion of the displayed content is based at least on an amplitude of the gesture.

15. The computer-implemented method of claim 14, wherein the content is segmented using a plurality of content segmentation boundaries located within the content and wherein the extent of the repositioning of the at least a portion of the boundary of the selected portion of the displayed content is additionally based on the content segmentation boundaries.

16. The computer-implemented method of claim 1, wherein at least a portion of the gesture of the user is detected inside the marked boundary of the selected portion of the displayed segmented content.

17. A non-transitory computer-readable medium embodying a set of computer-executable instructions, which, when executed in a computerized system comprising a central processing unit, a display device and a memory, cause the computerized system to perform a method comprising:

a. displaying a segmented content on the display device, wherein the content is segmented using a plurality of content segmentation boundaries comprising word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content;

b. detecting an initial selection of a portion of the displayed segmented content by a user and marking a boundary of the selected portion of the displayed segmented content on the display device;

c. while continuing to mark on the display device the boundary of the selected portion, detecting a swipe or flick gesture of the user outside of the marked boundary of the selected portion of the displayed segmented content, wherein a starting point of the detected user gesture is outside of a start portion of the boundary and an end portion of the boundary of the selected portion;

d. responsive to the detecting of the gesture, using the central processing unit to reposition at least a portion of the boundary of the initially selected portion of the displayed segmented content based on at least one characteristic of the gesture and the segmentation of the content, wherein a location of the detected gesture in relation to the selected portion of the displayed content determines whether the start portion of the boundary of the selected portion of the displayed content or the end portion of the boundary of the selected portion of the displayed content is repositioned, wherein when the gesture is detected above the selected portion of the displayed segmented content, the start portion of the boundary of the selected portion of the displayed content is repositioned and when the gesture is detected below the selected portion of the displayed segmented content, the end portion of the boundary of the selected portion of the displayed content is repositioned and wherein the portion of the boundary of the selected portion of the displayed content is repositioned based on the word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content; and

e. marking the repositioned boundary of the selected portion of the segmented content on the display device.

18. A computerized system comprising a processing unit, a display device and a memory, the memory storing:

a. a user interface generation module for causing a segmented content to be displayed on the display device, wherein the content is segmented using a plurality of content segmentation boundaries comprising word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content;

b. an initial selection module for detecting an initial selection of a portion of the displayed segmented content by a user and causing a boundary of the selected portion of the displayed segmented content to be marked on the display device;

c. a gesture analysis module for detecting a swipe or flick gesture of the user outside of the marked boundary of the selected portion of the displayed segmented content, wherein the boundary of the selected portion continues to be marked on the display device during the detecting, wherein a starting point of the detected user gesture is outside of a start portion of the boundary and an end portion of the boundary of the selected portion; and

d. a selection adjustment module for, responsively to the detecting of the gesture, causing the processing unit to reposition at least a portion of the boundary of the initially selected portion of the displayed segmented content based on at least one characteristic of the gesture and a segmentation of the content and causing the repositioned boundary of the selected portion of the segmented content to be marked on the display device, wherein a location of the detected gesture in relation to the selected portion of the displayed content determines whether the start portion of the boundary of the selected portion of the displayed content or the end portion of the boundary of the selected portion of the displayed content is repositioned, wherein when the gesture is detected above the selected portion of the displayed segmented content, the start portion of the boundary of the selected portion of the displayed content is repositioned and when the gesture is detected below the selected portion of the displayed segmented content, the end portion of the boundary of the selected portion of the displayed content is repositioned and wherein the portion of the boundary of the selected portion of the displayed content is repositioned based on the word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content.

19. A computer-implemented method performed in a computerized system comprising a central processing unit, a display device and a memory, the computer-implemented method comprising:

a. displaying a content on the display device, wherein the content is segmented using a plurality of content segmentation boundaries comprising word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content;

b. detecting an initial selection of a portion of the displayed content by a user and marking a boundary of the selected portion of the displayed content on the display device;

c. while continuing to mark on the display device the boundary of the selected portion, detecting a swipe or flick gesture of the user outside of the marked boundary of the selected portion of the displayed content, wherein a starting point of the detected user gesture is outside of a start portion of the boundary and an end portion of the boundary of the selected portion;

d. responsive to the detecting of the gesture, using the central processing unit to reposition at least a portion of the boundary of the selected portion of the displayed content based on at least one characteristic of the gesture wherein a location of the detected gesture in relation to the selected portion of the displayed content determines whether the start portion of the boundary of the selected portion of the displayed content or the end portion of the boundary of the selected portion of the displayed content is repositioned; and

e. marking the repositioned boundary of the selected portion of the content on the display device, wherein when the gesture is detected upstream of the selected portion of the displayed content, the start portion of the boundary of the initially selected portion of the displayed content is repositioned and wherein when the gesture is detected downstream of the selected portion of the displayed content, the end portion of the boundary of the selected portion of the displayed content is repositioned and wherein the portion of the boundary of the selected portion of the displayed content is repositioned based on the word boundaries, sentence boundaries, paragraph boundaries or chapter boundaries located within the content.