TOUCH BASED SELECTION OF GRAPHICAL ELEMENTS

• US 20140129985A1
• Filed: 11/02/2012
• Published: 05/08/2014
• Est. Priority Date: 11/02/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Winternitz, Fabian O., Lee, Craig Allen, Morozov, Vladimir Y.
• Granted Patent: US 9,690,449 B2
• Time in Patent Office: Days
• US Class Current: 715/823
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

The present invention extends to methods, systems, and computer program products for touch based selection of graphical elements. Embodiments of the invention improve the usability of interacting with data visualizations. When multiple graphical elements are covered by a finger touch, a gesture recognition component detects ambiguity in the selection. In response, a visualization component draws additional regions around the touched location. The additional regions correspond to the covered graphical elements covered. Thus, a user is made aware of the graphical elements covered by his or her finger. These additional regions are touch friendly and more easily selectable. A finger can be moved into one of the additional regions and released to finish the selection. The gesture recognition component interprets this gesture as if the user tapped directly and unambiguously on the corresponding graphical element.

Claims

1. At a computing device, the computing device including a touch screen display device, a method for selecting a graphical element, the method comprising:

presenting a data visualization on the touch screen display device, the data visualization including a plurality of graphical elements, each graphical element representing a portion of data;

detecting a touch based selection gesture on the touch screen display device, detecting the touch based selection gesture including detecting a finger touch area on the touch screen display device;

determining that there is ambiguity as to which of the plurality of graphical elements is being selected by calculating that the finger touch area covers at least a sub-plurality of the plurality of graphical elements;

in response to the determination, presenting a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area, each of the plurality of new corresponding graphical elements corresponding to one of the graphical elements in the at least a sub-plurality of graphical elements;

detecting a further touch based selection gesture on the touch screen display device, detecting the further touch based selection gesture including detecting a further finger touch area on the touch screen display device;

identifying an unambiguous intent to select one of the new corresponding graphical elements by calculating that the further finger touch area primarily covers the one of the new corresponding graphical elements from among the plurality of new corresponding graphical elements; and

in response identifying the unambiguous intent to select one of the new corresponding graphical elements;

selecting the corresponding graphical element from within the sub-plurality of graphical elements; and

performing an activity related to the corresponding graphical element.

2. The method of claim 1, wherein presenting a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprises presenting a touch optimized group of new corresponding graphical elements to assist with selection of one of the plurality of graphical elements.

3. The method of claim 1, wherein presenting a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprises presenting a group of new corresponding graphical elements in a semi-circular arrangement around the touch area.

4. The method claim 3, wherein detecting a finger touch selecting one of the new corresponding graphical elements comprises detecting a finger touch selecting one of the new corresponding graphical elements from the group of new corresponding graphical elements in the semi-circular arrangement.

5. The method of claim 3, wherein presenting a group of new corresponding graphical elements in a semi-circular arrangement around the touch area comprises presenting a further new graphical element representing additional graphical elements from within the sub-plurality of graphical elements, the further new graphical element visually indicating representation of the additional graphical elements from within the sub-plurality of graphical elements.

6. The method of claim 5, further comprising detecting a finger touch selecting the further new corresponding graphical element;

and

in response to detecting the finger touch selecting the further new corresponding graphical element, presenting additional new graphical elements in a list box arrangement in a region of the touch screen display device that is outside the touch area, the additional new graphical elements corresponding to other of the sub-plurality of graphical elements.

7. The method claim 3, further comprising detecting a swipe gesture on the semi-circular arrangement of graphical elements;

and

in response to the swipe gesture, rotating the semi-circular arrangement of graphical elements to expose other new graphical elements.

8. The method of claim 1, wherein presenting a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprises presenting a group of new corresponding graphical elements in a list box arrangement to a side of the touch area.

9. The method claim 8, wherein detecting a finger touch selecting one of the new corresponding graphical elements comprises detecting a finger touch selecting one of the new corresponding graphical elements from the group of new corresponding graphical elements in the list box arrangement.

10. The method of claim 1, wherein presenting a data visualization on the touch screen display device comprises presenting a data visualization selected from among:

a chart, a grid, one or more tiles, small multiples, a map, text, and a play axis.

11. The method of claim 1, wherein presenting a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprises presenting a new corresponding graphical element of a size large enough for unambiguous selection by a finger.

12. The method of claim 1, further comprising, in response to detecting the finger touch selecting one of the new corresponding graphical elements, removing any presented new corresponding graphical elements from the touch screen display device.

13. The method of claim 1, further comprising inferring that the finger is also obscuring the view of another portion of the touch screen display device outside of the touch area based on the orientation of the touch screen display device;

and

wherein presenting a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprises presenting a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is both outside the touch area and outside the inferred obscured portion of the touch screen display device.

14. The method of claim 1, further comprising:

prior to detecting a finger touch selecting one of the new corresponding graphical elements, detecting a finger touch highlighting the one of the new corresponding graphical elements; and

in response to highlighting the one of the new corresponding graphical elements, presenting a visual cue indicating the correspondence between the one of new corresponding graphic elements and the corresponding graphical element in the sub-plurality of graphical elements, the visual cue presented on the touch screen display device.

15. The method of claim 1, wherein presenting a plurality of new corresponding graphical elements on the touch screen display device comprises presenting a plurality of new corresponding graphical elements, each of the plurality of new corresponding graphical elements including additional details for a corresponding graphically element in the at least a sub-plurality of graphical elements.

16. A computer program product for use at computing device, the computing device including a touch screen display device, the computer program product for implementing a method for selecting a graphical element, the computer program product comprising one or more computer storage devices having stored thereon computer-executable instructions that, when executed at a processor, cause the computing device to perform the method, including the following

present a data visualization on the touch screen display device, the data visualization including a plurality of graphical elements, each graphical element representing a portion of data;

detect a touch based selection gesture on the touch screen display device, detecting the touch based selection gesture including detecting a finger touch area on the touch screen display device;

determine that there is ambiguity as to which of the plurality of graphical elements is being selected by calculating that the finger touch area covers at least a sub-plurality of the plurality of graphical elements;

in response to the determination, present a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area, each of the plurality of new corresponding graphical elements corresponding to one of the graphical elements in the at least a sub-plurality of graphical elements;

detect a further touch based selection gesture on the touch screen display device, detecting the further touch based selection gesture including detecting a further finger touch area on the touch screen display device;

identify an unambiguous intent to select one of the new corresponding graphical elements by calculating that the further finger touch area primarily covers the one of the new corresponding graphical elements from among the plurality of new corresponding graphical elements; and

in response identifying the unambiguous intent to select one of the new corresponding graphical elements;

select the corresponding graphical element from within the sub-plurality of graphical elements; and

perform an activity related to the corresponding graphical element.

17. The computer program product of claim 16, wherein computer-executable instructions that, when executed, cause the computing device to present a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprise computer-executable instructions that, when executed, cause the computing device to present a touch optimized group of new corresponding graphical elements to assist with selection of one of the plurality of graphical elements.

18. The computer program product of claim 16, wherein computer-executable instructions that, when executed, cause the computing device to present a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprise computer-executable instructions that, when executed, cause the computing device to present a radius control.

19. The computer program product of claim 16, wherein computer-executable instructions that, when executed, cause the computing device to present a plurality of new corresponding graphical elements on the touch screen display device in a region of the touch screen display device that is outside the touch area comprise computer-executable instructions that, when executed, cause the computing device to present a list box.

20. A computer system, the computer system comprising:

one or more processors;

system memory;

a touch screen display device; and

one or more computer storage devices having stored thereon computer-executable instructions representing rendering engine and a gesture recognition component, the rendering engine configured to;

present a data visualization on the touch screen display device by rendering a scene graph, the data visualization including a plurality of graphical elements, each graphical element representing a portion of data;

in response to a determination that there is ambiguity as to which of the plurality of graphical elements is being selected, present a plurality of new corresponding graphical elements on the touch screen display device by re-rendering the scene graph, the plurality of new corresponding graphical elements presented in a region of the touch screen display device that is outside the touch area, each of the plurality of new corresponding graphical elements corresponding to one of the graphical elements in the at least a sub-plurality of graphical elements; and

in response identifying an unambiguous intent to select one of the new corresponding graphical elements, visually indicate that the selected corresponding graphical element is selected by again re-rendering the scene graph; and

the gesture recognition component configured to;

detect a touch based selection gesture on the touch screen display device, detecting the touch based selection gesture including detecting a finger touch area on the touch screen display device;

determine that there is ambiguity as to which of the plurality of graphical elements is being selected by calculating that the finger touch area covers at least a sub-plurality of the plurality of graphical elements;

issue an event to the scene graph, the event triggering modification of the scene graph to include nodes for the plurality of new corresponding graphical elements;

detect a further touch based selection gesture on the touch screen display device, detecting the further touch based selection gesture including detecting a further finger touch area on the touch screen display device;

identify the unambiguous intent to select one of the new corresponding graphical elements by calculating that the further finger touch area primarily covers the one of the new corresponding graphical elements from among the plurality of new corresponding graphical elements; and

in response identifying the unambiguous intent to select one of the new corresponding graphical elements;

select the corresponding graphical element from within the sub-plurality of graphical elements; and

issue an further event to the scene graph, the further event triggering modification of a node corresponding to the selected graphical element to visually indicate that the selected corresponding graphical element is selected.