PROVIDING A GESTURE-BASED INTERFACE

• US 20140253431A1
• Filed: 11/12/2013
• Published: 09/11/2014
• Est. Priority Date: 03/08/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: GOSSWEILER, Richard Carl III, Mills, Michael I., Bharat, Krishna Asur, Dauber, Kenneth Wayne, Fontes, Paul L., Tam, Erwin S., Hoon, Vincent J.
• Granted Patent: US 9,519,351 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0485 Scrolling or panning

Abstract

Systems and methods for providing a gesture-based interface are provided. In some aspects, depth data indicative of a person interacting with one or more display devices is received. The one or more display devices display a plurality of columns. Each of the plurality of columns includes a plurality of icons. A column corresponding to a position of the person with respect to the one or more display devices is determined using the depth data. The column is from among the plurality of columns displayed at the one or more display devices. A signal for expanding a predetermined icon in the column corresponding to the position of the person with respect to the one or more display devices is provided.

Claims

1. A computer-implemented method for providing a gesture-based interface, the method comprising:

receiving depth data indicative of a person interacting with one or more display devices, wherein the one or more display devices display a plurality of columns, wherein each of the plurality of columns comprises a plurality of icons;

determining, using the depth data, a column corresponding to a position of the person with respect to the one or more display devices, the column being from among the plurality of columns displayed at the one or more display devices; and

providing a signal for expanding a predetermined icon in the column corresponding to the position of the person with respect to the one or more display devices.

2. The method of claim 1, further comprising:

providing a signal for decreasing a size of at least one icon in the column, wherein the at least one icon is different from the predetermined icon.

3. The method of claim 1, wherein the predetermined icon comprises a title of an article, and wherein expanding the predetermined icon comprises presenting text from the article.

4. The method of claim 1, wherein the predetermined icon comprises a thumbnail from a video, and wherein expanding the predetermined icon comprises playing at least a portion of the video.

5. The method of claim 1, wherein providing the signal for expanding the predetermined icon comprises:

firing an event corresponding to the person interacting with the one or more display devices and the column corresponding to the position of the person, wherein the predetermined icon is expanded in response to the fired event.

6. The method of claim 1, further comprising:

determining, based on the received depth data, that the person made a hand movement pointing a hand to an additional icon in the column corresponding to the position of the person with respect to the one or more display devices;

providing a signal for expanding the additional icon in response to the hand movement; and

providing a signal for decreasing a size of the predetermined icon in response to the hand movement.

7. The method of claim 1, further comprising:

determining, based on the received depth data, that the person made a hand movement pointing a hand to an additional icon in a second column different from the column corresponding to the position of the person with respect to the one or more display devices;

providing a signal for providing for display of an indication of selection of the additional icon;

determining, based on the received depth data, that the person made a hand movement for moving the additional icon into the column corresponding to the position of the person;

providing a signal for moving the additional icon into the column corresponding to the position of the person;

providing a signal for expanding the additional icon in the column corresponding to the position of the person; and

providing a signal for decreasing a size of the predetermined icon.

8. The method of claim 1, further comprising:

determining, based on the received depth data, that the person moved a hand across a spatial region corresponding to the expanded predetermined icon; and

providing a signal for scrolling in the expanded predetermined icon in response to the received indication of the moving of the hand by the person.

9. The method of claim 1, further comprising:

receiving an indication of a hand movement pulling a hand toward the person'"'"'s self; and

transmitting, to a mobile device, information related to an expanded icon in response to the received indication of the hand movement.

10. A non-transitory computer-readable medium for providing a gesture-based interface, the computer-readable medium comprising instructions which, when executed by one or more computers, cause the one or more computers to implement a method, the method comprising:

receiving depth data indicative of a first person and a second person interacting with one or more display devices, wherein the one or more display devices display a plurality of sets of icons;

determining, using the depth data, a first set of icons corresponding to a position of the first person with respect to the one or more display devices, the first set of icons being from among the plurality of sets of icons displayed at the one or more display devices;

determining, using the depth data, a second set of icons corresponding to a position of the second person with respect to the one or more display devices, the second set of icons being from among the plurality of sets of icons displayed at the one or more display devices;

providing a command for expanding a first predetermined icon in the first set of icons and for expanding a second predetermined icon in the second set of icons, while continuing to provide for display of the plurality of sets of icons.

11. The computer-readable medium of claim 10, wherein the first predetermined icon comprises a title of an article and a blurb of the article, and wherein expanding the first predetermined icon comprises presenting text from the article.

12. The computer-readable medium of claim 10, wherein the first predetermined icon comprises a thumbnail from a video, and wherein expanding the first predetermined icon comprises playing at least a portion of the video.

13. The computer-readable medium of claim 10, wherein providing the command for expanding the first predetermined icon comprises:

firing an event corresponding to the first person interacting with the one or more display devices and the first set corresponding to the position of the first person, wherein the first predetermined icon is expanded in response to the fired event.

14. The computer-readable medium of claim 10, the method further comprising:

determining, based on the received depth data, that the first person made a hand movement pointing a hand to an additional icon in the set corresponding to the position of the first person;

providing a command for expanding the additional icon in response to the hand movement; and

providing a command for decreasing a size of the first predetermined icon in response to the hand movement.

15. The computer-readable medium of claim 10, the method further comprising:

determining, based on the received depth data, that the first person made a hand movement for transferring the first predetermined icon to the second person;

providing a command for providing for display of an indication of the transferring within the second set of icons;

determining, based on the received depth data, that the second person made a hand movement for accepting the transferring of the first predetermined icon;

providing a command for copying the first predetermined icon into the second set;

providing a command for expanding the copy of the first predetermined icon in the second set; and

providing a command for decreasing a size of the second predetermined icon in the second set.

16. A system for providing a gesture-based interface, the system comprising:

one or more processors;

a memory comprising instructions which, when executed by the one or more processors, cause the one or more processors to implement a method, the method comprising;

receiving depth data indicative of a person interacting with one or more display devices, wherein the one or more display devices display a plurality of columns, wherein each of the plurality of columns comprises a plurality of icons;

determining, using the depth data, a column corresponding to a position of the person with respect to the one or more display devices, the column being from among the plurality of columns;

providing a signal for expanding a predetermined icon in the column corresponding to the position of the person with respect to the one or more display devices; and

providing a signal for decreasing a size of at least one other icon in the column corresponding to the position of the person with respect to the one or more display devices.

17. The system of claim 16, further comprising:

one or more depth cameras, wherein the depth data is received via the one or more depth cameras.

18. The system of claim 16, further comprising:

the one or more display devices.

19. The system of claim 16, wherein the depth data is received, via a network, from a remote computing device.

20. The system of claim 16, wherein the signal for expanding the predetermined icon is provided, via a network, to a remote computing device.