Foot gestures for computer input and interface control

• US 9,002,680 B2
• Filed: 03/18/2011
• Issued: 04/07/2015
• Est. Priority Date: 06/13/2008
• Status: Active Grant

• Current Assignee: Nike, Inc.
• Original Assignee: Nike, Inc.
• Inventors: Nurse, Matthew A., Meschter, James C., Pisciotta, Jeffrey C., Schrock, Allan M., Rauchholz, William F.
• Primary Examiner(s): Huynh, Phuong
• Application Number:
  US13/051,859
• Publication Number:
  US 20110199393A1
• Time in Patent Office: 1,481 Days
• Field of Search: 702/188, 702/41, 702/139, 702/160, 702/182, 702/187
• US Class Current: 702/188
• CPC Class Codes: A43B 3/00 Footwear characterised by the shape or the use
• CPC Class Codes: A43B 3/16 Overshoes
• CPC Class Codes: A43B 3/34 with electrical or electronic arrangements
• CPC Class Codes: A43B 3/50 with sound or music sources
• CPC Class Codes: A63B 2069/0006 for pitching
• CPC Class Codes: A63B 2071/0638 Displaying moving images of recorded environment, e.g. virtual environment
• CPC Class Codes: A63B 2102/02 Tennis
• CPC Class Codes: A63B 2102/18 Baseball, rounders or similar games
• CPC Class Codes: A63B 2102/22 Field hockey
• CPC Class Codes: A63B 2102/32 Golf
• CPC Class Codes: A63B 2220/12 Absolute positions, e.g. by using GPS
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/53 Force of an impact, e.g. blow or punch
• CPC Class Codes: A63B 2220/836 Sensors arranged on the body of the user
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 2243/0025 Football American football A63B2243/007
• CPC Class Codes: A63B 24/0062 Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: A63B 26/003 for improving balance or equilibrium balance beams A63B4/00; apparatus for rope dancing A63B7/08; balancing drums, balancing boards or the like A63B22/16
• CPC Class Codes: A63B 69/00 Training appliances or apparatus for special sports training of parachutists B64D23/00
• CPC Class Codes: A63B 69/0028 for running, jogging or speed-walking movable endless bands A63B22/02
• CPC Class Codes: A63F 2300/1012 : involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1031 : using a wireless connection, e.g. Bluetooth, infrared connections
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 : using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0334 : Foot operated pointing devices
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 : Scrolling or panning

Abstract

Non-video game computer user interfaces utilize human foot action to generate input for controlling the interface (e.g., a graphical user interface). Computer systems that use such input may include: (a) at least one input system including one or more sensing devices for sensing action by one or more feet; (b) a data transmission system for transmitting the sensed foot action to (c) a processing system that utilizes the foot action input to provide control data input for a user interface; and (d) an output device for producing a non-video game computer output based, at least in part, on data provided by the processing system. The control data relating to the foot action is used, at least in part, for controlling a user interface associated with the computer output.

Claims

1. A computer system, comprising:

an input system including a first sensing device for sensing foot action, wherein the first sensing device is engaged with or formed as part of an article of footwear, and wherein the sensed foot action includes at least one of;

foot movement, contact force between a foot and a contact surface, and contact between a foot and a contact surface;

a data transmission system for transmitting data relating to the sensed foot action;

a processing system for receiving the data transmitted by the data transmission system, wherein the processing system utilizes the data transmitted by the data transmission system, at least in part, to provide control data based, at least in part, on the sensed foot action; and

an output device for producing a non-video game computer output based, at least in part, on data provided by the processing system, wherein the control data relating to the sensed foot action is used, at least in part, for controlling a user interface associated with the computer output.

2. A computer system according to claim 1, wherein the data transmission system includes a wireless transmission system.

3. A computer system according to claim 1, wherein the data transmission system is engaged with or formed as part of an article of footwear.

4. A computer system according to claim 1, wherein the first sensing device senses contact of a first portion of a foot with a contact surface, and wherein the article of footwear further includes a second sensing device engaged with or formed as part of the article of footwear, wherein the second sensing device senses contact of a second portion of the foot with the contact surface.

5. A computer system according to claim 1, wherein the sensed foot action includes at least one member selected from the group consisting of:

a foot tap motion, a foot double tap motion, a foot press-and-hold, a heel tap motion, a toe tap motion, a lateral foot side tap motion, a medial foot side tap motion, a heel double tap motion, a toe double tap motion, a lateral foot side double tap motion, a medial foot side double tap motion, a heel press-and-hold, a toe press-and-hold, a lateral foot side press-and-hold, a medial foot side press-and-hold, a tap force, a press-and-hold press force, foot motion while in contact with a contact surface, heel motion while in contact with a contact surface, toe motion while in contact with a contact surface, lateral foot side motion while in contact with a contact surface, medial foot side motion while in contact with a contact surface, toe contact force, heel contact force, lateral foot side contact force, medial foot side contact force, a foot shaking motion, a direction of foot motion, a foot motion speed, a change in foot motion direction, a change in foot motion speed, a change in foot press force, a change in location of foot contact force, relative motion of two feet, relative motion direction of two feet, force differential between contact force of two feet, force differential between contact force of two consecutive foot contacts, and a foot hover action.

6. A computer system according to claim 1, wherein the data relating to the sensed foot action is used to provide the control data for controlling the user interface to perform at least one of the actions selected from the group consisting of:

an item select action;

a cursor positioning action;

a highlight action;

a switching action between open application programs;

a switching action between operating modes within a single application program;

launching a new application or program;

volume up;

volume down;

channel up;

channel down;

zoom in;

zoom out;

changing a view on a display device by moving upward;

changing a view on a display device by moving downward;

changing a view on a display device by moving leftward;

changing a view on a display device by moving rightward;

page up;

page down;

moving around on content displayed on a display device;

activating a text input panel;

moving locations on a listing of items;

incrementing a counter;

skipping a currently playing item;

replaying a currently playing item from its beginning;

chapter up;

chapter down;

activating fast forward of audio or video;

activating rewind of audio or video;

activating play of audio or video;

pausing playback of audio or video;

moving to a checkout sequence;

advancing to a top of a list or to a beginning of a display;

moving to a bottom of a list or to an end of a display;

draw a line or curve;

exiting an application program;

closing an individual document or other window element;

changing font properties;

changing line properties;

changing window sizes;

operating virtual switches, knobs, sliders, handles, doors, drawers, buttons, ON/OFF switches, and radio buttons;

selecting or changing ringtones;

initiating a telephone call;

ending a telephone call;

dialing a telephone call;

initiating a conference call; and

dropping one or more parties from a conference call.

7. A computer system according to claim 1, wherein the user interface controlled, at least in part, by the control data relating to the sensed foot action includes a graphical user interface.

8. A computer system according to claim 7, wherein the foot action sensed by the first sensing device includes moving the foot in a front-to-rear direction, and wherein the control data relating to this sensed foot action is used to scroll upward in displayed content.

9. A computer system according to claim 7, wherein the foot action sensed by the first sensing device includes moving the foot in a rear-to-front direction, and wherein the control data relating to this sensed foot action is used to scroll downward in displayed content.

10. A computer system according to claim 7, wherein the foot action sensed by the first sensing device includes moving the foot in a right-to-left direction, and wherein the control data relating to this sensed foot action is used to scroll rightward in displayed content.

11. A computer system according to claim 7, wherein the foot action sensed by the first sensing device includes moving the foot in a left-to-right direction, and wherein the control data relating to this sensed foot action is used to scroll leftward in displayed content.

12. A computer system according to claim 7, wherein the foot action sensed by the first sensing device includes a foot tap motion, and wherein the control data relating to this sensed foot action is used to make a selection action from information displayed by the output device.

13. A computer system according to claim 7, wherein the foot action sensed by the first sensing device includes a foot tap motion, and wherein the control data relating to this sensed foot action is used to position a cursor at a location of a displayed pointing element with respect to displayed content at the time of the foot tap motion.

14. A computer system, comprising:

a first input system including a first sensing system for sensing action of a first foot, wherein the first sensing system is engaged with or formed as part of a first shoe of a pair of shoes, and wherein the action includes at least one of;

first foot movement, contact force between the first foot and a contact surface, and contact between the first foot and a contact surface;

a second input system including a second sensing system for sensing action of a second foot, wherein the second sensing system is engaged with or formed as part of a second shoe of the pair of shoes, and wherein the action includes at least one of;

second foot movement, contact force between the second foot and a contact surface, and contact between the second foot and a contact surface;

a data transmission system for transmitting data relating to the sensed action by the first foot and the second foot;

a processing system for receiving the data transmitted by the data transmission system, wherein the processing system utilizes the data transmitted by the data transmission system, at least in part, to provide control data based, at least in part, on the sensed actions of the first foot and the second foot; and

an output device for producing a non-video game computer output based, at least in part, on data provided by the processing system, wherein the control data is used, at least in part, to control a user interface associated with the computer output.

15. A computer system according to claim 14, wherein the data transmission system includes a first data transmission device for transmitting data relating to the sensed action by the first foot and a second data transmission device for transmitting data relating to the sensed action by the second foot.

16. A computer system according to claim 15, wherein the first data transmission device is engaged with the first shoe of the pair of shoes and the second data transmission device is engaged with the second shoe of the pair of shoes.

17. A computer system according to claim 14, wherein the data transmission system includes at least one wireless transmission system.

18. A computer system according to claim 14, wherein the actions sensed by the first and second input systems include at least one member selected from the group consisting of:

a foot tap motion, a foot double tap motion, a foot press-and-hold, a heel tap motion, a toe tap motion, a lateral foot side tap motion, a medial foot side tap motion, a heel double tap motion, a toe double tap motion, a lateral foot side double tap motion, a medial foot side double tap motion, a heel press-and-hold, a toe press-and-hold, a lateral foot side press-and-hold, a medial foot side press-and-hold, a tap force, a press-and-hold press force, foot motion while in contact with a contact surface, heel motion while in contact with a contact surface, toe motion while in contact with a contact surface, lateral foot side motion while in contact with a contact surface, medial foot side motion while in contact with a contact surface, toe contact force, heel contact force, lateral foot side contact force, medial foot side contact force, a foot shaking motion, a direction of foot motion, a foot motion speed, a change in foot motion direction, a change in foot motion speed, a change in foot press force, a change in location of foot contact force, relative motion of two feet, relative motion direction of two feet, force differential between contact force of two feet, force differential between contact force of two consecutive foot contacts, and a foot hover action.

19. A computer system according to claim 14, wherein the data relating to the sensed actions by the first foot and the second foot is used to provide or produce the control data for controlling the user interface to perform at least one of the actions selected from the group consisting of:

an item select action;

a cursor positioning action;

a highlight action;

a switching action between open application programs;

a switching action between operating modes within a single application program;

launching a new application or program;

volume up;

volume down;

channel up;

channel down;

zoom in;

zoom out;

changing a view on a display device by moving upward;

changing a view on a display device by moving downward;

changing a view on a display device by moving leftward;

changing a view on a display device by moving rightward;

page up;

page down;

moving around on content displayed on a display device;

activating a text input panel;

moving locations on a listing of items;

incrementing a counter;

skipping a currently playing item;

replaying a currently playing item from its beginning;

chapter up;

chapter down;

activating fast forward of audio or video;

activating rewind of audio or video;

activating play of audio or video;

pausing playback of audio or video;

moving to a checkout sequence;

advancing to a top of a list or to a beginning of a display;

moving to a bottom of a list or to an end of a display;

draw a line or curve;

exiting an application program;

closing an individual document or other window element;

changing font properties;

changing line properties;

changing window sizes;

operating virtual switches, knobs, sliders, handles, doors, drawers, buttons, ON/OFF switches, and radio buttons;

selecting or changing ringtones;

initiating a telephone call;

ending a telephone call;

dialing a telephone call;

initiating a conference call; and

dropping one or more parties from a conference call.

20. A computer system according to claim 14, wherein the user interface controlled, at least in part, by the control data relating to the sensed action of the first foot and the second foot includes a graphical user interface.

21. A computer system according to claim 20, wherein the action sensed includes a foot tap motion by at least one of the first foot or the second foot, and wherein the control data relating to this sensed foot action is used to make a selection action from information displayed by the output device.

22. A computer system according to claim 20, wherein the action sensed includes an indication of the first foot and the second foot moving apart from one another, and wherein the control data relating to this sensed action is used to enlarge a view of an object displayed on a display device.

23. A computer system according to claim 20, wherein the action sensed includes an indication of the first foot and the second foot moving toward one another, and wherein the control data relating to this sensed action is used to shrink a view of an object displayed on a display device.

24. A computer system according to claim 20, wherein combination or simultaneous actions using both the first foot and the second foot provide control data for controlling the graphical user interface.

25. A computer system according to claim 14, wherein the first input system is engaged with or formed as part of a first shoe of a pair of shoes and senses contact of different regions of the first foot with a contact surface, and wherein the second input system is engaged with or formed as part of a second shoe of the pair and senses contact of different regions of the second foot with a contact surface.

26. A method of controlling a non-video game computer user interface, comprising:

receiving input data indicating sensed foot action from one or more sensors engaged with or formed as part of an article of footwear, wherein the sensed foot action includes at least one of;

foot movement, contact force between a foot and a contact surface, and contact between a foot and a contact surface;

processing the data indicating the sensed foot action to provide control data based, at least in part, on the sensed foot action; and

generating output data for controlling a non-video game computer user interface displayed by an output device, wherein the output data includes the control data.

27. A non-transitory computer readable medium containing computer executable instructions stored thereon for causing a computer processor to perform a method of controlling a non-video game computer user interface comprising:

receiving input data indicating sensed foot action from one or more sensors engaged with or formed as part of an article of footwear, wherein the sensed foot action includes at least one of;

foot movement, contact force between a foot and a contact surface and contact between a foot and a contact surface;

processing the data indicating foot action to provide control data based at least in part, on the sensed foot action; and

generating output data for controlling a non-video game computer user interface displayed by an output device, wherein the output data includes the control data.