HUMAN-COMPUTER USER INTERACTION

• US 20100261526A1
• Filed: 05/19/2010
• Published: 10/14/2010
• Est. Priority Date: 05/13/2005
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Aviles, Richard, Aviles, Walter, Jones, Jake, Anderson, Bill, Anderson, Thomas G., Grafe, V. Gerald
• Granted Patent: US 9,804,672 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Methods of and apparatuses for providing human interaction with a computer, including human control of three dimensional input devices, force feedback, and force input.

Claims

1. A method of providing interaction between a human and a computer, comprising causing the computer to perform an action responsive to manipulation by the human of an input device, wherein manipulation comprises motion of the input device in at least two dimensions, or application of force to the input device in at least two dimensions, or a combination thereof.

2. A method as in claim 1, wherein the human is to control an actor in a virtual environment implemented in the computer, wherein the human provides input commands to the computer, and wherein occurrences in the virtual environment affect the status of the actor, and wherein the human action required to supply an input command varies responsive to the status of the actor.

3. A method as in claim 2, wherein the status of the actor corresponds to a measure or damage, health, or fatigue of the actor.

4. A method as in claim 1, further comprising providing a sound feedback from the computer to the user representative of the motion of the input device and of a desired effect such as the squeak of a rusty lever.

5. A method as in claim 1, wherein the human controls the position of a cursor within a virtual environment implemented in the computer, comprising determining when the cursor is positioned within a threshold distance of a surface within the virtual environment, and then communicating a force to the user control of the cursor that encourages the cursor to contact the surface.

6. A method as in claim 1, wherein the human controls the position of a cursor within a virtual environment implemented in the computer, comprising defining a point in the virtual environment and applying a force to the user control of the cursor that encourages the cursor to the point, wherein the force is greater at a first distance from the point than at a second distance from the point, wherein the first distance is greater than the second distance.

7. A method as in claim 1, wherein the human experiences the effects of a collision in the virtual environment, comprising communicating a force to the user opposite the direction of motion initiating the collision for a first time period, then communicating a time-varying force to the user for a second time.

8. A method as in claim 1, wherein the human controls an input device that is configurable throughout a range of motion, comprising providing to the user a visible indication of a target region of a displayable region such that, when the user configures the input device to position a cursor within the target region, the input device will be configured to be at a starting region of its range of motion such that subsequent configuration of the input device will be unlikely to cause the input device to reach a limit of its range of motion.

9. A method as in claim 1, wherein the human controls an input device that is configurable throughout a range of motion, comprising communicating a force to the human that encourages the input device to a preferred region of the range of motion.

10. A method as in claim 9, wherein the force comprises a plurality of limited duration forces directing the input device toward the preferred region, applied while the input device is not within the preferred region.

11. A method as in claim 1, wherein the human controls the position of an input device whose position, motion, or a combination thereof, corresponds to the position, motion, or a combination thereof, of a cursor in a virtual space implemented in a computer, comprising defining a region within the virtual environment surrounding the position of the cursor, and determining when the user controls the input device to cause the cursor to encounter one or more surfaces of the region in a first predetermined manner and then initiating a first action of the computer responsive thereto.

12. A method as in claim 11, further comprising determining when the user controls the input device to cause the cursor to encounter one or more surfaces of the region in a second predetermined manner and then initiating a second action of the computer responsive thereto.

13. A method as in claim 11, wherein the first action is determined part from one or more of:

time taken by the user to complete the first predetermined manner, speed at which the user completes the first predetermined manner, positioning of the input device when the user began the first predetermined manner, size of the pattern of motion of the input device, orientation of the input device during motion, idle time during the user motion of the input device, force applied by the user to the input device.

14. A method as in claim 1, comprising determining if the user motion of the input device conforms to a predetermined pattern, and initiating an action on the computer if it does.

15. A method as in claim 14, wherein the action is determined in part from the degree of conformance of the motion to the predetermined pattern.

16. A method as in claim 14, wherein the action is determined in part from one or more of:

the speed of the motion, the size of the pattern, the position of a cursor in a virtual environment implemented on a computer when the pattern was initiated, the position of a cursor in a virtual environment implemented on a computer when the pattern was completed.

17. A method as in claim 14, further comprising communicating forces to the user corresponding to the pattern, the action, or a combination thereof.

18. A method as in claim 14, further comprising communicating a visual image to the user corresponding to the pattern, the action, or a combination thereof.

19. A method as in claim 14, further comprising communicating forces to the user that facilitate or resist completion of the pattern.

20. A method as in claim 1, comprising defining a plurality of input device patterns and actions, and allowing the user to initiate an action according to the method of claim 14, wherein the number of patterns in the plurality is determined from a measure of the human'"'"'s past achievements in the virtual environment.

21. A method as in claim 1, comprising accepting from the human a definition of a pattern and a corresponding computer action, and allowing interaction according to the pattern and the action according to claim 14.

22. A method as in claim 1, wherein the human controls a character in a virtual environment implemented on the computer, wherein the balance of the character is controlled by the human, comprising providing a user object whose position in the virtual environment is controlled at least in part by the user, and providing a balance object in the virtual environment, wherein the balance of the character in the virtual environment is determined from the relative positions of the user object and the balance object.

23. A method as in claim 22, wherein the balance of the character is determined from the relative position of the user object and the balance object in a first dimension but not in a second dimension.

24. A method as in claim 1, wherein the human controls a character in a virtual environment, and wherein the character can encounter collisions in the virtual environment, and wherein, after the character encounters a collision in the virtual environment, the human control of the character is made more difficult for at least a limited time thereafter, wherein “

more difficult”

comprises at least one of slower computer response to human input, resistive forces applied to a force-capable input device opposing human motion of the device, or scaling of the effort or motion required by the human to cause the computer to initiate actions.

25. A method as in claim 1, wherein the human controls the catching of an object in a virtual environment implemented in the computer, wherein the human control of the input device corresponds to a graphical representation of an apparatus that can catch the object, and wherein intersection of the patch of the object with a catching region of the virtual environment causes the computer to implement the virtual environment as though the object had been caught, and wherein the catching region is not completely coincident with the graphical representation of the apparatus.

26. A method as in claim 1, wherein the human controls a character in a virtual environment implemented in the computer, and wherein the human controls the motion of the input device, and wherein each of a plurality of cycles of repetitive motion of the input device causes the computer to implement one cycle of a climbing action of the character.

27. A method as in claim 1, wherein the input device is capable of transmitting forces to the human, and wherein the human experiences the actions of a character in a virtual environment implemented in the computer, and wherein, when the character is flipping in the virtual environment, the computer uses the input device to transmit to the human forces representative of the forces that would be in effect on the character in the virtual environment.

28. A method as in claim 1, wherein the human controls a character in a virtual environment implemented in the computer, and wherein the human controls the motion of the input device, and wherein each of a plurality of cycles of repetitive motion of the input device causes the computer to implement one cycle of a running, walking, pedaling, flying, or swimming action of the character.

29. A method as in claim 1, wherein the human controls a character in a virtual environment implemented in the computer, and wherein the human controls force applied to the input device, and wherein each of a plurality of cycles of repetitive force profiles applied to the input device causes the computer to implement one cycle of a running, walking, pedaling, or swimming action of the character.

30. A method as in claim 1, wherein the human controls a character in a virtual environment implemented in the computer, and wherein the computer displays little or no information about virtual environment in a region near the character, and wherein the computer communicates forces to the human representative of the character'"'"'s proximity to or collision with objects or surfaces in the virtual environment.

31. A method as in claim 1, wherein the human controls a character in a virtual environment implemented in the computer, and wherein the computer displays little or no information about virtual environment in a region near the character, and wherein the computer communicates forces to the human representative of a desired path for the character in the virtual environment to avoid collision with objects or surfaces in the virtual environment.

32. A method as in claim 1, wherein the human controls a gun in a virtual environment implemented in the computer, wherein the computer communicates force to the user corresponding to the firing of the gun, wherein the force is different for different types of guns.

33. A method as in claim 1, wherein the human controls an archer apparatus in a virtual environment implemented in the computer, wherein the human control of the input device controls the aiming of the archery apparatus in the virtual environment.

34. A method as in claim 14, wherein human controls a character in a virtual environment implemented in the computer, wherein the character is capable of casting a spell in the virtual environment, and wherein the action initiated is a spell.

35. A method as in claim 1, wherein the human controls the throwing of an object in a virtual environment implemented in the computer, and wherein the human controls at least one of the direction, the trajectory, or the velocity of a thrown object in the virtual environment by the human control of the input device.

36. A method as in claim 35, wherein the computer communicates forces to the user corresponding to at least one of the trajectory, the velocity, or the acceleration of the thrown object.

37. A method as in claim 1, wherein the computer implements a virtual environment, and wherein the human can control objects in the virtual environments, including by holding an object, and wherein the computer communicates forces to the human corresponding to at least one of the mass of a held object, the weight of a held object, the inertia of a held object, or the interaction of a held object with the virtual environment.

38. A method as in claim 1, wherein the human controls a character in a virtual environment implemented in the computer, wherein the character can bite or chew objects in the virtual environment, and wherein the computer communicates forces to the human corresponding to biting or chewing actions of the character.

39. A method as in claim 1, wherein the human controls a character in a virtual environment implemented in the computer, wherein the character can be treated as breathing in the virtual environment at a variable rate, intensity, or both, and wherein the computer communicates forces to the human corresponding to breathing of the character.

40. A method as in claim 1, wherein the input device provides to the computer an indication of the magnitude of force exerted by the human on the input device at a plurality of locations in the real world.

41. A method as in claim 40, wherein the human controls a character in a virtual environment implemented in the computer, and wherein the computer causes the character to walk or run in the virtual environment responsive to the human exerting force on the input device at one location and then at another location in the real word.

42. A method as in claim 1, comprising providing an input device moveable in three dimensions, providing a computer implementing a virtual boxing environment, wherein the computer communicates wirelessly with the input device, and causing the computer to initiate a punching action in the virtual environment responsive to motion of the input device having predefined characteristics.

43. A method as in claim 42, wherein the computer can implement a plurality of punching actions, and wherein the punching action to initiate is chosen responsive to characteristics of the motion of the input device.

44. A method as in claim 42, further comprising communicating a vibration to the human using the input device responsive to a hit by the punching action in the virtual environment.

45. A method as in claim 1, comprising providing an input device moveable in three dimensions, providing a computer implementing a virtual environment, wherein the computer communicates wirelessly with the input device, and causing the computer to initiate a throwing action in the virtual environment responsive to motion of the input device having predefined characteristics.

46. A method as in claim 1, comprising providing an input device moveable in three dimensions, providing a computer implementing a virtual environment, wherein the computer communicates wirelessly with the input device, and causing the computer to initiate a swinging action of a racket in the virtual environment responsive to motion of the input device having predefined characteristics.

47. A method as in claim 42, further comprising communicating a vibration to the human using the input device responsive to a hit by the swinging action of the racket in the virtual environment.

48. A method of allowing a user to control a character in a multidimensional computer environment, comprising:

a. Determining the user'"'"'s movement of a user-specified reference point by observation of the movements of an input device controlled by the user;

b. Providing a predetermined set of acceptable movements largely along a primary axis of movement in the multidimensional environment, each of which is mapped to a desired particular software event;

c. Comparing the user'"'"'s movement to the predetermined set of acceptable movements and selecting the best match from among the set;

d. Performing a corresponding particular software event based on the selection;

e. Varying the resultant Character'"'"'s actions by measuring variations in movement along the non-primary degrees of freedom.

49. A method of controlling a software simulation by user input in at least three dimensions, comprising

a. Determining at least the start and end point of a user'"'"'s movement;

b. Computing parameters of a likely movement from the determined start and end points;

c. Using the computed parameters or likely movement to trigger a software event in the software simulation.