Mouse with optical sensing surface

• US 20070152966A1
• Filed: 03/13/2006
• Published: 07/05/2007
• Est. Priority Date: 12/30/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Hotelling, Steve, Krah, Christoph
• Granted Patent: US 8,077,147 B2
• Time in Patent Office: Days
• US Class Current: 345/163
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen

Abstract

A mouse with an extended optical sensing surface is disclosed.

Claims

1. A configurable mouse with an extended sensing surface, which provides the mouse a customizable, programmable or adaptable way of generating inputs, the mouse including an arbitrarily shaped grippable member having a 3D shape, a position detection mechanism that detects the movement of the mouse along a surface, a multipoint touch detection mechanism that detects one or more objects that are contacting or in close proximity to a substantially large portion of the grippable member, and a controller for processing signals generated by the position detection mechanism and multipoint touch detection mechanism.

2. The configurable mouse as recited in claim 1 wherein the grippable member includes no fixed input areas.

3. The configurable mouse as recited in claim 1 wherein the grippable member is formed from a single member with a full continuous surface, which defines the front, sides, back and top surfaces of the mouse.

4. The configurable mouse as recited in claim 1 wherein the controller working in conjunction with the touch detection mechanism generates touch data based on the position and motion of the objects contacting or in close proximity to the substantially large portion of the grippable member, and wherein the touch data is converted to commands to be used by a host device operatively coupled to the mouse.

5. The configurable mouse as recited in claim 4 wherein a command generator identifies one or more hand positions or motions from the touch data and generates a command based on the identified hand position or hand motion.

6. The configurable mouse as recited in claim 1 wherein the grippable member is formed from an optically transmissive material, and wherein the multipoint touch detection mechanism is configured to optically image the substantially large portion of the grippable member, the multipoint touch detection mechanism including a light arrangement and an imaging arrangement that communicate with the controller, which directs the light arrangement to emit light into the grippable portion and the image arrangement to capture images of the substantially large portion of the grippable portion and any object interacting therewith in a controlled manner.

7. A method of operating a mouse coupled to a host device such as a personal computer, the method comprising:

obtaining images of a hand as it interacts with any portion of an arbitrarily shaped mouse housing;

obtaining touch patterns for each image;

extracting control information from the resulting touch patterns; and

controlling some aspect of a host device based on the control information.

8. The method as recited in claim 7 wherein obtaining images includes:

illuminating the arbitrarily shaped mouse housing formed from an optically transmissive material; and

optically imaging the arbitrarily shaped mouse housing when it is illuminated.

9. The method as recited in claim 7 wherein obtaining touch patterns includes:

correcting each of the images; and

locating distinct elements within each of the images.

10. The method as recited in claim 7 wherein extracting control information includes:

identifying discrete touches within the touch patterns;

assigning a function to touches and touch events associated with the touches;

monitoring each touch to determine if a touch event has been performed; and

generating control information based on the assigned function and the touches and touch events.

11. The method as recited in claim 7 wherein monitoring each touch includes:

obtaining a first set of touch characteristics for a particular touch;

obtaining a second set of touch characteristics for the particular touch;

comparing the first and second set of touch characteristics to determine if a touch event has been performed; and

obtaining characteristics associated with the touch event when a touch event has been performed.

12. A computer mouse, comprising

a mouse housing for enclosing mouse electronics and including a top shell that is connected to a base, the base being embodied as a flat member capable of traveling across a flat surface, the top shell being embodied as a single 3D shaped member that fits inside a user'"'"'s hand and that is optically transmissive, the base defining the bottom surface of the mouse, the top shell defining the top, front, back and side surfaces of the mouse;

and

a mouse tracking unit for tracking movements of the mouse along a flat surface;

an optical image system for imaging a substantially large portion of the 3D shaped top shell, the optical imaging system including an illumination source for illuminating the 3D shaped top shell, an image sensor arrangement for imaging the 3D shaped top shell when the 3D shaped top shell is illuminated, and a wide angle lens assembly that allows the substantially large portion of the 3D shaped top shell to be imaged by the image sensor arrangement, the illumination source including one or more light sources, the image sensor arrangement including one or more image sensors;

a controller operatively coupled to the mouse tracking unit and optical imaging system, and configured to control the operations of the mouse tracking unit and the optical imaging system, the controller including a mouse tracking module associated with the mouse tracking unit and an optical sensing module associated with the optical imaging system.

13. The mouse as recited in claim 12 wherein the optical image system images the entire top shell.

14. The mouse as recited in claim 12 wherein the illumination also colorizes the top shell.

15. The mouse as recited in claim 12 wherein the image sensor arrangement includes a single image sensor.

16. The mouse as recited in claim 12 wherein the image sensor is a VGA CMOS image sensor.

17. The mouse as recited in claim 12 wherein the controller is a digital signal processor.

18. The mouse as recited in claim 12 wherein the controller converts the images produced from the optical imaging system to touch data and sends the touch data to a host device.

19. The mouse as recited in claim 18 wherein input commands are extracted from the touch data.

20. The mouse as recited in claim 12 wherein the controller is configured to convert warped 3D images to flat space.

21. The mouse as recited in claim 12 wherein the controller is configured segment the image into discrete points.

22. The mouse as recited in claim 12 further comprising:

a feedback system operatively coupled to the controller, and configured to provide feedback to the user of the mouse so that the user is able to positively confirm that a user action has resulted in an input, the feedback system including one or more feedback generators selected from audio feedback device and haptics devices.

23. The mouse as recited in claim 22 wherein the top shell and base are interconnected by means of a haptics device.

24. The mouse as recited in claim 12 wherein the light sources illuminate an inner surface of the top shell.

25. The mouse as recited in claim 12 wherein the light sources illuminate an edge of the top shell.

26. The mouse as recited in claim 12 wherein the light sources illuminate an inner surface of the top shell and an edge of the top shell.

27. The mouse as recited in claim 12 wherein the mouse further includes a light diffuser configured to diffuse the light being emitted by the light sources.

28. A method of inputting from a mouse into a host device, comprising:

at the mouse, imaging the entire or substantially large portion of a 3D shaped top shell of a mouse;

at the mouse, mapping the 3D image to flat space;

at the mouse, segmenting the flat space image into one or more features, each of which represents a discrete touch on the surface of the top shell;

at the mouse, calculating contact patch parameters for each of the features; and

at the mouse, sending the contact patch parameters to the host device so that actions can be performed base don the contact patch parameters.

29. The method as recited in claim 28 wherein the contact patch parameters include identifiers, coordinates and area.

30. The method as recited in claim 28 further comprising:

at the host, extracting command data from the contact patch parameters.

31. The method as recited in claim 30 the command data being selected from buttoning, tracking, scrolling, rotating, paging, and sizing.

32. A mouse, comprising:

a housing forming a grippable surface and a sliding surface;

a mouse tracking unit built into the sliding surface; and

an input mechanism built into the grippable surface, the input mechanism including a touch sensitive protruding member that is supported by a force feedback unit that is attached to the housing, the force feedback unit producing force feedback at the touch sensitive protruding member in response to inputs generated at the touch sensitive surface.

33. The mouse as recited in claim 32 wherein the force feedback unit includes a housing made of ferro-magnetic material and a voice coil, the housing is suspended within the voice coil via a tension spring, the voice coil is operatively coupled to a controller that is capable of sending commands to the voice coil.

34. The mouse as recited in claim 33 wherein the touch sensitive protruding member is based on optical imaging.

35. The mouse as recited in claim 34 wherein the touch sensitive protruding member is a lens assembly that extends above the top shell and includes a rounded dome like shape, the lens assembly is in optical communication with a light source and an image sensor, both of which are operatively coupled to the controller, the controller is capable of sending commands to the light source and image sensor as well as receiving data from the image sensor.