User interface controller method and apparatus for a handheld electronic device

• US 20060267927A1
• Filed: 05/27/2005
• Published: 11/30/2006
• Est. Priority Date: 05/27/2005
• Status: Abandoned Application

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Jelley, Kevin W., Augustine, Bruce A., Mareachen, Russell D., Crenshaw, James E., Thiems, Michael S.
• Application Number:
  US11/139,052
• Publication Number:
  US 20060267927A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A user interface controller of a handheld electronic device (100) that has a camera that generates video images presents (1005) information on a display (105) of the handheld electronic device, processes (1010) the video images to track a three dimensional position of a directing object (260) that is within a field of view (225) of the camera, generates (1015) a two dimensional position of the directing object that is used to control a corresponding location in a scene on the display, and controls (1020) a function of the handheld electronic device in response to a comparison of the track of the directing device to a virtual surface (805, 810, 920, 925) that is defined relative to the handheld electronic device.

Claims

1. A user interface controller of a handheld electronic device, comprising:

a display,

a camera that generates video images; and

a processing function coupled to the display and the camera, that presents information on the display,

processes the video images to generate a three dimensional track of a position of a directing object that is within a field of view of the camera,

generates a two dimensional position of the directing object that is used to control a corresponding location within a scene on the display, and

controls a function of the handheld electronic device in response to a comparison of the track of the directing object to a virtual surface that is defined relative to the handheld electronic device.

2. The user interface controller according to claim 1, further comprising generating an event when a result of the comparison is that the track traverses the virtual surface, while generating the two dimensional position essentially simultaneously.

3. The user interface controller according to claim 2, wherein a direction of the traverse is associated with the event.

4. The user interface controller according to claim 2, wherein the event is used as a virtual mouse button event.

5. The user interface controller of a handheld electronic device according to claim 1, wherein the display has a viewing area that is less than 100 square centimeters.

6. The user interface controller of a handheld electronic device according to claim 1, wherein the camera has a depth of field range of at least 10 centimeters under lighting conditions expected for normal use.

7. The user interface controller of a handheld electronic device according to claim 1, wherein an axis of the field of view of the camera is oriented in a direction essentially perpendicular to the display.

8. The user interface controller of a handheld electronic device according to claim 1, wherein an axis of the field of view is oriented in a direction biased away from an expected direction of an operator'"'"'s face.

9. The user interface controller of a handheld electronic device according to claim 1, wherein an axis of the field of view of the camera can be moved by an operator of the handheld electronic device.

10. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function that processes the video images to generate the track of the position of the directing object is responsive to images of one or more directing object markers that have one or more of the group of characteristics comprising:

each object marker image is a projection of a defined shape that includes at least one defined point location,

each object marker image is small in size in relation to the field of view,

each object marker image has a high brightness contrast ratio compared to the immediate surroundings, and

each object marker image primarily comprises light in a particular light band.

11. The user interface controller according to claim 1, wherein, for at least one of a plurality of regions, the function is controlled according to which of the at least one of the plurality of regions the directing object is within, and wherein the plurality of regions are defined by a plurality of virtual surfaces that are defined with reference to the handheld electronic device.

12. The user interface controller according to claim 11, wherein when the directing object is within one of the at least one of the plurality of regions, the two dimensional position of the directing object is used for identifying one of a plurality of mutually exclusive sectors of the one of the at least one of the plurality of regions instead of being used for controlling a corresponding location within a scene on the display, and wherein the function is controlled according to a sector and region the directing object is within, for at least one of the plurality of regions.

13. The user interface controller according to claim 11, wherein the plurality of virtual surfaces is two virtual surfaces that are located at two distances from the plane of the display, and wherein the two virtual surfaces define a near region, an intermediate region, and a far region.

14. The user interface controller according to claim 13, wherein the near and far regions are used to control an increase and decrease in value of a function of the handheld electronic device.

15. The user interface controller according to claim 14, wherein the function includes one more of zoom control, scroll control, audio volume control, and channel selection.

16. A user interface method used in a handheld electronic device that has a camera that generates video images and has a display, comprising:

presenting information on the display;

processing the video images to generate a three dimensional track of a position of a directing object that is within a field of view of the camera;

generating a two dimensional position of the directing object that is used to control a corresponding location in a scene on the display; and

controlling a function of the handheld electronic device in response to a comparison of the track of the directing device to a virtual surface that is defined relative to the handheld electronic device.