User interface controller method and apparatus for a handheld electronic device

• US 20060036947A1
• Filed: 08/10/2004
• Published: 02/16/2006
• Est. Priority Date: 08/10/2004
• Status: Abandoned Application

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Crenshaw, James E., Jelley, Kevin W., Thiems, Michael Stephen
• Application Number:
  US10/916,384
• Publication Number:
  US 20060036947A1
• Time in Patent Office: Days
• US Class Current: 715/722
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A user interface controller of a handheld electronic device (100) that has a camera that generates video images presents (1105) information on a display (105) of the handheld electronic device, processes (1110) the video images to track at least one of a position and orientation of a directing object (260) that is within a field of view (225) of the camera, and modifies (1115) at least one scene presented on the display in response to a track of the directing object. Modification of scenes may include selecting one or more scene objects, moving a cursor object, and adjusting a viewing angle of successive scenes.

Claims

1. A user interface controller of a handheld electronic device, comprising:

a display,

a first camera that generates video images; and

a processing function coupled to the display and the first camera, that

presents information on the display,

processes the video images to track at least one of a position and an orientation of a directing object that is within a field of view of the first camera, and

modifies at least one scene presented on the display in response to a track of the directing object.

2. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function that modifies the at least one scene, performs at least one of the functions of

moving a cursor object within one or more successive scenes on the display in response to a track of the directing object;

selecting one or more scene objects within one or more successive scenes on the display in response to a track of the directing object; and

adjusting a viewing perspective of successive scenes on the display in response to a track of the directing object.

3. The user interface controller of a handheld electronic device according to claim 2, wherein for the function of selecting, the one or more scene objects are one of characters and icons.

4. The user interface controller of a handheld electronic device according to claim 2, wherein for the function of moving a cursor object, a response to the motion of the cursor object after the processor interprets a drawing command is drawing a scene object within the one or more successive scenes according to the motion of the cursor.

5. The user interface controller of a handheld electronic device according to claim 2, wherein for the function of moving a cursor object, the cursor object is an icon.

6. The user interface controller of a handheld electronic device according to claim 2, wherein the moving of a cursor object comprises modifying the cursor object in response to a three dimensional tracking of the directing object.

7. The user interface controller of a handheld electronic device according to claim 2, wherein the adjusting of a viewing perspective of successive scenes comprises modifying a three dimensional aspect of the successive scenes in response to a three dimensional tracking of the directing object.

8. The user interface controller of a handheld electronic device according to claim 2, wherein the

moving of the cursor object within one or more successive scenes on the display is performed in response to one or more positions and one or more orientations of the directing object, and includes modifications of the cursor object, and

adjusting of the viewing perspective of successive scenes on the display is performed in response to one or more positions and one or more orientations of the directing object.

9. The user interface controller of a handheld electronic device according to claim 8, further comprising at least one sensor by which the processor detects at least one user command that impacts the one of the functions of selecting, moving and adjusting.

10. The user interface controller of a handheld electronic device according to claim 9, wherein the at least one sensor comprises a touch sensitive detector.

11. The user interface controller of a handheld electronic device according to claim 9, wherein the at least one sensor is a microphone responsive to voice, and wherein the at least one user command is interpreted by a speech recognition function coupled to the sensor.

12. The user interface controller of a handheld electronic device according to claim 9, wherein the sensor is the first camera and wherein the at least one user command is detected in response to at least one particular track of the directing object.

13. The user interface controller of a handheld electronic device according to claim 9, wherein the at least one sensor is the first camera and wherein the at least one command is detected in response to a particular pattern within the video image.

14. The user interface controller of a handheld electronic device according to claim 1, wherein the display has a viewing area that is less than 100 square centimeters.

15. The user interface controller of a handheld electronic device according to claim 1, wherein the first camera has a depth of field range of at least 10 centimeters under lighting conditions expected for normal use.

16. The user interface controller of a handheld electronic device according to claim 1, wherein an axis of the field of view of the first camera is oriented in a direction essentially perpendicular to the display.

17. The user interface controller of a handheld electronic device according to claim 1, wherein an axis of the field of view is oriented in a direction biased away from an expected direction of an operator'"'"'s face.

18. The user interface controller of a handheld electronic device according to claim 1, wherein an axis of the field of view of the first camera can be moved by an operator of the electronic device.

19. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function that processes the video images to track the position of the directing object is responsive to images of one or more directing object markers that have one or more of the group of characteristics comprising:

each object marker image is a projection of a defined shape that includes at least one defined point location,

each object marker image is small in size in relation to the field of view,

each object marker image has a high brightness contrast ratio compared to the immediate surroundings, and

each object marker image primarily comprises light in a particular light band.

20. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function tracks the directing object using at least one image of one or more directing object markers.

21. The user interface controller of a handheld electronic device according to claim 20, wherein the handheld electronic device further comprises a light source and the image of at least one of the one or more directing object markers is a reflection of light from a light source in the handheld electronic device.

22. The user interface controller of a handheld electronic device according to claim 21, wherein at least one of the one or more directing object markers comprises a retro-reflector that causes the reflection of light.

23. The user interface controller of a handheld electronic device according to claim 20, wherein at least one of the one or more directing object markers comprises a light source that generates the image of the one of the one or more directing object markers.

24. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function tracks the directing object in two dimensions that are in the plane of the display.

25. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function tracks the directing object in three dimensions.

26. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function further processes the video images to track a position of the directing object and determines the position of the directing object from images of one or more directing object markers on the directing object.

27. The user interface controller of a handheld electronic device according to claim 26, wherein the processing function further processes the video images to track an orientation of the directing object and determines the orientation of the directing object from the images of one or more directing object markers that are on the directing object.

28. The user interface controller of a handheld electronic device according to claim 1, wherein the processing function further performs a function that transmits information related to at least a portion of a scene on the display to a communication device.

29. The user interface controller of a handheld electronic device according to claim 1, further comprising a second camera, wherein the information on the display comprises images captured by the second camera.

30. The user interface controller of a handheld electronic device according to claim 1, wherein the handheld electronic device further comprises a wireless telephone.

32. The user interface method according to claim 30, wherein the modifying further comprises at least one of:

selecting one or more scene objects within one or more successive scenes on the display in response to a track of the directing object,

moving a cursor object within one or more successive scenes on the display in response to a track of the directing object, and

adjusting a viewing angle of successive scenes on the display in response to a track of the directing object.

33. The user interface method according to claim 32, wherein the processing function tracks the directing object using images of one or more directing object markers.

31. A user interface method used in a handheld electronic device that has a camera that generates video images and has a display, comprising:

presenting information on the display;

processing the video images to track at least one of a position and orientation of a directing object that is within a field of view of the camera; and

modifying at least one scene presented on the display in response to a track of the directing object.