Mobile device with on-screen optical navigation

• US 20050033512A1
• Filed: 08/05/2004
• Published: 02/10/2005
• Est. Priority Date: 08/05/2003
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Research In Motion Limited (Blackberry Limited)
• Inventors: Skarine, Alexei
• Granted Patent: US 7,340,342 B2
• Time in Patent Office: Days
• US Class Current: 701/211
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface

Abstract

A mobile computing device, including a main body, a processor and associated memory housed within the main body, a display screen housed within the main body and responsive to signals from the processor, an optical sensor fixed to the main body for capturing successive images and providing image signals representative of the captured images to the processor, and a navigation module associated with the processor for determining, based on the image signals, a relative movement between the main body and a reference surface and moving a pointer on the display screen based on the determined relative movement.

Claims

1. A mobile device, comprising:

a main body;

a processor and associated memory housed within the main body;

a display screen housed within the main body and responsive to signals from the processor;

a color enabled high resolution camera fixed to the main body for capturing successive images and providing color image signals representative of the captured images to the processor; and

a navigation module associated with the processor for converting the color image signals into lower resolution black-and-white image signals and determining, based on the black-and-white image signals, a relative movement between the main body and a reference surface and moving a pointer on the display screen based on the determined relative movement.

2. The device of claim 1 wherein the navigation module compares the successive images captured by the camera at a predetermined capture rate to determine a relative direction, distance and speed of movement of the main body relative to the reference surface, and moves the pointer on the display screen based on the determined relative direction, distance and speed.

3. The device of claim 1 wherein the device further includes a camera module associated with the processor for displaying the captured images on the display screen, the camera module and the navigation module being selectively operable to cause the captured images to be displayed on the display screen in a camera mode of the device and the pointer to be moved on-screen based on the captured images in a navigation mode.

4. The device of claim 3 wherein the device includes a scanner module associated with the processor for, in a selectable scanner mode, converting the image signals received from the camera into alphanumeric data.

5. The device of claim 1 wherein the device includes user interface keys positioned on a front side of the main body for providing input signals to the processor, the camera being fixed to the main body for capturing images located behind a back side of the main body.

6. The device of claim 1 wherein the camera is housed within the main body and positioned behind a lens covered opening provided through a wall of the main body.

7. The device of claim 1 wherein the camera is pivotally mounted to the main body.

8. The device of claim 1 wherein the device is a handheld wireless communications device including a communications sub-system connected to the processor for exchanging signals with a wireless network and with the processor.

9. An on-screen navigation method for a mobile computing device having a display screen, the method including steps of:

(a) fixing a high resolution color enabled camera for movement with the display screen;

(b) displaying an on-screen pointer on the display screen;

(c) capturing successive images of a reference surface through the camera;

(d) converting the successive images into lower resolution grey scale images;

(e) comparing successive grey scale images to determine a relative movement between the camera and the reference surface; and

(f) moving the on-screen pointer on the display screen based on the determined movement.

10. The method of claim 9 wherein in step (e) a direction of the relative movement is determined, and in step (f) the on-screen pointer is moved in a same direction as the determined direction.

11. The method of claim 9 wherein in step (e) a direction of the relative movement is determined, and in step (f) the on-screen pointer is moved in an opposite direction as the determined direction.

12. The method of claim 9 wherein the method includes steps of selecting between at least a first mode of operation and a second mode of operation, wherein in the first mode of operation steps (b) to (f) are performed and wherein in the second mode of operation, in place of steps (b) to (f), the following steps are performed:

(g) capturing at least one image through the camera; and

(h) displaying a representation of the at least one captured image on the display screen.

13. The method of claim 10 wherein the mobile device is a wireless communication device in communication with a wireless network, the method including, after step (h), a step of transmitting a representation of the at least one captured image over the wireless network.

14. The method of claim 9 including selecting between a third mode of operation wherein in the third mode of operation in place of steps (b) to (f) and steps (g) and (h), the following steps are performed:

(i) scanning at least one image through the camera; and

(j) recognizing representations of alphanumeric characters within the scanned image and converting the recognised representations into alphanumeric characters.

15. A mobile device, comprising:

a main body;

a processor and associated memory housed within the main body;

a display screen housed within the main body and responsive to signals from the processor;

an optical sensor fixed to the main body for capturing successive images and providing image signals representative of the captured images to the processor;

a backlight fixed to the main body adjacent the optical sensor;

a navigation module associated with the processor for receiving the image signals and determining, based on the image signals, a relative movement between the main body and a reference surface and moving a pointer on the display screen based on the determined relative movement; and

a lens movable relative to the optical sensor between a first position in which the lens is located in a line of sight of the optical sensor and a second position in which the lens is not located in the line of sight of the optical sensor.

16. The device of claim 15 wherein the device includes a sensor operatively connected to the processor for determining if the lens is in the first position or the second position.

17. The device of claim 16 wherein the device further includes a camera module associated with the processor for displaying the captured images on the display screen, the camera module and the navigation module being selectively operable to cause the captured images to be displayed on the display screen in a camera mode of the device and the pointer to be moved on-screen based on the captured images in a navigation mode.

18. The device of claim 17 wherein the navigation mode is selected in response to the sensor detecting that the lens is in the first position and wherein the camera mode is selected in response to the sensor detecting that the lens is in the second position.

19. The device of claim 15 wherein the device includes user interface keys positioned on a front side of the main body for providing input signals to the processor, the optical sensor being fixed to the main body for capturing images located behind a back side of the main body.

20. The device of claim 15 wherein the device is a handheld wireless communications device including a communications sub-system connected to the processor for exchanging signals with a wireless network and with the processor.

21. A mobile device, comprising:

a main body;

a processor and associated memory housed within the main body;

a display screen housed within the main body and responsive to signals from the processor;

an optical sensor fixed to the main body for capturing successive images and providing image signals representative of the captured images to the processor;

a camera module associated with the processor for displaying the captured images on the display screen;

a navigation module associated with the processor for receiving the image signals and determining, based on the image signals, a relative movement between the main body and a reference surface and moving a pointer on the display screen based on the determined relative movement; and

a select mode module associated with the processor for selecting between a camera mode and a navigation mode, wherein in said camera module is active in said camera mode and said navigation module is active in said navigation mode.

22. The device of claim 21, wherein the device further includes a lens movable relative to the optical sensor between a first position in which the lens is located in a line of sight of the optical sensor and a second position in which the lens is not located in the line of sight of the optical sensor, and wherein said select mode module selects between said camera mode and said navigation mode based upon the position of said lens.

23. The device of claim 22 wherein the device includes a sensor operatively connected to the processor for determining if the lens is in the first position or the second position and wherein said select mode module is responsive to said sensor.

24. The device of claim 21 wherein said select mode module receives said captured images and determines, based upon said captured images, whether said optical sensor is within a threshold distance from a reference surface, and wherein said select mode module automatically selects between said navigation mode and said camera mode based upon said determination.

25. The device of claim 21 wherein the device is a handheld wireless communications device including a communications sub-system connected to the processor for exchanging signals with a wireless network and with the processor.

26. An on-screen navigation method for a mobile computing device having a display screen, the method including steps of:

(a) fixing a camera for movement with the display screen;

(b) displaying an on-screen pointer on the display screen;

(c) capturing successive images of a reference surface through the camera;

(d) automatically selecting between a navigation mode and a camera mode;

(e) in said navigation mode, comparing successive images to determine a relative movement between the camera and the reference surface; and

(f) in said navigation mode, moving the on-screen pointer on the display screen based on the determined movement.

27. The method of claim 26 wherein said step of automatically selecting includes determining whether a lens movable relative to the camera between a first position in which the lens is located in a line of sight of the camera and a second position in which the lens is not located in the line of sight of the camera.

28. The method of claim 27 wherein said step of determining the position of said lens includes receiving a signal from a proximity sensor.

29. The method of claim 26 wherein the step of automatically selecting includes determining, based upon said captured images, whether said camera is within a threshold distance from a reference surface, and automatically selecting between said navigation mode and said camera mode based upon said determination.

30. The method of claim 26 wherein the method further includes steps of, in said camera mode, capturing at least one image through the camera and displaying a representation of the at least one captured image on the display screen.