Coordinating images displayed on devices with two or more displays

• US 7,016,704 B2
• Filed: 04/01/2002
• Issued: 03/21/2006
• Est. Priority Date: 04/02/2001
• Status: Active Grant

• Current Assignee: Gula Consulting Limited Liability Company (Intellectual Ventures LLC)
• Original Assignee: Move Mobile Systems, Inc.
• Inventors: Pallakoff, Matthew G.
• Primary Examiner(s): Gesesse, Tilahun
• Assistant Examiner(s): Lee, John J.
• Application Number:
  US10/113,239
• Publication Number:
  US 20020151283A1
• Time in Patent Office: 1,450 Days
• Field of Search: 455/566, 455/414.1, 455/414.2, 455/414.3, 455/414.4, 455/419, 455/457, 455/556.2, 455/550.1, 455/557, 455/563, 455/418, 455/575.1, 455/90.3
• US Class Current: 455/566
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/165 the additional display being small, e.g. for presenting status information
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2340/145 related to small screens
• CPC Class Codes: H04M 1/0214 Foldable telephones, i.e. with body parts pivoting to an open position around an axis parallel to the plane they define in closed position combined with translation movement H04M1/0235
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/16 including more than one display unit

Abstract

A hand held electronic device having at least two displays is disclosed. At least one display is a direct-view display, like those on most cell phones or PDAs in use in 2001, for viewing at normal reading distance of approximately 12 to 24 inches (“arms'"'"'-length” viewing). At least one of the other displays is a microdisplay, a tiny display with magnifying optical elements, for viewing larger, higher-resolution images when the microdisplay is positioned close to the eye. The invention allows coordinating microdisplays and direct-view displays in ways that allow people to comfortably access and interact with full-page Web content on pocket-size devices. When a user is viewing a Web page (or other content) on a device'"'"'s microdisplay held near-to-eye, the device allows the user to position a cursor or a rectangular outline (or some other indication of a “region of interest”) on a particular part of the Web page, and then when the user moves the device out to arms'"'"'-length viewing, the user should be able to view that region of interest on the direct-view display—that is, view a subset of the larger image that appeared on the microdisplay.

Claims

1. A phone handset or other hand-held device which displays images and operates on functions transmitted remotely from said device, said device comprising a face, said face having two or more displays for displaying said images and control elements for controlling said displays, wherein at least one display is a direct-view display or a microdisplay, and such at least one of said functions is displayed in the direct-view display and a user can operate said control elements to affect visual changes on one or more of the displays, the displayed function including an attachment that can be displayed in the microdisplay and (1) the displayed function is taken from the group of functions consisting of instant messaging, email, contact management, schedule maintenance, calculation functions and placing telephone calls, viewing Internet content, and viewing other types of documents and (2) the attachment is an attachment taken from the group of attachments consisting of a Web page URL, a photo image, a spreadsheet, a presentation, an Adobe Acrobat file, a Word Processor document, and other types of content, and (3) the controls are capable of operation by a user to display that attachment in the microdisplay.

2. The device of claim 1 in which the device has exactly two displays.

3. The device of claim 1 in which the device includes one direct-view display and one microdisplay.

4. The device of claim 1 further including a distance sensor wherein the user affects displaying said attachment in the microdisplay by bringing the microdisplay to the user'"'"'s eye.

5. The device of claim 1 further including control elements which, when activated, cause the device to display an indication in the direct-view display that indicates to the user when all or part of said attachment is being downloaded or prepared for display on the microdisplay, or cause the device to display an indication in the direct-view display that indicates when the attachment is ready for viewing on the microdisplay.

6. A phone handset or other hand-held device with two or more displays, said device comprising:

one direct-view display and one microdisplay, in which device when a first display is displaying first content having items that can be selected or typed in, one of such items is in-focus;

controls operable by a user to (1) cause one of the displays to display first content that includes one or more references to other content, and (2) cause that referred-to content to be displayed in one of the other displays, wherein the item in-focus can be changed by the user typing, depressing buttons, speaking, or using other controls on the device, and in which, as the focus changes, if the microdisplay is displaying an image, the image shown in the microdisplay auto-scrolls to keep the in-focus item in view, and, if the direct-view display is displaying an image, the image shown in the direct-view display auto-scrolls to keep the in-focus item in view;

controls operable by a user that cause, if the device has prepared an image having one item in-focus for display in the microdisplay, a subset of said image surrounding the in-focus item to be displayed in the direct-view display; and

a computer processor in which both said item in-focus in said microdisplay and said image subset displayed in the direct-view display can be changed by software running on said computer processor within the device.

7. The device of claim 6 in which the device has exactly two displays.

8. The device of claim 6 including a microphone and voice recognition software that can allow a user to cause the in-focus item to change from one selectable item to another one by the user speaking into the device.

9. A phone handset or other hand-held device which displays images, said device comprising:

two or more displays for displaying said images and control elements for controlling said displays, wherein a user can operate said control elements to cause a microdisplay to display an image and cause a direct-view display to display a region of interest subset of said image, the microdisplay displaying a relatively high-resolution large apparent image and the direct-view display being a low-resolution bit-map direct-view display, said direct view display acting as a window into said larger apparent image, said device being capable of displaying a subset of said image in at least one of said displays and further having at least one of the group consisting of (1) a plurality of face-keys capable of being depressed by a user to generate text typed into a text-entry field shown on said one or more displays, and (2) arrow keys or other controls which, when actuated by a user, change said text-entry point, wherein if the text-entry point moves out of the subset of said image visible in any given display, then that display will automatically scroll the image so that the text-entry entry point will reappear in that display;

controls operable by a user to move the borders of said region of interest subset displayed in the direct-view display down or up or right or left or other directions across the image displayed in the microdisplay; and

face-keys for entering text, the image is a document or Web Page and, as the user operates said face-keys to enter text into said document or Web page, the subset of the document or Web Page that includes the text-entry point is automatically scrolled into view in both the direct view display and the microdisplay, wherein the subset displayed in the direct-view display is an area of bits containing all or part of the current in-focus area of the image displayed in the microdisplay.

10. The device of claim 9 having a plurality of face-keys capable of being depressed by a user to generate new text into a text-entry field displayed in a first display, which text-entry field is also shown on a second display such that the new text that appears in said first display will also appear in said second display.

11. The device of claim 9 including at least one control button capable of being depressed by a user to generate a changed state of a check box, radio-button, or other state control that appears in more than one display, and concurrently changing to said changed state all displays showing that same check box, radio-button or other state control.

12. A computer implemented process generating codes for interpreting the operation of control elements on a phone handset or other hand-held device which displays images, the control elements including first controls capable of operation by a user to display that attachment in the microdisplay, the control elements further including second control elements which, when activated by the user, display an indication in the direct-view display that indicates to the user that the attachment has been opened and is being displayed in the microdisplay and said codes effecting such display of such indication in response to said activation, said codes effecting such display of such attachment in response to operation of said second controls, said process operating on functions transmitted remotely from said device, wherein such functions are selected from the group consisting of the function is taken from the group of functions consisting of instant messaging, email, contact management, schedule maintenance, calculation functions, placing telephone calls, viewing Internet content, and viewing other types of documents, at least one of said functions being displayed in the direct-view display, wherein the displayed function includes an attachment that can be displayed in the microdisplay, said phone handset or other device having (1) a face, said face having two or more displays for displaying said images, wherein at least one display is a direct-view display or a microdisplay, and (2) control elements operable by the user for controlling said displays, said codes effecting visual changes on one or more of the displays in response to such operation, wherein said control elements include a distance sensor, said codes effecting displaying said attachment in the microdisplay in response to the user bringing said distance sensor to the user'"'"'s eye, the attachment being an attachment taken from the group of attachments consisting of a Web page URL or a photo image, a spreadsheet, a presentation, an Adobe Acrobat file, a Word Processor document, and other types of content;

said phone handset or other hand-held device further including a menu button and, if a user depresses said menu button while said distance sensor senses that the microdisplay on said device is close to an object, then the device will display a menu on the microdisplay, and otherwise the device will display said menu on the direct-view display.

13. A computer implemented process generating codes for interpreting the operation of control elements on a phone handset or other hand-held device which displays images, said phone handset or other device comprise:

two or more displays for displaying said images, the microdisplay displaying a relatively high-resolution large apparent image, the direct-view display is a low-resolution bit-map direct-view display, said direct-view display acting as a window into said larger apparent image;

control elements for controlling said displays, wherein a user can operate said control elements to cause a first display to display an image and cause a second display to display a region of interest subset of said image, and said codes effect displaying of said subset in response to the operation of said control elements, wherein said codes effecting said subset of said image are being so displayed in response to the operation of said third controls, said control elements further including third controls operable by a user to cause, if the device has prepared an image having one item in-focus for display in the microdisplay, a subset of said image surrounding the in-focus item to be displayed in the direct-view display, and the item in-focus can be changed by the user typing, depressing buttons, speaking, or using other controls on the device, and in which, as the focus changes, if the microdisplay is displaying an image, the image shown in the microdisplay auto-scrolls to keep the in-focus item in view, and, if the direct-view display is displaying an image, the image shown in the direct-view display auto-scrolls to keep the in-focus item in view, the device being capable of displaying a subset of said image in at least one of said displays and further having at least one of the group consisting of (1) a plurality of face-keys capable of being depressed by a user to generate text typed into a text-entry field shown on said one or more displays, and (2) arrow keys or other controls which, when actuated by a user, change said text-entry point, wherein if the text-entry point moves out of the subset of said image visible in any given display, then said codes effect said any given display automatically scrolling the image so that the text-entry point will reappear in said any given display;

a first display being a microdisplay and a second display being a direct-view display, and a subset of said control elements being operable by a user to move the borders of said region of interest subset displayed in the direct-view display down or up or right or left or other directions across the image displayed in the microdisplay, said codes effecting said movement of said borders in response to the operation of said subset of said control elements, and the subset displayed in the direct-view display is an area of bits containing all or part of the current in-focus area of the image displayed in the microdisplay;

a microphone and voice recognition software that allows a user to cause the in-focus item to be changed from one selectable item to another one by the user speaking into the device, said codes effecting said change in response to the user so speaking; and

face-keys operable by a user to enter text, the image is a document or Web Page having a text entry point, said codes effecting the subset of the document or Web Page that includes the text-entry point being automatically scrolled into view in both the direct-view display and the microdisplay in response to the user operating said face-keys to enter text into said document or Web page.

14. A storage medium having embedded therein a computer implemented process generating codes for interpreting the operation of control elements on a phone handset or other hand-held device which displays images, wherein said control elements include a distance sensor and first controls capable of operation by a user to display that attachment in the microdisplay, said control elements further including second control elements which, when activated by the user, display an indication in the direct-view display that indicates to the user that the attachment has been opened and is being displayed in the microdisplay and said codes effecting such display of such indication in response to said activation, said codes effecting such display of such attachment in response to operation of said second controls, said codes effecting displaying said attachment in the microdisplay in response to the user bringing said distance sensor to the user'"'"'s eye, the attachment being an attachment taken from the group of attachments consisting of a Web page URL or a photo image, a spreadsheet, a presentation, an Adobe Acrobat file, a Word Processor document, and other types of content, said phone handset or other device operating on functions transmitted remotely from said device, wherein such functions are selected from the group consisting of instant messaging, email, contact management, schedule maintenance, calculation functions, placing telephone calls, viewing Internet content, and viewing other types of documents, and at least one of said functions is displayed in the direct-view display, and the displayed function includes an attachment that can be displayed in the microdisplay, said device comprising:

a face, said face having two or more displays for displaying said images, wherein at least one display is a direct-view display or a microdisplay; and

control elements operable by a user for controlling said displays, said code effecting visual changes on one or more of the displays in response to such operation.

15. A storage medium having embedded therein a computer implemented process generating codes for interpreting the operation of control elements on a phone handset or other hand-held device which displays images, said phone handset or other device:

two or more displays for displaying said images, the microdisplay displaying a relatively high-resolution large apparent image, the direct-view display is a low-resolution bit-map direct-view display, said direct-view display acting as a window into said larger apparent image;

control elements for controlling said displays, wherein a user can operate said control elements to cause a first display to display an image and cause a second display to display a region of interest subset of said image, and said codes effect displaying of said subset in response to the operation of said control elements, wherein said codes effecting said subset of said image are being so displayed in response to the operation of said third controls, said control elements include third controls operable by a user to cause, if the device, has prepared an image having one item in-focus for display in the microdisplay, a subset of said image surrounding the in-focus item to be displayed in the direct-view display, said codes effecting said subset of said image being so displayed in response to the operation of said third controls, and the item in-focus can be changed by the user typing, depressing buttons, speaking, or using other controls on the device, and in which, as the focus changes, if the microdisplay is displaying an image, the image shown in the microdisplay auto-scrolls to keep the in-focus item in view, and, if the direct-view display is displaying an image, the image shown in the direct-view display auto-scrolls to keep the in-focus item in view, the device being capable of displaying a subset of said image in at least one of said displays and further having at least one of the group consisting of (1) a plurality of face-keys operable by a user to enter text, the image is a document or Web Page having a text entry point, said codes effecting the subset of the document or Web Page that includes the text-entry point being automatically scrolled into view in both the direct-view display and the microdisplay in response to the user operating said face-keys to enter text into said document or Web page, and (2) arrow keys or other controls which, when actuated by a user, change said text-entry point, wherein if the text-entry point moves out of the subset of said image visible in any given display, then said codes effect said any given display automatically scrolling the image so that the text-entry point will reappear in said any given display;

a first display being a microdisplay and a second display being a direct-view display, and a subset of said control elements being operable by a user to move the borders of said region of interest subset displayed in the direct-view display down or up or right or left or other directions across the image displayed in the microdisplay, said codes effecting said movement of said borders in response to the operation of said subset of said control elements, and the subset displayed in the direct-view display is an area of bits containing all or part of the current in-focus area of the image displayed in the microdisplay;

a microphone and voice recognition software that allows a user to cause the item in-focus to be changed from one selectable item to another one by the user speaking into the device, said codes effecting said change in response to the user so speaking; and

a computer processor in which both said item in-focus in said microdisplay and said image subset displayed in the direct-view display can be changed by software running on said computer processor.

16. A computer implemented process generating codes for interpreting the operation of control elements on a phone handset or other hand-held device which displays images, said phone handset or other device having (1) two or more displays for displaying said images and (2) control elements for controlling said displays, wherein a user can operate said control elements to cause a first display to display an image and cause a second display to display a region of interest subset of said image, and said codes effect displaying of said subset in response to the operation of said control elements, wherein said control elements include third controls operable by a user to cause, if the device has prepared an image having one item in-focus for display in the microdisplay, a subset of said image surrounding the in-focus item to be displayed in the direct-view display, said codes effecting said subset of said image being so displayed in response to the operation of said third controls, said device further having a microphone and voice recognition software that allows a user to cause the in-focus item to be changed from one selectable item to another one by the user speaking into the device, said codes effecting said change in response to the user so speaking.

17. The process of claim 16 wherein said control elements include first controls operable by a user to (1) display substantially a full Web page in one display and (2) display a close up view of one subset of that Web page in another of the displays, said codes effecting said displaying in said one display and said displaying in said another of the displays in response to the operation of said first controls.

18. The process of claim 16 in which the device includes one direct-view display and one microdisplay.

19. The process of claim 18 wherein the microdisplay displays a relatively high-resolution large apparent image, the direct-view display is a low-resolution bit-map direct-view display, said direct-view display acting as a window into said larger apparent image.

20. The process of claim 19 wherein the device further includes face-keys operable by a user to enter text, the image is a document or Web Page having a text entry point, said codes effecting the subset of the document or Web Page that includes the text-entry point being automatically scrolled into view in both the direct-view display and the microdisplay in response to the user operating said face-keys to enter text into said document or Web page.

21. The process of claim 19 in which the subset displayed in the direct-view display is an area of bits containing all or part of the current in-focus area of the image displayed in the microdisplay.

22. The process of claim 16 wherein said control elements include second controls operable by a user which, when a first display is displaying first content having items that can be selected or typed in, cause one of such items to be in-focus, said codes effecting said one of such items.

23. The process of claim 22 wherein, in response to the user operating said second controls, said codes effect said one in-focus item being highlighted by outlining that item in a visually recognizable shade.

24. The process of claim 23 wherein the device has a select function operable by a user to select said one in-focus item, said codes effecting selecting said one item into in response to the operation of said select function.

25. The process of claim 24 wherein the device further includes one or more face-keys-which, when depressed by a user, causes a character to be typed into said one item if said one item is an editable text field, said codes effecting said character being so typed in response to the user depressing said one or more face-keys.

26. The process of claim 16 in which the subset displayed in the direct-view display is a close up view of said one item.

27. The process of claim 16 in which the subset displayed in the direct-view display includes both the in-focus item and a text or bit-map representation of text or graphic labels or symbols corresponding to that item.

28. The process of claim 27 in which the device includes at least one control element from the group consisting of a roller control or up/down buttons or multi-speed up/down controls capable of operation by the user to cause the in-focus item to change from the current in-focus item to the previous or next item that can be selected or typed in.

29. The process of claim 28 wherein the process includes software that causes the image displayed in the direct-view display to concurrently change when the in-focus item changes from one selectable item to a new in-focus item.

30. The process of claim 29 in which said image is changed by displaying a subset of the image surrounding all or part of said new in-focus item.

31. The process of claim 16 in which the in-focus item is a “

Last Name”

text entry box on a Web-page form and the subset includes the label “

Last Name” and

the corresponding text-entry box into which the last name will be typed.

32. The process of claim 16 wherein the control elements include fourth controls operable by a user to cause the subset displayed in the direct-view display to be changed by user operation, said codes effecting said change in response to said user operation.

33. The process of claim 16 wherein the device includes a computer processor in which both said item in-focus in said microdisplay and said image subset displayed in the direct-view display can be changed by software running on said computer processor within the device.

34. The process of claim 16 wherein said change generates animations by having a large image made up of many smaller frames, and rapidly changing which subset of the larger image is displayed in the direct-view display.

35. The process of claim 16 in which the item in-focus can be changed by the user typing, depressing buttons, speaking, or using other controls on the device, and in which, as the focus changes, if the microdisplay is displaying an image, the image shown in the microdisplay auto-scrolls to keep the in-focus item in view, and, if the direct-view display is displaying an image, the image shown in the direct-view display auto-scrolls to keep the in-focus item in view.

36. The process of claim 16 wherein the device has a plurality of face-keys capable of being depressed by a user to generate new text into a text-entry field displayed in a first display, which text-entry field is also shown in a second display such that the new text that appears in said first display will also appear in said second display, said codes effecting said new text appearing in said first display an in said second display in response to the user so depressing said face keys.

37. The process of claim 16 wherein the device includes at least one control button capable of being depressed by a user to generate a changed state of a check box, radio-button, or other state control that appears in more than one display, and concurrently changing to said changed state all displays showing that same check box, radio-button or other state control.

38. The process of claim 16 in which the device is capable of displaying a subset of said image in at least one of said displays and further having at least one of the group consisting of (1) a plurality of face-keys capable of being depressed by a user to generate text typed into a text-entry field shown on said one or more displays, and (2) arrow keys or other controls which, when actuated by a user, change said text-entry point, wherein if the text-entry point moves out of the subset of said image visible in any given display, then said codes effect said any given display automatically scrolling the image so that the text-entry point will reappear in said any given display.

39. The process of claim 16 wherein a first display is a microdisplay and a second display is a direct-view display, and a subset of said control elements are operable by a user to move the borders of said region of interest subset displayed in the direct-view display down or up or right or left or other directions across the image displayed in the microdisplay, said codes effecting said movement of said borders in response to the operation of said subset of said control elements.

40. A computer implemented process generating codes for interpreting the operation of control elements on a phone handset or other hand-held device which displays images, said phone handset or other device having (1) one direct-view display and one microdisplay for displaying said images and (2) control elements for controlling said displays, wherein a user can operate said control elements to cause the microdisplay to display an image and cause the direct-view display to display a region of interest subset of said image, and said codes effect displaying of said subset in response to the operation of said control elements;

wherein the microdisplay displays a relatively high-resolution large apparent image, the direct-view display is a low-resolution bit-map direct-view display, said direct-view display acting as a window into said larger apparent image.

41. A computer implemented process generating codes for interpreting the operation of control elements on a phone handset or other hand-held device which displays images, said phone handset or other device having (1) one direct-view display and one microdisplay for displaying said images and (2) control elements for controlling said displays, wherein a user can operate said control elements to cause the microdisplay to display an image and cause the direct-view display to display a region of interest subset of said image, and said codes effect displaying of said subset in response to the operation of said control elements;

wherein a subset of said control elements are operable by a user to move the borders of said region of interest subset displayed in the direct-view display down or up or right or left or other directions across the image displayed in the microdisplay, said codes effecting said movement of said borders in response to the operation of said subset of said control elements.