Interaction techniques for flexible displays

• US 8,466,873 B2
• Filed: 09/09/2011
• Issued: 06/18/2013
• Est. Priority Date: 03/30/2006
• Status: Active Grant

• Current Assignee: David Holman, Roel Vertegaal
• Original Assignee: David Holman, Roel Vertegaal
• Inventors: Vertegaal, Roel, Holman, David
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US13/228,681
• Publication Number:
  US 20120112994A1
• Time in Patent Office: 648 Days
• Field of Search: 345156-184, 178 1801- 1901, 382/203, 382/285, 463/30, 463/37
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1652 the display being flexible, e.g. mimicking a sheet of paper, or rollable
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

The invention relates to a set of interaction techniques for obtaining input to a computer system based on methods and apparatus for detecting properties of the shape, location and orientation of flexible display surfaces, as determined through manual or gestural interactions of a user with said display surfaces. Such input may be used to alter graphical content and functionality displayed on said surfaces or some other display or computing system.

Claims

1. A method for providing input to a computer system that uses at least one property selected from shape, orientation, and location of a flexible display surface associated with the computer system, or a change in one or more said properties;

wherein the input is selected from the group consisting of;

a) Hold, wherein the flexible display is activated as a destination of computer commands, or activates associated computing commands, comprising holding the display with one or two hands, and wherein the display remains the active display until another such display is activated;

b) Flip or Turn, wherein rotating the flexible display around its horizontal axis or vertical axis wherein one edge of the display is lifted up and then folded over, is used as input to the computer system;

c) Fold, wherein folding the flexible display about one of its axes is used as input to the computer system;

d) Part-fold, wherein partly folding the flexible display is used as input to the computer system;

e) Semi-permanent fold, wherein folding the flexible display about one of its axes such that it remains in a folded state after release is used as input to the computing system;

f) Roll, wherein changing the shape of the flexible display such that the shape transitions from planar to cylindrical or vice versa is used as input to the computing system;

g) Rub, a rubbing gesture wherein a hand or finger or a tool moved back and forth over the flexible display is used as input to the computing system;

h) Staple, wherein using an impact of the flexible display with a second flexible display is used as input to the computing system; and

i) Pointing, wherein the location of a hand, finger, or tool relative to the flexible display is used as input to the computer system.

2. The method of claim 1, wherein input (g) is applied to a first flexible display collated with a second display, and wherein input (g) causes content of the first flexible display to be copied or otherwise moved onto the second display.

3. The method of claim 2, wherein the second display is a flexible, traditional, or rigid computer display terminal;

wherein content is moved from the second display to the first flexible display if the first flexible surface does not display an image;

or

wherein content is moved from the first flexible display to the second display when the first flexible display does display an image.

4. The method of claim 2, wherein the second display is associated with a computing peripheral that has a processing action, and wherein content is moved from the first flexible display to the computing peripheral for processing.

5. The method of claim 4, wherein the computing peripheral is a printer or network peripheral, and wherein content is moved to the printer or network peripheral, or to a remote location for printing or viewing on a computing system.

6. The method of claim 1, wherein the input to the computer system causes a command to be executed on the computer system and wherein the command is selected from:

a) Activate, wherein a file or computer content, image, selection, or window associated with or displayed on the flexible display is selected;

b) Zoom in, wherein an image or content of a file associated with the flexible display is enlarged;

c) Zoom out, wherein an image associated with the flexible display is reduced;

d) Organize, wherein a file, digital information, text, image, or other computer content associated with or displayed on the flexible display is organized or sorted digitally such that it matches organization or sorting on the flexible display;

e) Scroll, wherein an image or content of a file associated with the flexible display is scrolled, such that a portion of the image or content of the file is revealed that was previously not rendered, or that is contiguous to what is currently rendered on the flexible surface, or other display;

f) Page Down, wherein a section of the content of a file that is subsequent to the section of the content that is currently displayed on or associated with the flexible display, or other display, causes the subsequent section to render on the flexible display or other display;

g) Page Up, wherein a section of the content of a file that precedes the section of the content that is currently displayed on or associated with the flexible display, or other display, causes the subsequent section to render on the flexible display or other display;

h) Navigate, wherein navigation toward an arbitrary section of the content of a file associated with the flexible display, or an online content or hyperlink associated with the display, causes the content to render on the flexible display or other display;

i) Page Back or Page Forward, wherein navigation toward a section of the content of a file, or online content, webpage, or hyperlink that precedes or follows the content currently displayed or associated with the flexible display, causes the content to render on the flexible display or other display;

j) Open or Close, wherein a file or digital information associated with the flexible display is opened or closed, read into memory, or output to a permanent medium;

k) Sleep or Wake, wherein the flexible display is de-activated or activated from a state of reduced energy use;

l) Deformation, wherein shape of a three dimensional model associated with the flexible display is deformed such that it follows the deformation of the flexible display, in any dimension;

m) Save, wherein a file associated with the flexible display is saved to a hard drive or other permanent medium or online service;

n) Move or Copy, wherein a file or other digital information, or some portion or selection thereof, currently associated with the flexible display is transferred to another flexible display or computing device;

o) Duplicate, wherein information or a file associated with the flexible display is made identical to that of a second flexible display so as to clone or duplicate the information on the second flexible display;

p) Select, wherein items or images, or both, displayed on the flexible display are selected;

q) Stationary, wherein the flexible display or other display shows a set of icons indicating a set of computing applications, or potential functionality of the flexible display, and wherein the computer system refrains to the application functionality after selecting the icon of the application;

r) Focus and Context, wherein a second flexible display stacked beneath the flexible display displays a summary, menu, or set of thumbnails pertaining to content of the flexible display; and

s) Enlarge, wherein collocation of multiple flexible displays enlarges content across all collocated flexible displays.

7. The method of claim 1, comprising using an accelerometer embedded in or fixed to the flexible display to sense one or more of shape, orientation, and location of the flexible display, or to sense a change in one or more of shape, orientation, and location of the flexible display, and provide input to the computer system.

8. The method of claim 1, comprising detecting intensity of light passing through one or more optical fiber embedded in or fixed to the flexible display to sense shape of the flexible display, or to sense a change in shape of the flexible display, and provide input to the computer system.

9. The method of claim 1, wherein opening a hyperlink causes an online connection to be made by the computer system.

10. The method of claim 1, wherein the input is applied to a stack or pile of flexible displays.

11. The method of claim 1, comprising providing input to a computer system that uses at least one property selected from shape, orientation, and location of two or more flexible display surfaces associated with the computer system, or a change in one or more said properties, wherein the input is selected from the group consisting of:

a) Collocate, wherein collocating multiple flexible displays to create a larger flexible display is used as input to the computer system;

b) Collate, wherein stacking multiple flexible displays on top of one another is used as input to the computer system; and

c) Pile, wherein physical organization of a pile of flexible displays is used as input to the computing system.

12. The method of claim 1, comprising providing input to a computer system that uses at least one property selected from shape, orientation, and location of one or more flexible display surfaces associated with the computer system, or a change in one or more said properties, wherein the input comprises:

Bend, wherein bending the flexible display about one of its axes is used as input to the computing system;

wherein the input to the computer system causes a command to be executed on the computer system;

wherein the command is selected from;

a) Activate, wherein a file or computer content, image, selection, or window associated with or displayed on the flexible display is selected;

b) Zoom in, wherein an image or content of a file associated with the flexible display is enlarged;

c) Zoom out, wherein an image associated with the flexible display is reduced;

d) Organize, wherein a file, digital information, text, image, or other computer content associated with or displayed on the flexible display is organized or sorted digitally such that it matches organization or sorting on the flexible display;

e) Scroll, wherein an image or content of a file associated with the flexible display is scrolled, such that a portion of the image or content of the file is revealed that was previously not rendered, or that is contiguous to what is currently rendered on the flexible surface, or other display;

f) Navigate, wherein navigation toward an arbitrary section of the content of a file associated with the flexible display, or an online content or hyperlink associated with the display, causes the content to render on the flexible display or other display;

g) Open or Close, wherein a file or digital information associated with the flexible display is opened or closed, read into memory, or output to a permanent medium;

h) Sleep or Wake, wherein the flexible display is de-activated or activated from a state of reduced energy use;

i) Deformation, wherein shape of a three dimensional model associated with the flexible display is deformed such that it follows the deformation of the flexible display, in any dimension;

j) Save, wherein a file associated with the flexible display is saved to a hard drive or other permanent medium or online service;

k) Move or Copy, wherein a file or other digital information, or a portion or selection thereof, currently associated with the flexible display is transferred to another flexible display or computing device;

l) Duplicate, wherein information or a file associated with the flexible display is made identical to that of a second flexible display so as to clone or duplicate the information on the second flexible display;

m) Select, wherein items or images, or both, displayed on the flexible display are selected;

n) Stationary, wherein the flexible display shows a set of icons indicating a set of computing applications, or potential functionality of the flexible display, and wherein the computer system refrains to the application functionality after selecting the icon of the application;

o) Focus and Context, wherein a second flexible display stacked beneath the flexible display displays a summary, menu, or set of thumbnails pertaining to content of the flexible display; and

p) Enlarge, wherein collocation of multiple flexible displays enlarges content across all collocated flexible displays.