Interaction techniques for flexible displays

• US 20100045705A1
• Filed: 07/10/2009
• Published: 02/25/2010
• Est. Priority Date: 03/30/2006
• Status: Abandoned Application

• Current Assignee: Béhar, Yves, Justin Lee, Pichaya Puttorngul, Roel Vertegaal
• Original Assignee: Béhar, Yves, Justin Lee, Pichaya Puttorngul, Roel Vertegaal
• Inventors: Béhar, Yves, Lee, Justin, Vertegaal, Roel, Puttorngul, Pichaya
• Application Number:
  US12/459,973
• Publication Number:
  US 20100045705A1
• Time in Patent Office: Days
• US Class Current: 345/661
• CPC Class Codes: A47G 19/2227 with means for amusing or giving information to the user A47G23/16 takes precedence
• CPC Class Codes: A47G 19/2272 from drinking glasses or cups comprising lids or covers
• CPC Class Codes: A47G 2019/2238 with illumination means
• CPC Class Codes: A47G 2019/2244 with sound emitting means
• CPC Class Codes: A47G 2019/225 vessels with thermometers
• CPC Class Codes: G02F 1/133305 Flexible substrates, e.g. plastics, organic film
• CPC Class Codes: G02F 1/13338 Input devices, e.g. touch panels
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1643 the display being associated to a digitizer, e.g. laptops that can be used as penpads details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615
• CPC Class Codes: G06F 1/1652 the display being flexible, e.g. mimicking a sheet of paper, or rollable
• CPC Class Codes: G06F 1/1656 Details related to functional adaptations of the enclosure, e.g. to provide protection against EMI, shock, water, or to host detachable peripherals like a mouse or removable expansions units like PCMCIA cards, or to provide access to internal components for maintenance or to removable storage supports like CDs or DVDs, or to mechanically mount accessories mounting of accessories to a computer display G06F1/1607; display hoods G06F1/1603; cooling arrangements for portable computers G06F1/203
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 2203/04102 Flexible digitiser, i.e. constructional details for allowing the whole digitising part of a device to be flexed or rolled like a sheet of paper
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 : using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 : Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 : Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0425 : using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 : Scrolling or panning
• CPC Class Codes: G06F 3/0487 : using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/147 : using display panels
• CPC Class Codes: G06Q 10/02 : Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 30/0209 : Incentive being awarded or redeemed in connection with the playing of a video game
• CPC Class Codes: G06Q 50/12 : Hotels or restaurants
• CPC Class Codes: G09G 2380/02 : Flexible displays

Abstract

The invention relates to a set of interaction techniques for obtaining input to a computer system based on methods and apparatus for detecting properties of the shape, location and orientation of flexible display surfaces, as determined through manual or gestural interactions of a user with said display surfaces. Such input may be used to alter graphical content and functionality displayed on said surfaces or some other display or computing system. The invention also relates to interactive food or beverage container with associated computing apparatus inside its body, and a curved multitouch display on its surface, associated interaction techniques for curved multitouch displays, methods of use, and apparatus for refilling said electronic food or beverage container.

Claims

1. A reusable portable interactive apparatus comprising:

a. a customizable lid;

b. a selection of input and output devices;

c. a container section with curved display technology selected from a group consisting of;

Flexible E-Ink, Flexible Organic Light Emitting Diode, Flexible LED Arrays, Projection, Laser, and Paintable display;

d. a base comprising computing apparatus selected from a group consisting of;

battery, power connector, network connector, audiovisual connector, central processing unit, wireless network tranceiver, graphics circuit board, RAM memory, firmware ROM, flash and a hard disk drive.

2. The apparatus of claim 1 wherein input and output devices selected from the group consisting of one or more 6 DOF accelerometer(s), Gyroscope, Bend Sensor, Touch screen, Capacitive touch sensor, Heart rate sensor, Galvanic skin conductor sensor, Alpha Dial potentiometer, Video camera, Still camera, Hygrometer;

Liquid Level Sensor;

Potentiometric Liquid Chemical Sensor, Altimeter, Thermometer, Force sensor;

Pressure Sensor;

Microphone, GPS, Buttons, Photoelectric Sensor;

Proximity Sensor, Electronic payment system, One or more RFID tags, Fingerprint reader, A water purification system, Ultraviolet light purification system, Carbon filtration system, Chemical or organic content analyzer, Bacterial content analyzer, Amplification system Speaker system and Compass.

3. The apparatus of claim 2 wherein said input and output devices are disposed on said customizable lid.

4. The apparatus of claims 1 wherein the form factor of said container allows the containment and consumption of beverages.

5. The apparatus of claim 1 wherein the form factor of said container allows the containment and consumption of solid or semi-solid food items, or a combination of solid and semi-solid food items with liquids.

9. The methods of claim 6 wherein said computer system comprises the apparatus according to claim 1.

20. A product refilling station for the apparatus of claim 1 wherein:

a. The product refilling station provides power or software communications to said container upon placement of said container within or on said product refilling station and wherein

b. the container communicates product orders made on said container to said product refilling station upon placement of said container within or on said product refilling station and wherein

c. Optionally, said product refilling station cleans said container and wherein

d. Optionally, said order consists of a recipe of ingredients, and wherein said order is fulfilled by mixing ingredients on site according to said recipe and wherein

e. the product refilling station fulfills said order by filling said container with said order and wherein

f. the product refilling station arranges payment for said order through communications with said container, and through some electronic payment system associated with said container or product refilling station.

6. A method for providing input to a computer system by sensing manual interactions with a curved display surface through a sensor, wherein said interactions are selected from a group consisting of:

a. Holding, wherein holding the curved display surface with one or two hands serves as input to the computer system associated with said curved display;

b. Collocating or stacking, wherein collocating, collating or stacking multiple curved displays creates a single contiguous display surface consisting of individual displays, and wherein subsequent inputs operate on said larger display surface;

c. Turning or Rotating, wherein rotating said curved display around an axis serves as input to the computer system associated with said display;

d. Swirling, wherein moving said curved display in around an axis that is non-concentrical but parallel to some axis of said curved display serves as a means of input to the computer system associated with said curved display;

e. Non-planar Strip Swiping, wherein moving one or more fingers along the top or bottom extremities of a curved display, or just above or below said display, serves as input to the computer system associated with said display;

f. Three-finger Non-planar Pinching, wherein placing three fingers within a threshold proximity on a curved display serves as input to the computer system associated with said curved display;

g. Pining and Swiping, wherein placing one finger on a fixed location on a curved display, while subsequently placing a second finger on said display, and wherein said second finger is subsequently moved away from said first finger, serves as input to the computer system associated with said display;

h. Deforming, wherein deforming a curved display at one location serves as input to the computer system associated with said display;

i. Rubing, wherein providing a rubbing action on a curved display, in which the hand, finger, or some tool is moved in a sinusoidal pattern over its surface, serves as input to the computer system associated with said display;

j. Tilting, wherein tilting a curved display serves as input to the computer system associated with said display;

k. Flicking or Tossing, wherein rapidly tilting a curved display, stopping and optionally returning to its approximate original orientation serves as input to the computer system associated with said display;

l. Resting, wherein placing and releasing an electronic food or beverage container on a surface, serves as input to the computer system associated with said container;

m. Drinking, Filling and Fluid Level, wherein an action selected from a group consisting of;

bringing an electronic food or beverage container to the mouth;

drinking a beverage from said container;

or filling said container serves as input to the computer system associated with said container;

n. Opening and closing, wherein opening and closing the lid of an electronic food or beverage container serves as input to the computer system associated with said container;

o. Multi-device Pouring, wherein holding an electronic food or beverage container over a second said container, and subsequently tilting said first container, serves as input to the computer system associated with either or both containers;

p. Fingerprint scanning, wherein placing one or more fingers of a user on a designated part of a curved display surface causes associated fingerprints to be analyzed with the purpose of authenticating access by said user to information on said curved display surface;

q. Face detection, wherein the face of a user is identified by an electronic food or beverage container for the purpose of authenticating access of said the user to information on said container;

7. The method of claim 6 wherein said sensor is selected from a group consisting of:

one or more 6 DOF Accelerometer(s), Gyroscope, Bend sensor, Touch screen, Capacitive touch sensor, Heart rate sensor, Galvanic skin conductor sensor, Alpha Dial potentiometer, Infrared computer vision or motion capture system, Video camera, Still camera, Hygrometer;

Liquid Level Sensor, Potentiometric Liquid Chemical Sensor, Altimeter, Thermometer, Force sensor, Pressure Sensor, Microphone, GPS;

Buttons, Photoelectric Sensor, Proximity Sensor, One or more RFID tags, Fingerprint reader, and Compass.

8. The method of claim 6 wherein said input to said computer system causes a command to execute on said computer system and wherein said command is selected from a group consisting of:

a. Activate, wherein the software and display of said computer system awakes from sleep, disabling a screen saver or energy reduce state, or enabling advertisement activity, and

b. Deactivate, wherein in the software and display of said computer goes to sleep, enabling a screen saver or energy reduced state, or disabling advertisement activity, and

c. Zoom in or Enlarge, wherein an image or content of a file or document rendered on said display is enlarged or zoomed in on, and

d. Zoom out or Reduce, wherein an image or content of a file or document rendered on said display is reduced or zoomed out of, and

e. Organize, wherein some property of file(s), digital information, text, images, or other computer content associated with or displaying on said display surface(s) is organized or sorted digitally in a way that matches properties of the physical computer system, such as physical order, and

f. Scroll, wherein a segment of an image or content of a file, document or application is rendered on a display, said segment being not previously rendered, and said segment being spatially contiguous to the segment of said image or content that was previously rendered on said display, and

g. Page Down, wherein a segment of the content of a file subsequent to the section of said content of a file that is currently rendered on a display, is navigated to such that it causes said subsequent section to be rendered on said display, and

h. Page Up, wherein a segment of the content of a file that precedes the section of said content of a file that is currently rendered on a display, is navigated to such that it causes said preceding section to be rendered on said display, and

i. Navigate, wherein an arbitrary section of the content of a file on said computer system, or some online content, hyperlink, or menu is navigated to such that it causes said the associated content to render on a display, and

j. Page Back or Forward, wherein a section of the content of a file, or some online content, webpage or hyperlink that precedes or follows the section of said content currently rendered on a display, is navigated to such that it causes said content to be rendered on said display, and Open, Save or Close, wherein some file or digital information on said computer system is opened or closed, read into memory, or out to a permanent storage medium, and

k. Move, Copy or Paste, wherein a section of the content of a file, image, text or some other digital information associated with said computer system or display is transferred to another computer system or display, or some different logical location on said same computer system or display, and Select, where graphical objects rendered on a display is selected such that it becomes the recipient of a subsequent action, input or command to the associated computer system, and

l. Click, wherein an insertion point or cursor is moved to a specific location on a display, selecting or activating graphical objects underlying said location on said display, and

m. Erase, wherein selected information or images, or content associated with said images on a computer system, is erased from said display and/or from the memory of said computer system, and

n. Playback control, wherein a multimedia file, including graphics animation, video, sound or musical content on said computer system, is played at some speed, and wherein said speed is optionally controlled by said input, and

o. Connect, wherein said computer system is connected through a computer network to another computer system, online server, communication tool or social networking site, and

p. Share, wherein information on said computer system is placed on a computer server for the purpose of sharing said information with other users connected to said server, and

q. Online status, wherein information about the usage of said computer system by the user, or some arbitrary status or attribute of said user, is shared with a computer server for the purpose of sharing said information with other users connected to said server, and

r. Communicate, wherein said computer system serves as a communication device, and

s. Advertise, wherein an advertisement is rendered on a display, and

t. Order, wherein a beverage or food order selected on a display is processed and communicated to a vendor, vending machine, refilling station, or dispenser along with payment for said order, and

u. Gamble and Game, wherein said computer system is used to play games, promotional games of chance, lotteries or the like, and

v. Segmented Display, wherein said computer system displays an image across a multitude of displays, and

w. Authenticate, wherein said computer system provides access to a particular user or usage of information on said computer system.

10. A method for ordering beverages or food items from an interactive display disposed on an electronic food or beverage container wherein said beverages or food items are selected from a list provided by a vendor, by past a history of orders from the user, a history of orders received by a vendor, favorite orders by friends, or by celebrity favorites of said user, and said list being optionally made available to said display through some online social network.

11. The method of claim 10, wherein ordering comprises selecting specific recipes or mixes of ingredients.

12. A method for obtaining information on the product offerings, pricing or location of the nearest food or drink vendor, or vending machine, comprising the step of connecting to a user interface disposed on an electronic food or beverage container.

13. A method for paying or pre-paying a beverage or food order through an online system comprising the step of accessing a user interface disposed on an electronic food or beverage container.

14. A method for delivering promotional materials from a vendor or vending machine to a customer'"'"'s interactive food or beverage container comprising the following steps:

a. Optionally, identifying said container by said vendor or vending machine through said container being within threshold distance of said vendor or vending machine, and

b. Optionally, identifying said container by said customer contacting said vendor or vending machine through a user interface disposed on said container, and

c. Optionally, identifying said container by said customer placing an order with said vendor or vending machine, and wherein

d. Said vendor or vending machine selecting said promotional materials on the basis of chance, characteristics of said customer'"'"'s history of orders;

or characteristics of said customer'"'"'s order; and

e. Digitally uploading said promotional materials to said container by a wireless or wired network, and

f. Displaying or playing on said container of said promotional materials.

15. The method of claim 14, wherein said promotional materials are selected from a group consisting of:

advertisements, ringtones, videos, music, lotteries, food or beverage menus, games of chance or skill, slogans; and

company art work.

16. A method for tracking nutritional or caloric value, carbon credits or carbon offsets, or volume of beverages or food items ordered or consumed through an electronic food or beverage container, wherein:

a. Optionally, said container identifies the nutritional or caloric value of beverages or foods consumed through said container by measuring said nutritional or caloric value through a input device disposed on said container; and

b. Optionally, the vendor of said beverages or food items communicates information about the nutritional value, caloric value, carbon credit or carbon offset through some wired or wireless communications with said container upon placement of an order of said container; and

wherein

c. Said container, or some associated online database, maintains information on the beverages or food item, and the volume of said beverages or food items consumed through said container.

17. A method wherein the level of food or drink contained in an electronic food or beverage container is an incentive in an electronic game rendered on said container'"'"'s display.

18. A method for purchasing an electronic travel, event or admission ticket comprising the steps of engaging a user interface disposed on an electronic food or beverage container, wherein upon enacting rights associated with said ticket, and wherein access to said rights is provided only after electronic verification of said ticket by electronic communications with said container.

19. A method for presenting an image or movie display across a set of displays disposed on multiple electronic food or beverage containers, wherein each individual container'"'"'s display serves to display only one or a few pixels of said image or movie display.