INTERACTION TECHNIQUES FOR FLEXIBLE DISPLAYS

• US 20130127748A1
• Filed: 08/20/2012
• Published: 05/23/2013
• Est. Priority Date: 03/30/2006
• Status: Abandoned Application

• Current Assignee: Justin Lee, Pichaya Puttorngul, Roel Vertegaal, Yves Behar
• Original Assignee: Justin Lee, Pichaya Puttorngul, Roel Vertegaal, Yves Behar
• Inventors: VERTEGAAL, Roel, LEE, Justin, BEHAR, Yves, PUTTORNGUL, Pichaya
• Application Number:
  US13/589,732
• Publication Number:
  US 20130127748A1
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: A47G 19/2227 with means for amusing or giving information to the user A47G23/16 takes precedence
• CPC Class Codes: A47G 19/2272 from drinking glasses or cups comprising lids or covers
• CPC Class Codes: A47G 2019/2238 with illumination means
• CPC Class Codes: A47G 2019/2244 with sound emitting means
• CPC Class Codes: A47G 2019/225 vessels with thermometers
• CPC Class Codes: G02F 1/133305 Flexible substrates, e.g. plastics, organic film
• CPC Class Codes: G02F 1/13338 Input devices, e.g. touch panels
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1643 the display being associated to a digitizer, e.g. laptops that can be used as penpads details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615
• CPC Class Codes: G06F 1/1652 the display being flexible, e.g. mimicking a sheet of paper, or rollable
• CPC Class Codes: G06F 1/1656 Details related to functional adaptations of the enclosure, e.g. to provide protection against EMI, shock, water, or to host detachable peripherals like a mouse or removable expansions units like PCMCIA cards, or to provide access to internal components for maintenance or to removable storage supports like CDs or DVDs, or to mechanically mount accessories mounting of accessories to a computer display G06F1/1607; display hoods G06F1/1603; cooling arrangements for portable computers G06F1/203
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 2203/04102 Flexible digitiser, i.e. constructional details for allowing the whole digitising part of a device to be flexed or rolled like a sheet of paper
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 : using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 : Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 : Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0425 : using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 : Scrolling or panning
• CPC Class Codes: G06F 3/0487 : using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/147 : using display panels
• CPC Class Codes: G06Q 10/02 : Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 30/0209 : Incentive being awarded or redeemed in connection with the playing of a video game
• CPC Class Codes: G06Q 50/12 : Hotels or restaurants
• CPC Class Codes: G09G 2380/02 : Flexible displays

Abstract

The invention relates to a set of interaction techniques for obtaining input to a computer system based on methods and apparatus for detecting properties of the shape, location and orientation of flexible display surfaces, as determined through manual or gestural interactions of a user with said display surfaces. Such input may be used to alter graphical content and functionality displayed on said surfaces or some other display or computing system. The invention also relates to interactive food or beverage container with associated computing apparatus inside its body, and a curved multitouch display on its surface, associated interaction techniques for curved multitouch displays, methods of use, and apparatus for refilling said electronic food or beverage container.

Claims

1. A reusable portable interactive apparatus, comprising:

a container portion;

a removable lid for the container portion;

at least one interactive device that receives input from a user and outputs information to a user, and optionally communicates with another device;

wherein the at least one interactive device includes a curved display surface disposed on the container portion, the curved display surface comprising a technology selected from the group consisting of;

flexible e-ink, flexible organic light emitting diode, flexible light emitting diode, projection, laser, and paintable display;

a base portion; and

a computing device disposed in the base portion and associated with the curved display, the computing device comprising one or more members selected from the group consisting of;

battery, power connector, network connector, audiovisual connector, central processing unit, wireless network transceiver, graphics circuit board, RAM memory, firmware ROM, flash memory, and hard disk drive;

wherein user input is provided by touching the curved display surface and/or by gestural interactions with the apparatus.

2. The apparatus of claim 1, further comprising a device selected from the group consisting of:

accelerometer, gyroscope, bend sensor, touch screen, capacitive touch sensor, heart rate sensor, galvanic skin conductor sensor, alpha dial potentiometer, video camera, still camera, hygrometer;

liquid level sensor;

potentiometric liquid chemical sensor, altimeter, thermometer, force sensor;

pressure sensor;

microphone, GPS, photoelectric sensor;

proximity sensor, electronic payment system, RFID tag, fingerprint reader, water purification system, carbon filtration system, chemical or organic content analyzer, bacterial content analyzer, amplification system, speaker system, compass, and a combination thereof.

4. The apparatus of claim 1, wherein the container is adapted to contain a beverage and to allow a user to consume the beverage.

5. The apparatus of claim 1, wherein the container is adapted to contain solid or semi-solid food items and to allow a user to consume the food items.

6. The apparatus of claim 1, wherein the interactive device senses manual interactions of the user with the apparatus, wherein said interactions are selected from the group consisting of:

a. Holding, wherein holding the curved display surface with one or two hands serves as input to the computing device associated with said curved display;

b. Collocating or stacking, wherein collocating, collating or stacking multiple curved displays creates a single contiguous display surface consisting of individual displays, and wherein subsequent inputs operate on said larger display surface;

c. Turning or Rotating, wherein rotating said curved display around an axis serves as input to the computing device associated with said display;

d. Swirling, wherein moving said curved display in around an axis that is non-concentrical but parallel to some axis of said curved display serves as a means of input to the computing device associated with said curved display;

e. Non-planar Strip Swiping, wherein moving one or more fingers along the top or bottom extremities of a curved display, or just above or below said display, serves as input to the computing device associated with said display;

f. Three-finger Non-planar Pinching, wherein placing three fingers within a threshold proximity on a curved display serves as input to the computing device associated with said curved display;

g. Pinning and Swiping, wherein placing one finger on a fixed location on a curved display, while subsequently placing a second finger on said display, and wherein said second finger is subsequently moved away from said first finger, serves as input to the computing device associated with said display;

h. Deforming, wherein deforming a curved display at one location serves as input to the computing device associated with said display;

i. Rubbing, wherein providing a rubbing action on a curved display, in which the hand, finger, or some tool is moved in a sinusoidal pattern over its surface, serves as input to the computing device associated with said display;

j. Tilting, wherein tilting a curved display serves as input to the computer system computing device associated with said display;

k. Flicking or Tossing, wherein rapidly tilting a curved display, stopping and optionally returning to its approximate original orientation serves as input to the computing device associated with said display;

l. Resting, wherein placing and releasing an electronic food or beverage container on a surface, serves as input to the computing device associated with said container;

m. Drinking, Filling and Fluid Level, wherein an action selected from a group consisting of;

bringing an electronic food or beverage container to the mouth;

drinking a beverage from said container;

or filling said container serves as input to the computing device associated with said container;

n. Opening and closing, wherein opening and closing the lid of an electronic food or beverage container serves as input to the computing device associated with said container;

o. Multi-device Pouring, wherein holding an electronic food or beverage container over a second said container, and subsequently tilting said first container, serves as input to the computing device associated with either or both containers;

p. Fingerprint scanning, wherein placing one or more fingers of a user on a designated part of a curved display surface causes associated fingerprints to be analyzed with the purpose of authenticating access by said user to information on said curved display surface; and

q. Face detection, wherein the face of a user is identified by an electronic food or beverage container for the purpose of authenticating access of said the user to information on said container.

8. The apparatus of claim 6, wherein said input to said computing device causes a command to execute on said computing device and wherein said command is selected from the group consisting of:

a. Activate, wherein the software and display of said computer system awakes from sleep, disabling a screen saver or energy reduce state, or enabling advertisement activity;

b. Deactivate, wherein the software and display of said computer goes to sleep, enabling a screen saver or energy reduced state, or disabling advertisement activity;

c. Zoom in or Enlarge, wherein an image or content of a file or document rendered on said display is enlarged or zoomed in on;

d. Zoom out or Reduce, wherein an image or content of a file or document rendered on said display is reduced or zoomed out of;

e. Organize, wherein a property of file(s), digital information, text, images, or other computer content associated with or displaying on said display surface(s) is organized or sorted digitally in a way that matches properties of the physical computer system, such as physical order;

f. Scroll, wherein a segment of an image or content of a file, document or application is rendered on a display, said segment being not previously rendered, and said segment being spatially contiguous to the segment of said image or content that was previously rendered on said display;

g. Page Down, wherein a segment of the content of a file subsequent to the section of said content of a file that is currently rendered on a display, is navigated to such that it causes said subsequent section to be rendered on said display;

h. Page Up, wherein a segment of the content of a file that precedes the section of said content of a file that is currently rendered on a display, is navigated to such that it causes said preceding section to be rendered on said display;

i. Navigate, wherein an arbitrary section of the content of a file on said computer system, or online content, hyperlink, or menu is navigated to such that it causes said the associated content to render on a display;

j. Page Back or Forward, wherein a section of the content of a file, or online content, webpage or hyperlink that precedes or follows the section of said content currently rendered on a display, is navigated to such that it causes said content to be rendered on said display;

k. Open, Save or Close, wherein a file or digital information on said computing device is opened or closed, read into memory, or out to a permanent storage medium;

l. Move, Copy or Paste, wherein a section of the content of a file, image, text or other digital information associated with said computing device or display is transferred to another computer system or display, or a different logical location on said same computing device or display;

m. Select, where a graphical object rendered on a display is selected such that it becomes the recipient of a subsequent action, input or command to the associated computer system;

n. Click, wherein an insertion point or cursor is moved to a specific location on a display, selecting or activating graphical objects underlying said location on said display;

o. Erase, wherein selected information or images, or content associated with said images on a computer system, is erased from said display and/or from the memory of said computer system;

p. Playback control, wherein a multimedia file, including graphics animation, video, sound or musical content on said computing device, is played at a speed, and wherein said speed is optionally controlled by said input;

q. Connect, wherein said computing device is connected through a computer network to another computer system, online server, communication tool or social networking site;

r. Share, wherein information on said computing device is placed on a computer server for the purpose of sharing said information with other users connected to said server;

s. Online status, wherein information about the usage of said computing device by the user, or an arbitrary status or attribute of said user, is shared with a computer server for the purpose of sharing said information with other users connected to said server;

t. Communicate, wherein said computing device serves as a communication device;

u. Advertise, wherein an advertisement is rendered on a display;

v. Order, wherein a beverage or food order selected on a display is processed and communicated to a vendor, vending machine, refilling station, or dispenser along with payment for said order;

w. Gamble and Game, wherein said computing device is used to play games, promotional games of chance, lotteries or the like;

x. Segmented Display, wherein said computing device displays an image across a multitude of displays; and

y. Authenticate, wherein said computing device provides access to a particular user or usage of information on said computer system.

17. The apparatus of claim 1, further comprising at least one sensor disposed inside the container portion, wherein the at least one sensor senses a level of food or drink in the container portion, and the level is used as an incentive in an electronic game rendered on said the curved display.

19. The apparatus of claim 1, wherein the apparatus is adapted for presenting an image or movie across two or more curved displays disposed on two or more respective apparatuses, wherein each curved display serves to display only a portion of pixels of said image or movie;

such that the two or more curved displays together display substantially the complete image or movie.

20. A product refilling station for the apparatus of claim 1 wherein:

a. The product refilling station provides power or software communications to said container upon placement of said container within or on said product refilling station and wherein

b. the container communicates product orders made on said container to said product refilling station upon placement of said container within or on said product refilling station and wherein

c. Optionally, said product refilling station cleans said container and wherein

d. Optionally, said order consists of a recipe of ingredients, and wherein said order is fulfilled by mixing ingredients on site according to said recipe and wherein

e. the product refilling station fulfills said order by filling said container with said order and wherein

f. the product refilling station arranges payment for said order through communications with said container, and through some electronic payment system associated with said container or product refilling station.

21. The apparatus of claim 1, wherein the apparatus determines and/or tracks or maintains a database on one or more parameter of the contents of the container or an amount of the contents of the container.

22. The apparatus of claim 21, further comprising at least one sensor disposed inside the container portion, wherein the at least one sensor senses the one or more parameter of the contents of the container.

23. The apparatus of claim 22, wherein the at least one sensor comprises a liquid chemical sensor.

24. The apparatus of claim 22, wherein the one or more parameter is selected from the group consisting of nutritional value and caloric value.

25. The apparatus of claim 1, wherein the apparatus receives information about one or more parameter of the contents of the container from a vendor or provider of the contents,

wherein the one or more parameter is selected from the group consisting of nutritional value and caloric value.

3. (canceled)

7. (canceled)

9. (canceled)

10. A method for ordering beverages or food items from an interactive display disposed on an electronic food or beverage container wherein said beverages or food items are selected from a list provided by a vendor, by past a history of orders from the user, a history of orders received by a vendor, favorite orders by friends, or by celebrity favorites of said user, and said list being optionally made available to said display through some online social network.

11. The method of claim 10, wherein ordering comprises selecting specific recipes or mixes of ingredients.

12. A method for obtaining information on the product offerings, pricing or location of the nearest food or drink vendor, or vending machine, comprising the step of connecting to a user interface disposed on an electronic food or beverage container.

13. The method of claim 12, further comprising paying or pre-paying a beverage or food order through an online system by connecting to the user interface disposed on the electronic food or beverage container.

14. The method of claim 12, further comprising delivering promotional materials from a vendor or vending machine to a customer'"'"'s interactive food or beverage container comprising the following steps:

a. Optionally, identifying said container by said vendor or vending machine through said container being within threshold distance of said vendor or vending machine, and

b. Optionally, identifying said container by said customer contacting said vendor or vending machine through a user interface disposed on said container, and

c. Optionally, identifying said container by said customer placing an order with said vendor or vending machine, and wherein

d. Said vendor or vending machine selecting said promotional materials on the basis of chance, characteristics of said customer'"'"'s history of orders;

or characteristics of said customer'"'"'s order; and

e. Digitally uploading said promotional materials to said container by a wireless or wired network, and

f. Displaying or playing on said container of said promotional materials.

18. The method of claim 12, further comprising purchasing an electronic travel, event or admission ticket, comprising the steps of engaging the user interface disposed on the electronic food or beverage container, wherein upon enacting rights associated with said ticket, and wherein access to said rights is provided only after electronic verification of said ticket by electronic communications with said container.

15. (canceled)

16. (canceled)