Direct manipulation of displayed content

• US 6,507,349 B1
• Filed: 01/06/2000
• Issued: 01/14/2003
• Est. Priority Date: 01/06/2000
• Status: Expired due to Term

• Current Assignee: Implicit, LLC (Edward Balassanian)
• Original Assignee: BeComm Corporation
• Inventors: Balassanian, Edward
• Primary Examiner(s): Kincaid, Kristine
• Assistant Examiner(s): Joseph, Thomas J
• Application Number:
  US09/478,920
• Time in Patent Office: 1,104 Days
• Field of Search: 345/676,681,711-713,762-765,847,856,851-855,840,841,825,818-820,810-813,440.1 702/66-68,71,124
• US Class Current: 345/676
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop

Abstract

The disclosed information describes a method, system, and computer-readable medium for performing direct manipulation of displayed content (e.g., dragging the displayed content in a particular direction or dragging a particular part of the displayed content). In particular, content is displayed to a user, and a variety of direct manipulations of the displayed content are provided to allow the user to modify display of the content without the use of separate displayed content manipulation controls. The disclosed direct manipulation techniques are used to modify the display of content by altering the value of a content properties affecting the display. Such properties can include an amplitude affecting the size or level with which the information is presented (e.g., a volume level at which audio information is played, or a size at which visual information is displayed), an information history property that refers to a series of distinct groups of information that are related to each other over time, and a channel property that can specify one of multiple alternate groups of information that can be displayed. When the user selects the displayed content and performs a direct manipulation of the content, the value of a corresponding property is modified accordingly, and the display of the content is updated to reflect the new value of the property. Also, the user can map what direct manipulations correspond to what properties.

Claims

1. A computer-implemented method for controlling display of information based on direct manipulation of the displayed information without use of separate manipulation controls, the information having multiple channels each representing an alternative stream of groups of information available for display, having a history including a series of previously displayed groups of information, and having multiple amplitudes at which the information can be displayed, the method comprising:

receiving an indication of the information to be displayed;

determining a current channel of the information, a current group of information from the stream associated with the current channel, and a current amplitude;

displaying the current group of the information from the current channel at the current amplitude; and

for each of a series of received indications that a user has dragged the displayed information,

when the user has dragged the displayed information in a predefined direction associated with the channels, displaying a new group of information from the stream associated with a new channel, the new channel selected based on the dragging;

when the user has directly manipulated the displayed information in a predefined manner associated with the amplitudes, displaying the current group of information at a new amplitude, the new amplitude selected based on the direct manipulation; and

when the user has dragged the displayed information in a predefined direction associated with the history, displaying a new group of information from the series of previously displayed groups of information, the new group selected based on the dragging,

so that the user can directly manipulate the display of information without the use of separate manipulation controls.

2. The method of claim 1 wherein a distance of the dragging of the displayed information determines the selection of the new channel or new group.

3. The method of claim 2 wherein the selection of the new channel, new amplitude, or new group is made in a non-linear manner with respect to the distance of the dragging.

4. The method of claim 3 wherein the non-linear manner is an inertia-based manner.

5. The method of claim 1 wherein an indication of a distinct output device is displayed, and including:

receiving an indication that the user has dragged the displayed information to the displayed indication of the output device; and

in response, displaying the displayed information on the output device.

6. The method of claim 1 wherein when a previously displayed group of information is selected from the series of previously displayed groups of information to be the current group of information, the groups of information that are later in the series are removed from the history.

7. The method of claim 1 wherein the predefined direction of the dragging that is associated with the history is a lateral direction.

8. The method of claim 1 wherein the predefined direction of the dragging that is associated with the channels is a vertical direction.

9. The method of claim 1 wherein the predefined manner of direct manipulation of the displayed information that is associated with the amplitude involves modifying a size of a window in which the information is displayed.

10. The method of claim 1 wherein each channel is a video stream consisting of a series of images and wherein the amplitude at which the information is displayed corresponds to a size of the displayed images for the current channel.

11. The method of claim 1 wherein each channel is an audio stream and wherein the amplitude at which the information is displayed corresponds to an audio level at which the audio information is played.

12. The method of claim 1 wherein the information is displayed inside a displayed container, and including:

receiving an indication that the user has dragged a portion of the container; and

modifying a visual representation of the container based on the dragging.

13. The method of claim 12 wherein the container is a window.

14. The method of claim 12 wherein the dragging of the container modifies the size of the container, and including displaying the displayed content in the modified container at a new amplitude corresponding to the modified size.

15. The method of claim 12 wherein the dragging of the container modifies the size of the container such that the modified size is below a threshold, and including closing the container based on the modifying of the size.

16. The method of claim 12 wherein the displayed container does not includes any displayed content manipulation controls.

17. The method of claim 12 wherein the displayed container includes a displayed control for shrinking the displayed container to a diminished form, wherein the displayed control is the portion of the container that is dragged by the user, and wherein the modified visual representation of the container is the diminished form.

18. The method of claim 17 wherein the dragging of the displayed control is a short tug in a direction opposite to a direction in which the displayed container is shrunk.

19. The method of claim 1 wherein a separate content control is displayed, and including:

receiving an indication that the user has dragged the separate content control; and

modifying the displayed information based on the dragging.

20. The method of claim 18 including, before the receiving of the indication that the user has dragged the separate content control, receiving an indication that the separate content control is associated with the displayed information.

21. The method of claim 1 wherein the indicated information that is displayed is of a first type, and including:

displaying a current group of information from a current channel of distinct information at a current amplitude, the distinct information of a second type and having multiple channels, a history, and multiple amplitudes; and

for each of a series of received indications that a user has dragged the displayed distinct information,

when the user has dragged the displayed distinct information in a predefined direction associated with the channels of the distinct information, displaying a new group of information from the stream associated with a new channel, the new channel selected based on the dragging;

when the user has directly manipulated the displayed distinct information in a predefined manner associated with the amplitudes of the distinct information, displaying the current group of information at a new amplitude, the new amplitude selected based on the direct manipulation; and

when the user has dragged the displayed distinct information in a predefined direction associated with the history of the distinct information, displaying a new group of information from the history, the new group selected based on the dragging.

22. The method of claim 21 wherein the first type is audio information and wherein the second type is video information.

23. The method of claim 21 wherein the predefined dragging direction associated with the channels of the indicated information is the same as the predefined dragging direction associated with the channels of the distinct information, wherein the predefined dragging direction associated with the history of the indicated information is the same as the predefined dragging direction associated with the history of the distinct information, and wherein the predefined direct manipulation manner associated with the amplitudes of the indicated information is the same as the predefined direct manipulation manner associated with the amplitudes of the distinct information,

so that a single metaphor is used for the direct manipulation of the different types of information.

24. The method of claim 21 wherein the predefined dragging direction associated with the channels of the indicated information is distinct from the predefined dragging direction associated with the channels of the distinct information, wherein the predefined dragging direction associated with the history of the indicated information is distinct from the predefined dragging direction associated with the history of the distinct information, and wherein the predefined direct manipulation manner associated with the amplitudes of the indicated information is distinct from the predefined direct manipulation manner associated with the amplitudes of the distinct information,

such that each distinct collection of information can have different direct manipulations of the displayed information associated with different types of modifications of the displayed information.

25. The method of claim 24 including, before the displaying of the current group of information from the current channel of the distinct information at the current amplitude, receiving an indication of mappings for dragging directions and direct manipulation manners to properties of the distinct information, the mappings including a first dragging direction that is associated with the channels property of the distinct information, a second dragging direction that is associated with the history property of the distinct information, and a first direct manipulation manner that is associated with the amplitudes property of the distinct information, the included first and second dragging directions and first direct manipulation manner all distinct.

26. The method of claim 25 wherein the mappings are specified by displaying indications of multiple dragging directions and direct manipulation manners to a user, displaying indications of multiple display properties of the distinct information to the user, and receiving indications from the user of mappings of the dragging directions and direct manipulation manners to the display properties.

27. The method of claim 1 wherein the direct manipulation of the displayed information in the predefined manner associated with the amplitudes includes dragging a predefined portion of the displayed information.

28. The method of claim 27 wherein dragging of a portion of the displayed information other than the predefined portion causes a display property of the displayed information other than the amplitudes to be modified.

29. The method of claim 1 wherein the direct manipulation of the displayed information in the predefined manner includes a gesture of the user representing a throwing of the displayed information.

30. The method of claim 1 wherein the direct manipulation of the displayed information in the predefined manner includes a gesture of the user representing a pushing of the displayed information.

31. The method of claim 1 wherein the direct manipulation of the displayed information in the predefined manner includes a gesture of the user representing a tugging of the displayed information.

32. The method of claim 1 wherein the direct manipulation of the displayed information in the predefined manner includes selection by the user of a key on the keyboard during a dragging of the displayed information.

33. The method of claim 1 wherein a visual appearance of a displayed cursor is changed during the dragging.

34. The method of claim 33 wherein the visual appearance of the cursor is specific to the dragging direction or manner.

35. The method of claim 33 wherein the visual appearance of the cursor is specific to the display property of the displayed information that is to be modified by the dragging.

36. The method of claim 1 wherein indications of multiple distinct resources are displayed, at least some of the distinct resources being remote, and including:

receiving an indication that the user has dragged the displayed information to the displayed indication of one of the displayed resources; and

in response, transmitting the displayed information to the one resource.

37. The method of claim 36 wherein the one resource is a storage device, and wherein the transmitting of the displayed information to the one resource includes storing the displayed information on the one resource.

38. The method of claim 36 wherein the one resource is an executing program, and wherein the transmitting of the displayed information to the one resource includes providing the displayed information to the executing program as input.

39. The method of claim 36 wherein the displayed information is of a type which the one resource is unable to process, and wherein the transmitting of the displayed information to the one resource includes converting the displayed information to a type which the one resource is able to process.

40. A computer-readable medium containing instructions that when executed on a computer controls display of information based on direct manipulation of the displayed information without use of separate manipulation controls, the information having multiple channels each representing an alternative stream of groups of information available for display, having a history including a series of previously displayed groups of information, and having multiple amplitudes at which the information can be displayed, by:

receiving an indication of the information to be displayed;

determining a current channel of the information, a current group of information from the stream associated with the current channel, and a current amplitude;

displaying the current group of the information from the current channel at the current amplitude; and

for each of a series of received indications that a user has directly manipulated the displayed information,

when the user has directly manipulated the displayed information in a predefined manner associated with the channels, displaying a new group of information from the stream associated with a new channel, the new channel selected based on the direct manipulation;

when the user has directly manipulated the displayed information in a predefined manner associated with the amplitudes, displaying the current group of information at a new amplitude, the new amplitude selected based on the direct manipulation; and

when the user has directly manipulated the displayed information in a predefined manner associated with the history, displaying a new group of information from the series of previously displayed groups of information, the new group selected based on the direct manipulation, so that the user can directly manipulate the display of information without the use of separate manipulation controls.

41. The computer-readable medium of claim 40 wherein the computer-readable medium is a transmission medium on which is transmitted a generated data signal containing the instructions.

42. A computing device for controlling display of information based on direct manipulation of the displayed information without use of separate manipulation controls, the information having multiple channels each representing an alternative stream of groups of information available for display, having a history including a series of previously displayed groups of information, and having multiple amplitudes at which the information can be displayed, comprising:

a first component capable of receiving an indication of the information to be displayed, of determining a current channel of the information, a current group of information from the stream associated with the current channel, and a current amplitude, and of displaying the current group of the information from the current channel at the current amplitude; and

a second component capable of, for each of a series of received indications that a user has directly manipulated the displayed information, displaying a new group of information from the stream associated with a new channel that is selected based on the direct manipulation if the direct manipulation is in a predefined manner associated with the channels, displaying the current group of information at a new amplitude that is selected based on the direct manipulation if the direct manipulation is in a predefined manner associated with the amplitudes, and displaying a new group of information from the series of previously displayed groups of information that is selected based on the direct manipulation if the direct manipulation is in a predefined manner associated with the history,

so that the user can directly manipulate the display of information without the use of separate manipulation controls.