System and method for unifying hotspots subject to non-linear transformation and interpolation in heterogeneous media representations

• US 6,693,649 B1
• Filed: 05/12/2000
• Issued: 02/17/2004
• Est. Priority Date: 05/27/1999
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.), Hybritech Polymers
• Original Assignee: International Business Machines Corporation
• Inventors: Zhang, Liang-Jie, Snible, Edward C., Menon, Jai, Lipscomb, James S., Kumar, Keeranoor G., Chen, Shu-Chun Jeane, Chang, Sih-Pin Subrina, Ramchandra, Arun
• Primary Examiner(s): NGUYEN, CAO H
• Application Number:
  US09/569,875
• Time in Patent Office: 1,376 Days
• Field of Search: 345/762, 345/803, 345/853, 345/708, 345/854, 345/978
• US Class Current: 715/762
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

A system and method for unifying hotspot subject to non-linear transformation and interpolation in heterogeneous media representations includes a content creator for creating Rich Media, e.g. stills, animation, panorama and a delivery system including an action handler for responding to a hotspot in the Rich Media when selected by a user interacting with a projected image display. The hotspot is expressed in the media as an area defined in terms of a Euclidean, rectangular coordinate system for transformation and mapping of the media image hotspot coordinates into corresponding coordinates of a hotspot location in the projected image display. Unfortunately, the image coordinates in the media do not necessarily map directly into corresponding coordinates in the projected image. Flat media images can be linearly transformed to projected image coordinates. However, panorama media require non-linear transformation of media coordinated to projected image coordinates. The coordinates for the media hotspot are stored in a centralized action handler independent of the non-linearities in the media. As a result of a user selecting or picking a hotspot in the projected image, the pick may fall outside of the corresponding hotspot in the case of non-linear media. The hotspot in the media and the projected image may be unified by increasing the number of points in the area geometry which makes more reliable that a pick in the projected image hotspot will fall within the hotspot area in the media. When the projected image hotspot is picked by a mouse or other input device, a signal is transmitted to the media indicating the coordinates of the input device with respect to the media. The media transforms the input coordinates and queries the action handler for a stored hotspot at the location of the input device. The action handler returns the hotspot, if any at the queried location The media displays the hotspot, if any at the location and the input device location on the media. When the pick falls within the media hotspot area and is actuated, the action handler initiates events associated with the picked hotspot in the projected image.

Claims

1. A system for unifying an action-initiating area subject to non-linear transformation and interpolation in heterogeneous media representations, comprising:

a) means for storing linear and non-linear media;

b) means for defining an action-initiating area in the media in terms of action-initiating area coordinates;

c) means for transforming and mapping media action-initiating area coordinates into corresponding coordinates of an action-initiating area for display in a projected image; and

d) means for unifying the action-initiating area in the media and the projected image regardless of media type.

2. The system of claim 1 further comprising:

e) means for holding the projected image in the display.

3. The system of claim 2 further comprising:

f) means for selecting an action-initiating area in the projected image and signaling the media of the selection.

4. The system of claim 3 further comprising:

g) means responsive to the selection to determine if the selected image action-initiating area is within the coordinates of the corresponding media action-initiating area.

5. The system of claim 4 further comprising:

h) means for initiating an event associated with the media action-initiating area if the selected image action-initiating area is within the coordinates of the corresponding media action-initiating area.

6. In a system including media containing action-initiating areas for linking to other media, a display for displaying a projected image in the media containing an action-initiating area, an input device for signaling a selection of an action-initiating area in the projected image and a processor for converting coordinates of the media action-initiating area into corresponding coordinates of the action-initiating area in the projected image, a method for unifying an action-initiating area subject to non-linear transformation and interpolation in heterogeneous media representations, comprising the steps of:

a) defining the action-initiating area in the media in terms of action-initiating area coordinates;

b) storing linear and non-linear media and an action-initiating area within the media,

c) transforming and mapping media action-initiating area coordinates into corresponding coordinates of an action-initiating area in a projected image of a display;

d) unifying the action-initiating area in the media and the projected image;

e) holding the projected image in the display; and

f) selecting an action-initiating area in the projected image and signaling the media of the selection.

7. The method of claim 6 further comprising the step of:

g) responding to the selection to determine if the selected image action-initiating area is within the coordinates of the corresponding media action-initiating area.

8. The method of claim 7 further comprising the step of:

h) initiating an event associated with the media action-initiating area if the selected image action-initiating area is within the coordinates of the corresponding media action-initiating area.

9. In an information system including an action enabler and an action handler for transferring media programs to a projected image display, a method for storing and handling action-initiating areas in the media, comprising the steps of:

a) collecting values from an action-initiating area location in the media programs stored in the information system in a first coordinate system;

b) transforming the values to an action-initiating area coordinate system;

c) querying an action enabler for an action-initiating area at the location in the projected image display;

d) returning the action-initiating area at the query location; and

e) displaying the action-initiating area in the media in the action-initiating area coordinate system.

10. The method of claim 9 wherein action-initiating area coordinate system characteristics are known to the media programs.

11. The method of claim 10 wherein the action-initiating areas are stored in an action handler, which communicates action-initiating area information between itself and the media programs.

12. The method of claim 10 wherein the action-initiating area coordinate system is a Euclidean, rectangular coordinate system.

13. The method of claim 9 further comprising the step of:

f) picking in the projected image display and converting to the action-initiating area coordinate system by the media programs before sending queries to the action handler; and

g) reporting action-initiating area information back to the media programs in the action-initiating area coordinate system by the action handler.

14. The method of claim 13 wherein one or more events or semantic actions on encountering or picking the action-initiating area are decoupled from the media.

15. The method of claim 14 wherein the media programs request for the action handler to perform events on the prior communication with the action handler.

16. The method of claim 13 wherein picking in the projected image display is performed as a point-in-polygon test in the action-initiating area coordinate system.

17. The method of claim 13 wherein picks are made as point-in-polygon tests both at a current position of an input device and a position of the input device in wrapped space off the end of media space.

18. The method of claim 9 wherein the media supply callback functions to do the transformations which enables other programs to convert to and from the action-initiating area coordinate system.

19. The method of claim 9 wherein the action-initiating area coordinate system is not the same as media space, media space being the space in which the media are stored and from which media are transformed to the projected image, a action-initiating area being attached to the media space if the media space is the same as the action-initiating area coordinate system.

20. The method of claim 9 wherein the media draw and process a action-initiating area coordinate system that is not in media space.

21. The method of claim 9 wherein shapes in a projected image display are approximated by shapes in the action-initiating area coordinate system.

22. The method of claim 21 wherein rectangles or other polygons in the projected image are approximated by other polygons, possibly with more vertices in order to make a bilinear approximation of straight lines with the number of vertices allowed to be determined by the amount of non-linearity at that place in the media.

23. The method of claim 21 wherein shapes in the projected image display are approximated by other shapes in the action-initiating area coordinate system.

24. The method of claim 21 wherein the intersection of shapes in the projected image display is detected as shape intersections in the action-initiating area coordinate system.

25. The method of claim 21 wherein the action-initiating areas that cover singularities in media space are converted to complex shapes in the action-initiating area coordinate system so as to avoid a singularity in the action-initiating area coordinate system.

26. The method of claim 21 wherein the action-initiating area in a region of wraparound in media space is represented as a shape that extends past one end of the unwrapped media space.

27. The method of claim 26 wherein the action-initiating area that overlaps the wraparound in media space and each other are all pushed off the end of media space, including those that do not overlap the wraparound.

28. A method of generating and processing an action-initiating area subject to non-linear transformation associated with media representations, comprising the steps of:

a) obtaining non-linear media;

b) defining an action-initiating area in the non-linear media in terms of action-initiating area coordinates;

c) transforming and mapping media action-initiating area coordinates into corresponding coordinates of an action-initiating area for display in a projected image; and

d) unifying the action-initiating area in the media and the projected image regardless of media type.

29. The method of claim 28, wherein the action-initiating area comprises a hotspot.