Method and apparatus for electronically distributing motion panoramic images

• US 6,337,708 B1
• Filed: 04/21/2000
• Issued: 01/08/2002
• Est. Priority Date: 06/24/1996
• Status: Expired due to Term

• Current Assignee: Cedar Lane Technologies Inc.
• Original Assignee: Be Here Corporation
• Inventors: Furlan, John Louise Warpakovoski, Driscoll, Edward Jr.
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US09/556,496
• Time in Patent Office: 627 Days
• Field of Search: 348/36-61, 382/232-233
• US Class Current: 348/36
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The present invention discloses an electronic image distribution apparatus for distributing motion panoramic images. The main portion of the apparatus is a motion panoramic image server that has a source of motion panoramic images. The source of panoramic image frames may be a motion panoramic image playback device or a connection to a motion panoramic camera system. The motion panoramic image server transforms the panoramic image frames into an intermediate format that is more conducive for electronic transmission. The motion panoramic image server then transmits the motion panoramic images to client systems. A first panoramic image transmission system divides the intermediate planar representation into individual tiles and transmits tiles that are requested by client systems. A second panoramic image transmission system divides the intermediate planar representation into individual tiles and transmits the individual tiles on different multicast channels. Client systems then subscribe to the multicast channels needed to render an image in the direction of a desired view.

Claims

1. A computer controlled method for panning a view over a distributed electronic image comprising steps of:

creating said distributed electronic image at a server system by dividing an image into a plurality of tiles, said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view on a client system;

sending a selected tile from said plurality of tiles to said client system;

determining a portion of said selected tile to be rendered as said view;

rendering said view from said portion of said selected tile;

redetermining said portion of said selected tile to be rendered as said view responsive to a shift-of-view; and

repeating the steps of rendering and redetermining.

2. The computer controlled method of claim 1 wherein each of said plurality of tiles does not substantially overlap another of said plurality of tiles, said method further comprising steps of:

anticipating that said view will be rendered using information from both said selected tile and an adjacent tile responsive to said shift-of-view;

sending said adjacent tile to said client system; and

wherein the step of determining further includes a step of determining an adjacent portion of said adjacent tile, and the step of rendering said view includes rendering from said adjacent portion of said adjacent tile.

3. The computer controlled method of claim 1 wherein each of said plurality of tiles substantially overlaps at least one other of said plurality of tiles, said method further comprising steps of:

anticipating that said view can be rendered using information from either said selected tile or an adjacent tile responsive to said shift-of-view;

sending said adjacent tile to said client system; and

wherein the step of redetermining further includes steps of;

determining an overlapping portion of said adjacent tile that overlaps said portion;

designating said adjacent tile as said selected tile and said overlapping portion as said portion.

4. The computer controlled method of claim 1 wherein said distributed electronic image is a panoramic image frame.

5. The computer controlled method of claim 1 further comprising transforming said portion of said selected tile by said client system whereby said view is corrected.

6. A computer controlled method for panning a view over a distributed electronic image comprising steps of:

requesting a selected tile from one of a plurality of tiles representing an image, each of said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view, said plurality of tiles representing said distributed electronic image;

receiving said selected tile;

determining a portion of said selected tile to be rendered as said view;

rendering said view from said portion of said selected tile;

redetermining said portion of said selected tile to be rendered as said view responsive to a shift-of-view; and

repeating the steps of rendering and redetermining.

7. The computer controlled method of claim 6 wherein each of said plurality of tiles does not substantially overlap another of said plurality of tiles, said method further comprising steps of:

anticipating that said view will be rendered using information from both said selected tile and an adjacent tile responsive to said shift-of-view;

requesting said adjacent tile;

receiving said adjacent tile; and

wherein the step of determining further includes a step of determining an adjacent portion of said adjacent tile, and the step of rendering said view includes rendering from said adjacent portion of said adjacent tile.

8. The computer controlled method of claim 6 wherein each of said plurality of tiles substantially overlaps at least one other of said plurality of tiles, said method further comprising steps of:

anticipating that said view can be rendered using information from either said selected tile or an adjacent tile responsive to said shift-of-view;

requesting said adjacent tile;

receiving said adjacent tile; and

wherein the step of redetermining further includes steps of;

determining an overlapping portion of said adjacent tile that overlaps said portion;

designating said adjacent tile as said selected tile and said overlapping portion as said portion.

9. The computer controlled method of claim 6 wherein said distributed electronic image is a panoramic image frame.

10. The computer controlled method of claim 6 further comprising transforming said portion of said selected tile whereby said view is corrected.

11. The computer controlled method of claim 6 wherein said plurality of tiles are accessed from a computer readable media.

12. The computer controlled method of claim 6 wherein said plurality of tiles are accessed from a server system.

13. A computer controlled method for panning a view over a distributed electronic image comprising steps of:

creating, at a server system, said distributed electronic image by dividing an image into a plurality of tiles, said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view on a client system;

receiving, by said server system, a first tile request for a selected tile from said plurality of tiles; and

sending said selected tile to said client system.

14. The computer controlled method of claim 13 wherein each of said plurality of tiles does not substantially overlap another of said plurality of tiles.

15. The computer controlled method of claim 13 wherein each of said plurality of tiles substantially overlaps another of said plurality of tiles.

16. The computer controlled method of claim 13 wherein said distributed electronic image is a panoramic image frame.

17. A computer controlled system, having a server system in communication with a client system over a network, for panning a view over a distributed electronic image comprising:

a tiling mechanism configured to create said distributed electronic image at said server system by dividing an image into a plurality of tiles, said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view on said client system;

a sending mechanism, at said server system, configured to send a selected tile from said plurality of tiles to said client system responsive to a first tile request from said client system;

a first receiving mechanism, at said client system, configured to receive said selected tile from said sending mechanism;

a determination mechanism, at said client system, configured to determine a portion of said selected tile to be rendered as said view;

a rendering mechanism, at said client system, configured to render said view from said portion of said selected tile;

a panning mechanism, at said client system, configured to redetermine said portion of said selected tile to be rendered as said view responsive to a shift-of-view; and

an iterative mechanism configured to repetitively invoke the rendering mechanism and the panning mechanism responsive to said shift-of-view.

18. The computer controlled system of claim 17 wherein each of said plurality of tiles does not substantially overlap another of said plurality of tiles, said system further comprising:

a tile anticipation mechanism, at said client system, configured to anticipate that said view will be rendered using information from both said selected tile and an adjacent tile responsive to said shift-of-view;

a second receiving mechanism, at said client system, configured to receive said adjacent tile; and

wherein the determination mechanism further includes an adjacent determination mechanism configured to determine an adjacent portion of said adjacent tile, and the rendering mechanism further includes a second rendering mechanism configured to render some of said view from said adjacent portion of said adjacent tile.

19. The computer controlled system of claim 17 wherein each of said plurality of tiles substantially overlap at least one other of said plurality of tiles, said system further comprising:

a tile anticipation mechanism, at said client system, configured to anticipate that said view can be rendered using information from either said selected tile or an adjacent tile responsive to said shift-of-view;

a second receiving mechanism, at said client system, configured to receive said adjacent tile; and

wherein the determination mechanism further includes;

an overlap detection mechanism configured to determine whether an overlapping portion of said adjacent tile overlaps said portion; and

a selection mechanism configured to designate said adjacent tile as said selected tile and said overlapping portion as said portion responsive to the overlap detection mechanism.

20. The computer controlled system of claim 17 wherein said distributed electronic image is a panoramic image frame.

21. An apparatus having a central processing unit (CPU) and a memory coupled to said CPU for panning a view over a distributed electronic image comprising:

a tile selection mechanism configured to request a selected tile from one of a plurality of tiles representing an image, each of said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view, said plurality of tiles representing said distributed electronic image;

a first receiving mechanism configured to receive said selected tile;

a determination mechanism configured to determine a portion of said selected tile to be rendered as said view;

a rendering mechanism configured to render said view from said portion of said selected tile;

a panning mechanism configured to redetermine said portion of said selected tile to be rendered as said view responsive to a shift-of-view; and

an iterative mechanism configured to repetitively invoke the rendering mechanism and the panning mechanism responsive to said shift-of-view.

22. The apparatus of claim 21 wherein each of said plurality of tiles does not substantially overlap another of said plurality of tiles, said apparatus further comprising:

a tile anticipation mechanism configured to anticipate that said view will be rendered using information from both said selected tile and an adjacent tile responsive to said shift-of-view;

a second receiving mechanism configured to receive said adjacent tile;

wherein the determination mechanism further includes an adjacent determination mechanism configured to determine an adjacent portion of said adjacent tile, and wherein the rendering mechanism further includes a second rendering mechanism configured to render some of said view from said adjacent portion of said adjacent tile.

23. The apparatus of claim 21 wherein each of said plurality of tiles substantially overlaps at least one other of said plurality of tiles, said apparatus further comprising:

a tile anticipation mechanism configured to anticipate that said view can be rendered using information from either said selected tile or an adjacent tile responsive to said shift-of-view;

a second receiving mechanism configured to receive said adjacent tile;

wherein the determination mechanism further includes;

an overlap detection mechanism configured to determine whether an overlapping portion of said adjacent tile overlaps said portion; and

a selection mechanism configured to designate said adjacent tile as said selected tile and said overlapping portion as said portion responsive to the overlap detection mechanism.

24. The apparatus of claim 21 wherein said distributed electronic image is a panoramic image frame.

25. The apparatus of claim 21 wherein said distributed electronic image is accessed from a computer readable media.

26. The apparatus of claim 21 wherein said distributed electronic image is accessed from a server system in communication with said apparatus through a network.

27. An apparatus having a central processing unit (CPU) and a memory coupled to said CPU for panning a view over a distributed electronic image comprising:

a tiling mechanism configured to create said distributed electronic image by dividing an image into a plurality of tiles, said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view on a client system;

a network interface for communication with said client system;

a first receiving mechanism configured to receive a first tile request for a selected tile from said plurality of tiles over the network interface; and

a sending mechanism configured to send said selected tile over said network interface to said client system.

28. The apparatus of claim 27 wherein each said plurality of tiles does not substantially overlap another of said plurality of tiles.

29. The apparatus of claim 27 wherein each of said plurality of tiles substantially overlaps at least one other of said plurality of tiles.

30. The apparatus of claim 27 wherein said distributed electronic image is a panoramic image frame.

31. A computer program product including:

a computer usable storage medium having computer readable code embodied therein for causing a computer to pan a view over a distributed electronic image, said computer readable code comprising;

computer readable program code configured to cause said computer to effect a tile request mechanism configured to request a selected tile from one of a plurality of tiles representing an image, each of said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view, said plurality of tiles representing said distributed electronic image;

computer readable program code configured to cause said computer to effect a first receiving mechanism configured to receive said selected tile;

computer readable program code configured to cause said computer to effect a determination mechanism configured to determine a portion of said selected tile to be rendered as said view;

computer readable program code configured to cause said computer to effect a rendering mechanism configured to render said view from said portion of said selected tile;

computer readable program code configured to cause said computer to effect a panning mechanism configured to redetermine said portion of said selected tile to be rendered as said view responsive to a shift-of-view; and

computer readable program code configured to cause said computer to effect an iterative mechanism configured to repetitively invoke the rendering mechanism and the panning mechanism responsive to said shift-of-view.

32. The computer program product of claim 31 wherein each of said plurality of tiles does not substantially overlap another of said plurality of tiles, said product further comprising:

computer readable program code configured to cause said computer to effect a tile anticipation mechanism configured to anticipate that said view will be rendered using information from both said selected tile and an adjacent tile responsive to said shift-of-view;

computer readable program code configured to cause said computer to effect a second receiving mechanism configured to receive said adjacent tile;

wherein the determination mechanism further includes an adjacent determination mechanism configured to determine an adjacent portion of said adjacent tile, and wherein the rendering mechanism further includes a second rendering mechanism configured to render some of said view from said adjacent portion of said adjacent tile.

33. The computer program product of claim 31 wherein each of said plurality of tiles substantially overlaps at least one other of said plurality of tiles, said product further comprising:

computer readable program code configured to cause said computer to effect a tile anticipation mechanism configured to anticipate that said view can be rendered using information from either said selected tile or an adjacent tile responsive to said shift-of-view;

computer readable program code configured to cause said computer to effect a second receiving mechanism configured to receive said adjacent tile;

wherein the determination mechanism further includes;

computer readable program code configured to cause said computer to effect an overlap detection mechanism configured to determine whether an overlapping portion of said adjacent tile overlaps said portion; and

computer readable program code configured to cause said computer to effect a selection mechanism configured to designate said adjacent tile as said selected tile and said overlapping portion as said portion responsive to the overlap detection mechanism.

34. The computer program product of claim 31 wherein said distributed electronic image is a panoramic image frame.

35. The computer program product of claim 31 wherein said distributed electronic image is accessed from a computer readable media.

36. The computer program product of claim 31 wherein said distributed electronic image is accessed from a server system in communication with said product through a network.

37. A computer program product including:

a computer usable storage medium having computer readable code embodied therein for causing a computer to pan a view over a distributed electronic image, said computer readable code comprising;

computer readable program code configured to cause said computer to effect a tiling mechanism configured to create said distributed electronic image by dividing an image into a plurality of tiles, said plurality of tiles being spatially variant and wherein each of said plurality of tiles contains more information than required to render said view on a client system;

computer readable program code configured to cause said computer to effect a network interface for communication with said client system;

computer readable program code configured to cause said computer to effect a first receiving mechanism configured to receive a first tile request for a selected tile from said plurality of tiles over the network interface; and

computer readable program code configured to cause said computer to effect a sending mechanism configured to send said selected tile over said network interface to said client system.

38. The computer program product of claim 37 wherein each of said plurality of tiles does not substantially overlap another of said plurality of tiles.

39. The computer program product of claim 37 wherein each of said plurality of tiles substantially overlaps at least one of said plurality of tiles.

40. The computer program product of claim 37 wherein said distributed electronic image is a panoramic image frame.