Panoramic viewing system with a composite field of view

• US 6,700,711 B2
• Filed: 03/08/2002
• Issued: 03/02/2004
• Est. Priority Date: 11/30/1995
• Status: Expired due to Term

• Current Assignee: FullView, Inc.
• Original Assignee: FullView, Inc.
• Inventors: Nalwa, Vishvjit Singh
• Primary Examiner(s): NGUYEN, THONG Q
• Application Number:
  US10/093,568
• Publication Number:
  US 20020089765A1
• Time in Patent Office: 725 Days
• Field of Search: 359/362-363, 359/367, 359/399, 359/402-403, 359/725-732, 359/831-837, 348/36-39, 353/30, 353/94, 353/98-99, 352/69-71
• US Class Current: 359/725
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 27/143 using macroscopically faceted or segmented reflective surfaces
• CPC Class Codes: G02B 5/04 Prisms
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/74 Projection arrangements for image reproduction, e.g. using eidophor
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Cameras are positioned so that at least a portion of the field of view of at least one camera is redirected by a reflective surface. Images from these cameras are merged to produce a composite image while disregarding at least a portion of a camera'"'"'s field of view such that there is at least one blind region in the composite field of view. Advantageously, such blind regions can be made to encompass edges of mirror surfaces that would otherwise produce image artifacts in the composite image; this can be accomplished without introducing gaps in the angular span of the composite field of view. Data representative of the composite image is stored in a memory from which it may be retrieved selectively using a control means to display a portion of the composite image. The image and control data may be transmitted over a communication network to facilitate remote control and display.

Claims

1. A method of producing a composite image with a plurality of sensors each having an individual field of view, comprising the steps of:

for at least one of the plurality of sensors, redirecting at least a portion of its individual field of view with a reflective area; and

merging images corresponding to the individual fields of view to produce the composite image having a corresponding field of view, wherein each one of at least two fields of view corresponding to images that are merged has a portion, where the images are merged, that has viewing directions that are substantially similar to the viewing directions of the other portion, and wherein the viewing directions within each one of such two portions appear to originate substantially from a point that is offset from the point for the other one of such two portions.

2. The method of claim 1, wherein the field of view corresponding to the composite image has at least one blind region that encompasses at least a portion of an edge of a reflective area and that lies between the two portions of the two individual fields of view.

3. The method of claim 2, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

4. The method of claim 2, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

5. The method of claim 1, wherein images corresponding to the individual fields of view are merged electronically by merging data representative of the fields of view to produce data representative of the composite image and data representative of the composite image is stored by electronic means.

6. The method of claim 5, further comprising displaying the composite image selectively using a control means to select a portion of the composite image to be displayed.

7. The method of claim 6, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

8. The method of claim 6, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

9. The method of claim 5, further comprising displaying at least a portion of the composite image remotely after transmitting this portion over a communication network.

10. The method of claim 9, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

11. The method of claim 9, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

12. The method of claim 5, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

13. The method of claim 5, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

14. The method of claim 1, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

15. The method of claim 1, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

16. A viewing apparatus having a composite field of view, comprising:

at least two image sensors, each having an individual field of view;

at least one reflective area that redirects at least part of the individual field of view associated with one of the image sensors,

the individual fields of view of at least two image sensors being merged to produce the composite field of view, at least portions of each of the two individual merged fields of view having substantially similar viewing directions where they are merged, and the merged fields of view creating in the composite field of view at least one blind region that encompasses at least a portion of an edge of the reflective area and that lies between portions of two individual fields of view; and

a memory for storing data representative of said composite field of view electronically.

17. The viewing apparatus of claim 16, wherein the number of sensors each of whose individual field of view is redirected at least in part by a reflective area is at least three.

18. The viewing apparatus of claim 16, wherein the number of sensors each of whose individual field of view is redirected at least in part by a reflective area is at least five.

19. A viewing apparatus having a composite field of view, comprising:

at least two image sensors, each having an individual field of view;

at least one reflective area that redirects at least part of the individual field of view associated with one of the image sensors,

the individual fields of view of at least two image sensors being merged to produce the composite field of view, at least portions of each of the two merged individual fields of view having substantially similar viewing directions where they are merged, and the merged fields of view creating in the composite field of view at least one blind region that encompasses at least a portion of an edge of the reflective area and that lies between portions of two individual fields of view;

a memory for storing data representative of said composite field of view electronically; and

control means for selectively retrieving said data from said memory in response to user signals produced in response to a user'"'"'s input.

20. The viewing apparatus of claim 19, wherein the number of sensors each of whose individual field of view is redirected at least in part by a reflective area is at least three.

21. The viewing apparatus of claim 19, wherein the number of sensors each of whose individual field of view is redirected at in part by a reflective area is at least five.

22. A viewing apparatus having a composite field of view, comprising:

at least two image sensors, each having an individual field of view;

at least one reflective area that redirects at least part of the individual field of view associated with one of the image sensors,

the individual fields of view of at least two images sensors being merged to produce the composite field of view, at least portions of each of the two merged individual fields of view having substantially similar viewing directions where they are merged, and the merged fields of view creating in the composite field of view at least one blind region that encompasses at least a portion of an edge of the reflective area and that lies between portions of two individual fields of view;

a memory for storing data representative of said composite field of view electronically;

control means for selectively retrieving said data from said memory in response to user signals produced in response to a users input; and

communication network interface means for transmitting said data retrieved from said memory over a communications network and for receiving said user signals over said communications network.

23. The viewing apparatus of claim 22, wherein the number of sensors each of whose individual field of view is redirected at least in part by a reflective area is at least three.

24. The viewing apparatus of claim 22, wherein the number of sensors each of whose individual field of view is redirected at least in part by a reflective area is at least five.

25. A composite image produce with a plurality of sensors each having an individual field of view, by following the method of:

for at least one of the plurality of sensors, redirecting at least a portion of its individual field of view with a reflective area; and

merging images corresponding to the individual fields of view to produce the composite image having a corresponding field of view, wherein each one of at least two fields of view corresponding to images that are merged has a portion, where the images are merged, that has viewing directions that are substantially similar to the viewing directions of the other portion, and wherein the viewing directions within each one of such two portions appear to originate substantially from a point that is offset from the point for the other one of such two portions.

26. The composite image of claim 25, wherein the field of view corresponding to the composite image has at least one blind region that encompasses at least a portion of an edge of a reflective area and that lies between the two portions of the two individual fields of view.

27. The method of claim 26 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

28. The method of claim 26 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

29. The composite image of claim 25, wherein images corresponding to the individual fields of view are merged electronically by merging data representative of the fields of view to produce data representative of the composite image and data representative of the composite image is stored by electron means.

30. The composite image of claim 29, wherein a portion of the composite image is displayed selectively using a control means to select the portion of the composite image to be displayed.

31. The method of claim 30 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

32. The method of claim 30 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

33. The composite image of claim 29, wherein at least a portion of the composite image is displayed remotely after this portion is transmitted over a communication network.

34. The method of claim 33 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

35. The method of claim 33 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

36. The method of claim 29 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

37. The method of claim 29 wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.

38. The method of claim 25, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least three.

39. The method of claim 25, wherein the number of sensors each of whose field of view is redirected at least in part by a reflective area is at least five.