Electronic camera and device for panoramic imaging and object searching

• US 5,262,867 A
• Filed: 06/12/1991
• Issued: 11/16/1993
• Est. Priority Date: 06/20/1990
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kojima, Kiyonobu
• Primary Examiner(s): Razavi, Michael T.
• Assistant Examiner(s): GARBER, WENDY RAE
• Application Number:
  US07/714,241
• Time in Patent Office: 888 Days
• Field of Search: 358/909, 358/209, 358/227, 358/906, 354/105, 354/109, 354/94, 352/69, 352/70, 352/93, 352/94
• US Class Current: 348/39
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00323 with a measuring, monitoring or signaling apparatus, e.g. for transmitting measured information to a central location
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3277 The additional information being stored in the same storage device as the image data
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: Y10S 358/906 Hand-held camera with recorder in a single unit

Abstract

An electronic camera has sensors for detecting an image of an object, the three dimensional position and azimuth of the camera, the focusing amount and the zoom amount. The image is recorded on a recording medium together with information concerning the three dimensional position and azimuth, the focusing amount and the zoom amount. An image reproducing device for the camera reproduces the recorded image on the basis of the information so that an interrelationship between a plurality of the recorded images is clearly reproduced. Accordingly, the camera and the reproducing device are suitable for combining the recorded images so as to prepare a panoramic image, or to locate a desired article within an image of several different articles.

Claims

1. An electronic camera comprising:

a lens section capable of focusing incident light having an image of a subject to be recorded;

first detecting means for detecting the image of the subject in response to the incident light passing through said lens section, on the basis of which respective image data is formed for each said image;

second detecting means for detecting, for each said image, data representing the three dimensional position at which the camera is located and the pointing direction of the camera;

third detecting means for detecting, for each said image, data representing the the focusing amount of the electronic camera from which a distance between the camera and the subject can be derived;

fourth detecting means for detecting, for each said image, data representing the zoom amount of the electronic camera from which zoom magnification can be derived;

a processing unit for generating auxiliary data in association with the image data, on the basis of said data detected by said second, third and fourth detecting means; and

magnetic recording means for recording said image data with said auxiliary data thereof on a recording medium within said electronic camera.

2. The electronic camera according to claim 1, wherein

said lens section includes control means for controlling said focusing amount and said zoom amount, and

said fourth detecting means is mounted on said lens section.

3. An image reproducing device for the electronic camera according to claim 1, comprising:

a disk drive for reading out said image data and said auxiliary data which are recorded on said recording medium;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored; and

a display unit for displaying said image data stored in said frame memory.

4. The image reproducing device according to claim 3, wherein said computer is used as said processing unit for the electronic camera.

5. The electronic camera according to claim 1, wherein said magnetic recording means are electrically connected to said first detecting means and through said processing unit to said second, third and fourth detecting means.

6. The electronic camera according to claim 1, wherein said second detecting means comprises global positioning means responsive to satellite signals for said detecting of said data representing said three dimensional position of the electronic camera.

7. The electronic camera according to claim 6, wherein said second detecting means comprises geomagnetic sensing means for said detecting of said pointing direction of the electronic camera.

8. The electronic camera according to claim 1, wherein said second detecting means comprises ultrasonic means for said detecting of said data representing said three dimensional position and pointing direction of the electronic camera.

9. The electronic camera according to claim 1, wherein said second detecting means comprises electromagnetic coil means for said detecting of said data representing said three dimensional position and pointing direction of the electronic camera.

10. The electronic camera according to claim 1, wherein:

said processing unit reduces said detected data from said second, third and fourth detecting means to values indicative of said three dimensional position, pointing direction, focus amount and zoom amount, and

said indicative values of said three dimensional position, pointing direction, focus amount and zoom amount being stored on said magnetic recording medium.

11. The image reproducing device according to claim 3, wherein:

said electronic camera is a video camera for generating a moving sequence of said images and the corresponding auxiliary data; and

wherein, for searching for a desired object among a plurality of different objects, said disk drive, computer, memory, frame memory and display unit are such that;

a sequence of displays can be constructed corresponding to free navigation in three dimensions;

whereby a desired article can be searched for among a plurality of articles without direct visual detection of the desired article at the time of searching.

12. The image reproducing device according to claim 11, wherein said free navigation at searching for said desired article comprise use of the formulas of equations (1) to (4):

##EQU3## wherein;

x, y, and z denote mutually perpendicular axes;

xo, yo, and zo denote coordinates on the x, y and z axes;

a, b, c denote camera direction vectors;

α

denotes the horizontal direction angle of camera view;

β

denotes the vertical direction of camera view.

13. An electronic camera comprising:

a lens section capable of focusing incident light having an image of a subject to be recorded;

first detecting means for detecting the image of the subject in response to the incident light passing through said lens section, on the basis of which respective image data is formed for each said image;

second detecting means for detecting, for each said image, data representing the three dimensional position at which the camera is located and the pointing direction of the camera;

third detecting means for detecting, for each said image, data representing the focusing amount of the electronic camera from which a distance between the camera and the subject can be derived;

fourth detecting means for detecting, for each image, data representing the zoom amount of the electronic camera from which the zoom magnification can be derived;

a processing unit for generating auxiliary data in association with the image data, on the basis of said data detected by said second, third and fourth detecting means; and

magnetic recording means for recording said image data with said auxiliary data thereof on a recording medium within said electronic camera;

a disk drive for reading out said image data and said auxiliary data which are recorded on said recording medium;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processes or searched by said computer are temporarily stored;

a display unit for displaying said image data stored in said frame memory;

further comprising a pointing device, wherein, for searching for a desired object among a first plurality of different objects, said disk drive, computer, memory, frame memory and display unit are such that;

a first display can be constructed comprising a reduced image of each said object, with all of said reduced images arranged in table form, each said reduced image having been obtained from a respective image of said electronic camera with the corresponding auxiliary data;

said desired one of said objects can be selected from said display with said pointing device; and

a second display is produced showing a second plurality of objects which includes said desired object, said second display being at least in part an image detected by said camera and selected by said computer on the basis of its respective auxiliary data; and

the location where said desired object is located in said image of said second display is indicated at that location by a predetermined marking at that position, as a result of said selecting by said pointing device from said first display.

14. An electronic camera comprising:

a lens section capable of focusing incident light having an image of a subject to be recorded;

first detecting means for detecting the image of the subject in response to the incident light passing through said lens section, on the basis of which respective image data is formed for each said image;

second detecting means for detecting, for each said image, data representing the three dimensional position at which the camera is located and the pointing direction of the camera;

third detecting means for detecting, for each said image, data representing the focusing amount of the electronic camera from which a distance between the camera and the subject can be derived;

fourth detecting means for detecting, for each image, data representing the zoom amount of the electronic camera from which the zoom magnification can be derived;

a processing unit for generating auxiliary data in association with the image data, on the basis of said data detected by said second, third and fourth detecting means; and

magnetic recording means for recording said image data with said auxiliary data thereof on a recording medium within said electronic camera;

a disk drive for reading out said image data and said auxiliary data which are recorded on said recording medium;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored;

a display unit for displaying said image data stored in said frame memory; and

wherein said disk drive, computer, memory, frame memory and display unit are such that said display unit can show a panoramic image formed from a plurality of the images obtained with said auxiliary data by said electronic camera.

15. The image reproducing device according to claim 14, wherein said panoramic image can be scrolled.

16. The image reproducing device according to claim 15, wherein:

said electronic camera is a video camera for generating a moving sequence of said images and the corresponding auxiliary data; and

wherein, for searching for a desired object among a plurality of different objects, said disk drive, computer, memory, frame memory and display unit are such that;

a sequence of displays can be constructed corresponding to free navigation in three dimensions;

whereby a desired article can be searched for among a plurality of articles without direct visual detection of the desired article at the time of searching.

17. The image reproducing device according to claim 16, wherein said free navigation at searching for said desired article comprise use of the formulas of equations (1) to (4):

##EQU4## wherein;

x, y, and z denote mutually perpendicular axes;

xo, yo, and zo denote coordinates on the x, y and z axes;

a, b, c denote camera direction vectors;

α

denotes the horizontal direction angle of camera view;

β

denotes the vertical direction of camera view.

18. An electronic camera comprising:

a lens section capable of focusing incident light having an image of a subject to be recorded;

first detecting means for detecting the image of the subject in response to the incident light passing through said lens section, on the basis of which respective image data is formed for each said image;

second detecting means for detecting, for each said image, data representing the three dimensional position at which the camera is located and the pointing direction of the camera;

third detecting means for detecting, for each said image, data representing the focusing amount of the electronic camera from which a distance between the camera and the subject can be derived;

fourth detecting means for detecting, for each image, data representing the zoom amount of the electronic camera from which the zoom magnification can be derived;

a processing unit for generating auxiliary data in association with the image data, on the basis of said data detected by said second, thrid and fourth detecting means; and

magnetic recording means for recording said image data with said auxiliary data thereof on a recording medium within said electronic camera;

a disk drive for reading out said image data and said auxiliary data which are recorded on said recording medium;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored;

a display unit for displaying said image data stored in said frame memory; and

wherein said panoramic image can be zoomed.

19. An image reproducing device comprising:

a disk drive for reading out image data of each of a plurality of images and corresponding auxiliary data for each said image which are recorded on a recording medium, said auxiliary data comprising information for determining position, pointing direction, focus amount and zoom amount of a camera taking each said image to provide said image data, and;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer is temporarily stored; and

a display unit for displaying said image data stored in said frame memory;

said image reproducing device further comprising a pointing device, wherein, for searching for a desired object among a first plurality of different objects, said disk drive, computer, memory, frame memory and display unit are such that;

a first display can be constructed comprising at least a reduced image of each said object, with all of said reduced images arranged in table form, each said reduced image having been obtained from a respective image of said camera with the corresponding auxiliary data;

said desired one of said objects can be selected from said first display with said pointing device; and

a second display is produced showing a second plurality of objects which includes said desired object, said second display being at least in part an image detected by said camera and selected by said computer on the basis of its respective auxiliary data; and

the location where said desired object is located in said image of said second display is indicated at that location by a predetermined marking at that position, as a result of said selecting by said pointing device from said first display.

20. The image reproducing device according to claim 19, wherein:

said camera is a video camera for generating a moving sequence of said images and the corresponding auxiliary data; and

wherein, for searching for a desired object among a plurality of different objects, said drive, computer, memory, frame memory and display unit are such that;

a sequence of displays can be constructed corresponding to free navigation in three dimensions;

whereby a desired article can be searched for among a plurality of articles without direct visual detection of the desired article at the time of searching.

21. An image reproducing device comprising:

a disk drive for reading out image data of each of a plurality of images and corresponding auxiliary data for each said image which are recorded on a recording medium, said auxiliary data comprising information for determining at least one of the position, pointing direction, focus amount or zoom amount of a camera taking each said image to provide said image data, and;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored; and

a display unit for displaying said image data stored in said frame memory; and

wherein said disk drive, computer, memory, frame memory and display unit are such that said display unit can show a panoramic image formed from a plurality of the images obtained with said auxiliary data by said electronic camera.

22. The image reproducing device according to claim 21, wherein said panoramic image can be zoomed.

23. The image reproducing device according to claim 21, wherein said panoramic image can be scrolled.

24. An image reproducing device comprising:

a disk drive for reading out image data of each of a plurality of images and corresponding auxiliary data for each said image which are recorded on a recording medium, said auxiliary data comprising information for determining at least one of the position, pointing direction, focus amount or zoom amount of a camera taking each said image to provide said image data, and;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored; and

a display unit for displaying said image data stored in said frame memory,

wherein;

said electronic camera is a video camera for generating a moving sequence of said images and the corresponding auxiliary data; and

wherein, for searching for a desired object among a plurality of different objects, said disk drive, computer, memory, frame memory and display unit are such that;

a sequence of displays can be constructed corresponding to free navigation in three dimensions;

whereby a desired article can be searched for among a plurality of articles without direct visual detection of the desired article at the time of searching; and

, wherein said free navigation at searching for said desired article comprise use of the formulas of equations (1) to (4);

##EQU5## wherein;

x, y, and z denote mutually perpendicular axes;

x₀, y₀, and z₀ denote coordinates on the x, y and z axes;

a, b, c denote camera direction vectors;

α

denotes the horizontal direction angle of camera view;

β

denotes the vertical direction of camera view.

25. An image reproducing device comprising:

a disk drive for reading out image data of each of a plurality of images and corresponding auxiliary data for each said image which are recorded on a recording medium, said auxiliary data comprising information for determining at least one of the position, pointing direction, focus amount or zoom amount of a camera taking each said image to provide said image data, and;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored;

a display unit for displaying said image data stored in said frame memory,

wherein;

said electronic camera is a video camera for generating a moving sequence of said images and the corresponding auxiliary data;

wherein, for searching for a desired object among a plurality of different objects, said disk drive, computer, memory, frame memory and display unit are such that;

a sequence of displays can be constructed corresponding to free navigation in three dimensions;

whereby a desired article can be searched for among a plurality of articles without direct visual detection of the desired article at the time of searching; and

wherein said free navigation at searching for said desired article comprise use of the formulas of equations (1) to (4) ##EQU6## wherein;

x, y, and z denote mutually perpendicular axes;

x₀, y₀, and z₀ denote coordinates on the x, y and z axes;

a, b, c denote camera direction vectors;

α

denotes the horizontal direction angle of camera view;

β

denotes the vertical direction of camera view.

26. An apparatus comprising an electronic camera and an image reproducing device, said electronic camera comprising:

a lens section capable of focusing incident light having an image of a subject to be recorded;

first detecting means for detecting the image of the subject provided by said lens section, on the basis of which respective image data is formed for each said image,

second, third and fourth detecting means, said second detecting means being for detecting, for each said image, data representing the three dimensional position at which the camera is located and the pointing direction of the camera, said third detecting means being for detecting, for each said image, data representing the focusing amount of the electronic camera from which a distance between the camera and the subject can be derived, said fourth detecting means for detecting, for each said image, data representing the zoom amount of the electronic camera from which zoom magnification can be derived;

a processing unit for generating auxiliary data in association with the image data, on the basis of said data detected by said each said second, third and fourth detecting means;

magnetic recording means for recording said image data with said auxiliary data thereof on a recording medium within said electronic camera; and

said image reproducing device comprising;

a disk drive for reading out said image data and said auxiliary data which are recorded on said recording medium;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored; and

a display unit for displaying said image data stored in said frame memory.

27. The apparatus according to claim 26, wherein said computer is used as said processing unit for the electronic camera.

28. An electronic camera and an image reproducing device, said electronic camera comprising:

a lens section capable of focusing incident light having an image of a subject to be recorded;

first detecting means for detecting the image of the subject provided by said lens section, on the basis of which respective image data is formed for each said image,

second, third and fourth detecting means, said second detecting means being for detecting, for each said image, data representing the three dimensional position at which the camera is located and the pointing direction of the camera, said third detecting means being for detecting, for each said image, data representing the focusing amount of the electronic camera from which a distance between the camera and the subject can be derived, said fourth detecting means for detecting, for each said image, data representing the zoom amount of the electronic camera from which zoom magnification can be derived;

a processing unit for generating auxiliary data in association with the image data, on the basis of said data detected by said each said second, third and fourth detecting means; and

magnetic recording means for recording said image data with said auxiliary data thereof on a recording medium within said electronic camera.

29. The electronic camera according to claim 28, comprising at least two of said second, third and fourth means, said auxiliary data comprising the respective data from said at least two of said second, third and fourth means.

30. An image reproducing device comprising:

a disk drive for reading out image data of each of a plurality of images and corresponding auxiliary data for each said image which are recorded on a recording medium, said auxiliary data comprising information for determining at least two of the position, pointing direction, focus amount or zoom amount of a camera taking each said image to provide said image data, and;

a computer for editing, processing or searching said image data read out by said disk drive, on the basis of said auxiliary data read out by said disk drive;

a memory in which said read-out image data and said read-out auxiliary data are stored;

a frame memory in which said image data edited, processed or searched by said computer are temporarily stored;

a display unit for displaying said image data stored in said frame memory; and

a pointing device, wherein, for searching for a desired object among a first plurality of different objects, said disk drive, computer, memory, frame memory and display unit are such that;

a first display can be constructed comprising a reduced image of each said object, with all of said reduced images arranged in table form, each said reduced image having been obtained from a respective image of said electronic camera with the corresponding auxiliary data;

said desired one of said objects can be selected from said first display with said pointing device; and

a second display is produced showing a second plurality of objects which includes said desired object, said second display being at least in part an image detected by said camera and selected by said computer on the basis of its respective auxiliary data; and

the location where said desired object is located in said image of said second display is indicated at that location by a predetermined marking at that position, as a result of said selecting by said pointing device from said first display.