Omnidirectional camera with movement detection

• US 9,888,174 B2
• Filed: 10/15/2015
• Issued: 02/06/2018
• Est. Priority Date: 10/15/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kankaanp, Esa, Soni, Shahil
• Primary Examiner(s): ELAHI, SHAN E
• Application Number:
  US14/884,299
• Publication Number:
  US 20170111579A1
• Time in Patent Office: 845 Days
• Field of Search: 348 37
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An omnidirectional camera is presented. The camera comprises: at least one lens coupled to an image sensor, a controller coupled to the image sensor, and a movement detection element coupled to the controller. The movement detection element is configured to detect a speed and direction of movement of the camera, the camera is configured to capture images via the at least one lens coupled to the image sensor, and the controller is configured to select a digital viewpoint in the captured images. A central point of the digital viewpoint is selected on the basis of direction of movement, and a field of view of the digital viewpoint is based on the speed of movement. A system and method are also presented.

Claims

1. An omnidirectional camera, comprising:

at least one lens coupled to an image sensor,

a controller coupled to the image sensor, and

a movement detection element coupled to the controller;

wherein

the movement detection element is configured to detect a speed and direction of movement of the omnidirectional camera, the speed corresponding to a speed of a moving object to which the omnidirectional camera is attached,

the omnidirectional camera is configured to capture images via the at least one lens coupled to the image sensor,

the controller is configured to select a digital viewpoint in the captured images, wherein a central point of the digital viewpoint is based on the direction of movement, and a field of view of the digital viewpoint is based on the speed of movement of the moving object.

2. The omnidirectional camera as claimed in claim 1, wherein the field of view of the digital viewpoint is an angle which determines a width of the digital viewpoint and the controller is configured to:

select and adjust the central point of the digital viewpoint based on the direction of movement,

narrow the field of view of the digital viewpoint when the speed of movement increases, and

widen the field of view of the digital viewpoint when the speed of movement decreases.

3. The omnidirectional camera as claimed in claim 1, comprising at least two lenses coupled to image sensors, wherein the controller is coupled to each of the image sensors.

4. The omnidirectional camera as claimed in claim 3, configured to capture a horizontally and vertically omnidirectional panoramic image via the at least two lenses coupled to the image sensors.

5. The omnidirectional camera as claimed in claim 1, configured to capture omnidirectional video footage via the at least one lens coupled to the image sensor.

6. The omnidirectional camera as claimed in claim 5, wherein the selected digital viewpoint in the captured images is a default digital viewpoint for video playback.

7. The omnidirectional camera as claimed in claim 1, comprising a gaze detection element configured to detect a gaze direction of a camera user, wherein the controller is configured to select a central point of the digital viewpoint based on the direction of movement and the detected gaze direction only after a pre-defined speed threshold of the moving object has been reached, wherein the controller is further configured to prioritize a direction of a gaze of a user for a period of time and then return the central point to the direction of movement.

8. The omnidirectional camera as claimed in claim 1, comprising a fastening element configured to attach the omnidirectional camera to the moving object.

9. The omnidirectional camera as claimed in claim 1, comprising an accelerometer coupled to the controller.

10. The omnidirectional camera as claimed in claim 9, comprising a gyroscopic element coupled to the controller, the gyroscopic element configured to provide information on an angle by which the digital viewpoint is to be adjusted, wherein the controller is configured to adjust the orientation of the digital viewpoint based on the accelerometer reading and the gyroscopic element reading in real configured to record the digital viewpoint selection as part of data related to the captured video or sequence of images, the data being recorded as metadata.

11. The omnidirectional camera as claimed in claim 1, wherein the image sensor comprises a charge-coupled device.

12. The omnidirectional camera as claimed in claim 1, wherein the image sensor comprises an active pixel sensor.

13. The omnidirectional camera as claimed in claim 1, comprising at least one mirror configured to reflect a panoramic image into the at least one lens coupled to the image sensor.

14. The omnidirectional camera as claimed in claim 1, configured to capture images at a framerate between 5 and 120 frames per second.

15. A system comprising:

an omnidirectional camera, the omnidirectional camera comprising at least one lens coupled to an image sensor;

a controller coupled to the omnidirectional camera, and

a movement detection unit coupled to the controller;

wherein

the movement detection unit is configured to detect a speed and direction of movement of the omnidirectional camera, the speed corresponding to a speed of a moving object to which the omnidirectional camera is attached,

the omnidirectional camera is configured to capture images via the at least one lens coupled to the image sensor, and

the controller is configured to select a digital viewpoint in the captured images, wherein a central point of the digital viewpoint is based on the direction of movement, and a field of view of the digital viewpoint is based on the speed of movement of the moving object.

16. A system as claimed in claim 15, wherein the system comprises a gaze detection unit configured to detect a gaze direction of a camera user, wherein the controller is configured to select a central point of the digital viewpoint based on the direction of movement and the detected gaze direction.

17. A system as claimed in claim 15, wherein the controller comprises a processor.

18. A method for selecting a digital viewpoint in an omnidirectional video, the method comprising:

receiving a video stream from an omnidirectional camera,

receiving a signal from a movement detection element, the signal comprising information including speed and direction of movement of the omnidirectional camera, the speed corresponding to a speed of a moving object to which the omnidirectional camera is attached,

defining a central point in the received video stream based on the direction of movement of the omnidirectional camera,

defining a field of view based on the speed of movement of the moving object, and

selecting a digital viewpoint in the received video stream with the defined central point and the defined field of view.

19. A method as claimed in claim 18, wherein defining a field of view based on the speed of movement of the omnidirectional camera comprises narrowing the field of view when the speed of movement increases, and widening the field of view when the speed of movement decreases.

20. A method as claimed in claim 18, comprising:

receiving a signal from a gaze detection element, the signal comprising information including gaze direction of a camera user

defining an additional central point based on the detected gaze direction, and

selecting an additional digital viewpoint in the received video stream with the additional defined central point.