Object detection for video camera self-calibration

• US 10,366,263 B2
• Filed: 12/30/2016
• Issued: 07/30/2019
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Original Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Inventors: Bataller, Cyrille, Astrom, Anders
• Primary Examiner(s): Labaze, Edwyn
• Application Number:
  US15/395,770
• Publication Number:
  US 20180189532A1
• Time in Patent Office: 942 Days
• Field of Search: 23546211, 235375, 2354621, 235487, 23546241, 23546224
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06K 19/06037 multi-dimensional coding
• CPC Class Codes: G06K 7/10811 Focalisation
• CPC Class Codes: G06K 7/1417 2D bar codes
• CPC Class Codes: G06K 7/1443 locating of the code in an image
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

This document describes systems, methods, devices, and other techniques for video camera self-calibration based on video information received from the video camera. In some implementations, a computing device receives video information characterizing a video showing a scene from a field of view of a video camera; detects an object that appears in the scene of the video; identifies a visual marking that appears on the detected object; determines a particular visual marking among a plurality of pre-defined visual markings that matches the visual marking that appears on the detected object; identifies one or more object characteristics associated with the particular visual marking; evaluates one or more features of the video with respect to the one or more object characteristics; and based on a result of evaluating the one or more features of the video with respect to the one or more object characteristics, sets a parameter of the video camera.

Claims

1. A computer-implemented method for performing video camera self-calibration, comprising:

receiving, by a computing system, video information that characterizes a video showing a scene from a field of view of a video camera that is to be self-calibrated;

detecting, by the computing system and based on analysis of the video information, an object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera, the object having an attached, computer-readable, visual marking that identifies the object to which it is attached;

identifying, by the computing system, the visual marking that is attached to the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera;

determining, by the computing system, a particular visual marking, among a plurality of pre-defined visual markings available to the computing system, that matches the visual marking that appears on the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera;

identifying, based on stored data that correlates the plurality of pre-defined visual markings with characteristics of respective objects on which the pre-defined visual markings can appear and which are used for assisting with self-calibration of video cameras, one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera, wherein the one or more real-world object characteristics comprise at least one of a physical dimension of the object, a real-world location of the object, a color of the object, or an orientation of the object;

comparing one or more characteristics of the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera as they appear in the video to respective ones of the one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object;

automatically self-calibrating the video camera based on a result of comparing one or more characteristics of the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera as they appear in the video to respective ones of the one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object, comprising;

selecting a setting for an operational parameter of the video camera, wherein the operational parameter of the video camera comprises at least one of a focal length, a principal point, an aspect ratio, a skew, a colorisation, a lens distortion, a height of the video camera, a tilt angle of the video camera, or a roll angle of the video camera; and

setting the operational parameter of the video camera according to the selected setting.

2. The method of claim 1, wherein the visual marking comprises a quick-response (QR) code.

3. The method of claim 1, wherein the visual marking comprises an encoded pattern printed on the detected object that uniquely identifies the detected object.

4. The method of claim 1, wherein each of the plurality of pre-defined visual markings identify a unique set of multiple real-world object characteristics.

5. The method of claim 4, wherein the unique set of multiple real-world object characteristics comprises one or more of (i) a physical dimension of the detected object or (ii) a real-world location of the detected object.

6. The method of claim 5, comprising comparing a dimension of the detected object as it appears in the video to the physical dimension of the detected object to determine a distance of the video camera from the detected object.

7. The method of claim 6, wherein selecting a setting for an operational parameter of the video camera comprises selecting a setting for the focal length of the video camera, and wherein setting the operational parameter of the video camera according to the selected setting comprises setting the focal length of the video camera based on the determined distance of the video camera from the detected object.

8. The method of claim 6, wherein selecting a setting for an operational parameter of the video camera comprises selecting a real-world location of the video camera, and wherein setting the operational parameter of the video camera according to the selected setting comprises storing an indication of the geographic location of the video camera based on the determined distance of the video camera from the detected object and the real-world location of the detected object.

9. The method of claim 4, wherein the visual marking comprises one or more colors, and

wherein the unique set of multiple real-world object characteristics that are defined for objects having the visual marking identifies the one or more colors of the visual marking.

10. The method of claim 9, comprising comparing a colorization of the visual marking as it appears in the video to the one or more colors indicated by the unique set of multiple real-world object characteristics that are defined for objects having the visual marking.

11. The method of claim 10, wherein selecting a setting for an operational parameter of the video camera comprises selecting a color setting of the video camera, and wherein setting the operational parameter of the video camera comprises adjusting a color setting of the video camera.

12. The method of claim 4, wherein the unique set of multiple real-world object characteristics indicate that the detected object is located on a ground plane.

13. The method of claim 12, wherein selecting a setting for an operational parameter of the video camera comprises selecting an orientation of the video camera, and wherein setting the operational parameter of the video camera comprises determining an orientation of the camera relative to the ground plane.

14. The method of claim 13, wherein the unique set of multiple real-world object characteristics comprise a height of the object above the ground plane, and wherein the method comprises modeling the ground plane within the scene.

15. The method of claim 1, wherein (i) the detected object comprises a card, and (ii) the visual marking that appears on the detected object comprises a glyph printed on the card.

16. The method of claim 1, wherein identifying one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object comprises querying a remotely stored database.

17. The method of claim 1, comprising determining a spatial relationship between the video camera and the detected object, and wherein selecting a setting for an operational parameter of the video camera comprises selecting a setting for an operational parameter based on the determined spatial relationship between the video camera and the detected object.

18. The method of claim 1, comprising:

comparing a colorization of the visual marking as it appears in the video to the one or more colours indicated by the unique set of real world object characteristics that are defined for objects having the visual marking; and

comparing dimensions of the object as they appear in the video to the real-world dimensions of the object to determine a distance of the camera from the object; and

selecting a setting for an operational parameter of the video camera comprises;

selecting a color setting of the video camera, and wherein setting the operational parameter of the video camera comprises adjusting a colour setting of the video camera; and

selecting a setting for the focal length of the video camera, and wherein setting the operational parameter of the video camera according to the selected setting comprises setting the focal length of the video camera based on the determined distance.

19. A system for performing video camera self-calibration, the system comprising:

one or more processors; and

one or more computer-readable media having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving, by a computing system, video information that characterizes a video showing a scene from a field of view of a video camera that is to be self-calibrated;

detecting, by the computing system and based on analysis of the video information, an object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera, the object having an attached, computer-readable, visual marking that identifies the object to which it is attached;

identifying, by the computing system, the visual marking that is attached to the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera;

determining, by the computing system, a particular visual marking, among a plurality of pre-defined visual markings available to the computing system, that matches the visual marking that appears on the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera;

identifying, based on stored data that correlates the plurality of pre-defined visual markings with characteristics of respective objects on which the pre-defined visual markings can appear and which are used for assisting with self-calibration of video cameras, one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera, wherein the one or more real-world object characteristics comprise at least one of a physical dimension of the object, a real-world location of the object, a color of the object, or an orientation of the object;

comparing one or more characteristics of the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera as they appear in the video to respective ones of the one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object;

automatically self-calibrating the video camera based on a result of comparing one or more characteristics of the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera as they appear in the video to respective ones of the one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object, comprising;

selecting a setting for an operational parameter of the video camera, wherein the operational parameter of the video camera comprises at least one of a focal length, a principal point, an aspect ratio, a skew, a colorisation, a lens distortion, a height of the video camera, a tilt angle of the video camera, or a roll angle of the video camera; and

setting the operational parameter of the video camera according to the selected setting.

20. A non-transitory computer-readable storage device storing software for performing video camera self-calibration, the software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a computing system, video information that characterizes a video showing a scene from a field of view of a video camera that is to be self-calibrated;

detecting, by the computing system and based on analysis of the video information, an object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera, the object having an attached, computer-readable, visual marking that identifies the object to which it is attached;

identifying, by the computing system, the visual marking that is attached to the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera;

determining, by the computing system, a particular visual marking, among a plurality of pre-defined visual markings available to the computing system, that matches the visual marking that appears on the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera;

identifying, based on stored data that correlates the plurality of pre-defined visual markings with characteristics of respective objects on which the pre-defined visual markings can appear and which are used for assisting with self-calibration of video cameras, one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera, wherein the one or more real-world object characteristics comprise at least one of a physical dimension of the object, a real-world location of the object, a color of the object, or an orientation of the object;

comparing one or more characteristics of the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera as they appear in the video to respective ones of the one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object;

automatically self-calibrating the video camera based on a result of comparing one or more characteristics of the detected object that has been placed in the scene of the video for use in assisting with self-calibration of the video camera as they appear in the video to respective ones of the one or more real-world object characteristics that are defined for objects having the particular visual marking that appears on the detected object, comprising;

selecting a setting for an operational parameter of the video camera, wherein the operational parameter of the video camera comprises at least one of a focal length, a principal point, an aspect ratio, a skew, a colorisation, a lens distortion, a height of the video camera, a tilt angle of the video camera, or a roll angle of the video camera; and

setting the operational parameter of the video camera according to the selected setting.