DYNAMICALLY CONFIGURING AN IMAGE PROCESSING FUNCTION

• US 20150029222A1
• Filed: 11/29/2011
• Published: 01/29/2015
• Est. Priority Date: 11/29/2011
• Status: Abandoned Application

• Current Assignee: Layar BV (Candy Capital Ltd.)
• Original Assignee: Layar BV (Candy Capital Ltd.)
• Inventors: Hofmann, Klaus Michael
• Application Number:
  US14/361,592
• Publication Number:
  US 20150029222A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks

Abstract

Method and systems for dynamically configuring an image processing function into at least a first and second detection state on the basis of function parameters are described, wherein transitions between said detection states are determined by at least a first state transition condition and wherein said image processing function includes extracting features from an image frame, matching features with features associated with one or more target objects and estimating pose information on the basis of matched features and wherein method comprises: configuring said image processing function in a detection state on the basis of a first set of function parameter values; processing a first image frame in said first detection state; monitoring said image processing function for occurrence of said at least state transition condition; and, if said at least one state transition condition is met, configuring said image processing function in said second detection state.

Claims

1. A method for dynamically configuring an image processing function into at least a first and second detection state on the basis of function parameters, wherein transitions between said first and second detection states are determined by at least a first state transition condition and wherein said image processing function includes extracting features from an image frame, matching extracted features with reference features associated with one or more target objects and estimating pose information on the basis of matched features, said method comprising:

configuring said image processing function in a first detection state on the basis of a first set of function parameter values;

processing a first image frame in said first detection state;

monitoring said image processing function for occurrence of said at least first state transition condition; and

,

if said at least one state transition condition is met, configuring said image processing function in said second detection state on the basis of a second set of function parameter values for processing a second image frame in said second detection state.

2. The method according to claim 1 wherein said at least first state transition condition is:

the detection of at least one target object in said first image frame, the detection of a predetermined number of objects in said first image frame, the absence in said first image frame of at least one previously recognized target object; and

/or, the generation of pose information according to a predetermined accuracy and/or within a certain processing time.

3. The method according to claim 1 wherein said first detection state is determined by a first set of function parameter values so that said image processing function is configured for fast detection one or more objects in said first image frame.

4. The method according to claim 1 wherein said second detection state is determined by a second set of function parameter values so that the image processing function is configured for accurate determination of pose information of at least one object in a second image frame, said object previously being detected by said image processing function in said first detection state.

5. The method according to claim 1 wherein said second detection state is determined by a second set of function parameter values so that the image processing function is configured for accurate determination of pose information of at least one object in a second image frame, said at least one object previously being detected in said first image frame by said image processing function in said first detection state;

and, for fast detection of one or more objects in said second frame that were not previously detected in said first image frame by said image processing function in said first detection state.

6. The method according to claim 1, wherein said first and second set of parameter values are configured such that in the first detection mode a smaller number of extracted features is used than in the second detection mode.

7. The method according to claim 1, wherein said first and second set of parameter values are configured such that in the first detection mode image frames of a lower resolution are used than the image frames used in the second detection mode.

8. The method according to claim 1, wherein said first and second set of parameter values are configured such that in the first detection mode the maximum computation time and/or the (maximum) number of iterations for pose estimation is smaller than the computation time and/or (maximum) number of iterations spent on pose estimation in the second detection mode.

9. The method according to claim 1, wherein said first and second set of parameter values are configured such that in the first detection mode a larger error margin and/or lower number of inlier data points for pose estimation is used than the error margin and/or number of inlier data points for pose estimation in said second detection mode.

10. The method according to claim 1, wherein said image processing function is configurable in a further third state, wherein transitions between said first and third detection states are determined by at least a second transition condition, said method further comprising:

monitoring said image processing function for occurrence of said at least second transition condition; and

,

if said at least second state transition condition is met, configuring said image processing function in said third detection state on the basis of said third set of function parameter values for processing a second image frame in said second detection state.

11. The method, according to claim 1, wherein said processing of said first and second image frames further comprises:

providing sets of reference features, each set being associated with a target object;

determining corresponding features pairs by matching said extracted features with said reference features;

determining the detection of said target object on the basis of said corresponding features.

12. The method according to claim 1, wherein image processing function is part of an augmented reality device comprising an image sensor for generating image frames and a graphics generator for generating a graphical item associated with at least one detected target object on the basis of pose information.

13. The method according to claim 1, wherein a state manager is configured to configure said image processing function into at least said first or second detection state and to monitor said first state transition conditions, wherein function parameters values associated with said detection states and information associated with said first state transition condition is stored in a memory.

14. The method according to claim 1 wherein function parameters include parameters for determining and/or controlling:

the number of features to be extracted from an image, the number or maximum number of iterations and/or processing time for processing features, at least one threshold value for deciding whether or not a certain condition in said image processing function is met, the resolution an image is to be processed in by said image processing function.

20. The method according to claim 7, said lower resolution images being a downscaled version of one or more images originating from an image sensor.

15. A dynamically configurable image processing module comprising:

a processor configured to execute a processing function configurable into at least a first and second detection state on the basis of function parameters, wherein said image processing function includes extracting features from an image frame, matching extracted features with reference features associated with one or more target objects and estimating pose information on the basis of matched features;

a state manager operably connected to and configured to configure said image processing function in one of detection states and configured to manage transitions between said detection states on the basis of least a first state transition condition, said state manager being configured to;

configure said image processing function in a first detection state on the basis of a first set of function parameter values for processing a first image frame;

monitor said image processing function for occurrence of said at least first state transition condition; and

, if said at least one state transition condition is met; and

,

configure said image processing function in said second detection state on the basis of a second set of function parameter values for processing a second image frame in said second detection state.

16. An augmented reality device comprising:

image sensor configured to generate image frames;

a dynamically configurable image processing module connected to the image sensor and configured to detect one or more target objects in an image frame and configured to generate pose information associated with at least one detected object the dynamically configurable image processing module;

a processor configured to execute a processing function configurable into at least a first and second detection state on the basis of function parameters, wherein said image processing function includes extracting features from an image frame, matching extracted features with reference features associated with one or more target objects and estimating pose information on the basis of matched features;

a state manager configured to configure said image processing function in one of detection states and configured to manage transitions between said detection states on the basis of least a first state transition condition, said state manager being configured to;

configure said image processing function in a first detection state on the basis of a first set of function parameter values for processing a first image frame;

monitor said image processing function for occurrence of said at least first state transition condition; and

, if said at least one state transition condition is met; and

,

configure said image processing function in said second detection state on the basis of a second set of function parameter values for processing a second image frame in said second detection state

a graphics generator connected to the dynamically configurable image processing module and configured to generate a graphical item associated with said detected object on the basis of said pose information.

18. The augmented reality device according to claim 16, further comprising a communication module configured to access said content database and/or said feature database via a data communication network.

17. An augmented reality system comprising:

a feature database comprising reference features associated with one or more target objects, said one or more target objects being identified by object identifiers;

a content database comprising one or more content items associated with said target objects, said one or more content items being stored together with one or more object identifiers;

at least one augmented reality device, wherein said augmented reality device is connected to said feature database and configured to;

retrieve reference features from said feature database on the basis of one or more object identifiers; and

,

retrieve one or more content items associated with one or more objects on the basis of said object identifiers.

19. A computer program product, implemented on computer-readable non-transitory storage medium, the computer program product configured for, when run on a computer, executing a method for dynamically configuring an image processing function into at least a first and second detection state on the basis of function parameters, wherein transitions between said first and second detection states are determined by at least a first state transition condition and wherein said image processing function includes extracting features from an image frame, matching extracted features with reference features associated with one or more target objects and estimating pose information on the basis of matched features, said method comprising:

configuring said image processing function in a first detection state on the basis of a first set of function parameter values;

processing a first image frame in said first detection state;

monitoring said image processing function for occurrence of said at least first state transition condition; and

,

if said at least one state transition condition is met, configuring said image processing function in said second detection state on the basis of a second set of function parameter values for processing a second image frame in said second detection state.