System, methods, and apparatus for in-vehicle fiducial mark tracking and interpretation

• US 9,600,728 B2
• Filed: 12/29/2011
• Issued: 03/21/2017
• Est. Priority Date: 12/29/2011
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Montesinos, Carlos, Graumann, David L., Healey, Jennifer
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): UHL, LINDSAY JANE KILE
• Application Number:
  US13/977,611
• Publication Number:
  US 20150317527A1
• Time in Patent Office: 1,909 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Certain embodiments of the invention may include systems, methods, and apparatus for in vehicle fiducial mark tracking and interpretation According to an example embodiment of the invention, a method is provided for executing computer executable instructions by one or more processors. The method includes receiving the one or more images from at least one camera attached to a vehicle; selecting, from the one or more received images, a region of interest associated with an object in the vehicle; identifying fiducial marks within the selected region of interest by comparing the identified fiducial marks with one or more stored fiducial representations; determining, based at least in part on the comparison, spatial information associated with the object or an occupant of the vehicle; and sending information based at least in part on the determined spatial information.

Claims

1. A method comprising executing computer-executable instructions by one or more processors for:

receiving one or more images from at least one camera attached to a vehicle;

selecting, from the one or more received images, a region of interest associated with an object in the vehicle, wherein the region of interest includes fiducial marks;

determining a fiducial mark representation by using the selected region of interest;

determining an alignment grid from the fiducial mark representation;

determining an offset between the alignment grid and an alignment pattern;

adjusting a portion of the fiducial mark representation using the offset;

identifying at least one occlusion pattern within the selected region of interest by comparing the portion of the fiducial mark representation with one or more stored fiducial representations;

determining, based at least in part on the at least one occlusion pattern, spatial information associated with the object or an occupant of the vehicle; and

sending information based at least in part on the determined spatial information.

2. The method of claim 1, wherein the spatial information comprises one or more of:

position, angle, orientation, linear velocity, or angular velocity associated with the object.

3. The method of claim 1, wherein the spatial information comprises one or more of hand position, hand grip, hand pressure, hand gesture, driving response, driving accuracy, safety belt tension, body position, body movement, or intent associated with the occupant of the vehicle.

4. The method of claim 1, wherein the fiducial marks comprise an optically identifiable pattern that varies with respect to a frame of reference associated with the object.

5. The method of claim 4, wherein at least a portion of the pattern represents a position or angle associated with the object.

6. The method of claim 1, further comprising estimating the intent of the occupant of the vehicle based at least in part on the determined spatial information.

7. The method of claim 1, wherein selecting a region of interest comprises selecting at least a portion of a steering wheel or at least a portion of a safety belt.

8. The method of claim 1, wherein sending information comprises sending one or more of steering wheel position, steering wheel velocity, hand position on the steering wheel, fingertip location on the steering wheel, safety belt shape, or safety belt position.

9. A vehicle comprising:

at least one camera;

one or more objects comprising fiducial marks that are in a field of view of the at least one camera; and

one or more processors configured for;

receiving one or more images from the at least one camera;

selecting, from the one or more received images, a region of interest associated with the one or more objects, wherein the region of interest include fiducial marks;

determining a fiducial mark representation by using the selected region of interest;

determining an alignment grid from the fiducial mark representation;

determining an offset between the alignment grid and an alignment pattern;

adjusting a portion of the fiducial mark representation using the offset;

identifying at least one occlusion pattern within the selected region of interest by comparing the portion of the fiducial mark representation with one or more stored fiducial representations;

determining, based at least in part on the at least one occlusion pattern, spatial information associated with the one or more objects or an occupant of the vehicle; and

sending information based at least in part on the determined spatial information.

10. The vehicle of claim 9, wherein the one or more objects comprise one or more of a steering wheel or a safety belt.

11. The vehicle of claim 9, wherein the at least one camera is one or more of a visible-light sensitive camera, a visible-light sensitive video camera, an infrared-sensitive camera, or an infrared-sensitive video camera.

12. The vehicle of claim 9, wherein the spatial information comprises one or more of:

position, angle, orientation, linear velocity, or angular velocity associated with the one or more objects.

13. The vehicle of claim 9, wherein the spatial information comprises one or more of hand position, hand grip, hand pressure, hand gesture, driving response, driving accuracy, safety belt tension, body position, body movement, or intent associated with the occupant of the vehicle.

14. The vehicle of claim 9, wherein the fiducial marks comprise an optically identifiable pattern that varies with respect to a frame of reference associated with the one or more objects.

15. The vehicle of claim 14, wherein at least a portion of the pattern represents a position or angle associated with the one or more objects.

16. The vehicle of claim 9, wherein the one or more processors are further configured for estimating the intent of the occupant of the vehicle based at least in part on the determined spatial information.

17. The vehicle of claim 9, wherein sending the information comprises sending one or more or steering wheel position, steering wheel velocity, hand position on the steering wheel, fingertip location on the steering wheel, safety belt shape, or safety belt position.

18. An apparatus comprising:

at least one memory for storing data and computer-executable instructions; and

one or more processors configured to access the at least one memory and further configured to execute computer-executable instructions for;

receiving one or more images from a camera;

selecting, from the one or more received images, a region of interest associated with one or more objects in a field of view of the camera, wherein the region of interest include fiducial marks;

determining a fiducial mark representation by using the selected region of interest;

determining an alignment grid from the fiducial mark representation;

determining an offset between the alignment grid and an alignment pattern;

adjusting a portion of the fiducial mark representation using the offset;

identifying at least one occlusion pattern within the selected region of interest by comparing the portion of the fiducial mark representation with one or more stored fiducial representations;

determining, based at least in part on the at least one occlusion pattern, spatial information associated with the one or more objects or an occupant of the vehicle; and

sending information based at least in part on the determined spatial information.

19. The apparatus of claim 18, wherein the one or more objects comprise one or more of a steering wheel or a safety belt.

20. The apparatus of claim 18, wherein the camera is a video camera.

21. The apparatus of claim 18, wherein the spatial information comprises one or more of:

position, angle, orientation, linear velocity, or angular velocity associated with the one or more objects.

22. The apparatus of claim 18, wherein the spatial information comprises one or more of hand position, hand grip, hand pressure, hand gesture, driving response, driving accuracy, safety belt tension, body position, body movement, or intent associated with the occupant of the vehicle.

23. The apparatus of claim 18, wherein the fiducial marks comprise an optically identifiable pattern that varies with respect to a frame of reference associated with the one or more objects.

24. The apparatus of claim 23, wherein at least a portion of the pattern represents a position or angle associated with the one or more objects.

25. The apparatus of claim 18, wherein the one or more processors are further configured for estimating the intent of the occupant of the vehicle based at least in part on the determined spatial information.

26. One or more computer-readable media comprising computer-executable instructions that, when executed by one or more processors, configure the one or more processors to:

receive one or more images from a camera;

select, from the one or more received images, a region of interest associated with an object in the vehicle, wherein the region of interest include fiducial marks;

determine a fiducial mark representation by using the selected region of interest;

determine an alignment grid from the fiducial mark representation;

determine an offset between the alignment grid and an alignment pattern;

adjust a portion of the fiducial mark representation using the offset;

identify at least one occlusion pattern within the selected region of interest by comparing the portion of the fiducial mark representation with one or more stored fiducial representations;

determine, based at least in part on the at least one occlusion pattern, spatial information associated with the object or an occupant of the vehicle; and

send information based at least in part on the determined spatial information.

27. The computer-readable media of claim 26, wherein the spatial information comprises one or more of position, angle, orientation, linear velocity, or angular velocity associated with the object.

28. The computer-readable media of claim 26, wherein the spatial information comprises one or more of hand position, hand grip, hand pressure, hand gesture, driving response, driving accuracy, safety belt tension, body position, body movement, or intent associated with the occupant of the vehicle.

29. The computer-readable media of claim 26, wherein the fiducial marks comprise an optically identifiable pattern that varies with respect to a frame of reference associated with the object, wherein at least a portion of the pattern represents a position or angle associated with the object.

30. The computer-readable media of claim 26, wherein the computer-executable instructions further configure the one or more processors to estimate the intent of the occupant of the vehicle based at least in part on the determined spatial information.