Video feature tagging

• US 10,129,530 B2
• Filed: 09/25/2015
• Issued: 11/13/2018
• Est. Priority Date: 09/25/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Bleiweiss, Amit
• Primary Examiner(s): Pham, Nam D
• Application Number:
  US14/865,403
• Publication Number:
  US 20170094252A1
• Time in Patent Office: 1,145 Days
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters

Abstract

An activity recording system is provided. The activity recording system includes a three-dimensional camera, a sensor arrangement that is fitted to a subject being recorded, and an activity recording device. The activity recording device receives image information from the three-dimensional camera and sensor arrangement information from the sensor arrangement. Both the image information and the sensor arrangement information include location measurements. The sensor arrangement information is generated by location sensors that are positioned at target features of the subject to be tracked. The sensor arrangement information is a key to the image information that specifies where, in any given image, the target features of the subject lie. Activity data having these characteristics may be applied to solve a variety of system development problems. Such activity data can be used to training machine learning components or test computer vision components for a fraction of the cost of using conventional techniques.

Claims

1. An activity recording system comprising an activity recording device including:

a memory; and

at least one processor coupled to the memory and configured to;

receive image information comprising at least one first three-dimensional measurement of at least one location of a subject;

receive sensor arrangement information comprising at least one second three-dimensional measurement of at least one sensor positioned on the subject at the at least one location; and

store at least one association between the at least one first three-dimensional measurement and the at least one second three-dimensional measurement.

2. The activity recording system of claim 1, wherein the image information depicts a human hand including one or more fingers and the at least one location includes one or more locations of the one or more fingers.

3. The activity recording system of claim 1, wherein the at least one processor is further configured to:

store, in the memory, image data based on the image information;

store, in the memory, sensor arrangement data based on the sensor arrangement information; and

synchronize the sensor arrangement data to the image data.

4. The activity recording system of claim 3, wherein the at least one processor is further configured to synchronize the sensor arrangement data to the image data at least in part by being configured to generate one or more fabricated sensor readings.

5. The activity recording system of claim 1, wherein the at least one processor is further configured to:

receive initial image information;

receive initial sensor arrangement information; and

generate calibration data using the initial image information and the initial sensor arrangement information.

6. The activity recording system of claim 1, further comprising a three-dimensional camera configured to communicate data with the activity recording system.

7. The activity recording system of claim 1, further comprising a sensor arrangement configured to communicate data with the activity recording system, the sensor arrangement comprising the at least one sensor.

8. The activity recording system of claim 7, wherein the sensor arrangement includes 22 sensors.

9. The activity recording system of claim 7, wherein the sensor arrangement includes 6 sensors.

10. A method of recording activity using a device, the method comprising:

receiving image information comprising at least one first three-dimensional measurement of at least one location of a subject;

receiving sensor arrangement information comprising at least one second three-dimensional measurement of at least one sensor positioned on the subject at the at least one location; and

storing at least one association between the at least one first three-dimensional measurement and the at least one second three-dimensional measurement.

11. The method of claim 10, wherein receiving the image information includes receiving image information depicting a human hand including one or more fingers and indicating one or more locations of the one or more fingers.

12. The method of claim 10, further comprising:

storing, in a memory of the device, image data based on the image information;

storing, in the memory, sensor arrangement data based on the sensor arrangement information; and

synchronizing the sensor arrangement data to the image data.

13. The method of claim 12, wherein synchronizing the sensor arrangement data to the image data includes to generating one or more fabricated sensor readings.

14. The method of claim 12, further comprising training a machine learning component with the image data, the sensor arrangement data, and the at least one association.

15. The method of claim 12, further comprising testing a computer vision component with the image data, the sensor arrangement data, and the at least one association.

16. The method of claim 10, further comprising:

receiving initial image information;

receiving initial sensor arrangement information; and

generating calibration data using the initial image information and the initial sensor arrangement information.

17. The method of claim 10, wherein receiving the image information includes receiving image information from a three-dimensional camera.

18. A non-transitory computer program product encoded with instructions that when executed by one or more processors cause a process for recording activity to be carried out, the process comprising:

receiving image information comprising at least one first three-dimensional measurement of at least one location of a subject;

receiving sensor arrangement information comprising at least one second three-dimensional measurement of at least one sensor positioned on the subject at the at least one location; and

storing at least one association between the at least one first three-dimensional measurement and the at least one second three-dimensional measurement.

19. The computer program product of claim 18, wherein receiving the image information includes receiving image information depicting a human hand including one or more fingers and indicating one or more locations of the one or more fingers.

20. The computer program product of claim 18, the process further comprising:

storing, in a memory, image data based on the image information;

storing, in the memory, sensor arrangement data based on the sensor arrangement information; and

synchronizing the sensor arrangement data to the image data.

21. The computer program product of claim 20, wherein synchronizing the sensor arrangement data to the image data includes to generating one or more fabricated sensor readings.

22. The computer program product of claim 20, the process further comprising training a machine learning component with the image data, the sensor arrangement data, and the at least one association.

23. The computer program product of claim 20, the process further comprising testing a computer vision component with the image data, the sensor arrangement data, and the at least one association.

24. The computer program product of claim 18, the process further comprising:

receiving initial image information;

receiving initial sensor arrangement information; and

generating calibration data using the initial image information and the initial sensor arrangement information.

25. The computer program product of claim 18, wherein receiving the image information includes receiving image information from a three-dimensional camera.