Image capture system

• US 9,072,929 B1
• Filed: 11/30/2012
• Issued: 07/07/2015
• Est. Priority Date: 12/01/2011
• Status: Active Grant

• Current Assignee: Nebraska Global Investment Company, LLC
• Original Assignee: Nebraska Global Investment Company, LLC
• Inventors: Rush, Benjamin D., Brown-Kramer, Joshua M., Sabalka, Lucas A., Lowry, Nathan H.
• Primary Examiner(s): Yentrapati, Avinash
• Application Number:
  US13/690,261
• Time in Patent Office: 949 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A63B 2071/0652 Visualisation or indication relating to symmetrical exercise, e.g. right-left performance related to spinal column
• CPC Class Codes: A63B 21/0724 Bar-bells; Hand bars
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/10 Positions
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

A system is described that allows for the tracking of an exercise occurring within a physical environment. In one or more implementations, the system includes a depth sensing device configured to obtain depth values associated with a plurality of pixels. The depth values indicating distances from one or more physical objects in a physical environment to the depth sensing device. The system also includes a computing device in communication with the depth sensing device. The computing device includes a memory and a processor. The processor is configured to execute the one or more modules to cause the processor to: identify a point corresponding to at least one pixel representing a portion of at least one physical object within the physical environment based upon the pixel depth values; track the point through a plurality of image frames; and determine whether at least one repetition has occurred based upon the tracked point.

Claims

1. A computer implemented method having access to memory, the method providing a real-time three-dimensional representation of a physical environment captured by a depth sensing device, the computer implemented method comprising:

obtaining depth values associated with a plurality of pixels indicating distances from one or more physical objects in the physical environment to the depth sensing device;

identifying a point corresponding to at least one pixel of the plurality of pixels as representing a portion of at least one physical object of the one or more physical objects within the physical environment based upon the pixel depth values, the at least one object utilized to perform an exercise, wherein identifying the point comprises selecting a starting pixel from the plurality of pixels;

selecting a plurality pixels from the plurality of pixels;

identifying at least two offset pixels of the plurality of offset pixels that are at least substantially equidistant and at least substantially collinear with the starting pixel; and

selecting the starting pixel as the point;

tracking the point through a plurality of image frames, the image frames representing the physical environment; and

determining whether at least one repetition has occurred based upon the tracked point.

2. The computer implemented method as recited in claim 1, wherein the at least one physical object comprises a weight bar.

3. The computer implemented method as recited in claim 1, further comprising determining an instantaneous velocity associated with the at least one physical object.

4. The computer implemented method as recited in claim 3, wherein

determining an instantaneous velocity comprises determining a change in position of the identified point between a first image frame and a second image frame;

and dividing the change in position by a time difference between the first image frame and the second image frame.

5. The computer implemented method as recited in claim 1, applying a line through the starting pixel and the at least two offset pixels to represent the object.

6. The computer implemented method as recited in claim 1, wherein the plurality of image frames comprise a plurality of successive image frames.

7. A system comprising:

a depth sensing device configured to obtain depth values associated with a plurality of pixels, the depth values indicating distances from one or more physical objects in a physical environment to the depth sensing device;

at least one computing device in communication with the depth sensing device, the at least one computing device including;

a memory configured to store one or more modules;

a processor coupled to the memory, the processor is configured to execute the one or more modules to cause the processor to;

identify a point corresponding to at least one pixel of the plurality of pixels as representing a portion of at least one physical object of the one or more physical objects within the physical environment based upon the pixel depth values, the at least one object utilized to perform an exercise, wherein identify the point comprises selecting a starting pixel from the plurality of pixels;

selecting a plurality of offset pixels from the plurality pixels;

identify at least two offset pixels of the plurality of offset pixels that are at least substantially equidistant and at least substantially collinear with the starting pixel; and

select the starting pixel as the point;

track the point through a plurality of image frames, the image frames including the plurality of pixels representing the physical environment; and

determine whether at least one repetition has occurred based upon the tracked point.

8. The system as recited in claim 7, wherein the one or more modules, when executed by the processor, are configured to cause the processor to determine an instantaneous velocity associated with the at least one physical object.

9. The system as recited in claim 7, wherein the instantaneous velocity is determined by determining a change in position of the identified point between a first image frame and a second image frame;

and dividing the change in position by a time difference between the first image frame and the second image frame.

10. The system as recited in claim 7, wherein the one or more modules,

when executed by the processor, are configured to apply a line through the starting pixel and the at least two offset pixels to represent the object.

11. The system as recited in claim 7, wherein the plurality of image frames comprise a plurality of successive image frames.

12. The system as recited in claim 7, wherein the depth sensing device comprises red-green-blue depth (RGB-D) cameras.

13. The system as recited in claim 7, wherein the object is a weight bar.

14. A non-transitory computer-readable medium embodying a program executable in at least one computing device, the program comprising:

code that generates a depth mask that represents a captured physical environment via a depth sensing device, the depth mask associated with a plurality of pixels representing the physical environment;

code to identify one or more pixels as representing a bed within the physical environment;

code that identifies one or more pixels associated with the depth mask that, the one or more pixels representing a subject located within the physical environment, wherein the subject is identified by comparing a depth value of the one or more pixels representing the subject to a depth value of the one or more pixels representing the bed;

code that segments at least one subset of pixels from the one or more pixels to isolate at least one grouping of pixels that represents a body segment of the subject; and

code that tracks movement of the body segment through a plurality of image frames, the movement corresponding to an exercise performed by the subject.

15. The non-transitory computer-readable medium as recited in claim 14, further comprising code to generate at least one near real-time instruction at a display, the at least one near real-time instruction corresponding to the movement of the body segment.

16. The non-transitory computer-readable medium as recited in claim 14, further comprising code to define at least one pixel of the one or more pixels as representing at least one of a wall within the physical environment or a floor within the physical environment.

17. The non-transitory computer-readable medium as recited in claim 14, wherein the depth sensing device comprises red-green-blue depth (RGB-D) cameras.