Method system and associated modules and software components for providing image sensor based human machine interfacing

• US 8,548,258 B2
• Filed: 10/25/2009
• Issued: 10/01/2013
• Est. Priority Date: 10/24/2008
• Status: Expired due to Fees

• Current Assignee: Extreme Reality Ltd.
• Original Assignee: Extreme Reality Ltd.
• Inventors: Givon, Dor, Elichai, Yoram
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US13/125,375
• Publication Number:
  US 20110286673A1
• Time in Patent Office: 1,437 Days
• Field of Search: 382/173, 382/224, 382/225
• US Class Current: 382/224
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

Disclosed is an image based human machine interface which comprises an image acquisition assembly which may acquire one or more of two-dimensional images of a user, wherein substantially each image may be associated with a different point in time. The human machine interface further comprises a processing unit which may classify one or more groups of pixels as either background or foreground.

Claims

1. An image processing system comprising:

an image processing unit configured to receive two or more images of a scene acquired by an imaging assembly and to classify a first pixel group of one of the two or more images as background or foreground, wherein classification performed by the processing unit is at least partially based on a spatiotemporal deviation value (SDV), which SDV is indicative of a spatiotemporal deviation between the first pixel group and a corresponding pixel group in a different image.

2. The system according to claim 1, wherein said processing unit is further configured to determine whether the first pixel group is within a defined spatial proximity to a previously classified pixel group appearing in the same frame as the first pixel group.

3. The system according to claim 2, wherein said processing unit is further configured to compare at least one visual attribute of the first pixel group against corresponding visual attributes of the previously classified pixel group.

4. The system according to claim 1, wherein said processing unit is configured to determine whether the first pixel group is located within a region between at least two other previously classified pixel groups appearing in the same frame as the first pixel group.

5. The system according to claim 4, wherein said processing unit is further configured to compare at least one visual attribute of the first pixel group against corresponding visual attributes of the at least two other previously classified pixel groups.

6. The system according to claim 1, wherein said processing unit is further configured to classify a second pixel group as background or foreground at least partially based on a proximity of the second pixel group to a previously classified pixel group appearing in the same frame as the second pixel group.

7. The system according to claim 6, wherein said processing unit is configured to classify the second pixel group as foreground when the at least one other previously classified pixel groups is classified as foreground.

8. The system according to claim 7, wherein the at least one other previously classified pixel group is classified as foreground because of an associated spatiotemporal deviation value being above a threshold.

9. The system according to claim 7, wherein the at least one other previously classified pixel group is classified as foreground based on feature detection.

10. An image based human machine interface comprising:

an image acquisition assembly configured to acquire a set of two-dimensional images of a user, wherein substantially two or more images of the set are each associated with a different point in time; and

a processing unit configured to derive one or more estimated user body positions by classifying one or more pixel group(s) as background or foreground at least partially based on a spatiotemporal deviation value (SDV), which SDV is indicative of a spatiotemporal deviation between the one or more pixel groups and corresponding pixel groups in a different image of the set.

11. The image based human machine interface according to claim 10, wherein classification of a first pixel group as either background or foreground is at least partially based on an SDV of the first pixel group and on a spatial relation between the first pixel group and at least one other previously classified pixel group appearing in the same frame as the first pixel group.

12. The image based human machine interface according to claim 11, wherein an SDV of a given pixel group being above a threshold results in the given pixel group being classified as foreground unless the SDV is characterized as the result of a High SDV Background Exception (“

HSBE”

).

13. The image based human machine interface according to claim 12, wherein the HSBE is one or more phenomenon selected from the group consisting of:

static motion in the frame, dynamic motion in the frame, non-unified motion in the frame, unified motion in the frame, a shift in the image acquisition assembly and a lighting shift.

14. The image based human machine interface according to claim 13, wherein a pixel group whose SDV is characterized as a result of a shift in the image acquisition assembly triggers said processing unit to reinitiate background classification processing.

15. The image based human machine interface according to claim 11, wherein an SDV of a given pixel group being below a threshold results in the given pixel group being classified as background unless the SDV is characterized as the result of a Low SDV Foreground Exception (“

LSFE”

).

16. The image based human machine interface according to claim 15, wherein the LSFE is one or more phenomenon selected from the group consisting of a shift in the image acquisition assembly, a lighting shift, a near predefined element exception and a near foreground exception.

17. The image based human machine interface according to claim 11, wherein said processing unit is configured to classify the first pixel group as foreground when a second pixel group appearing in the same frame as the first pixel group is characterized as foreground and the first pixel group and the second pixel group both have one or more similar traits or visual attributes.

18. The image based human machine interface according to claim 11, wherein said processing unit is configured to classify the first pixel group as foreground when a second pixel group appearing in the same frame as the first pixel group is consistent with a predefined element or feature and the first pixel group and the second pixel group are within a predefined proximity of one another.

19. The image based human machine interface according to claim 13, wherein a pixel group whose SDV is characterized as a result of a lighting shift is not reclassified.

20. The image based human machine interface according to claim 13, wherein a pixel group whose SDV is characterized as a result of a shift in the image acquisition assembly triggers said unit to perform pixel offset compensation based on a detected change of position of said image acquisition assembly.

21. The image based human machine interface according to claim 20, wherein pixel offset compensation includes correlation of pixel group data associated with a first assembly position with a second assembly position.

22. An image processing method comprising:

acquiring a set of two-dimensional images of a user, wherein substantially each image of the set is associated with a different point in time; and

processing one or more estimated user body position by classifying one or more pixel group(s) as background or foreground at least partially based on spatiotemporal deviation values (SDVs) determined for each of the one or more pixel groups, which SDVs are indicative of spatiotemporal deviations between each of the one or more pixel groups and corresponding pixel groups in a different image.

23. The method according to claim 22, wherein classifying a given pixel group of the one or more pixel groups as either background or foreground is at least partially based on the measured SDV of the given pixel group and a spatial relationship between the given pixel group and at least one other previously classified pixel group appearing in the same frame as the given pixel group.

24. The method according to claim 23, wherein detecting the SDV of the given pixel group above a threshold results in the given pixel group being classified as foreground unless the SDV is characterized as the result of a High SDV Background Exception (“

HSBE”

).

25. The method according to claim 24, wherein the HSBE is one or more phenomenon selected from the group consisting of:

static motion in the frame, dynamic motion in the frame, non-unified motion in the frame, unified motion in the frame, a shift in the image acquisition assembly and a lighting shift.