Wearable Behavior-Based Vision System

• US 20140198017A1
• Filed: 01/12/2013
• Published: 07/17/2014
• Est. Priority Date: 01/12/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lamb, Mathew J., Kipman, Alex Aben-Athar
• Granted Patent: US 9,395,543 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

A see through display apparatus includes a see-through, head mounted display and sensors on the display which detect audible and visual data in a field of view of the apparatus. A processor cooperates with the display to provide information to a wearer of the device using a behavior-based real object mapping system. At least a global zone and an egocentric behavioral zone relative to the apparatus are established, and real objects assigned behaviors that are mapped to the respective zones occupied by the object. The behaviors assigned to the objects can be used by applications that provide services to the wearer, using the behaviors as the foundation for evaluation of the type of feedback to provide in the apparatus.

Claims

1. A see through display apparatus, comprising:

a see-through, head mounted display;

a plurality of sensors cooperating with the display to detect audible and visual data in a field of view of the apparatus;

one or more processing devices in communication with display and the sensors, the one or more processing devices automatically

establish a plurality of behavioral zones relative to the apparatus;

determine at least one real object proximate to the apparatus, the at least one real object associated with one of the plurality of behavioral zones;

assign one or more behaviors with the real object based on the association; and

provide the one or more behaviors and the object to a feedback application for the apparatus.

2. The apparatus of claim 1 wherein the one or more processing devices establish at least a global behavioral zone and an egocentric behavioral zone.

3. The apparatus of claim 2 wherein the one or more processing devices determine the egocentric behavioral zone based on input data from the plurality of sensors.

4. The apparatus of claim 2 wherein the global behavioral zone includes real object data mapped to geographical position data for real objects.

5. The apparatus of claim 1 wherein each behavior comprises an interaction rule for a real object relative to a wearer of the apparatus.

6. The apparatus of claim 1 wherein each behavioral zone includes at one or more unique behaviors associated with said behavioral zone.

7. The apparatus of claim 1 wherein the one or more processing devices receive feedback generated by the application relative to real object data.

8. The apparatus of claim 1 wherein the one or more processing devices render the feedback in the display based on instructions from the application.

9. A computer readable medium including instructions operable on a processing device to perform a method of providing a vision system comprising the steps of:

identifying a plurality of real objects in a global behavioral zone and in an egocentric behavioral zone;

assigning one or more behaviors to each real object identified, the behavior comprising a primitive interaction rule for a wearer of a see through head mounted display and being dependent on an expected user behavior relative to the object in each behavioral zone;

determining real objects proximate to a location of the see through head mounted display; and

for at least the real object proximate to the location, providing the behaviors and the real object to a feedback application for the system.

10. The computer readable medium of claim 9 wherein the behaviors comprise one of danger, avoidance, touch, and look.

11. The computer readable medium of claim 9 further including the step of creating a data definition for each of the plurality of real objects, the data definition including data identifying the real object to the device.

12. The computer readable medium of claim 11 wherein the data identifying the real object to the device is geographical position data for real objects.

13. The computer readable medium of claim 12 wherein data identifying the real object to the device is input data from a plurality of sensors on the device.

14. The computer readable medium of claim 13 the egocentric zone includes interactive behaviors with real objects.

15. The computer readable medium of claim 14 further including providing feedback generated by the application relative to one or more of the real objects.

16. A computer implemented method defining a plurality of real objects relative to a processing device, the processing device including a display, comprising:

defining a plurality of behaviors for a global behavioral zone;

defining a plurality of behaviors for an egocentric behavioral zone;

identifying a plurality of real objects in a global behavioral zone and in an egocentric behavioral zone;

assigning one or more of the plurality of behaviors for the global behavioral zone to each real object identified in the global behavioral zone

determining real objects proximate to a location of the processing device

assigning one or more of the plurality of behaviors for the egocentric behavioral zone to each real object determined to be in the egocentric behavioral zone; and

providing the behaviors and the real object to a feedback application.

17. The method of claim 16 further including creating a data definition for each of the plurality of real objects, the data definition including data identifying the real object to the device.

18. The method of claim 17 wherein the data definition identifying the real object in the global behavioral zone is geographical position data for real objects.

19. The method of claim 18 wherein data identifying the real object to the device is input data from a plurality of sensors providing imaging data to the processing device.

20. The method of claim 19 the egocentric zone includes interactive behaviors with real objects.