Wearable behavior-based vision system
First Claim
1. A see through display apparatus, comprising:
- a see-through, head mounted display;
a plurality of sensors cooperating with the display to detect at least visual data in a field of view of the apparatus; and
one or more processors in communication with the display and the sensors, the one or more processors automatically;
establish a plurality of behavioral zones representing a scene mapped to a physical environment that is relative to the apparatus such that each of the plurality of behavioral zones are segmented into different types of information having different defined behaviors that relate to at least one real object within a corresponding zone, wherein a first of the behavioral zones surrounds a wearer of the see-through head mounted display and is proximate to the wearer compared to a second of the behavioral zones, and the second of the behavioral zones encompasses the first behavioral zone and includes objects registered to a global space;
determine the at least one real object proximate to the apparatus using one of geographic position data and data from the sensors, the at least one real object associated with one of the plurality of behavioral zones;
assign one or more of the behaviors with the at least one real object based on the association; and
provide a data definition, for the at least one real object, comprising the one or more behaviors to a feedback application for the apparatus.
2 Assignments
0 Petitions
Accused Products
Abstract
A see through display apparatus includes a see-through, head mounted display and sensors on the display which detect audible and visual data in a field of view of the apparatus. A processor cooperates with the display to provide information to a wearer of the device using a behavior-based real object mapping system. At least a global zone and an egocentric behavioral zone relative to the apparatus are established, and real objects assigned behaviors that are mapped to the respective zones occupied by the object. The behaviors assigned to the objects can be used by applications that provide services to the wearer, using the behaviors as the foundation for evaluation of the type of feedback to provide in the apparatus.
21 Citations
19 Claims
-
1. A see through display apparatus, comprising:
-
a see-through, head mounted display; a plurality of sensors cooperating with the display to detect at least visual data in a field of view of the apparatus; and one or more processors in communication with the display and the sensors, the one or more processors automatically; establish a plurality of behavioral zones representing a scene mapped to a physical environment that is relative to the apparatus such that each of the plurality of behavioral zones are segmented into different types of information having different defined behaviors that relate to at least one real object within a corresponding zone, wherein a first of the behavioral zones surrounds a wearer of the see-through head mounted display and is proximate to the wearer compared to a second of the behavioral zones, and the second of the behavioral zones encompasses the first behavioral zone and includes objects registered to a global space; determine the at least one real object proximate to the apparatus using one of geographic position data and data from the sensors, the at least one real object associated with one of the plurality of behavioral zones; assign one or more of the behaviors with the at least one real object based on the association; and provide a data definition, for the at least one real object, comprising the one or more behaviors to a feedback application for the apparatus. - View Dependent Claims (2, 3, 4, 5, 6, 16, 17, 18, 19)
-
-
7. A computer storage device including instructions to perform a method of providing a vision system, the instructions when executed by a processor comprising:
-
identifying a plurality of real objects in a global behavioral zone using geographic position data and in an egocentric behavioral zone using sensor data, such that the global behavioral zone corresponds to a physical environment and the egocentric behavioral zone is a sub-zone of the global behavioral zone, the global behavioral zone and egocentric behavioral zones are segmented into different types of information having different defined behaviors that relate to at least one real object within a corresponding zone, the defined behaviors constrained within one of the global and egocentric behavioral zones and interaction with a real object such that the egocentric behavioral zone surrounds a wearer wearing a see-through head mounted display and the global behavioral zone encompasses the egocentric behavioral zone; assigning one or more of a first plurality of behaviors and a second plurality of behaviors to each real object identified in the global behavior zone and the egocentric behavioral zone, respectively, the behavior comprising a behavioral primitive rule for the wearer of the see through head mounted display and being dependent on an expected user behavior relative to the object in each behavioral zone; and for at least the real object proximate to a location, providing a data definition, for the at least one real object, comprising the one or more behaviors to a feedback application for the system. - View Dependent Claims (8, 9, 10, 11, 12, 13)
-
-
14. A computer implemented method for a see through display apparatus comprising:
-
receiving visual sensor data from a plurality of sensors cooperating with the display; identifying a plurality of real objects in a global behavioral zone using geographic position data, and in an egocentric behavioral zone using the sensor data, such that the global behavioral zone and the egocentric behavioral zone are defined by a proximate and physical environment and are segmented into different types of information having different defined behaviors that relate to at least one real object within a corresponding zone, the defined behaviors constrained within one of the global and egocentric behavioral zones and interaction with a real object such that the egocentric behavioral zone surrounds a wearer wearing a see-through head mounted display and the global behavioral zone encompasses the egocentric behavioral zone; assigning one or more of a first plurality of behaviors for the global behavioral zone to each real object identified in the global behavioral zone; assigning one or more of a second plurality of behaviors for the egocentric behavioral zone to each real object determined to be in the egocentric behavioral zone; and providing a data definition, for the at least one real object, comprising the behaviors to a feedback application. - View Dependent Claims (15)
-
Specification