Intelligent translations in personal see through display
First Claim
1. A method for presenting a translation of a real world expression to a wearer of a see-through head mounted display apparatus, comprising:
- determining a three dimensional location of one or more objects in a field of view of the wearer through the see-through head mounted display apparatus, the determining of the three dimensional location of the object is performed using one or more sensors;
receiving a selection of data for translation in the field of view of the wearer by reference to a gaze of the wearer at the item of data;
analyzing the data for translation to provide input data, the input data having one or more of a linguistic translation input, supplemental information input and contextual input;
analyzing the input data to determine whether each of a linguistic translation, an informational translation, and a context translation should be provided based on the input data and a wearer profile, wherein the linguistic translation is a translation from one language to another, the information translation provides additional information regarding the selected item and the contextual translation is a translation based at least in part on a context of another user providing the input data, wherein the linguistic, informational and context translations are different types of translations;
determining wearer knowledge of subject matter in the analyzed input data from the wearer profile;
determining a complexity of the translation based on the determined wearer knowledge;
determining one of a plurality of output formats for the translation based on the input data;
translating the input data into a translated form for the wearer, the translation based on the determined translation type and the determined translated complexity, the translated form based on the determined one of the plurality of output formats; and
rendering the translation in at least one of the plurality of output formats determined, the output format selected from at least an audio translation format and a visual translation format rendered in the see-through head mounted display.
2 Assignments
0 Petitions
Accused Products
Abstract
A see-through, near-eye, mixed reality display apparatus for providing translations of real world data for a user. A wearer'"'"'s location and orientation with the apparatus is determined and input data for translation is selected using sensors of the apparatus. Input data can be audio or visual in nature, and selected by reference to the gaze of a wearer. The input data is translated for the user relative to user profile information bearing on accuracy of a translation and determining from the input data whether a linguistic translation, knowledge addition translation or context translation is useful.
87 Citations
13 Claims
-
1. A method for presenting a translation of a real world expression to a wearer of a see-through head mounted display apparatus, comprising:
-
determining a three dimensional location of one or more objects in a field of view of the wearer through the see-through head mounted display apparatus, the determining of the three dimensional location of the object is performed using one or more sensors; receiving a selection of data for translation in the field of view of the wearer by reference to a gaze of the wearer at the item of data; analyzing the data for translation to provide input data, the input data having one or more of a linguistic translation input, supplemental information input and contextual input; analyzing the input data to determine whether each of a linguistic translation, an informational translation, and a context translation should be provided based on the input data and a wearer profile, wherein the linguistic translation is a translation from one language to another, the information translation provides additional information regarding the selected item and the contextual translation is a translation based at least in part on a context of another user providing the input data, wherein the linguistic, informational and context translations are different types of translations; determining wearer knowledge of subject matter in the analyzed input data from the wearer profile; determining a complexity of the translation based on the determined wearer knowledge; determining one of a plurality of output formats for the translation based on the input data; translating the input data into a translated form for the wearer, the translation based on the determined translation type and the determined translated complexity, the translated form based on the determined one of the plurality of output formats; and rendering the translation in at least one of the plurality of output formats determined, the output format selected from at least an audio translation format and a visual translation format rendered in the see-through head mounted display. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for presenting a translation of information at a location to a wearer of a see-through head mounted display apparatus, comprising:
-
receiving from the wearer a selection of input data for translation at the location of the wearer; determining three dimensional locations of objects within a field of view of the wearer at the location; receiving a selection of input data based on an object within the field of view for translation in the field of view of the wearer; retrieving wearer profile information bearing on an accuracy of a translation; analyzing the input data to select a type of translation to perform, the types of translations comprising each of a linguistic translation from a first language to a second language, an informational translation which provides additional information regarding a particular subject or scene in the input data and a context translation which provides information on an emotional state of a person in the field of view should be provided based on the input data, wherein the linguistic, informational and context translations are different types of translations; determining wearer proficiency in a language of subject matter in the analyzed input data from the wearer profile information; determining a complexity of the translation based on the determined wearer knowledge; determining one of a plurality of output formats for the translation; performing one or more of the linguistic translation, the informational translation or the context translation on the input data based on determining a translation type and the determined translation complexity, the translation based on wearer feedback on translation accuracy of previous translations; and rendering the translation in at least one of the plurality of output formats, the plurality of output formats comprising an audio output format and a visual output format in the see-through head mounted display. - View Dependent Claims (11, 12, 13)
-
Specification