CORRELATING USER REACTIONS WITH AUGMENTATIONS DISPLAYED THROUGH AUGMENTED VIEWS

• US 20140101555A1
• Filed: 12/28/2012
• Published: 04/10/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr., Tegreene, Clarence T.
• Granted Patent: US 9,077,647 B2
• Time in Patent Office: Days
• US Class Current: 715/733
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]

Abstract

Computationally implemented methods and systems include receiving first data that at least identifies one or more augmentations that were remotely displayed in one or more remotely displayed augmented views of one or more actual scenes, receiving second data indicating one or more user reactions of one or more users in response to the remote display of the one or more remotely displayed augmented views; and correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1. A computationally-implemented method, comprising:

receiving first data that at least identifies one or more augmentations that were remotely displayed in one or more remotely displayed augmented views of one or more actual scenes;

receiving second data indicating one or more user reactions of one or more users in response to the remote display of the one or more remotely displayed augmented views; and

correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views.

2-90. -90. (canceled)

91. A computationally-implemented system, comprising:

means for receiving first data that at least identifies one or more augmentations that were remotely displayed in one or more remotely displayed augmented views of one or more actual scenes;

means for receiving second data indicating one or more user reactions of one or more users in response to the remote display of the one or more remotely displayed augmented views; and

means for or correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views.

111. The computationally-implemented system of claim 91, wherein said means for receiving first data that at least identifies one or more augmentations that were remotely displayed in one or more remotely displayed augmented views of one or more actual scenes comprises:

means for receiving the first data including receiving first data that indicates one or more aspects of one or more non-augmentation scene elements that were included in the one or more actual scenes that were obfuscated in the one or more remotely displayed augmented views by the inclusion of the one or more augmentations in the one or more remotely displayed augmented views.

116. The computationally-implemented system of claim 91, wherein said means for receiving second data indicating one or more user reactions of one or more users in response to the remote display of the one or more remotely displayed augmented views comprises:

means for receiving the second data by receiving second data that indicates one or more user reactions of the one or more users proximate to the remote display of the one or more remotely displayed augmented views.

118. The computationally-implemented system of claim 116, wherein said means for receiving the second data by receiving second data that indicates one or more user reactions of the one or more users proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for receiving the second data by receiving second data that indicates one or more physiological reactions of the one or more users proximate to the remote display of the one or more remotely displayed augmented views.

119. The computationally-implemented system of claim 118, wherein said means for receiving the second data by receiving second data that indicates one or more physiological reactions of the one or more users proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for receiving the second data that indicates the one or more physiological reactions of the one or more users by receiving second data that indicates one or more eye characteristics of the one or more users proximate to the remote display of the one or more remotely displayed augmented views.

128. The computationally-implemented system of claim 119, wherein said means for receiving the second data that indicates the one or more physiological reactions of the one or more users by receiving second data that indicates one or more eye characteristics of the one or more users proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for receiving the second data that indicates the one or more eye characteristics of the one or more users by receiving second data that indicates one or more pupil characteristics of one or more eyes of the one or more users proximate to the remote display of the one or more remotely displayed augmented views.

136. The computationally-implemented system of claim 91, wherein said means for receiving second data indicating one or more user reactions of one or more users in response to the remote display of the one or more remotely displayed augmented views comprises:

means for receiving the second data indicating the one or more user reactions of the one or more users by receiving second data that indicates one or more facial expressions, head positions, and/or head tilts of the one or more users proximate to the display of the one or more remotely displayed augmented views.

137. The computationally-implemented system of claim 91, wherein said means for correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views comprises:

means for correlating the one or more user reactions with the one or more augmentations by determining one or more relationships between the one or more user reactions and the one or more augmentations, the one or more user reactions having occurred proximate to the remote display of the one or more remotely displayed augmented views.

138. The computationally-implemented system of claim 137, wherein said means for correlating the one or more user reactions with the one or more augmentations by determining one or more relationships between the one or more user reactions and the one or more augmentations, the one or more user reactions having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more augmentations including determining one or more temporal relationships between the one or more user reactions and the one or more augmentations.

139. The computationally-implemented system of claim 137, wherein said means for correlating the one or more user reactions with the one or more augmentations by determining one or more relationships between the one or more user reactions and the one or more augmentations, the one or more user reactions having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more augmentations including determining one or more spatial relationships between the one or more user reactions and the one or more augmentations.

140. The computationally-implemented system of claim 137, wherein said means for correlating the one or more user reactions with the one or more augmentations by determining one or more relationships between the one or more user reactions and the one or more augmentations, the one or more user reactions having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more augmentations including determining one or more relationships between one or more physiological reactions of the one or more users and the one or more augmentations, the one or more physiological reactions having occurred proximate to the remote display of the one or more remotely displayed augmented views.

141. The computationally-implemented system of claim 140, wherein said means for determining the one or more relationships between the one or more user reactions and the one or more augmentations including determining one or more relationships between one or more physiological reactions of the one or more users and the one or more augmentations, the one or more physiological reactions having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between one or more physiological reactions of the one or more users and the one or more augmentations by determining one or more relationships between one or more eye characteristics of the one or more users and the one or more augmentations, the one or more eye characteristics having occurred proximate to the remote display of the one or more remotely displayed augmented views.

142. The computationally-implemented system of claim 141, wherein said means for determining the one or more relationships between one or more physiological reactions of the one or more users and the one or more augmentations by determining one or more relationships between one or more eye characteristics of the one or more users and the one or more augmentations, the one or more eye characteristics having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more eye characteristics of the one or more users and the one or more augmentations by determining one or more relationships between one or more eye movements of one or more eyes of the one or more users and the one or more augmentations, the one or more eye movements having occurred proximate to the remote display of the one or more remotely displayed augmented views.

143. The computationally-implemented system of claim 142, wherein said means for determining the one or more relationships between the one or more eye characteristics of the one or more users and the one or more augmentations by determining one or more relationships between one or more eye movements of one or more eyes of the one or more users and the one or more augmentations, the one or more eye movements having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more eye movements of the one or more eyes of the one or more users and the one or more augmentations by determining one or more relationships between one or more dwell paths of one or more eyes of the one or more users with respect to the one or more remotely displayed augmented views and the one or more augmentations, the one or more dwell paths of the one or more eyes of the one or more users having occurred proximate to the remote display of the one or more remotely displayed augmented views.

144. The computationally-implemented system of claim 143, wherein said means for determining the one or more relationships between the one or more eye movements of the one or more eyes of the one or more users and the one or more augmentations by determining one or more relationships between one or more dwell paths of one or more eyes of the one or more users with respect to the one or more remotely displayed augmented views and the one or more augmentations, the one or more dwell paths of the one or more eyes of the one or more users having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more dwell paths of one or more eyes of the one or more users and the one or more augmentations by determining one or more relationships between one or more dwell paths of the one or more eyes of the one or more users and the one or more augmentations, the one or more dwell paths having been as a result of eye focus of the one or more eyes of the one or more users moving away from the one or more augmentations included in the one or more remotely displayed augmented views.

146. The computationally-implemented system of claim 141, wherein said means for determining the one or more relationships between one or more physiological reactions of the one or more users and the one or more augmentations by determining one or more relationships between one or more eye characteristics of the one or more users and the one or more augmentations, the one or more eye characteristics having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more eye characteristics of the one or more users and the one or more augmentations by determining one or more relationships between eye focus of the one or more eyes of the one or more users with respect to the one or more remotely displayed augmented views and the one or more augmentations, the eye focus of the one or more eyes of the one or more users having occurred proximate to the remote display of the one or more remotely displayed augmented views.

149. The computationally-implemented system of claim 146, wherein said means for determining the one or more relationships between the one or more eye characteristics of the one or more users and the one or more augmentations by determining one or more relationships between eye focus of the one or more eyes of the one or more users with respect to the one or more remotely displayed augmented views and the one or more augmentations, the eye focus of the one or more eyes of the one or more users having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more eye focus of the one or more eyes of the one or more users with respect to the one or more remotely displayed augmented views and the one or more augmentations by determining one or more relationships between dwell time of the one or more eyes of the one or more users and the one or more augmentations, the dwell time of the one or more eyes of the one or more users being with respect to the one or more remotely displayed augmented views, and the dwell time having occurred proximate to the remote display of the one or more remotely displayed augmented views.

150. The computationally-implemented system of claim 141, wherein said means for determining the one or more relationships between one or more physiological reactions of the one or more users and the one or more augmentations by determining one or more relationships between one or more eye characteristics of the one or more users and the one or more augmentations, the one or more eye characteristics having occurred proximate to the remote display of the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more eye characteristics of the one or more users and the one or more augmentations by determining one or more relationships between one or more pupil characteristics of one or more eyes of the one or more users and the one or more augmentations, the one or more pupil characteristics having occurred proximate to the remote display of the one or more remotely displayed augmented views.

152. The computationally-implemented system of claim 91, wherein said means for correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views comprises:

means for correlating the one or more user reactions with the one or more augmentations by determining one or more relationships between one or more facial expressions, and/or head positions of the one or more users and the one or more augmentations, the one or more facial expressions, and/or head positions of the one or more users having occurred proximate to the remote display of the one or more remotely displayed augmented views.

153. The computationally-implemented system of claim 91, wherein said means for correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views comprises:

means for correlating the one or more user reactions with the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more aspects of the one or more augmentations.

154. The computationally-implemented system of claim 153, wherein said means for correlating the one or more user reactions with the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more aspects of the one or more augmentations comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more aspects of the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more forms of the one or more augmentations.

155. The computationally-implemented system of claim 154, wherein said means for determining the one or more relationships between the one or more user reactions and the one or more aspects of the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more forms of the one or more augmentations comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more forms of the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more shapes and/or sizes of the one or more augmentations.

156. The computationally-implemented system of claim 154, wherein said means for determining the one or more relationships between the one or more user reactions and the one or more aspects of the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more forms of the one or more augmentations comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more forms of the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more colors, brightness, and/or hue of the one or more augmentations.

157. The computationally-implemented system of claim 154, wherein said means for determining the one or more relationships between the one or more user reactions and the one or more aspects of the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more forms of the one or more augmentations comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more forms of the one or more augmentations by determining one or more relationships between the one or more user reactions and one or more locations of the one or more augmentations in the one or more remotely displayed augmented views.

158. The computationally-implemented system of claim 91, wherein said means for correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views comprises:

means for correlating the one or more user reactions with the one or more augmentations including determining one or more relationships between the one or more user reactions and one or more aspects of one or more non-augmentation scene elements included in the one or more remotely displayed augmented views.

159. The computationally-implemented system of claim 158, wherein said means for correlating the one or more user reactions with the one or more augmentations including determining one or more relationships between the one or more user reactions and one or more aspects of one or more non-augmentation scene elements included in the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more aspects of one or more non-augmentation scene elements by determining one or more relationships between the one or more user reactions and one or more forms of the one or more non-augmentation scene elements included in the one or more remotely displayed augmented views.

160. The computationally-implemented system of claim 159, wherein said means for determining the one or more relationships between the one or more user reactions and the one or more aspects of one or more non-augmentation scene elements by determining one or more relationships between the one or more user reactions and one or more forms of the one or more non-augmentation scene elements included in the one or more remotely displayed augmented views comprises:

means for determining the one or more relationships between the one or more user reactions and the one or more forms of the one or more non-augmentation scene elements by determining one or more relationships between the one or more user reactions and one or more locations of the one or more non-augmentation scene elements in the one or more remotely displayed augmented views.

161. The computationally-implemented system of claim 160, wherein said means for determining the one or more relationships between the one or more user reactions and the one or more forms of the one or more non-augmentation scene elements by determining one or more relationships between the one or more user reactions and one or more locations of the one or more non-augmentation scene elements in the one or more remotely displayed augmented views comprises:

means for determining one or more relationships between the one or more user reactions and the one or more locations of the one or more non-augmentation scene elements in the one or more remotely displayed augmented views by determining one or more relationships between the one or more user reactions and one or more locations of the one or more non-augmentation scene elements relative to one or more locations of the one or more augmentations in the one or more remotely displayed augmented views.

162. The computationally-implemented system of claim 91, wherein said means for correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views comprises:

means for correlating the one or more user reactions with the one or more augmentations including determining one or more relationships between the one or more user reactions and one or more non-augmentation scene elements obfuscated by the one or more augmentations in the one or more remotely displayed augmented views.

163. The computationally-implemented system of claim 91, further comprising:

means for developing one or more second augmentations for displaying through one or more augmented views, the one or more second augmentations being based, at least in part, the correlating.

164. The computationally-implemented system of claim 163, wherein said means for developing one or more second augmentations for displaying through one or more augmented views, the one or more second augmentations being based, at least in part, the correlating comprises:

means for developing the one or more second augmentations for displaying through the one or more augmented views by developing one or more second augmentations having one or more forms that are based, at least in part, on the correlating.

165. The computationally-implemented system of claim 164, wherein said means for developing the one or more second augmentations for displaying through the one or more augmented views by developing one or more second augmentations having one or more forms that are based, at least in part, on the correlating comprises:

means for developing the one or more second augmentations having the one or more forms by developing one or more second augmentations having one or more visual features, shapes, and/or dimensions that are selected, based at least in part, on the correlating.

166. The computationally-implemented system of claim 164, wherein said means for developing the one or more second augmentations for displaying through the one or more augmented views by developing one or more second augmentations having one or more forms that are based, at least in part, on the correlating comprises:

means for developing the one or more second augmentations having the one or more forms by developing one or more second augmentations having one or more colors, brightness, and/or hue that are selected based, at least in part, on the correlating.

167. The computationally-implemented system of claim 164, wherein said means for developing the one or more second augmentations for displaying through the one or more augmented views by developing one or more second augmentations having one or more forms that are based, at least in part, on the correlating comprises:

means for developing the one or more second augmentations having the one or more forms by developing one or more second augmentations designed to be located at one or more specific locations in the one or more augmented views, the one or more specific locations being based, at least in part, on the correlating.

92-110. -110. (canceled)

112-115. -115. (canceled)

117. (canceled)

120-127. -127. (canceled)

129-135. -135. (canceled)

145. (canceled)

147-148. -148. (canceled)

151. (canceled)

168-180. -180. (canceled)

181. A system, comprising:

circuitry for receiving first data that at least identifies one or more augmentations that were remotely displayed in one or more remotely displayed augmented views of one or more actual scenes;

circuitry for receiving second data indicating one or more user reactions of one or more users in response to the remote display of the one or more remotely displayed augmented views; and

circuitry for correlating the one or more user reactions with the one or more augmentations that were remotely displayed through the one or more remotely displayed augmented views.