DISPLAYING IN RESPONSE TO DETECTING ONE OR MORE USER BEHAVIORS ONE OR MORE SECOND AUGMENTATIONS THAT ARE BASED ON ONE OR MORE REGISTERED FIRST AUGMENTATIONS

• US 20140098137A1
• Filed: 02/19/2013
• Published: 04/10/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr., Tegreene, Clarence T.
• Granted Patent: US 10,665,017 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Computationally implemented methods and systems include registering one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown at least at end of a segment of time, detecting, following the showing of the one or more first augmentations up to the end of the segment of time, one or more user behaviors of the user that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations; and displaying, in response at least in part to said detecting, one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene, the one or more second augmentations to be displayed being based, at least in part, on the registering of the one or more first augmentations. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1-165. -165. (canceled)

166. A system, comprising:

an augmentation recording module configured to record one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown up until end of a segment of time;

a user behavior ascertaining module configured to ascertain one or more user behaviors of the user following the showing of the one or more first augmentations that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations; and

an augmentation presenting module including a user directing augmentation presenting module, the user directing augmentation presenting module configured to present, prior to presenting by the augmentation presenting module of one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene in response to the ascertaining of the one or more user behaviors, one or more third augmentations that directs the user to execute one or more acts in order to view the one or more second augmentations, the one or more second augmentations having been based, at least in part, on the recording of the one or more first augmentations.

167. The system of claim 166, wherein said augmentation recording module configured to record one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown up until end of a segment of time comprises:

an augmentation aspect recording module configured to record one or more aspects of the one or more first augmentations that were shown up until the end of the segment of time, the one or more first augmentations being, at least temporality, not being shown beyond the end of the segment of time.

168. The system of claim 167, wherein said augmentation aspect recording module configured to record one or more aspects of the one or more first augmentations that were shown up until the end of the segment of time, the one or more first augmentations being, at least temporality, not being shown beyond the end of the segment of time comprises:

a visual format recording module configured to record one or more visual formats of the one or more first augmentations at the end of the segment of time.

169. The system of claim 167, wherein said augmentation aspect recording module configured to record one or more aspects of the one or more first augmentations that were shown up until the end of the segment of time, the one or more first augmentations being, at least temporality, not being shown beyond the end of the segment of time comprises:

a location recording module configured to record one or more locations of the one or more first augmentations in the first augmented view at the end of the segment of time.

171. The system of claim 167, wherein said augmentation aspect recording module configured to record one or more aspects of the one or more first augmentations that were shown up until the end of the segment of time, the one or more first augmentations being, at least temporality, not being shown beyond the end of the segment of time comprises:

a content recording module configured to record one or more contents of the one or more first augmentations that were shown at the end of the segment of time.

172. The system of claim 171, wherein said content recording module configured to record one or more contents of the one or more first augmentations that were shown at the end of the segment of time comprises:

a content recording module configured to record one or more textual contents of the one or more first augmentations that were shown at the end of the segment of time.

173. The system of claim 171, wherein said content recording module configured to record one or more contents of the one or more first augmentations that were shown at the end of the segment of time comprises:

a content recording module configured to record one or more image contents of the one or more first augmentations that were shown at the end of the segment of time.

174. The system of claim 166, wherein said augmentation recording module configured to record one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown up until end of a segment of time comprises:

a non-augmentation aspect recording module configured to record one or more aspects of one or more non-augmentation scene elements included in the first augmented view at the end of the segment of time.

178. The system of claim 166, wherein said augmentation recording module configured to record one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown up until end of a segment of time comprises:

a dynamic augmentation recording module configured to record one or more dynamic augmentations that change in response to one or more environmental factors.

181. The system of claim 178, wherein said dynamic augmentation recording module configured to record one or more dynamic augmentations that change in response to one or more environmental factors comprises:

a dynamic augmentation recording module configured to record one or more dynamic augmentations that changes in response to passage of time or in response to one or more changes to one or more non-time factors over time.

182. The system of claim 181, wherein said dynamic augmentation recording module configured to record one or more dynamic augmentations that changes in response to passage of time or in response to one or more changes to one or more non-time factors over time comprises:

a rotating augmentation recording module configured to record a rotating augmentation that is designed to rotate its form and/or content in response to the passage of time or in response to one or more changes to one or more non-time factors over time.

202. The system of claim 166, wherein said user behavior ascertaining module configured to ascertain one or more user behaviors of the user following the showing of the one or more first augmentations that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a user movement ascertaining module configured to ascertain one or more movements of the user that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations.

203. The system of claim 202, wherein said user movement ascertaining module configured to ascertain one or more movements of the user that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a location relative user movement ascertaining module configured to ascertain one or more movements of the user relative to one or more specific geographical locations, the one or more specific geographical locations being where the one or more augmentations were shown to the user, and the one or more movements of the user relative to the one or more specific geographical locations being one or more movements that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations.

204. The system of claim 203, wherein said location relative user movement ascertaining module configured to ascertain one or more movements of the user relative to one or more specific geographical locations, the one or more specific geographical locations being where the one or more augmentations were shown to the user, and the one or more movements of the user relative to the one or more specific geographical locations being one or more movements that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a location relative user movement ascertaining module configured to ascertain the user spatially moving towards the one or more specific geographical locations.

205. The system of claim 203, wherein said location relative user movement ascertaining module configured to ascertain one or more movements of the user relative to one or more specific geographical locations, the one or more specific geographical locations being where the one or more augmentations were shown to the user, and the one or more movements of the user relative to the one or more specific geographical locations being one or more movements that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a location relative user movement ascertaining module configured to ascertain the user initially moving away from the one or more specific geographical locations before turning and moving towards the one or more specific geographical locations.

206. The system of claim 203, wherein said location relative user movement ascertaining module configured to ascertain one or more movements of the user relative to one or more specific geographical locations, the one or more specific geographical locations being where the one or more augmentations were shown to the user, and the one or more movements of the user relative to the one or more specific geographical locations being one or more movements that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a location relative user movement ascertaining module configured to ascertain the user'"'"'s head and/or body turning towards the one or more specific geographical locations.

207. The system of claim 166, wherein said user behavior ascertaining module configured to ascertain one or more user behaviors of the user following the showing of the one or more first augmentations that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

an ocular characteristic ascertaining module configured to ascertain one or more ocular characteristics of the user that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations.

208. The system of claim 207, wherein said ocular characteristic ascertaining module configured to ascertain one or more ocular characteristics of the user that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a dwell path ascertaining module configured to ascertain one or more dwell paths of one or more eyes of the user relative to the first actual scene or to another actual scene that at least overlaps the first actual scene, the one or more dwell paths being one or more dwell paths that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations.

209. The system of claim 208, wherein said dwell path ascertaining module configured to ascertain one or more dwell paths of one or more eyes of the user relative to the first actual scene or to another actual scene that at least overlaps the first actual scene, the one or more dwell paths being one or more dwell paths that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a dwell path ascertaining module configured to ascertain one or more dwell paths of the one or more eyes of the user relative to one or more equivalent locations in the first actual scene or in the other actual scene, the one or more equivalent locations being where the one or more first augmentations were previously shown to be located at in the first augmented view.

210. The system of claim 207, wherein said ocular characteristic ascertaining module configured to ascertain one or more ocular characteristics of the user that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a focus point ascertaining module configured to ascertain one or more focus points of the one or more eyes of the user that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations.

211. The system of claim 210, wherein said focus point ascertaining module configured to ascertain one or more focus points of the one or more eyes of the user that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a focus point ascertaining module configured to ascertain one or more focus points of the one or more eyes of the user relative to one or more equivalent locations in the first actual scene or in another actual scene that at least overlaps the first actual scene, the first actual scene being where the one or more first augmentations were previously shown to be located at in the first augmented view.

212. The system of claim 210, wherein said focus point ascertaining module configured to ascertain one or more focus points of the one or more eyes of the user that when detected as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

a focus point ascertaining module configured to ascertain one or more focus points of the one or more eyes of the user that indicates that the user is visually focused proximate to or at one or more locations in the first actual view or in a second actual view that overlaps the first actual view, the one or more locations of the first actual view or the second actual view being one or more locations of the first actual view or the second actual view where the one or more first augmentations were previously shown to be located at in the first augmented view.

213. The system of claim 166, wherein said user behavior ascertaining module configured to ascertain one or more user behaviors of the user following the showing of the one or more first augmentations that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations comprises:

an actual scene ascertaining module configured to ascertain the first actual scene or the second actual scene that when detected as being at least partially displayed to the user infers user'"'"'s interest in seeing the one or more first augmentations.

214. The system of claim 213, wherein said actual scene ascertaining module configured to ascertain the first actual scene or the second actual scene that when detected as being at least partially displayed to the user infers user'"'"'s interest in seeing the one or more first augmentations comprises:

an actual scene ascertaining module configured to ascertain a second actual scene that includes one or more visual cues that when detected as being at least partially displayed to the user infers user'"'"'s interest in seeing the one or more first augmentations.

215. The system of claim 214, wherein said actual scene ascertaining module configured to ascertain a second actual scene that includes one or more visual cues that when detected as being at least partially displayed to the user infers user'"'"'s interest in seeing the one or more first augmentations comprises:

an actual scene ascertaining module configured to ascertain one or more non-augmentation scene elements in the second actual scene that were previously detected in the first actual scene that the first augmented view was based on.

220. The system of claim 166, wherein said augmentation presenting module including a user directing augmentation presenting module, the user directing augmentation presenting module configured to present, prior to presenting by the augmentation presenting module of one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene in response to the ascertaining of the one or more user behaviors, one or more third augmentations that directs the user to execute one or more acts in order to view the one or more second augmentations, the one or more second augmentations having been based, at least in part, on the recording of the one or more first augmentations comprises:

a user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to execute the one or more acts in order to view the one or more second augmentations.

221. The system of claim 220, wherein said user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to execute the one or more acts in order to view the one or more second augmentations comprises:

a user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to execute one or more movements in order to view the one or more second augmentations.

222. The system of claim 221, wherein said user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to execute one or more movements in order to view the one or more second augmentations comprises:

a user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to relocate to one or more locations in order to view the one or more second augmentations.

223. The system of claim 220, wherein said user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to execute the one or more acts in order to view the one or more second augmentations comprises:

a user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to visually focus at one or more visual locations in order to view the one or more second augmentations.

224. The system of claim 220, wherein said user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to execute the one or more acts in order to view the one or more second augmentations comprises:

a user directing augmentation presenting module configured to present one or more symbolic or textual augmentations that directs the user to visually focus at one or more visual directions in order to view the one or more second augmentations.

227. The system of claim 166, wherein said augmentation presenting module including a user directing augmentation presenting module, the user directing augmentation presenting module configured to present, prior to presenting by the augmentation presenting module of one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene in response to the ascertaining of the one or more user behaviors, one or more third augmentations that directs the user to execute one or more acts in order to view the one or more second augmentations, the one or more second augmentations having been based, at least in part, on the recording of the one or more first augmentations comprises:

an augmentation presenting module configured to present one or more second augmentations that are at least substantially same as the one or more first augmentations.

228. The system of claim 227, wherein said augmentation presenting module configured to present one or more second augmentations that are at least substantially same as the one or more first augmentations comprises:

an augmentation presenting module configured to present one or more second augmentations having one or more visual formats that are same as one or more visual formats of the one or more first augmentations.

229. The system of claim 227, wherein said augmentation presenting module configured to present one or more second augmentations that are at least substantially same as the one or more first augmentations comprises:

an augmentation presenting module configured to present one or more second augmentations that have content the same as content of the one or more first augmentations.

231. The system of claim 166, wherein said augmentation presenting module including a user directing augmentation presenting module, the user directing augmentation presenting module configured to present, prior to presenting by the augmentation presenting module of one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene in response to the ascertaining of the one or more user behaviors, one or more third augmentations that directs the user to execute one or more acts in order to view the one or more second augmentations, the one or more second augmentations having been based, at least in part, on the recording of the one or more first augmentations comprises:

a modified first augmentation presenting module configured to present one or more second augmentations that are modified version or versions of the one or more first augmentations.

232. The system of claim 231, wherein said modified first augmentation presenting module configured to present one or more second augmentations that are modified version or versions of the one or more first augmentations comprises:

a first augmentation with modified visual format presenting module configured to present the modified version or versions of the one or more first augmentations that have one or more modified visual formats that are different from one or more unmodified visual formats of the one or more first augmentations displayed through the first augmented view.

233. The system of claim 232, wherein said first augmentation with modified visual format presenting module configured to present the modified version or versions of the one or more first augmentations that have one or more modified visual formats that are different from one or more unmodified visual formats of the one or more first augmentations displayed through the first augmented view comprises:

a first augmentation with modified visual format presenting module configured to present the modified version or versions of the one or more first augmentations that have one or more modified visual formats that are designed to increase visibility of the one or more first augmentations.

234. The system of claim 233, wherein said first augmentation with modified visual format presenting module configured to present the modified version or versions of the one or more first augmentations that have one or more modified visual formats that are designed to increase visibility of the one or more first augmentations comprises:

a first augmentation with modified visual format presenting module configured to present the modified version or versions of the one or more first augmentations that are modified to have at least one of increased brightness, increased visual size, change in bordering, or change in coloring that are designed to increase visibility of the one or more first augmentations.

238. The system of claim 231, wherein said modified first augmentation presenting module configured to present one or more second augmentations that are modified version or versions of the one or more first augmentations comprises:

a user behavior based modified first augmentation presenting module configured to present the modified version or versions one or more first augmentations in the second augmented view, the one or more modified version or versions of the one or more first augmentations to be displayed being based, at least in part, on one or more user behaviors of the user that were detected following the showing of the one or more first augmentations.

240. The system of claim 238, wherein said user behavior based modified first augmentation presenting module configured to present the modified version or versions one or more first augmentations in the second augmented view, the one or more modified version or versions of the one or more first augmentations to be displayed being based, at least in part, on one or more user behaviors of the user that were detected following the showing of the one or more first augmentations comprises:

a movement based modified first augmentation presenting module configured to present modified version or versions of the one or more first augmentations, the modified version or versions of the one or more first augmentations being based, at least in part, on one or more movements of the user that were detected following the showing of the one or more first augmentations.

242. The system of claim 238, wherein said user behavior based modified first augmentation presenting module configured to present the modified version or versions one or more first augmentations in the second augmented view, the one or more modified version or versions of the one or more first augmentations to be displayed being based, at least in part, on one or more user behaviors of the user that were detected following the showing of the one or more first augmentations comprises:

an ocular characteristic based modified first augmentation presenting module configured to present modified version or versions of the one or more first augmentations in response to one or more ocular characteristics of the user that were detected following the showing of the one or more first augmentations.

170. (canceled)

175-177. -177. (canceled)

179-180. -180. (canceled)

183-201. -201. (canceled)

216-219. -219. (canceled)

225-226. -226. (canceled)

230. (canceled)

235-237. -237. (canceled)

239. (canceled)

241. (canceled)

243-244. -244. (canceled)

245. An article of manufacture, comprising:

a non-transitory storage medium bearing;

one or more instructions for recording one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown up to end of a segment of time;

one or more instructions for ascertaining one or more user behaviors of the user following the showing of the one or more first augmentations that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations; and

one or more instructions presenting, prior to presenting of one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene in response to the ascertaining of the one or more user behaviors, one or more third augmentations that directs the user to execute one or more acts in order to view the one or more second augmentations, the one or more second augmentations having been based, at least in part, on the recording of the one or more first augmentations.

246. A wearable augmented reality (AR) device, comprising:

a coupling device for coupling the wearable AR device to a user'"'"'s head;

one or more display monitors;

an augmentation recording module configured to record one or more first augmentations that were shown to a user through a first augmented view of a first actual scene, the one or more first augmentations having been shown up until end of a segment of time;

a user behavior ascertaining module configured to ascertain one or more user behaviors of the user following the showing of the one or more first augmentations that when ascertained as occurring infers user'"'"'s interest in seeing the one or more first augmentations; and

an augmentation presenting module including a user directing augmentation presenting module, the user directing augmentation presenting module configured to present, prior to presenting by the augmentation presenting module of one or more second augmentations through a second augmented view of the first actual scene or of a second actual scene in response to the ascertaining of the one or more user behaviors, one or more third augmentations that directs the user to execute one or more acts in order to view the one or more second augmentations, the one or more second augmentations having been based, at least in part, on the recording of the one or more first augmentations.