PRESENTING AN AUGMENTED VIEW IN RESPONSE TO ACQUISITION OF DATA INFERRING USER ACTIVITY

• US 20140098127A1
• Filed: 11/08/2012
• Published: 04/10/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr., Tegreene, Clarence T.
• Granted Patent: US 9,141,188 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Computationally implemented methods and systems include obtaining visual data of an actual view of a scene from a real environment, determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired, and presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1. A computationally-implemented method, comprising:

obtaining visual data of an actual view of a scene from a real environment;

determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired; and

presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data.

2-70. -70. (canceled)

71. A computationally-implemented system, comprising:

means for obtaining visual data of an actual view of a scene from a real environment;

means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired; and

means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data.

78. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data have been acquired by determining whether the visual data that was obtained includes activity-inferring visual data that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment

79. The computationally-implemented system of claim 78, wherein said means for determining whether the activity-inferring data have been acquired by determining whether the visual data that was obtained includes activity-inferring visual data that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment comprises:

means for determining whether the visual data that was obtained includes the activity-inferring visual data that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment by determining whether the activity-inferring visual data that was obtained includes the activity-inferring visual data that provides multiple visual cues that when detected in combination infers the at least initial occurrence of the one or more user activities.

80. The computationally-implemented system of claim 79, wherein said means for determining whether the visual data that was obtained includes the activity-inferring visual data that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment by determining whether the activity-inferring visual data that was obtained includes the activity-inferring visual data that provides multiple visual cues that when detected in combination infers the at least initial occurrence of the one or more user activities comprises:

means for determining whether the activity-inferring visual data that was obtained includes the activity-inferring visual data that provides the multiple visual cues that when detected in combination are linked to the at least initial occurrence of the one or more user activities by determining whether the activity-inferring visual data that was obtained includes the activity-inferring visual data that indicates presence of multiple visible items in the scene from the real environment that when detected in combination infers the at least initial occurrence of the one or more user activities.

81. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data have been acquired by determining whether activity-inferring audio data have been acquired that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment.

82. The computationally-implemented system of claim 81, wherein said means for determining whether the activity-inferring data have been acquired by determining whether activity-inferring audio data have been acquired that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment comprises:

means for determining whether the activity-inferring audio data have been acquired by determining whether the activity-inferring audio data have been acquired that includes data for one or more identifiable sounds that when detected infers the at least initial occurrence of the one or more user activities.

84. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data have been acquired by determining whether movement data have been acquired that indicate one or more movements that infer the at least initial occurrence of the one or more user activities associated with the scene from the real environment.

85. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data have been acquired by determining whether personal information manager (PIM) data have been acquired that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment

86. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data have been acquired by determining whether sensor-provided data have been acquired that infers occurrence of the one or more user activities associated with the scene from the real environment.

87. The computationally-implemented system of claim 86, wherein said means for determining whether the activity-inferring data have been acquired by determining whether sensor-provided data have been acquired that infers occurrence of the one or more user activities associated with the scene from the real environment comprises:

means for determining whether the sensor-provided data have been acquired by determining whether sensor-provided data indicating one or more user physiological characteristics have been acquired that when detected as occurring infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment.

88. The computationally-implemented system of claim 86, wherein said means for determining whether the activity-inferring data have been acquired by determining whether sensor-provided data have been acquired that infers occurrence of the one or more user activities associated with the scene from the real environment comprises:

means for determining whether the sensor-provided data have been acquired by determining whether sensor-provided data that indicates one or more characteristics associated with one or more user-employed machines that when detected as occurring infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment.

89. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data have been acquired by determining whether social networking data have been acquired that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment.

90. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data have been acquired by determining whether the visual data that was obtained includes activity-inferring visual data that infers the at least initial occurrence of the one or more user activities and determining whether one or more of activity-inferring audio data, location and/or orientation data, movement data, personal information manager (PIM) data, social networking data, and/or sensor-provided data have been acquired that infers the at least initial occurrence of the one or more user activities associated with the scene from the real environment.

92. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data that infers the at least initial occurrence of the one or more user activities have been acquired by determining whether activity-inferring data that infers at least initial occurrence of one or more user leisure activities have been acquired.

93. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data that infers the at least initial occurrence of the one or more user activities have been acquired by determining whether activity-inferring data that infers at least initial occurrence of one or more user machine operating activities have been acquired.

95. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data that infers the at least initial occurrence of the one or more user activities have been acquired by determining whether activity-inferring data that infers at least initial occurrence of one or more user work or professional activities have been acquired.

96. The computationally-implemented system of claim 71, wherein said means for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired comprises:

means for determining whether the activity-inferring data that infers the at least initial occurrence of the one or more user activities have been acquired by determining whether activity-inferring data that infers at least initial occurrence of one or more user academic activities have been acquired.

107. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for presenting the augmented view including the one or more augmentations by presenting an augmented view including one or more dynamic functional augmentations that provide one or more dynamically-changing functional information.

116. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for presenting the augmented view including the one or more augmentations in response to determining that the activity-inferred data have at least been acquired by presenting an augmented view including one or more augmentations that provide one or more time information and/or user physiological information in response to determining that activity-inferring data that infers one or more exercise and/or athletic activities have been acquired.

117. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for presenting the augmented view including the one or more augmentations in response to determining that the activity-inferred data have at least been acquired by presenting an augmented view including one or more augmentations that provide one or more retail information in response to determining that activity-inferring data that infers one or more shopping activities have been acquired.

118. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for presenting the augmented view including the one or more augmentations in response to determining that the activity-inferred data have at least been acquired by presenting an augmented view including one or more augmentations that provide one or more dietary information in response to determining that activity-inferring data that infers one or more dining activities have been acquired.

119. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for presenting the augmented view including the one or more augmentations in response to determining that the activity-inferred data have at least been acquired by presenting an augmented view including one or more augmentations that provide one or more business, financial, and/or market information in response to determining that activity-inferring data that infers one or more employment or professional activities have been acquired.

120. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for presenting the augmented view including the one or more augmentations in response to determining that the activity-inferred data have at least been acquired by presenting an augmented view including one or more augmentations that provide one or more environmental and/or weather information in response to determining that activity-inferring data that infers one or more leisure activities have been acquired.

121. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for presenting the augmented view including the one or more augmentations in response to determining that the activity-inferred data have at least been acquired by presenting an augmented view including one or more augmentations that provide one or more reference information in response to determining that activity-inferring data that infers one or more educational and/or academic activities have been acquired.

122. The computationally-implemented system of claim 71, wherein said means for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data comprises:

means for soliciting for and receiving the one or more augmentations to be included in the augmented view in response, at least in part, to determining that the activity-inferred data have at least been acquired.

123. The computationally-implemented system of claim 122, wherein said means for soliciting for and receiving the one or more augmentations to be included in the augmented view in response, at least in part, to determining that the activity-inferred data have at least been acquired comprises:

means for soliciting for and receiving the one or more augmentations to be included in the augmented view via one or more wireless and/or wired networks.

124. The computationally-implemented system of claim 123, wherein said means for soliciting for and receiving the one or more augmentations to be included in the augmented view via one or more wireless and/or wired networks comprises:

means for soliciting for and receiving the one or more augmentations to be included in the augmented view from one or more parties determined to be associated with the scene from the real environment.

125. The computationally-implemented system of claim 124, wherein said means for soliciting for and receiving the one or more augmentations to be included in the augmented view from one or more parties determined to be associated with the scene from the real environment comprises:

means for soliciting for and receiving the one or more augmentations to be included in the augmented view from one or more parties determined to be associated with one or more locations associated with the scene from the real environment.

126. The computationally-implemented system of claim 124, wherein said means for soliciting for and receiving the one or more augmentations to be included in the augmented view from one or more parties determined to be associated with the scene from the real environment comprises:

means for soliciting for and receiving the one or more augmentations to be included in the augmented view from one or more parties determined to be associated with one or more items detected in the scene from the real environment.

127. The computationally-implemented system of claim 71, further comprising:

means for obtaining visual data of a second scene from the real environment, wherein the scene from the real environment is a first scene from the real environment;

means for determining whether a second activity-inferring data that infers at least initial occurrence of one or more second user activities associated with the second scene from the real environment have at least been acquired, wherein the activity-inferring data that infers the at least initial occurrence of the one or more user activities of the first scene from the real environment is a first activity-inferring data; and

means for presenting, in response at least in part to determining that the second activity-inferring data have at least been acquired, a second augmented view of the second scene from the real environment, the second augmented view including one or more second augmentations that have been included into the second augmented view based, at least in part, on the second activity-inferring data, and wherein the augmented view including the one or more augmentations of the scene from the real environment is a first augmented view including one or more first augmentations of the first scene from the real environment.

131. The computationally-implemented system of claim 127, wherein said means for presenting, in response at least in part to determining that the second activity-inferring data have at least been acquired, a second augmented view of the second scene from the real environment, the second augmented view including one or more second augmentations that have been included into the second augmented view based, at least in part, on the second activity-inferring data, and wherein the augmented view including the one or more augmentations of the scene from the real environment is a first augmented view including one or more first augmentations of the first scene from the real environment comprises:

means for presenting the second augmented view including the one or more second augmentations by presenting a second augmented view including one or more second augmentations that is or are modified version or versions of the one or more first augmentations.

132. The computationally-implemented system of claim 131, wherein said means for presenting the second augmented view including the one or more second augmentations by presenting a second augmented view including one or more second augmentations that is or are modified version or versions of the one or more first augmentations comprises:

means for presenting the second augmented view including the one or more second augmentations that is or are modified version or versions of the one or more first augmentations by presenting a second augmented view including one or more second augmentations that is or are same or similar as the one or more first augmentations except for location or locations of the one or more second augmentations in the second augmented view being different from location or locations of the one or more first augmentations in the first augmented view.

133. The computationally-implemented system of claim 131, wherein said means for presenting the second augmented view including the one or more second augmentations by presenting a second augmented view including one or more second augmentations that is or are modified version or versions of the one or more first augmentations comprises:

means for presenting the second augmented view including the one or more second augmentations that is or are modified version or versions of the one or more first augmentations by presenting a the second augmented view including one or more second augmentations that is or are same or similar as the one or more first augmentations except for the one or more second augmentations in the second augmented view being more transparent than the one or more first augmentations in the first augmented view.

134. The computationally-implemented system of claim 131, wherein said means for presenting the second augmented view including the one or more second augmentations by presenting a second augmented view including one or more second augmentations that is or are modified version or versions of the one or more first augmentations comprises:

means for presenting the second augmented view including the one or more second augmentations that is or are modified version or versions of the one or more first augmentations by presenting a second augmented view including one or more second augmentations that is or are same or similar as the one or more first augmentations except for the one or more second augmentations in the second augmented view being less transparent than the one or more first augmentations in the first augmented view.

135. The computationally-implemented system of claim 131, wherein said means for presenting the second augmented view including the one or more second augmentations by presenting a second augmented view including one or more second augmentations that is or are modified version or versions of the one or more first augmentations comprises:

means for presenting the second augmented view including the one or more second augmentations that is or are modified version or versions of the one or more first augmentations by presenting a second augmented view including one or more second augmentations that is or are one or more disabled functional augmentations of one or more functional augmentations included in the first augmented view.

136. The computationally-implemented system of claim 135, wherein said means for presenting the second augmented view including the one or more second augmentations that is or are modified version or versions of the one or more first augmentations by presenting a second augmented view including one or more second augmentations that is or are one or more disabled functional augmentations of one or more functional augmentations included in the first augmented view comprises:

means for presenting the second augmented view including the one or more second augmentations that is or are the one or more disabled functional augmentations of the one or more functional augmentations included in the first augmented view in response to a request made by a party associated with the second scene to disable the one or more functional augmentations.

137. The computationally-implemented system of claim 127, wherein said means for presenting, in response at least in part to determining that the second activity-inferring data have at least been acquired, a second augmented view of the second scene from the real environment, the second augmented view including one or more second augmentations that have been included into the second augmented view based, at least in part, on the second activity-inferring data, and wherein the augmented view including the one or more augmentations of the scene from the real environment is a first augmented view including one or more first augmentations of the first scene from the real environment comprises:

means for presenting the second augmented view including the one or more second augmentations by presenting one or more second augmentations that is or are one or more indicators that indicate that the one or more first augmentations have been removed.

138. The computationally-implemented system of claim 137, wherein said means for presenting the second augmented view including the one or more second augmentations by presenting one or more second augmentations that is or are one or more indicators that indicate that the one or more first augmentations have been removed comprises:

means for presenting the one or more second augmentations that is or are one or more indicators that indicate that the one or more first augmentations have been removed by presenting one or more second augmentations that is or are one or more indicators that can be selected or activated in order to reinstate the one or more first augmentations into the second augmented view.

139. The computationally-implemented system of claim 71, further comprising:

means for obtaining visual data of a second scene from the real environment, wherein the scene from the real environment is a first scene from the real environment;

means for determining whether a second activity-inferring data that infers at least initial occurrence of one or more second user activities associated with the second scene from the real environment have at least been acquired, wherein the activity-inferring data that infers at least initial occurrence of the first scene from the real environment is a first activity-inferring data; and

means for presenting, in response at least in part to determining that the second activity-inferring data have at least been acquired, a view of the second scene that does not include the one or more augmentations.

140. The computationally-implemented system of claim 71, further comprising:

means for transmitting via one or more of a wireless and/or wired networks a notification that notifies a party that the one or more augmentations have been removed.

72-77. -77. (canceled)

83. (canceled)

91. (canceled)

94. (canceled)

97-106. -106. (canceled)

108-115. -115. (canceled)

128-130. -130. (canceled)

141. A system, comprising:

circuitry for obtaining visual data of an actual view of a scene from a real environment;

circuitry for determining whether activity-inferring data that infers at least initial occurrence of one or more user activities associated with the scene from the real environment have at least been acquired; and

circuitry for presenting, in response at least in part to determining that the activity-inferring data have at least been acquired, an augmented view of the scene from the real environment, the augmented view including one or more augmentations that have been included into the augmented view based, at least in part, on the activity-inferring data.