Synchronized, interactive augmented reality displays for multifunction devices
First Claim
1. A non-transitory machine-readable storage medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to:
- capture live video of a real-world, physical environment, and display the live video on a touch sensitive surface of the computing device;
combine an information layer and the live video, the information layer related to one or more objects in the live video;
generate computer-generated imagery based on the live video;
display the computer-generated imagery representing one or more objects in the live video on the touch sensitive surface;
overlay the information layer on the computer-generated imagery;
receive sensor data from one or more onboard motion sensors indicating that the computing device is in motion; and
synchronize the display of the live video, the computer-generated imagery, and the information layer on the touch sensitive surface using the sensor data.
0 Assignments
0 Petitions
Accused Products
Abstract
A device can receive live video of a real-world, physical environment on a touch sensitive surface. One or more objects can be identified in the live video. An information layer can be generated related to the objects. In some implementations, the information layer can include annotations made by a user through the touch sensitive surface. The information layer and live video can be combined in a display of the device. Data can be received from one or more onboard sensors indicating that the device is in motion. The sensor data can be used to synchronize the live video and the information layer as the perspective of video camera view changes due to the motion. The live video and information layer can be shared with other devices over a communication link.
-
Citations
25 Claims
-
1. A non-transitory machine-readable storage medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to:
-
capture live video of a real-world, physical environment, and display the live video on a touch sensitive surface of the computing device; combine an information layer and the live video, the information layer related to one or more objects in the live video; generate computer-generated imagery based on the live video; display the computer-generated imagery representing one or more objects in the live video on the touch sensitive surface; overlay the information layer on the computer-generated imagery; receive sensor data from one or more onboard motion sensors indicating that the computing device is in motion; and synchronize the display of the live video, the computer-generated imagery, and the information layer on the touch sensitive surface using the sensor data. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. An augmented reality system, comprising:
-
a touch sensitive surface configured for receiving touch input; one or more onboard motion sensors configured for sensing motion; a video camera for capturing live video of a real-world, physical environment, for display on the touch sensitive surface; a processor coupled to the touch sensitive surface, the motion sensor and the video camera, the processor configured for, combining an information layer and the live video for display on the touch sensitive surface, modeling computer-generated imagery based on the live video, displaying computer-generated imagery representing one or more objects in the live video on the touch sensitive surface, overlaying the information layer on the computer-generated imagery for display on the touch sensitive surface, receiving sensor data from the one or more onboard motion sensors indicating that the apparatus is in motion, and synchronizing the live video, computer-generated imagery, and the information layer using the sensor data; and a communication interface configured for sharing the live video over a communication link. - View Dependent Claims (20, 21, 22, 23, 24)
-
-
25. A mobile device comprising:
-
a touch sensitive display configured for displaying a graphical user interface and receiving touch input; a sensor configured to generate data related to position of the mobile device; a video camera to capture live video for display on the touch sensitive display; and a processor coupled to the touch sensitive display, the sensor and the video camera, the processor configured to, generate an information layer including data related to an object within the live video captured by the video camera, integrate the information layer and the live video for display on the touch sensitive display, model computer-generated imagery based at least in part on the live video, display the computer-generated imagery representing one or more objects in the live video on the touch sensitive display, overlay the information layer on the computer-generated imagery for display on the touch sensitive display, process sensor data from the sensors indicating that the mobile device is in motion, and synchronize the live video, computer-generated imagery, and the information layer using the sensor data.
-
Specification