DATA MANIPULATION BASED ON REAL WORLD OBJECT MANIPULATION

• US 20140354686A1
• Filed: 06/03/2013
• Published: 12/04/2014
• Est. Priority Date: 06/03/2013
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: DAQRI, LLC
• Inventors: Mullins, Brian
• Granted Patent: US 9,639,984 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

A system and method for data manipulation based on real world object manipulation is described. A device captures an image of a physical object. The image is communicated via a network to a remote server. The remote server includes virtual object data associated with the image and a communication notification for a user of the computing device. The device receives the virtual object data and displays the virtual image in a virtual landscape using the virtual object data. In response to relative movement between the computing device and the physical object caused by the user, the virtual image is modified.

Claims

1. A device comprising:

a camera configured to capture an image of a physical object; and

a processor comprising an augmented reality application and an augmented reality notification module,

the augmented reality application configured to communicate the image via a network to a remote server, the remote server including virtual object data associated with the image and a communication notification for a user of the computing device, to receive the virtual object data at the computing device, and

the augmented reality notification module configured to display the virtual image in a virtual landscape using the virtual object data, and in response to relative movement between the computing device and the physical object caused by the user, to modify the virtual image.

2. The device of claim 1, wherein the processor further comprises:

a physical world manipulation module configured to receive virtual functionalities associated with physical manipulations of the physical object, and to generate a virtual function corresponding to a physical manipulation of the physical object.

3. The device of claim 1, wherein the processor further comprises:

a notification module configured to receive the communication notification from a notification server associated with the user of the computing device, the communication notification comprising an email notification, a calendar reminder notification, or a social network notification.

4. The device of claim 1, wherein the augmented reality notification module comprises a reference identifier module, a notification visualization module, and a virtual object generator,

the reference identifier module configured to access reference identifiers associated with the communication notification,

the notification visualization module configured to access an augmented reality visualization associated with the reference identifiers, and

the virtual object generator configured to generate the virtual object engaged with a reference identifier in the image of the physical object using the augmented reality visualization associated with the reference identifier in the image of the physical object.

5. The device of claim 4, wherein the notification visualization module is configured to change a feature of the virtual object, and to replace the virtual object with another virtual object based on the augmented reality visualization associated with the reference identifier in the image of the physical object.

6. The device of claim 2, wherein the physical world manipulation module comprises a physical object identifier, a physical object manipulation detector, and a virtual actuator,

the physical object identifier configured to identify the physical object and virtual functionalities associated with physical manipulations of the physical object,

the physical object manipulation detector configured to identify a physical manipulation of the physical object, and

the virtual actuator configured to generate the virtual function corresponding to the identified physical manipulation of the physical object.

7. The device of claim 6, wherein the virtual actuator is configured to generate a virtual user interface corresponding to the identified physical manipulation of the physical object.

8. The device of claim 6, wherein the virtual actuator is configured activate a function on the device in response to the identified physical manipulation of the physical object.

9. The device of claim 6, wherein the virtual actuator is configured to generate a command for the virtual function to control another device in response to the identified physical manipulation of the physical object, the other device having no communication with the physical object.

10. The device of claim 6, wherein the physical manipulation of the physical object comprises moving or rotating the physical object.

11. A method comprising:

using a computing device, capturing an image of a physical object;

communicating the image via a network to a remote server, the remote server including virtual object data associated with the image and a communication notification for a user of the computing device;

receiving the virtual object data at the computing device;

displaying the virtual image in a virtual landscape using the virtual object data; and

in response to relative movement between the computing device and the physical object caused by the user, modifying the virtual image.

12. The method of claim 11, further comprising:

receiving virtual functionalities associated with physical manipulations of the physical object; and

generating a virtual function corresponding to a physical manipulation of the physical object.

13. The method of claim 11, further comprising:

receiving the communication notification from a notification server associated with the user of the computing device, the communication notification comprising an email notification, a calendar reminder notification, or a social network notification.

14. The method of claim 11, further comprising:

accessing reference identifiers associated with the communication notification;

accessing an augmented reality visualization associated with the reference identifiers; and

generating the virtual object engaged with a reference identifier in the image of the physical object using the augmented reality visualization associated with the reference identifier in the image of the physical object.

15. The method of claim 14, further comprising:

changing a feature of the virtual object, or replacing the virtual object with another virtual object based on the augmented reality visualization associated with the reference identifier in the image of the physical object.

16. The method of claim 12, further comprising:

identifying the physical object and the virtual functionalities associated with physical manipulations of the physical object;

identifying a physical manipulation of the physical object; and

generating the virtual function corresponding to the identified physical manipulation of the physical object.

17. The method of claim 16, further comprising:

generating a virtual user interface corresponding to the identified physical manipulation of the physical object.

18. The method of claim 16, further comprising:

activating a function on the computing device in response to the identified physical manipulation of the physical object.

19. The method of claim 16, further comprising:

generating a command for the virtual function to control another device in response to the identified physical manipulation of the physical object, the other device having no communication with the physical object.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

capturing an image of a physical object;

communicating the image via a network to a remote server, the remote server including virtual object data associated with the image and a communication notification for a user of the computing device;

receiving the virtual object data at the computing device;

displaying the virtual image in a virtual landscape using the virtual object data; and

in response to relative movement between the computing device and the physical object caused by the user, modifying the virtual image.