Sensor Based Display Environment

• US 20120007850A1
• Filed: 07/07/2010
• Published: 01/12/2012
• Est. Priority Date: 07/07/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Piemonte, Patrick
• Granted Patent: US 8,907,943 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A three-dimensional (“3D”) display environment for mobile device is disclosed that uses orientation data from one or more onboard sensors to automatically determine and display a perspective projection of the 3D display environment based on the orientation data without the user physically interacting with (e.g., touching) the display.

Claims

1. A computer-implemented method performed by one or more processors onboard a handheld mobile device, comprising:

generating a three-dimensional (3D) display environment;

receiving first sensor data from one or more sensors onboard the mobile device, the first sensor data indicative of a first orientation of the mobile device relative to a reference coordinate frame;

determining a first perspective projection of the 3D display environment based on the first sensor data;

displaying the first perspective projection of the 3D display environment on a display of the mobile device;

receiving second sensor data from the one or more sensors onboard the mobile device, the second sensor data indicative of a second orientation of the mobile device relative to the reference coordinate frame;

determining a second perspective projection of the 3D display environment based on the second sensor data; and

displaying the second perspective projection of the 3D display environment on the display of the mobile device.

2. The method of claim 1, wherein receiving first sensor data or second sensor data further comprises:

receiving gyro sensor data.

3. The method of claim 1, wherein determining a first perspective projection or second perspective projection, further comprises:

determining a camera view of the 3D display environment by transforming points comprising the 3D display environment using a camera transform computed from the first or second sensor data; and

displaying the 3D display environment from the camera view based on results of the camera transform.

4. The method of claim 3, further comprising:

processing the first or second sensor data to determine if a threshold value has been exceeded;

determining that the threshold value has been exceeded; and

changing the position of the camera in the display environment.

5. The method of claim 4, where changing the position of the camera in the display environment, further comprises:

snapping to a new camera position.

6. The method of claim 1, further comprising:

translating objects in the display environment between a foreground and a background of the display environment.

7. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, causes the one or more processors to perform operations, comprising:

generating a three-dimensional (3D) display environment on a display of a mobile device;

receiving first sensor data from one or more sensors onboard the mobile device, the first sensor data indicative of a first orientation of the mobile device relative to a reference coordinate frame;

determining a first perspective projection of the 3D display environment based on the first sensor data;

displaying the first perspective projection of the 3D display environment on a display of the mobile device;

receiving second sensor data from the one or more sensors onboard the mobile device, the second sensor data indicative of a second orientation of the mobile device relative to the reference coordinate frame;

determining a second perspective projection of the 3D display environment based on the second sensor data; and

displaying the second perspective projection of the 3D display environment on the display of the mobile device.

8. The computer-readable storage medium of claim 7, wherein receiving first sensor data or second sensor data, further comprises:

receiving gyro sensor data.

9. The computer-readable storage medium of claim 7, wherein determining a first perspective projection or second perspective projection, further comprises:

determining a camera view of the 3D display environment by transforming points comprising the 3D display environment using a camera transform computed from the first or second sensor data; and

displaying the 3D display environment from the camera view based on results of the camera transform.

10. The computer-readable storage medium of claim 9, further comprising:

processing the first or second sensor data to determine if a threshold value has been exceeded;

determining that the threshold value has been exceeded; and

changing the position of the camera in the display environment.

11. The computer-readable storage medium of claim 10, where changing the position of the camera in the display environment, further comprises:

snapping to a new camera position.

12. The computer-readable storage medium of claim 7, further comprising:

translating objects in the display environment between a foreground and a background of the display environment.

13. A system, comprising:

a processor;

a computer-readable storage medium coupled to the processor and storing instructions, which, when executed by a processor, causes the processor to perform operations comprising;

generating a three-dimensional (3D) display environment;

receiving first sensor data from one or more sensors onboard the mobile device, the first sensor data indicative of a first orientation of the mobile device relative to a reference coordinate frame;

determining a first perspective projection of the 3D display environment based on the first sensor data;

displaying the first perspective projection of the 3D display environment on a display of the mobile device;

receiving second sensor data from the one or more sensors onboard the mobile device, the second sensor data indicative of a second orientation of the mobile device relative to the reference coordinate frame;

determining a second perspective projection of the 3D display environment based on the second sensor data; and

displaying the second perspective projection of the 3D display environment on the display of the mobile device.

14. The system of claim 13, wherein receiving first sensor data or second sensor data, further comprises:

receiving gyro sensor data.

15. The system of claim 13, wherein determining a first perspective projection or second perspective projection, further comprises:

determining a camera view of the 3D display environment by transforming points comprising the 3D display environment using a camera transform computed from the first or second sensor data; and

displaying the 3D display environment from the camera view based on results of the camera transform.

16. The system of claim 15, further comprising:

processing the first or second sensor data to determine if a threshold value has been exceeded;

determining that the threshold value has been exceeded; and

changing the position of the camera in the display environment.

17. The system of claim 16, where changing the position of the camera in the display environment, further comprises:

snapping to a new camera position.

18. The system of claim 13, further comprising:

translating objects in the display environment between a foreground and a background of the display environment.