Sensor based display environment

• US 8,907,943 B2
• Filed: 07/07/2010
• Issued: 12/09/2014
• Est. Priority Date: 07/07/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Piemonte, Patrick
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Du, Haixia
• Application Number:
  US12/831,722
• Publication Number:
  US 20120007850A1
• Time in Patent Office: 1,616 Days
• Field of Search: 345/419
• US Class Current: 345/419
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A three-dimensional (“3D”) display environment for mobile device is disclosed that uses orientation data from one or more onboard sensors to automatically determine and display a perspective projection of the 3D display environment based on the orientation data without the user physically interacting with (e.g., touching) the display.

Claims

1. A computer-implemented method performed by one or more processors onboard a handheld mobile device, comprising:

generating a three-dimensional (3D) display environment having an imaginary camera located at a first position and a first camera view of the 3D display environment at the first camera position, the 3D display environment including two or more walls, the first camera view directly facing a first wall;

receiving sensor data from the one or more sensors onboard the mobile device, the sensor data indicative of an orientation of the mobile device relative to a reference coordinate frame;

determining that the orientation exceeds a threshold value;

snapping the camera position from the first camera position to a second camera position;

determining a second camera view at the second camera position of the 3D display environment, the second camera view directly facing a second wall; and

displaying the 3D display environment from the second camera view.

2. The method of claim 1, wherein receiving sensor data further comprises:

receiving gyro sensor data.

3. The method of claim 1, further comprising:

translating objects in the 3D display environment between a foreground and a background of the 3D display environment.

4. The method of claim 1, where the first and second camera view or position is determined by a camera transform.

5. The method of claim 1, where determining that the orientation exceeds a threshold value includes determining that an angle of rotation about an axis in the reference coordinate frame exceeds a defined angle.

6. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, causes the one or more processors to perform operations, comprising:

generating a three-dimensional (3D) display environment having an imaginary camera located at a first position and a first camera view of the 3D display environment at the first camera position, the 3D display environment including two or more walls, the first camera view directly facing the first wall;

receiving sensor data from the one or more sensors onboard the mobile device, the sensor data indicative of an orientation of the mobile device relative to a reference coordinate frame;

determining that the orientation exceeds a threshold value;

snapping the camera position from the first camera position to a second camera position;

determining a second camera view at the second camera position of the 3D display environment, the second camera view directly facing a second wall; and

displaying the 3D display environment from the second camera view.

7. The non-transitory computer-readable storage medium of claim 6, wherein receiving sensor data, further comprises:

receiving gyro sensor data.

8. The non-transitory computer-readable storage medium of claim 6, further comprising:

translating objects in the 3D display environment between a foreground and a background of the 3D display environment.

9. The non-transitory computer-readable storage medium of claim 6, where the first and second camera view or position is determined by a camera transform.

10. The non-transitory computer-readable storage medium of claim 6, where determining that the orientation exceeds a threshold value includes determining that an angle of rotation about an axis in the reference coordinate frame exceeds a defined angle.

11. A system, comprising:

a processor;

a computer-readable storage medium coupled to the processor and storing instructions, which, when executed by a processor, causes the processor to perform operations comprising;

generating a three-dimensional (3D) display environment having an imaginary camera located at a first position and a first camera view of the 3D display environment at the first camera position, the 3D display environment including two or more walls, the first camera view directly facing a first wall;

receiving sensor data from the one or more sensors onboard the mobile device, the sensor data indicative of an orientation of the mobile device relative to a reference coordinate frame;

determining that the orientation exceeds a threshold value;

snapping the camera position from the first camera position to a second camera position;

determining a second camera view at the second camera position of the 3D display environment, the second camera view directly facing a second wall; and

displaying the 3D display environment from the second camera view.

12. The system of claim 11, wherein receiving sensor data, further comprises:

receiving gyro sensor data.

13. The system of claim 11, further comprising:

translating objects in the 3D display environment between a foreground and a background of the 3D display environment.

14. The system of claim 11, where the first and second camera view or position is determined by a camera transform.

15. The system of claim 11, where determining that the orientation exceeds a threshold value includes determining that an angle of rotation about an axis in the reference coordinate frame exceeds a defined angle.