Enhanced Display Rotation

• US 10,228,766 B2
• Filed: 09/12/2014
• Issued: 03/12/2019
• Est. Priority Date: 09/12/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bhesania, Firdosh Kersy, Smahi, Ahmed, Hendri, Pradipta Ariyo Bhaskoro
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US14/484,647
• Publication Number:
  US 20160077592A1
• Time in Patent Office: 1,642 Days
• Field of Search: 345650
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/20 for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
• CPC Class Codes: G09G 5/32 : with means for controlling the display position see provisionally G09G5/42

Abstract

Enhanced display rotation techniques are described in which arrangement of on-screen elements and user interfaces may be controlled based upon a combination of inputs from a sensor system of a device indicative of positional relationships. Output positions for elements are controlled in dependence upon a calculated position of a user in relation to the device and alignment of the device with a magnetic direction. The position and alignment may be derived based on sensor data, including at least a camera to obtain eye tracking data and a compass used to determine an alignment of the device. Other factors may also be considered such as data from an accelerometer, facial tracking by the camera, gaze detection, a proximity sensor, and other sensor data. Responsive to manipulation of the device, rotation and/or location of the elements may be controlled in accordance with the positional relationships that are ascertained.

Claims

1. A computer-implemented method comprising:

receiving an indication of at least one of a selectable type of auto-rotation, a number of pre-defined rotational positions, and corresponding angular increments, wherein the angular increments are selectively varied for different devices, scenarios, applications, or contexts;

collecting sensor data from multiple sensors associated with a sensor system of a device;

detecting changing viewing positions of a user relative to the device, the changing viewing positions detected based in part on a viewing plane and a viewing angle with respect to the user;

calculating, as one of the changing viewing positions, a viewing position of the user relative to the device and alignment of the device as coordinates in a three-dimensional coordinate system;

rotating a user interface on a display of the device through incremental rotational positions that are each one of a selected number of the pre-defined rotational positions that correspond to the angular increments; and

controlling arrangement of one or more displayed elements in the user interface on the display of the device based on the calculated viewing position of the user of the device and the alignment of the device, type of auto-rotation, and a selected device, scenario, application, or context.

2. The computer-implemented method of claim 1, wherein collecting the sensor data comprises tracking eyes of the user via a camera of the device, the viewing position of the user relative to the device calculated at least in part based on the tracking.

3. The computer-implemented method of claim 1, wherein collecting the sensor data comprises obtaining data via a magnetometer of the device used to determine the alignment of the device relative to a magnetic direction.

4. The computer-implemented method of claim 1, wherein controlling the arrangement of the one or more displayed elements comprises causing rotation of the user interface between two or more of the pre-defined rotational positions.

5. The computer-implemented method of claim 1, wherein controlling the arrangement of the one or more displayed elements comprises alignment of the one or more displayed elements with the viewing plane established by the calculated viewing position of the user of the device.

6. The computer-implemented method of claim 1, wherein controlling the arrangement of the one or more displayed elements comprises switching display of the one or more displayed elements between multiple different display locations provided by the device to compensate for manipulation of the device.

7. The computer-implemented method of claim 1, wherein controlling the arrangement of the one or more displayed elements comprises causing rotation of the one or more displayed elements to multiple different angular positions to compensate for manipulation of the device, the multiple different angular positions corresponding to the pre-defined rotational positions.

8. The computer-implemented method of claim 7, wherein the multiple different angular positions correspond to degree increments of less than forty-five degrees.

9. The computer-implemented method of claim 7, wherein the multiple different angular positions correspond to degree increments of one degree that enable arrangement of the one or more elements into three-hundred and sixty angular positions.

10. The computer-implemented method of claim 1, wherein the device comprises a computing device having a slate form factor.

11. The computer-implemented method of claim 1, wherein the device comprises a wearable computing device.

12. A computing device comprising:

a display device;

a sensor system having multiple sensors including a camera and a magnetometer; and

a display manager configured to;

receive an indication of at least one of a selectable type of auto-rotation, a number of pre-defined rotational positions, and corresponding angular increments, wherein the angular increments are selectively varied for different devices, scenarios, applications, or contexts;

obtain sensor data collected from the multiple sensors including tracking eyes of a user of the computing device via the camera and obtaining directional data via the magnetometer;

calculate a viewing plane and a viewing angle based at least in part on the tracking the eyes of the user;

calculate a viewing position of a user relative to the computing device based upon the calculated viewing plane and viewing angle and alignment of the device relative to a magnetic direction based on the directional data; and

automatically rotate a user interface exposed on the display device to a rotational position of a user selected number of the pre-defined rotational positions of the computing device that corresponds to the calculated viewing position of the user of the computing device and the alignment of the computing device, type of auto-rotation, and a selected device, scenario, application, or context, the pre-defined rotational positions corresponding to the angular increments.

13. A computing device as recited in claim 12, wherein the display manager is configured to perform the rotation of the user interface to compensate for manipulation of the computing device using sensor data that is supplied by the sensor system.

14. A computing device as recited in claim 12, wherein the user interface is rotatable through the incremental rotational positions in the angular increments of about ten degrees or less.

15. A computing device as recited in claim 12, wherein the viewing position of the user relative to the computing device indicates that the computing device is placed in a flat position upon a surface, the display manager configured to cause rotation of the user interface while the computing device is in the flat position.

16. A computing device as recited in claim 12, wherein the viewing position of the user relative to the computing indicates that user is lying in a flat position upon a surface, the display manager configured to control rotation of the user interface while the user is in the flat position to align content in the user interface relative to a head position of the user.

17. A wearable device comprising:

a sensor system having one or more sensors;

a display manager module configured to;

receive an indication of at least one of a selectable type of auto-rotation, a number of ore-defined rotational positions, and corresponding angular increments, wherein the angular increments are selectively varied for different devices, scenarios, applications, or contexts;

collect sensor data from the one or more sensors, the sensor data including indications regarding alignment of the wearable device relative to a magnetic direction;

ascertain a changing spatial orientation of the wearable device with respect to a user interacting with the wearable device based on the collected sensor data, the changing spatial orientation of the wearable device ascertained based in part on a viewing plane and a viewing angle with respect to the user, and based on a circular rotation of the wearable device that includes a display extending circularly in a circular band of the wearable device; and

modify output positions for content displayed on the display integrated in the circular band of the wearable device from a first display region of the display to a second display region of the display to align with the changing spatial orientation, based on the type of auto-rotation and a selected device, scenario, application, or context.

18. The wearable device of claim 17, wherein the display manager module is implemented at least partially by hardware elements of the wearable device.

19. The wearable device of claim 17, wherein the display manager module is configured to modify the output positions by rotation of a user interface exposed on the display of the wearable device into one of eight or more different angular rotations supported by the display manager module that corresponds to the changing spatial orientation.

20. The wearable device of claim 17, wherein:

the display that is integrated in the circular band of the wearable device comprising individual display devices at separate display locations in the circular band; and

the display manager is configured to modify the output positions for the content displayed by selectively switching output of the content between the separate display locations of the wearable device in response to manipulation of the wearable device.