Volume adjustment on hinged multi-screen device

• US 10,481,856 B2
• Filed: 06/30/2017
• Issued: 11/19/2019
• Est. Priority Date: 05/15/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sadak, Christian Michael, Hernandez Santisteban, Adolfo, Muehlhausen, Andrew Frederick
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): Morehead, III, John H
• Application Number:
  US15/640,318
• Publication Number:
  US 20180329672A1
• Time in Patent Office: 872 Days
• CPC Class Codes: G06F 1/1605 Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1618 the display being foldable up to the back of the other housing with a single degree of freedom, e.g. by 360° rotation over the axis defined by the rear edge of the base enclosure
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 1/1649 the additional display being independently orientable, e.g. for presenting information to a second user
• CPC Class Codes: G06F 1/1677 for detecting open or closed state or particular intermediate positions assumed by movable parts of the enclosure, e.g. detection of display lid position with respect to main body in a laptop, detection of opening of the cover of battery compartment
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1688 the I/O peripheral being integrated loudspeakers
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: H04M 1/0216 Foldable in one direction, i.e. using a one degree of freedom hinge
• CPC Class Codes: H04M 1/0243 using the relative angle between housings
• CPC Class Codes: H04M 1/6008 in the transmitter circuit
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264

Abstract

A mobile computing device including a housing having a first part and a second part, the first part including a first display and a first forward facing camera, and the second part including a second display and a second forward facing camera, at least one speaker mounted in the housing, and a processor mounted in the housing and configured to display a first graphical user interface element having an associated first audio stream on the first display and to display a second graphical user interface element having an associated second audio stream on the second display, wherein the processor is configured to perform face detection on a first and a second image, adjust an audio setting based on a result of the face detection, and play the first and second audio streams out of the at least one speaker based on the adjusted audio setting.

Claims

1. A mobile computing device comprising:

a housing having a first part and a second part, the first part including a first display and a first forward facing camera, and the second part including a second display and a second forward facing camera, wherein the first part and the second part are coupled by a hinge configured to permit the first and second displays to rotate between angular orientations from a face-to-face angular orientation to a back-to-back angular orientation;

a sensor mounted in the housing and configured to measure a relative angular displacement between the first and second parts of the housing;

at least one speaker mounted in the housing; and

a processor mounted in the housing and configured to display a first graphical user interface element having an associated first audio stream on the first display and to display a second graphical user interface element having an associated second audio stream on the second display;

wherein the first and second forward facing cameras are configured to capture respective first and second images; and

the processor is configured to perform face detection on the first and second images, adjust an audio setting based on a result of the face detection, and play the first and second audio streams out of the at least one speaker based on the adjusted audio setting.

2. The mobile computing device of claim 1, wherein the audio setting includes a volume balance setting that controls a volume balance between the first and second audio streams.

3. The mobile computing device of claim 2, wherein the processor is configured to determine which display has a user focus, and adjust the volume balance setting to increase a relative volume of an audio stream associated with a user focused display.

4. The mobile computing device of claim 3, wherein the result of the face detection includes a gaze angle estimation, and the processor is configured to determine the user focused display based on the gaze angle estimation.

5. The mobile computing device of claim 4, wherein the gaze angle estimation is computed by performing image processing on the first and second images and comparing a position of eyes of the user in each of the first and second images, taking into account the relative angular displacement of the first part and the second part detected by the sensor.

6. The mobile computing device of claim 3, wherein the processor is further configured to determine whether the result of the face detection detected a face in the first image captured by the first camera and not in the second image, and if so, then determine that the first display is the user focused display.

7. The mobile computing device of claim 3, wherein the processor is further configured to determine whether the result of face detection detected a face in the second image captured by the second camera and not in the first image, and if so, then determine that the second display is the user focused display.

8. The mobile computing device of claim 3, wherein

the processor is configured to detect which display has user focus by determining that the relative angular displacement is greater than a predetermined threshold via the sensor;

and

responsive to determining that the relative angular displacement is greater than the predetermined threshold, decrease a relative volume of the display that is not the user focused display.

9. The mobile computing device of claim 3, wherein the predetermined threshold is 180 degrees.

10. The mobile computing device of claim 1, wherein the audio setting includes an overall audio volume setting for the first and second audio streams.

11. The mobile computing device of claim 10, wherein the processor is configured to decrease the overall audio volume setting based on the result of the face detection indicating that no face was detected in front of the first display and the second display.

12. The mobile computing device of claim 10, wherein the result of the face detection includes an interpupillary distance estimation, and the processor is configured to determine a distance between the user and the mobile computing device based on the interpupillary distance estimation, and adjust the overall audio volume setting based on the determined distance.

13. The mobile computing device of claim 1, wherein the at least one speaker is a first speaker in the first part of the housing, and the second part of the housing includes a second speaker.

14. A mobile computing device comprising:

a housing having a first part and a second part, the first part including a first display and a first forward facing camera, and the second part including a second display and a second forward facing camera, wherein the first part and the second part are coupled by a hinge configured to permit the first and second displays to rotate between angular orientations from a face-to-face angular orientation to a back-to-back angular orientation;

at least one speaker mounted in the housing;

a sensor mounted in the housing and configured to detected relative angular displacement between the first and second parts of the housing;

a processor configured to adjust a parameter of audio played back on the at least one speaker based upon the detected relative angular displacement; and

wherein the audio includes a first audio stream and a second audio stream, and the parameter adjusted is a volume balance of the first audio stream associated with a graphical user interface element displayed on the first display and the second audio stream associated with a graphical user interface element displayed on the second display.

15. The mobile computing device of claim 14, wherein the parameter adjusted is an overall volume of the audio played back on the at least one speaker.

16. The mobile computing device of claim 14, wherein the at least one speaker is a first speaker in a first part of the housing, and the second part of the housing includes a second speaker.

17. The mobile computing device of claim 16, wherein the parameter adjusted is a balance of the audio played back on the first speaker and the second speaker.

18. The mobile computing device of claim 14, wherein the sensor is a hinge angle sensor coupled to the hinge.

19. A method comprising:

displaying a first graphical user interface element having an associated first audio stream on a first display included in a first part of a housing of a mobile computing device;

displaying a second graphical user interface element having an associated second audio stream on a second display included in a second part of the housing, wherein the first part and the second part are coupled by a hinge configured to permit the first and second displays to rotate between angular orientations from a face-to-face angular orientation to a back-to-back angular orientation;

capturing respective first and second images via a first forward facing camera included in the first part of the housing and a second forward facing camera included in the second part of the housing;

performing face detection on the first and second images;

adjusting an audio setting based on a result of the face detection; and

playing the first and second audio streams out of at least one speaker mounted in the housing based on the adjusted audio setting.