Synchronized display on hinged multi-screen device

• US 10,440,263 B2
• Filed: 06/19/2017
• Issued: 10/08/2019
• Est. Priority Date: 05/12/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Long, Donna Katherine, Jackson, Andrew Austin, Jeune, Charlene, Sadak, Christian Michael, Maltezos, Mario Emmanuel, Hawthorne, Bryant Daniel
• Primary Examiner(s): Monk, Mark T
• Application Number:
  US15/627,341
• Publication Number:
  US 20180332216A1
• Time in Patent Office: 841 Days
• Field of Search: None
• CPC Class Codes: G06F 1/1618 the display being foldable up to the back of the other housing with a single degree of freedom, e.g. by 360° rotation over the axis defined by the rear edge of the base enclosure
• CPC Class Codes: G06F 1/1641 the display being formed by a plurality of foldable display components G06F1/1647 takes precedence
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 1/1677 for detecting open or closed state or particular intermediate positions assumed by movable parts of the enclosure, e.g. detection of display lid position with respect to main body in a laptop, detection of opening of the cover of battery compartment
• CPC Class Codes: G06F 1/1681 Details related solely to hinges hinge details related to the transmission of signals or power are classified in G06F1/1683
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 23/531 being rotatable or detachable
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 5/2222 Prompting

Abstract

A mobile computing device is provided that comprises a housing having a first part and a second part coupled by a hinge, the first part including a first display and the second part including a second display, wherein the hinge is configured to permit the first and second displays to rotate between angular orientations from a face-to-face angular orientation to a back-to-back angular orientation. The mobile computing device further includes a camera mounted in the first part of the housing and configured to capture image data, the camera and the first display both facing a first direction, and a processor mounted in the housing. In the back-to-back angular orientation, the processor is configured to cause the second display to display the image data while simultaneously causing the first display to display the image data and secondary content.

Claims

1. A mobile computing device comprising:

a housing having a first part and a second part coupled by a hinge, the first part including a first display and the second part including a second display, wherein the hinge is configured to permit the first and second displays to rotate between angular orientations from a face-to-face angular orientation to a back-to-back angular orientation;

a camera mounted in the first part of the housing and configured to capture image data, the camera and the first display both facing a first direction; and

a processor mounted in the housing, configured to;

in the back-to-back angular orientation, to cause the first display to display the image data and speech prompts for a subject recorded in the image data while concurrently causing the second display to display the image data and not the speech prompts,

recognize speech from at least one of the image data or audio data that is captured by a microphone contemporaneously with the image data,

correlate the speech prompts to the recognized speech,

determine whether or not a mistake is made in the recognized speech compared to the speech prompts, and

when a number of mistakes made reaches a predetermined mistake threshold, change the speech prompts to include phonetic spelling for designated words.

2. The mobile computing device of claim 1, further comprising a sensor mounted in the housing and configured to detect an angular orientation of the pair of displays.

3. The mobile computing device of claim 1, wherein the processor is further configured to:

adjust a display progression of the speech prompts according to a speed of the recognized speech.

4. The mobile computing device of claim 1, wherein the processor is further configured to:

generate subtitle data from the correlated speech prompts and recognized speech.

5. The mobile computing device of claim 1, wherein the processor is further configured to:

determine portions of the audio data that are not the recognized speech;

filter the portions from the audio data; and

store or transmit a remaining portion of the audio data that does not include the filtered portions.

6. The mobile computing device of claim 1, wherein the processor is further configured to:

estimate a change in distance of the subject recorded in the image data from the camera over time; and

adjust a size of the speech prompts according to the change in the distance over time.

7. The mobile computing device of claim 1, wherein the processor is further configured to cause the second display to display videographer prompts.

8. A method of operating a mobile computing device comprising a housing having a first part and a second part coupled by a hinge, the first part including a first display and the second part including a second display, the method comprising:

permitting the first and second displays to rotate between angular orientations from a face-to-face angular orientation to a back-to-back angular orientation;

capturing image data with a camera mounted in the first part of the housing, the camera and the first display both facing a first direction;

in the back-to-back angular orientation, displaying the image data and speech prompts for a subject recorded in the image data on the first display while concurrently displaying the image data and not the speech prompts on the second display;

recognizing speech from at least one of the image data or audio data that is captured by a microphone contemporaneously with the image data;

correlating the speech prompts to the recognized speech;

determining whether or not a mistake is made in the recognized speech compared to the speech prompts; and

when a number of mistakes made reaches a predetermined mistake threshold, changing the speech prompts to include phonetic spelling for designated words.

9. The method of claim 8, further comprising:

detecting an angular orientation of a hinge with a sensor mounted in the housing; and

determining a current angular orientation of the pair of displays based on the angular orientation of the hinge.

10. The method of claim 8, further comprising:

adjusting a display progression of the speech prompts according to a speed of the recognized speech.

11. The method of claim 8, further comprising:

generating subtitle data from the correlated speech prompts and recognized speech.

12. The method of claim 8, further comprising:

determining portions of the audio data that are not the recognized speech;

filtering the portions from the audio data; and

storing or transmitting a remaining portion of the audio data that does not include the filtered portions.

13. The method of claim 8, further comprising:

estimating a change in distance of the subject recorded in the image data from the camera over time; and

adjusting a size of the speech prompts according to the change in the distance over time.

14. The method of claim 8, further comprising causing the second display to display videographer prompts.

15. A mobile computing device comprising:

a housing having a first part and a second part coupled by a hinge, the first part including a first display and the second part including a second display, wherein the hinge is configured to permit the first and second displays to rotate between angular orientations from a face-to-face angular orientation to a back-to-back angular orientation;

a camera mounted in the first part of the housing and configured to capture image data, the camera and the first display both facing a first direction;

a sensor mounted in the housing and configured to detect an angular orientation of the pair of displays; and

a processor mounted in the housing, configured to;

in the back-to-back angular orientation, cause the first display to display the image data and speech prompts for a subject recorded in the image data while concurrently causing the second display to display the image data and not the speech prompts,

recognize speech from at least one of the image data or audio data that is captured by a microphone contemporaneously with the image data,

correlate the speech prompts to the recognized speech, and

generate subtitle data from the correlated speech prompts and recognized speech.

16. The mobile computing device of claim 15, wherein the processor is further configured to:

adjust a display progression of the speech prompts according to a speed of the recognized speech.

17. The mobile computing device of claim 15, wherein the processor is further configured to:

determine whether or not a mistake is made in the recognized speech compared to the speech prompts; and

when a number of mistakes made reaches a predetermined mistake threshold, change the speech prompts to include phonetic spelling for designated words.

18. The mobile computing device of claim 15, wherein the processor is further configured to:

determine portions of the audio data that are not the recognized speech;

filter the portions from the audio data; and

store or transmit a remaining portion of the audio data that does not include the filtered portions.

19. The mobile computing device of claim 15, wherein the processor is further configured to:

estimate a change in distance of the subject recorded in the image data from the camera over time; and

adjust a size of the speech prompts according to the change in the distance over time.

20. The mobile computing device of claim 15, wherein the processor is further configured to cause the second display to display videographer prompts.