Front-facing camera and maximized display screen of a mobile device

• US 10,353,426 B2
• Filed: 06/28/2018
• Issued: 07/16/2019
• Est. Priority Date: 06/29/2017
• Status: Active Grant

• Current Assignee: Inodyn Newmedia GmbH
• Original Assignee: Inodyn Newmedia GmbH
• Inventors: Pantel, Lothar
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Noh, Jae N
• Application Number:
  US16/021,930
• Publication Number:
  US 20180307269A1
• Time in Patent Office: 383 Days
• Field of Search: 348 36
• CPC Class Codes: G02F 1/13338 Input devices, e.g. touch panels
• CPC Class Codes: G06F 1/1605 Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2340/145 related to small screens
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/2092 Details of a display terminals using a flat panel, the details relating to the control arrangement of the display terminal and to the interfaces thereto suitable for both CRT and flat panel G09G5/003; specific for a CRT G09G1/165
• CPC Class Codes: H04M 1/0264 for a camera module assembly
• CPC Class Codes: H04M 1/0266 for a display module assembly
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H10K 59/40 OLEDs integrated with touch screens

Abstract

A mobile electronic device, for example a smartphone, may feature thin or curved borders at the left, right, top, and/or bottom edge of the display screen. To accommodate front-facing optical sensor(s) and/or emitter(s) despite a border that is too thin (e.g., too thin for a front camera module and lens), two or more corners of the display panel have a cutout, and the optical sensor(s) and/or emitter(s) are disposed, at least in part, in the areas of these cutouts at the corners. The cutouts may have a convex shape for the most part, thereby maximizing the remaining screen area. At least one embodiment may display scrollable screen content that extends over the entire height of the screen, such that the uppermost part is displayed between two cutouts. To adjust the layout, the elements displayed on the screen may be categorized by relevance.

Claims

1. A mobile electronic device featuring an enlarged screen area for scrollable screen content, said mobile electronic device comprising:

a) a display panel having at least a first cutout at the upper-left corner and a second cutout at the upper-right corner,

b) at least one front-facing optical sensor disposed, at least in part, in the area of said first cutout or in the area of said second cutout, wherein at least within the area of said first or second cutout, said front-facing optical sensor has an unobstructed light path,

c) a computer system connected to said display panel and said front-facing optical sensor, said computer system being configured or programmed to;

i) display, on said display panel, a scrollable screen content that seamlessly extends over the entire height of said display panel, such that the uppermost part of said screen content is displayed between said first cutout and said second cutout,

ii) determine, based on an user input, that said screen content is to be scrolled vertically,

iii) determine, based on the current vertical scrolling position of said screen content, object categories of one or more screen objects that overlap with the area of said first cutout or said second cutout, said object categories comprising at least text lines and pictures,

iv) render, while scrolling, a repeatedly updated screen content, wherein;

each screen object that belongs to the text line category and overlaps with the area of said first cutout or said second cutout is modified by reducing the character width while keeping the original height of the characters and the original line spacing, the length of each text line being tailored to a predetermined shape derived, at least in part, from the shape of said first cutout or said second cutout,

and

each screen object that belongs to the picture category is rendered unchanged, such that parts of the screen object that overlap with the area of said first cutout or said second cutout are invisible.

2. The mobile electronic device of claim 1, wherein said scrollable screen content is a scrollable web page, a scrollable document, a scrollable e-book, or scrollable content of an application that is running on said mobile electronic device.

3. The mobile electronic device of claim 1, wherein each screen object that belongs to the text line category and overlaps with the area of said first cutout or said second cutout is modified by reducing the character width, the character spacing, or a combination thereof while keeping the original height of the characters and the original line spacing.

4. The mobile electronic device of claim 1, wherein said computer system is further configured or programmed to:

a) categorize screen objects that overlap with the area of said first cutout or said second cutout by relevance,

b) change the layout of the displayed screen content if at least one screen object belongs to an essential category and overlaps with the area of said first cutout or said second cutout.

5. The mobile electronic device of claim 1, wherein said computer system is further configured or programmed to:

a) determine the relevance of at least one affected screen object that overlaps with the area of said first cutout or said second cutout,

b) determine the relevance of at least one blocking screen object that blocks the relocatability of said at least one affected screen object,

c) determine the least relevant screen objects from a list of screen objects, said list comprising at least said at least one affected screen object and said at least one blocking screen object,

d) reduce the size of at least one less relevant screen object, and shift at least one affected screen object that overlaps with the area of said first cutout or said second cutout.

6. The mobile electronic device of claim 1, wherein said computer system is further configured or programmed to display a setup option on said display panel, wherein said setup option allows the user to configure the relevance of object categories as a user preference.

7. The mobile electronic device of claim 1, wherein said computer system is further configured or programmed to determine by means of optical character recognition whether an image that overlaps with the area of said first cutout or said second cutout contains relevant symbols or characters in the image area that overlaps with said first or second cutout.

8. The mobile electronic device of claim 7, wherein said computer system is further configured or programmed to move relevant symbols or characters contained in the image area that overlaps with said first or second cutout to another location within said image by means of image processing at the bitmap level.

9. The mobile electronic device of claim 1, wherein said computer system is further configured or programmed to distort the screen content near said first or second cutout by scaling down horizontal lines of pixels, vertical lines of pixels, or a combination thereof to a shorter length, such that the length of each line of pixels is tailored to a predetermined shape derived, at least in part, from the shape of said first cutout or said second cutout.

10. The mobile electronic device of claim 1, wherein said front-facing optical sensor at the location of said first or second cutout fills the void at the level of said display panel at least in part.

11. The mobile electronic device of claim 1, wherein:

a) said front-facing optical sensor is a first front-facing camera and is disposed, at least in part, in the area of said first cutout,

b) a second front-facing camera is disposed, at least in part, in the area of said second cutout, said second front-facing camera being connected to said computer system,

c) said computer system is configured or programmed to process at least a part of the image data captured by said first front-facing camera and said second front-facing camera, wherein the interaxial separation between said first front-facing camera and said second front-facing camera is used to generate data based on stereoscopic information.

12. The mobile electronic device of claim 1, wherein said front-facing optical sensor is a front-facing camera, and wherein said computer system is further configured or programmed to display a status indicator on said display panel, wherein:

a) said status indicator indicates the operating state of said front-facing camera,

b) said status indicator is located adjacent to said first cutout or said second cutout, and

c) said status indicator has the shape of a quarter circle that surrounds said first cutout or said second cutout, such that the endpoints of said quarter circle abut the vertical and horizontal edges of said display panel.

13. The mobile electronic device of claim 1, wherein:

a) at least one optical infrared emitter is disposed, at least in part, in the area of said first cutout or in the area of said second cutout, said optical infrared emitter being capable of illuminating the face of a user at least partially,

b) said computer system is configured or programmed to perform a face recognition using at least a part of the data of said front-facing optical sensor.

14. The mobile electronic device of claim 1, further comprising a touch key located in the area of said first or second cutout in front of said front-facing optical sensor, said touch key being connected to said computer system, wherein said computer system is configured or programmed to execute a function as soon as said touch key is tapped.

15. The mobile electronic device of claim 14, further comprising a printed symbol that is printed on a transparent layer in front of said front-facing optical sensor in the area of said first or second cutout, wherein said printed symbol surrounds said front-facing optical sensor at least in part.

16. The mobile electronic device of claim 1, wherein the height of said first cutout substantially is equal to the width of said first cutout, said first cutout having, at least in part, the shape of a quarter circle, and wherein the height of said second cutout substantially is equal to the width of said second cutout, said second cutout having, at least in part, the shape of a quarter circle, whereby the remaining area of said display panel is maximized.

17. The mobile electronic device of claim 16, wherein said front-facing optical sensor is a front camera module with a front-facing lens, said front-facing lens being disposed, at least in part, in the area of said first cutout or in the area of said second cutout, wherein said front-facing lens is partially surrounded by the border of said first or second cutout substantially at a constant or minimized distance, whereby the remaining area of said display panel is maximized.

18. The mobile electronic device of claim 17, further comprising a casing with thin borders, wherein, in front view, said casing has rounded corners with a curvature radius that is greater than the width of said thin borders, and wherein the curvature radius of the rounded corners of said casing is substantially equal to the curvature radius of said first cutout or said second cutout, such that said front-facing lens essentially is wrapped symmetrically between two rounded corners of said casing and said first or second cutout, whereby the size or the volume of said casing is minimized while the area of said display panel is maximized.

19. The mobile electronic device of claim 1, wherein said display panel has a cutout at all four corners of said display panel, and wherein an acoustic emitter is disposed at the location or close to the location of each cutout, each of said acoustic emitters being connected to said computer system, whereby the user receives sound from all four corners of said mobile electronic device.

20. The mobile electronic device of claim 1, further comprising:

a) a casing with a thin border, said thin border being located at the upper edge of said display panel between said first cutout and said second cutout,

b) a mechanical keyboard located below said display panel, said mechanical keyboard being connected to said computer system,

whereby the height of the display panel is maximized despite the mechanical keyboard.

21. A method of making a mobile electronic device so that the screen area of said mobile electronic device is enlarged for scrollable screen content, said method comprising:

a) providing a display panel with at least a first cutout at the upper-left corner and a second cutout at the upper-right corner,

b) providing at least one front-facing optical sensor,

c) providing a computer system connected to said display panel and said front-facing optical sensor,

d) disposing said front-facing optical sensor, at least in part, in the area of said first cutout or in the area of said second cutout, wherein at least within the area of said first or second cutout, said front-facing optical sensor has an unobstructed light path,

e) configuring or programming said computer system to;

i) display, on said display panel, a scrollable screen content that seamlessly extends over the entire height of said display panel, such that the uppermost part of said screen content is displayed between said first cutout and said second cutout,

ii) determine, based on an user input, that said screen content is to be scrolled vertically,

iii) determine, based on the current vertical scrolling position of said screen content, object categories of one or more screen objects that overlap with the area of said first cutout or said second cutout, said object categories comprising at least text lines and pictures,

iv) render, while scrolling, a repeatedly updated screen content, wherein;

each screen object that belongs to the text line category and overlaps with the area of said first cutout or said second cutout is modified by reducing the character width while keeping the original height of the characters and the original line spacing, the length of each text line being tailored to a predetermined shape derived, at least in part, from the shape of said first cutout or said second cutout,

and

each screen object that belongs to the picture category is rendered unchanged such that parts of the screen object that overlap with the area of said first cutout or said second cutout are invisible.