Method and system for operating a display apparatus

• US 9,811,155 B2
• Filed: 03/27/2015
• Issued: 11/07/2017
• Est. Priority Date: 03/27/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: SensoMotoric Instruments GmbH (Apple Inc.)
• Inventors: Schmidt, Eberhard
• Primary Examiner(s): GIESY, ADAM
• Application Number:
  US15/129,010
• Publication Number:
  US 20170108920A1
• Time in Patent Office: 956 Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/18 of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/391 Resolution modifying circuits, e.g. variable screen formats

Abstract

The invention relates to a method for operating a display apparatus (12), comprising the following steps: capture of image data from at least one part of the head of a user (14) (step 110), ascertainment of at least one parameter, which the user (14) can alter by moving at least one part of the head, from the image data captured in step a) (step 120), control of a visual representation on a display apparatus (12) on the basis of the ascertained at least one parameter (step 130), wherein the capture in step a) is taken as a basis for ascertaining at least one quality parameter (step 140) and the at least one quality parameter is taken as a basis for varying at least one resolution of at least one part of the visual representation (step 150). Moreover, the invention relates to a corresponding computer program product and to a corresponding system for operating a display apparatus.

Claims

1. A method for operating a display apparatus, including the following steps:

a) capturing image data of at least one part of the head of a user;

b) ascertaining at least one parameter value of at least one parameter from the image data captured in step a), wherein the at least one parameter value is alterable in response to movement of at least one part of the head of the user;

c) controlling a visual representation for operating an application by the user on a display apparatus depending on the ascertained at least one parameter;

d) ascertaining at least one quality parameter based on the capture in step a); and

e) varying at least one resolution of at least one part of the visual representation depending on the at least one quality parameter;

wherein the parameter ascertained in step b) represents a measured quantity and the quality parameter represents the accuracy of measurement associated with the measured quantity or a measure of the accuracy of measurement associated with the measured quantity.

2. The method according to claim 1,

wherein

for ascertaining the at least one quality parameter, a quality and/or magnitude of the parameter ascertained in step b) and/or a quality of capture of the image data and/or for determining the at least one quality parameter the image data captured in step a) is analyzed.

3. The method according to claim 1,

wherein

the visual representation includes at least one movable element movable by movement of at least one part of the head, wherein in step e) the resolution of a movement speed of the at least one movable element is varied depending on the at least one quality parameter.

4. The method according to claim 1,

wherein

in step e) the resolution of the entire visual representation is varied depending on the at least one quality parameter.

5. The method according to claim 4,

wherein

the visual representation includes at least one static and/or dynamic element, wherein in step e) the resolution of the static and/or dynamic element is varied depending on the at least one quality parameter.

6. The method according to claim 4,

wherein

in step e) the resolution of the entire visual representation is varied.

7. The method according to claim 4,

wherein

as a part of the visual representation, the resolution of at least one element from the following group is varied:

an area according to a magnifier function;

the content of a presettable working window;

a mouse pointer;

a cursor;

an operating element of an application;

a content element or an area of the data represented by an application;

or

logical segments of an application, which the visual representation generates.

8. The method according to claim 1,

wherein

for ascertaining the at least one quality parameter, at least one quantity from the following group is ascertained:

parameters of the image data captured in step a), in particular noise portions, frequency, intensity distributions in the image and/or of at least one ascertained object;

or

parameters in the evaluation of the image data captured in step a), in particular in intermediate or result data calculated from the captured image data.

9. The method according to claim 1,

wherein

for ascertaining the at least one quality parameter, the image data captured in step a) is analyzed with regard to at least one of the following quantities:

frequency portions in presettable frequency ranges, in particular by spatio-temporal frequency analysis or wavelets;

contrast values in presettable image areas, in particular gradients, edges, intensities, rates/distributions in image areas;

magnitude values of presettable parameters;

magnitude values of result data;

the presence of at least one presettable object; and

/or

a rate of change or acceleration of intermediate or result data ascertained from the image data captured in step a).

10. The method according to claim 1,

wherein

the resolution is varied according to a linear, a non-linear, a continuous and/or a discontinuous characteristic line.

11. The method according to claim 1,

wherein

the variation of the resolution can be turned on and off by a user in particular by a key command, a voice command or a gesture command.

12. The method according to claim 1,

wherein

at least one resolution once set is stored and again retrieved by a user depending on or independently of an application generating the visual representation, wherein the storage and retrieval is effected by pre-adjusted interactions or interactions presettable by the user, in particular by key commands, voice commands or gesture commands, or that a context is co-stored and the retrieval of a stored resolution is automatically effected upon occurrence of the context.

13. The method according to claim 12,

wherein

the context is determined from one or more of the following parameters:

a presettable application;

a presettable file;

a presettable view of an application, in particular of the toolbar, windows, tools;

a presettable view of a file, in particular with respect to the visible data, the processing settings, the layout, the visible structures;

or

a certain process step in a flow, which is offered by an application.

14. The method according to claim 1,

wherein

the at least one quality parameter is ascertained:

after step a), in particular with step b) or between step b) and step c);

continuously, i.e. after step a) and further in presettable temporal distances and/or depending on at least one presettable event, preferably upon exceeding and/or falling below at least one presettable threshold value for image, intermediate or result data, in particular of at least one quality parameter derived from this data.

15. The method according to claim 1,

wherein

the adaptation of the resolution is effected according to at least one of the following conditions:

continuously, whenever the quality parameter obtains a new value;

if the quality parameter exceeds at least one presettable threshold value;

if the context changes;

or

if the user executes a presettable action, in particular presses a presettable key, executes a presettable gesture and/or delivers a presettable acoustic signal.

16. The method according to claim 1,

wherein

as the at least one part of the head of the user, his head and/or one and/or two eyes of the user is/are moved.

17. The method according to claim 1,

wherein

the quality parameter relates to a quality of the image data or a quality of intermediate or result data calculated from the image data.

18. The method according to claim 1,

wherein

the quality parameter relates to a quality of the parameter ascertained from the image data or of the image data.

19. A non transitory computer program product with a program for a computer device, with program code sections for executing and/or causing the steps according to claim 1 when the program is executed on the computer device.

20. A system for operating a display apparatus including:

a capturing device for capturing image data of at least one part of the head of a user;

at least one computing device, which is adapted to ascertain at least one parameter value of at least one parameter from the image data captured by the capturing device, wherein the at least one parameter value is alterable in response to movement of at least one part of the head of the user;

a control device for controlling a visual representation on a display apparatus depending on the at least one parameter ascertained by the computing device for operating an application by the user;

wherein

at least one computing device is provided, which is configured to ascertain at least one quality parameter based on the image data captured by the capturing device, wherein the control device is further adapted to vary at least one resolution of at least one part of the visual representation depending on the at least one quality parameter, wherein the parameter ascertained by the at least one computing device represents a measured quantity and the quality parameter represents the accuracy of measurement associated with the measured quantity or a measure of the accuracy of measurement associated with the measured quantity.

21. The system according to claim 20,

wherein

the quality parameter relates to a quality of the image data or a quality of intermediate or result data calculated from the image data.

22. The system according to claim 20,

wherein

the quality parameter relates to a quality of the parameter ascertained from the image data or of the image data.