Method of and apparatus for displaying image

• US 5,455,601 A
• Filed: 02/28/1994
• Issued: 10/03/1995
• Est. Priority Date: 04/21/1993
• Status: Expired due to Term

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Ozaki, Tohru
• Primary Examiner(s): Kuntz, Curtis
• Assistant Examiner(s): HARVEY, MINSUN OH
• Application Number:
  US08/202,579
• Time in Patent Office: 582 Days
• Field of Search: 345/157, 345/156, 345/123, 345/121, 345/146
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/30 Control of display attribute
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers

Abstract

A display section displays an image to be displayed on a screen. An image processing section detects an opening/closing of an operator'"'"'s eyelids on the basis of an image of the operator'"'"'s face. A display controlling section changes the video data displayed on the display screen through a display section on the basis of the eyelid opening/closing state detected by the image processing section in accordance with a timing at which the operator does not look at the display screen.

Claims

1. A method of displaying an image formed by video data on a screen and viewed by an operator, said method comprising:

a displaying step of sequentially displaying the video data on the screen;

an image inputting step of inputting facial image information indicative of information of a face of the operator;

an image processing step of detecting opening and closing of eyelids of the operator based on the image information of the face;

a signal generating step of generating a rewrite signal indicative of rewriting the video data displayed on the screen; and

a display controlling step of performing a principal change of the video data displayed on the screen when the rewrite signal is generated in said signal generating step and when the operator fails to look at the screen on the basis of the eyelid opening/closing state obtained by said image processing step.

2. The method according to claim 1, wherein:

said image inputting step comprises a step of inputting the facial image information in a period at least as short as a period for which the eyelids are closed by blinking; and

said image processing step comprises a step of detecting the opening and closing of the eyelids in a period at least as short as the eyelid-closed period.

3. The method according to claim 1, wherein said display controlling step comprises a step of rewriting the displayed video data in a substantially-closed state of the eyelids of the operator.

4. The method according to claim 1, wherein said display controlling step comprises a step of rewriting the displayed video data at a proper timing by alternately writing change video data, which is the video data to replace the video data displayed on the screen, in two video memories and reading to-be-displayed video data, which is the change video data already written in the two video memories, from one of the two video memories different from the other one of the two video memories into which the change video data is written.

5. The method according to claim 4, wherein said display controlling step further comprises a step of reading the to-be-displayed video data from the one of the two video memories to the screen for display during an eyelid-closed state only when rewriting the whole screen.

6. The method according to claim 4, wherein said display controlling step further comprises a step of reading the to be displayed video data from the one of the two video memories to the screen for display at any time irrespective of the opening/closing of the eyelids when rewriting only a part of the screen.

7. The method according to claim 6, wherein said display controlling step further comprises a step of reading the to-be-displayed video data from the one of the two video memories to the screen for display during the eyelid-closed state only when rewriting the whole screen.

8. The method according to claim 1, wherein said image processing step comprises an edge extracting step of generating a facial edge image by extracting edges of the facial image information.

9. The method according to claim 8, wherein said edge extracting step comprises a step of performing a spatial differential process on the facial image information to obtain the facial edge image.

10. The method according to claim 9, wherein said image processing step further comprises a step of determining the eyelid opening/closing state by detecting at least one of a shape and a size of an iris part of an eye of the face from the facial edge image.

11. The method according to claim 8, wherein said image processing step further comprises an image cutout step of cutting out an edge image of an eye-part segmented area from the facial edge image.

12. The method according to claim 8, wherein said image processing step further comprises a blink detecting step of detecting an eyelid opening/closing state from the facial edge image.

13. The method according to claim 12, wherein said blink detecting step comprises a step of detecting a blink at a proper timing by writing and reading the facial image information to be processed through an alternate use of two image memories.

14. The method according to claim 1, wherein said image processing step comprises a step of detecting the opening/closing state of the eyelids by intermittently processing the facial image information in a period shorter than an eyelid-closed period due to blinking.

15. An apparatus, run by an operator, for displaying an image formed by video data, said apparatus comprising:

displaying means, including a screen, for sequentially displaying video data on the screen;

imaging means for imaging at least a face of the operator to generate facial image information;

image processing means for detecting opening and closing of eyelids of the operator based on the facial image information;

signal generating means for generating a rewrite signal which indicates rewriting of the video data displayed on the screen; and

display controlling means for performing a principal change of the video data displayed on the screen by said displaying means when the rewrite signal is generated by said signal generating means and when the operator fails to look at the screen on the basis of the eyelid opening/closing state obtained by said image processing means.

16. The apparatus according to claim 15, wherein:

said imaging means further comprises means for generating the facial image information in a period at least as short as a period for which the eyelids are closed by blinking; and

said image processing means comprises means for detecting the opening and closing of the eyelids in a period at least as short as the eyelid-closed period.

17. The apparatus according to claim 15, wherein said display controlling means comprises means for rewriting the displayed video data in a substantially-closed state of the eyelids of the operator.

18. The apparatus according to claim 15, wherein said display controlling means comprises:

two video memories for alternately storing change video data, which is the video data to replace the video data displayed on the screen; and

means for rewriting the displayed video data at a proper timing by writing the change video data to one of the two video memories and reading to-be-displayed video data, which is the change video data already written into the two video memories, from the other one of the two video memories to the displaying means.

19. The apparatus according to claim 18, wherein said display controlling means further comprises means for reading the to-be-displayed video data from the one of the two video memories to the screen for display during an eyelid-closed state only in case of rewriting the whole screen.

20. The apparatus according to claim 18, wherein said display controlling means further comprises means for reading the to-be-displayed video data from the one of the two video memories at any time irrespective of the opening/closing of the eyelids in case of rewriting only a part of the screen.

21. The apparatus according to claim 20, wherein said display controlling step further comprises means for reading the to-be-displayed video data from the one of the two video memories to the screen for display during the eyelid-closed state only in case of rewriting the whole screen.

22. The apparatus according to claim 15, wherein said image processing means comprises edge extracting means for generating a facial edge image by extracting edges of the facial image information.

23. The apparatus according to claim 22, wherein said edge extracting means comprises means for obtaining the facial edge image by performing a spatial differential process on the facial image information.

24. The apparatus according to claim 23, wherein said image processing step further comprises means for determining opening/closing state of the eyelids by detecting at least one of a shape and a size of an iris part of an eye of the face from the facial edge image.

25. The apparatus according to claim 22, wherein said image processing means further comprises image cutout means for cutting out an edge image of an eye-part segmented area from the facial edge image.

26. The apparatus according to claim 22, wherein said image processing means further comprises blink detecting means for detecting an eyelid opening/closing state from the facial edge image.

27. The apparatus according to claim 26, wherein said blink detecting means comprises:

two video memories for storing the facial edge image to be processed; and

means for detecting a blink at a proper timing by writing and reading the facial edge image to be processed through an alternate use of two video memories.

28. The apparatus according to claim 15, wherein said image processing means comprises means for detecting the opening/closing state of the eyelids by intermittently processing the facial image information in a period shorter than an eyelid-closed period due to blinking.

29. A method of displaying an image formed by video data on a screen and viewed by an operator, said method comprising:

a displaying step of sequentially displaying the video data on the screen;

an image inputting step of inputting facial image information of a face of the operator;

an image processing step of detecting opening and closing of eyelids of the operator based on the image information of the face, said image processing step having,

an edge extracting step of generating a facial edge image by extracting edges of the facial image information, said edge extracting step obtaining the facial edge image by effecting a spatial differential process on the facial image information; and

a display controlling step of performing a principal change of the video data displayed on the screen when the operator fails to look at the screen on the basis of the eyelid opening/closing state obtained by said image processing step.

30. The method according to claim 29, wherein said image processing step further comprises a step of determining the eyelid opening/closing state by detecting at least one of a shape and size of an iris part of an eye of the face from the facial edge image.