Multi-image communications system

• US 4,695,975 A
• Filed: 10/23/1984
• Issued: 09/22/1987
• Est. Priority Date: 10/23/1984
• Status: Expired due to Fees

• Current Assignee: Profit Technology, Inc.
• Original Assignee: Profit Technology, Inc.
• Inventors: Bedrij, Orest J.
• Primary Examiner(s): Heckler, Thomas M.
• Application Number:
  US06/663,893
• Time in Patent Office: 1,064 Days
• Field of Search: 364/200 MS File, 364/900 MS File
• US Class Current: 715/202
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A device for automatic translation of natural language words into corresponding visual images has an addressable video disc memory storing a large number of images, an image dictionary memory for storing the video disc addresses for sets of images for respective natural language words along with information controlling the selection and display of the respective images, and a computer for addressing the image diictionary memory and interpreting the selection and display information. The translation device is advantageously used in an audio-visual presentation system which also includes a speech synthesizer and a multiple display synchronized to the pronunciation of the speech synthesizer. Preferably, the selection of images is responsive to viewer preferences and the context of the translated words, and redundancy of image presentation is eliminated by random or indexed selection.

Claims

1. A visual communication device for translating a natural language into visual images to be viewed by a human receiver comprising, in combination,

video memory means for storing a plurality of predetermined video images at predetermined image addresses,

video display means for displaying selected images from said video memory to said human receiver,

image dictionary memory means for storing predetermined sets of said image addresses associated with respective ones of a predetermined set of natural language words and addressable to obtain the respective stored set of image addresses for a selected natural language word,

input means for receiving selected natural language words, and

control means for repetitively obtaining individual ones of the selected natural language words from said input means, addressing said image dictionary memory means to obtain the respective sets of image addresses for the selected natural language words, and addressing said video memory means at the respective sets of image addresses to obtain images for display by said video display for viewing by said human receiver, so that the selected natural language words are automatically translated into a succession of visual images for viewing by said human receiver.

2. The visual communication device as claimed in claim 1, further comprising multiple image buffer means for receiving a selected plurality of images from said video memory means and wherein said video display means has a plurality of display areas for displaying respective ones of the images stored in said multiple image buffer means.

3. The visual communication device as claimed in claim 2, wherein said image dictionary memory means includes predetermined display rules stored for respective ones of the sets of said image addresses, said display rules specifying particular methods by which their respective images are displayed by said video display means.

4. The visual communication device as claimed in claim 3, wherein said display rules specify the time sequence in which particular images in respective sets of images are displayed.

5. The visual communication device as claimed in claim 3, wherein said display rules specify the time durations for which particular images in respective sets of images are displayed.

6. The visual communication device as claimed in claim 3, wherein said display rules specify the particular display areas and image placements at which particular images in respective sets of images are displayed.

7. The visual communication device as claimed in claim 3, wherein said display rules include priority descriptions for particular images in respective sets of images and wherein said particular images are selectively displayed in response to comparison of their respective priority descriptions.

8. The visual communication device as claimed in claim 3, wherein said display areas are arranged in a rectangular matrix.

9. The visual communication device as claimed in claim 3, wherein the control unit includes a display information buffer for temporarily storing the display rules obtained by addressing the image dictionary memory means in response to a sequence of words received by said input means, and a display rule processor for controlling the addressing of the video display means in response to the display rules stored in said display information buffer.

10. The visual communication device as claimed in claim 9, wherein said control unit further includes a display state register storing the control information generated by said display rule processor, and wherein said rule processor is responsive to the information stored in said display state register.

11. The visual communications device as claimed in claim 3, wherein said control means includes means for recognizing a predetermined set of control words interspersed in the selected natural language words received by said input means, and upon recognizing said control words executing said control words to override specified ones of the display rules.

12. The visual communications device as claimed in claim 11, wherein said predetermined set of control words includes control words for inhibiting the display of a specified image and displaying a specified image.

13. The visual communications device as claimed in claim 1, wherein at least some of said natural language words each have a plurality of image addresses associated therewith and stored in said image dictionary memory means, and said control means includes selector means for choosing for display a particular one of the plurality of images when a plurality of such image addresses are obtained by the addressing of said image dictionary memory means.

14. The visual communications device as claimed in claim 13, wherein said selector means chooses a particular one of the plurality of images at random.

15. The visual communications device as claimed in claim 13, wherein said image dictionary memory means has stored along with at least some of said plurality of images a pointer to a predetermined random access memory location, and said selector means including means for addressing said predetermined random access memory location, using the contents at said memory location to choose a particular one of the plurality of said images, and changing the contents at said memory location to avoid the redundant selection of a particular one of the plurality of images.

16. The visual communications device as claimed in claim 15, wherein the predetermined memory locations are initially set to predetermined initial values and iteratively cycled through permissible values by said means for choosing.

17. The visual communications system as claimed in claim 13, further comprising text buffer means for storing a plurality of said natural language words occurring in time sequence over a duration including the natural language word currently addressing said image dictionary memory means, wherein said image dictionary memory means has stored along with the image addresses of at least some of said plurality of images a context description, and wherein the choosing by said selection means is responsive to a comparison between the context description and the contents of said text buffer means.

18. The visual communications device as claimed in claim 13, further comprising means for receiving viewer preferences, wherein said image dictionary memory means has stored along with the image addresses of at least some of said plurality of images a viewer preference description, and wherein the choosing by said selector means is responsive to a comparison between the viewer preferences and the viewer preference description.

19. The visual communication device as claimed in claim 1, further comprising speech synthesizer means for pronouncing said natural language words received from said input means coincident with the display of the corresponding video images, and means for transmitting said pronunciations to said human receiver.

20. The visual communications device as claimed in claim 19, further comprising means for synchronizing the pronunciations of said natural language words with the display of corresponding images.

21. The visual communications device as claimed in claim 20, wherein said speech synthesizer means generated a signal at the start of the pronunciation of a word, and further comprising buffer means responsive to said signal for delaying the display of said video images until said signal is received from said speech synthesizer.

22. A multi-image communciations system for translating a sequence of natural language words into a corresponding audio-visual presentation directed to at least one human receiver, said system comprising, in combination,

input means for receiving said sequence of natural language words,

speech synthesizer means for sequentially receiving said natural language words and pronouncing them,

video memory means for storing a plurality of predetermined video images corresponding to a substantial portion of the natural language words used in said natural language,

video display means for simultaneously displaying a plurality of selected ones of said video images,

image dictionary memory means for storing for each word in said substantial portion of the natural language words the set of addresses in said video memory means of corresponding ones of said video images and for storing display rules for respective ones of said video images, said display rules specifying particular methods by which their respective images are displayed by said video display means, and

control means for obtaining individual ones of said natural language words received by said input means, addressing said image dictionary means to obtain the respective sets of image addresses for the selected natural language words and their respective display rules, addressing said video memory means at the respective sets of image addresses to obtain images for display by said video display, and controlling the video display in accordance with the respective display rules so that said sequence of natural language words is translated in real time to a corresponding audio visual presentation.

23. The system as claimed in claim 22 wherein said video memory means includes a laser video disc.

24. The visual communications system as claimed in claim 22, further comprising means for synchronizing the pronunciations of said natural language words with the display of corresponding images.

25. The system as claimed in claim 22, wherein said speech synthesizer means generates a signal at the start of the pronunciation of a word, said system further comprising buffer means responsive to said signal whereby the display of said video images is delayed until said signal is received from said speech synthesizer.

26. The system as claimed in claim 22, wherein said display areas are arranged in a rectangular matrix.

27. The system as claimed in claim 22, wherein said display rules specify the time sequence in which particular images in respective sets of images are displayed.

28. The system as claimed in claim 22, wherein said display rules specify the time durations for which particular images in respective sets of images are displayed.

29. The system as claimed in claim 22, wherein said display rules specify the particular display areas at which particular images in respective sets of images are displayed.

30. The system as claimed in claim 22, wherein said display rules include priority descriptions for particular images in respective sets of images and wherein said particular images are selectively displayed in response to comparison of their respective priority descriptions.

31. The system as claimed in claim 22, wherein the control unit includes a display information buffer for temporarily storing the display rules obtained by addressing the image dictionary memory means in response to a sequence of words received by said input means, and a display rule processor for controlling the addressing of the video display means in response to the display rules stored in said display information buffer.

32. The system as claimed in claim 22, wherein said control unit further includes a display state register storing the control information generated by said display rule processor, and wherein said rule processor is responsive to the information stored in said display state register.

33. The system as claimed in claim 22, wherein said control means includes means for recognizing a predetermined set of control words interspersed in the selected natural language words received by said input means, and upon recognizing said control words executing said control words to override specified ones of the display rules.

34. The system as claimed in claim 33, wherein said predetermined set of control words includes control words for inhibiting the display of a specified image and displaying a specified image.

35. The system as claimed in claim 22, wherein at least some of said natural language words each have a plurality of image addresses associated therewith and stored in said image dictionary memory means, and said control means includes selector means for choosing for display a particular one of the plurality of images when a plurality of such image addresses are obtained by the addressing of said image dictionary memory means.

36. The system as claimed in claim 35, wherein said selector means chooses a particular one of said plurality of images at random.

37. The system as claimed in claim 35, wherein said image dictionary memory means has stored along with at least some of said plurality of images a pointer to a predetermined random access memory location, and said selector means including means for addressing said predetermined random access memory location, using the contents at said memory location to choose a particular one of the plurality of said images, and changing the contents at said memory location to avoid the redundant selection of a particular one of the plurality of images.

38. The system as claimed in claim 37, wherein the predetermined memory locations are initially set to predetermined initial values and iteratively cycled through permissible values by said means for choosing.

39. The system as claimed in claim 35, further comprising text buffer means for storing a plurality of said natural language words occurring in time sequence over a duration including the natural language word currently addressing said image dictionary memory means, wherein said image dictionary memory means has stored along with the image addresses of at least some of said plurality of images a context description, and wherein the choosing by said selection means is responsive to a comparison between the context description and the contents of said text buffer means.

40. The system as claimed in claim 35, further comprising means for receiving viewer preferences, wherein said image dictionary memory means has stored along with the image addresses of at least some of said plurality of images a viewer preference description, and wherein the choosing by said selector means is responsive to a comparison between the viewer preference and the viewer preferences descriptions.

41. A multi-image communciations system for translating a sequence of natural language words into a corresponding audio-visual presentation directed to at least one human receiver, said system comprising, in combination,

input means for receiving said sequence of natural language words,

speech synthesizer means for sequentially receiving said natural language words and pronouncing them and including a text buffer storing a plurality of sequential ones of said natural language words,

video memory means for storing a plurality of predetermined video images corresponding to a substantial portion of the natural language words used in said natural language,

video display means for simultaneously displaying a plurality of selected ones of said video images,

image dictionary memory means for storing for each word in said substantial portion of the natural language words the set of addresses in said video memory means of corresponding ones of said video images, for storing display rules for respective ones of said images for specifying particular methods by which their respective images are displayed by said video display means, for storing context rules for specifying particular methods by which respective images are selected for display for some of the words in said substantial portion of the natural language words in response to the words currently stored in said text buffer, and for storing viewer preference descriptions for specifying whether respective images meet predetermined viewer preferences,

means for receiving viewer preferences,

image selector means for obtaining individual ones of said natural language words received by said input means, addressing said image dictionary memory means to obtain the respective sets of image addresses for the selected natural language words and their associated display and context rules and viewer preference descriptions, applying said context rules and said viewer preference descriptions to select image addresses of respective images in said video memory means for display by said video display means, and

display rule processor for applying said display rules and controlling said video display to display the respective video images from said video memory means in accordance with said display rules.