HUMAN READABLE SENTENCES TO REPRESENT COMPLEX COLOR CHANGES

• US 20110288854A1
• Filed: 05/20/2010
• Published: 11/24/2011
• Est. Priority Date: 05/20/2010
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Philippi, Paul W., Griffith, Jeremy, Glass, Brian R., Stevens, Michael T.
• Granted Patent: US 8,510,100 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

Methods and a system for a natural language control interface are provided to enable a user to modify various properties of a document. The modifications comprise building sentences from modification words, and combining them together in one display. The modifications are displayed in real time for a user to observe as they are inputted. The order of the modifications is managed by the user and is configured to be changed, added and/or removed.

Claims

1. A method for a user interface system for modifying a document with imaging components using a natural language control system executed via a processor with a memory storing executable instructions having the method, comprising:

receiving the document from an image input device;

presenting an image modification in real time within an image that is displayed in a first view of a display while the image modification is received as input from a user; and

presenting a text-based interface in a second view, the text-based interface presenting text-based categories corresponding to portions of a human readable sentence used to generate the image modification.

2. The method of claim 1, further comprising:

presenting a first text-based category in the text-based interface having various color selections for receiving a color to be modified in the document from the user;

presenting a second text-based category in the text-based interface having selections for receiving a magnitude of the image modification or a resultant image modification;

presenting a third text-based category in the text-based interface having selections different from the second text-based category for receiving the magnitude or the resultant image modification;

receiving a selection respectively from the first, second and third text-based category; and

compiling and presenting the human readable sentence in a window of the text-based interface representing the image modification while the modification is displayed in the image.

3. The method of claim 1, wherein the image comprises a graphic image preselected from among a database of graphic images to represent the document, a thumbnail representation of the document, or the document to be modified.

4. The method of claim 1, wherein the text-based categories comprise questions presented in the user interface for obtaining a color property selection from the user comprising a color to be modified, a magnitude of the modification, and a resultant image modification.

5. The method of claim 2, wherein presenting the first text-based category comprises presenting a first question to the user for a color to be selected for the modification having a drop down menu with various color selections;

wherein presenting the second text-based category comprises presenting a second question to the user for the magnitude or the resultant image modification having a drop down menu with various text-based selections comprising the magnitude or resultant image modification selections;

wherein presenting the third text-based category comprises presenting a third question to the user for the magnitude or the resultant image modification having a drop down menu with various text-based selections comprising or resultant image modification selections;

wherein the second question and third question correspond respectively to the magnification or the resultant color selections in particular and are different from one another, and the resultant image modification is indicative of a direction by which the image modification is directed towards.

6. The method of claim 2, wherein the second text-based category is actively presented to the user upon receiving a selection from the first text-based category, and the third text-based category is actively presented to the user upon receiving a selection from the second text-based category.

7. The method of claim 2, wherein the first text-based category comprises a first portion of the human readable sentence, the second text-based category comprises a second portion of the human readable sentence, and the third text-based category comprises a third portion of the human readable sentence.

8. The method of claim 1, wherein the image modification presented in real time is presented within a portion of the image that is displayed in the first view, the portion comprising the color selected for the modification and/or objects within the image selected by the user to be modified.

9. The method of claim 1, further comprising:

receiving the color selected to be modified via a touch sensed directly on the color in the image within the display.

10. The method of claim 1, further comprising:

displaying in the window more than one human readable sentence corresponding respectively to more than one image modification generated;

upon receiving a specified ordering with a corresponding priority of the more than one human readable sentence for the respective one or more image modification generated, sorting the more than one human readable sentence in the window for display according to the specified ordering, wherein sentences sorted with a higher priority in the specified order generate the respective image modifications in the image first before other image modifications.

11. The method of claim 1, further comprising:

mapping the document into a working color space;

wherein the human readable sentence comprises a natural language command;

receiving the natural language command to modify the color in a portion of the photo image selected by the user, wherein the natural language command specifies the color selected for the image modification in the photo image, and the modification associated therewith in natural language;

parsing the natural language command into a description of a color property to be modified, the magnitude of the modification, and the resultant image modification;

converting the parsed natural language command into a transform in the working color space;

applying the transform to the color selected by the user of the at least a portion of the document in the working color space; and

modifying the selected color of the photo image based upon application of the transform in the working color space.

12. A method for a user interface system for modifying a document with imaging components using a natural language control system executed via a processor with a memory storing executable instructions having the method, comprising:

receiving the document from an image input device;

presenting an image modification in real time within an image that is displayed in a first view of a display while the image modification is received by a user;

presenting a text-based interface in a second view concurrently with the first view having the image, the text-based interface presenting text-based categories corresponding to portions of a human readable sentence used to generate the image modification;

displaying in the second view more than one human readable sentence corresponding respectively to more than one image modification generated;

upon receiving a specified ordering with a corresponding priority of the more than one human readable sentence for the respective image modifications generated, sorting the more than one human readable sentence in the window for display according to the specified ordering.

13. The method of claim 12, wherein sentences sorted with a higher priority in the specified ordering generate the respective image modifications in the image first thereby affecting the image modification of subsequent sentences.

14. The method of claim 12, the text-based categories comprising user selections respectively for color properties to be modified comprising one or more colors to be modified, one or more magnitudes of the modification, and one or more resultant modifications, the method further comprising:

presenting the selections from the categories selected by the user in a window of the second view to display a human readable sentence representing the image modification while the modification is displayed in the image.

15. A user interface system of an imaging system for modification of a document having a natural language control system, comprising:

a user display comprising a first view area having an image displayed therein for the image modification which is selected by a user and configured to present the image modification in real time to the user in the image within the view of the display at portions of the photo image having the color; and

a textual interface comprising text-based categories in a second view with text-based selections comprising color selections, magnitude selections, and resultant image modification selections respectively for the image modification in the document;

a window of the second view for displaying one or more human readable sentences comprising portions within each sentence respectively representing a color selection, a magnitude selection, and a resultant image modification; and

a sorting engine configured to sort the one or more human readable sentences that respectively correspond to image modifications, the sorting based on a specified priority inputted to the engine of the one or more human readable sentences for the respective image modifications.

16. The system of claim 15, wherein sentences sorted with a higher priority in the specified priority generate the respective image modifications in the image first thereby affecting the image modification of sentences with subsequent priority.

17. The system of claim 15, wherein the text-based categories comprise a first text-based category having a first portion of a human readable sentence, the second text-based category having a second portion of the human readable sentence, and the third text-based category having a third portion of the human readable sentence.

18. The system of claim 15, wherein the image comprises a graphic image preselected from among a database of graphic images to represent the document, a thumbnail representation of the document, or the document to be modified.

19. The method of claim 15, wherein the text-based categories comprise questions presented in the user interface for obtaining a color property selection from the user comprising a color to be modified, a magnitude of the modification, and a resultant image modification comprising a color for the modification to be directed towards.

20. The system of claim 15, wherein the natural language control system comprises:

a dictionary of ordinary language color terms corresponding to the text-based selections, wherein each portion of a working color space is mapped to at least one term in the color term dictionary;

an ordinary language command lexicon and syntax for use with the ordinary language color terms that correspond to transformations in the working color space; and

a processor configured to parse the ordinary language command related to colors in the document selected for modification, to determine which portions of the image to modify by consulting the dictionary of ordinary language terms in view of the parsed ordinary language command, and to determine how to adjust the image based on the color to be modified by consulting the ordinary language lexicon and syntax.