Human readable sentences to represent complex color changes

• US 8,510,100 B2
• Filed: 05/20/2010
• Issued: 08/13/2013
• Est. Priority Date: 05/20/2010
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Glass, Brian R., Griffith, Jeremy, Philippi, Paul W., Stevens, Michael T.
• Primary Examiner(s): Hudspeth, David R
• Assistant Examiner(s): Nguyen, Timothy
• Application Number:
  US12/783,633
• Publication Number:
  US 20110288854A1
• Time in Patent Office: 1,181 Days
• Field of Search: 704/9, 434/98
• US Class Current: 704/9
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

Methods and a system for a natural language control interface are provided to enable a user to modify various properties of a document. The modifications comprise building sentences from modification words, and combining them together in one display. The modifications are displayed in real time for a user to observe as they are inputted. The order of the modifications is managed by the user and is configured to be changed, added and/or removed.

Claims

1. A method for a user interface system for modifying a document with imaging components using a natural language control system executed via a processor with a memory storing executable instructions having the method, comprising:

receiving the document from an image input device;

presenting an image modification in real time within an image that is displayed in a first view of a display while the image modification is received as input from a user; and

presenting a text-based interface in a second view, the text-based interface presenting text-based categories corresponding to portions of a human readable sentence used to generate the image modification;

displaying in a window more than one human readable sentence corresponding respectively to more than one image modification generated;

upon receiving a user specified ordering with a corresponding priority of the more than one human readable sentence for the respective one or more image modification generated, sorting the one or more image modification according to a user specified input of the more than one human readable sentence displayed in the window, wherein sentences with a higher user specified priority in the user specified order generate the respective image modifications in the image before other image modifications displayed in descending order of priority in the window, and wherein the order is dynamically changed by the user through a user interface.

2. The method of claim 1, further comprising:

presenting a first text-based category in the text-based interface having various color selections for receiving a color to be modified in the document from the user;

presenting a second text-based category in the text-based interface having selections for receiving a magnitude of the image modification or a resultant image modification;

presenting a third text-based category in the text-based interface having selections different from the second text-based category for receiving the magnitude or the resultant image modification;

receiving a selection respectively from the first, second and third text-based category; and

compiling and presenting the human readable sentence in a window of the text-based interface representing the image modification while the modification is displayed in the image.

3. The method of claim 1, wherein the image comprises a graphic image preselected from among a database of graphic images to represent the document, a thumbnail representation of the document, or the document to be modified.

4. The method of claim 1, wherein the text-based categories comprise questions presented in the user interface for obtaining a color property selection from the user comprising a color to be modified, a magnitude of the modification, and a resultant image modification.

5. The method of claim 2, wherein presenting the first text-based category comprises presenting a first question to the user for a color to be selected for the modification having a drop down menu with various color selections;

wherein presenting the second text-based category comprises presenting a second question to the user for the magnitude or the resultant image modification having a drop down menu with various text-based selections comprising the magnitude or resultant image modification selections;

wherein presenting the third text-based category comprises presenting a third question to the user for the magnitude or the resultant image modification having a drop down menu with various text-based selections comprising or resultant image modification selections;

wherein the second question and third question correspond respectively to the magnification or the resultant color selections in particular and are different from one another, and the resultant image modification is indicative of a direction by which the image modification is directed towards.

6. The method of claim 2, wherein the second text-based category is actively presented to the user upon receiving a selection from the first text-based category, and the third text-based category is actively presented to the user upon receiving a selection from the second text-based category.

7. The method of claim 2, wherein the first text-based category comprises a first portion of the human readable sentence, the second text-based category comprises a second portion of the human readable sentence, and the third text-based category comprises a third portion of the human readable sentence.

8. The method of claim 1, wherein the image modification presented in real time is presented within a portion of the image that is displayed in the first view, the portion comprising the color selected for the modification and/or objects within the image selected by the user to be modified.

9. The method of claim 1, further comprising:

receiving the color selected to be modified via a touch sensed directly on the color in the image within the display.

10. The method of claim 1, further comprising:

mapping the document into a working color space;

wherein the human readable sentence comprises a natural language command;

receiving the natural language command to modify the color in a portion of the photo image selected by the user, wherein the natural language command specifies the color selected for the image modification in the photo image, and the modification associated therewith in natural language;

parsing the natural language command into a description of a color property to be modified, the magnitude of the modification, and the resultant image modification;

converting the parsed natural language command into a transform in the working color space;

applying the transform to the color selected by the user of the at least a portion of the document in the working color space; and

modifying the selected color of the photo image based upon application of the transform in the working color space.

11. A method for a user interface system for modifying a document with imaging components using a natural language control system executed via a processor with a memory storing executable instructions having the method, comprising:

receiving the document from an image input device;

presenting an image modification in real time within an image that is displayed in a first view of a display while the image modification is received by a user;

presenting a text-based interface in a second view concurrently with the first view having the image, the text-based interface presenting text-based categories corresponding to portions of a complete human readable sentence used to generate the image modification;

displaying in the second view more than one human readable sentence corresponding respectively to more than one image modification generated;

upon receiving a user specified ordering with a corresponding user specified priority of the more than one human readable sentence for the respective image modifications generated, sorting the one or more image modifications described in the more than one human readable sentence in the window according to the user specified ordering, wherein sentences with a higher user specified priority in the user specified ordering generate the respective image modifications in the image first thereby affecting the image modification of subsequent sentences with a lower user specified priority and wherein the ordering is dynamically changed by the user through a user interface.

12. The method of claim 11, the text-based categories comprising user selections respectively for color properties to be modified comprising one or more colors to be modified, one or more magnitudes of the modification, and one or more resultant modifications, the method further comprising:

presenting the selections from the categories selected by the user in a window of the second view to display a human readable sentence representing the image modification while the modification is displayed in the image.

13. A user interface system of an imaging system for modification of a document having a natural language control system, comprising:

a user display comprising a first view area having an image displayed therein for the image modification which is selected by a user and configured to present the image modification in real time to the user in the image within the view of the display at portions of the photo image having the color; and

a textual interface comprising text-based categories in a second view with text-based selections comprising color selections, magnitude selections, and resultant image modification selections respectively for the image modification in the document;

a window of the second view for displaying one or more human readable sentences comprising portions within each sentence respectively representing a color selection, a magnitude selection, and a resultant image modification; and

a sorting engine configured to sort the one or more human readable sentences that respectively correspond to image modifications, the sorting based on a specified priority inputted by the user to the engine of the one or more complete human readable sentences for the respective image modifications, wherein sentences with a higher user specified priority in the specified priority generate the respective image modifications in the image first thereby affecting the image modification of sentences with subsequent lower priority and wherein the priority is dynamically changed by the user through a user interface.

14. The system of claim 13, wherein the text-based categories comprise a first text-based category having a first portion of a complete human readable sentence, the second text-based category having a second portion of the complete human readable sentence, and the third text-based category having a third portion of the complete human readable sentence.

15. The system of claim 13, wherein the image comprises a graphic image preselected from among a database of graphic images to represent the document, a thumbnail representation of the document, or the document to be modified.

16. The system of claim 13, wherein the text-based categories comprise questions presented in the user interface for obtaining a color property selection from the user comprising a color to be modified, a magnitude of the modification, and a resultant image modification comprising a color for the modification to be directed towards.

17. The system of claim 13, wherein the natural language control system comprises:

a dictionary of ordinary language color terms corresponding to the text-based selections, wherein each portion of a working color space is mapped to at least one term in the color term dictionary;

an ordinary language command lexicon and syntax for use with the ordinary language color terms that correspond to transformations in the working color space; and

a processor configured to parse the ordinary language command related to colors in the document selected for modification, to determine which portions of the image to modify by consulting the dictionary of ordinary language terms in view of the parsed ordinary language command, and to determine how to adjust the image based on the color to be modified by consulting the ordinary language lexicon and syntax.