SYSTEM AND METHOD FOR MACHINE-ASSISTED HUMAN LABELING OF PIXELS IN AN IMAGE

• US 20110007964A1
• Filed: 07/10/2009
• Published: 01/13/2011
• Est. Priority Date: 07/10/2009
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Lin, Jing, Sarkar, Prateek, Saund, Eric
• Granted Patent: US 8,452,086 B2
• Time in Patent Office: Days
• US Class Current: 382/162
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/148 Segmentation of character regions
• CPC Class Codes: G06V 30/19133 Interactive pattern learning with a human teacher
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps

Abstract

A user interface and method is embodied on a computer readable medium and executable on a computer. The user interface is a labeler which labels only foreground pixels of an image stored in a computing environment. The labeler operates in a Region mode/state and Brush mode/state, and includes a Tentative mode that permits an assigned label to be changed after pixels have been selected. Groups of pixels may be selected for labeling at once by a point-and-click command, and a pixel may belong to one or more groups of pixels which are stored in memory as image layers. The groups are formed dynamically by user selection actions, and/or through automatic recognition algorithms. Pixels already labeled with certain labels may be locked to not be altered by additional labeling operations. Unassigned pixels may be highlighted to increase the ease at which they are identified in an image. Comparisons between labeled images are undertaken to indicate differences between different groundtruth labeling.

Claims

1. A groundtruth user interface labeler embodied on a computer readable medium and executable on a computer, the user interface comprising:

a labeling mechanism for assigning labels on a pixel basis to only foreground pixels of an image stored in a computing environment; and

a tentative operation mode permitting the assigned labels to be changed after pixels have been selected but not committed to by a user.

2. The user interface according to claim 1 wherein pixel selection is accomplished using either a region mode or a brush mode, the region mode configured to select pixels by enclosure of the selected pixels, and the brush mode configured to select pixels in a painting operation.

3. The user interface according to claim 1 wherein the labels identify handwritten text, machine print text, machine print graphics and speckle noise.

4. The user interface according to claim 1 wherein selection in the tentative selection operation, generates a tentative label color which has an augmented appearance of a final label color, wherein the final label color is displayed when the commitment is made to the selection made in the tentative operation.

5. The user interface according to claim 1 further including a bootstrapping facility, which permits other labeling systems to operate with the labeler.

6. The user interface according to claim 1, wherein groups of pixels may be selected for labeling at a same time by a point and click command.

7. The user interface according to claim 1, wherein a pixel may belong to one or more groups of pixels which are stored in memory as image layers.

8. The user interface according to claim 1, wherein groups of pixels are formed dynamically by user selection actions.

9. The user interface according to claim 1, wherein groups of pixels are formed through automatic recognition algorithms.

10. The user interface according to claim 1, further including a locking operation wherein pixels already labeled are locked to not be altered by additional labeling operations.

11. The user interface according to claim 1, wherein multiple labeling outputs are compared and differences highlighted.

12. A groundtruth user interface labeler embodied on a computer readable medium and executable on a computer, the user interface comprising:

a labeling mechanism for assigning labels on a pixel basis to only foreground pixels of an image stored in a computing environment; and

a grouping mechanism for forming groups of pixels that may be selected for labeling at a same time, wherein a pixel may belong to one or more groups of pixels which are stored in memory as image layers, and wherein the groups of pixels are formed dynamically by user selection actions and/or by automatic recognition algorithms.

13. The user interface according to claim 12, further including:

a tentative operation mode permitting the assigned labels to be changed after pixels have been selected but not committed to by a user.

14. The user interface according to claim 12, wherein pixel selection is accomplished using either a region mode or a brush mode, the region mode configured to select pixels by enclosure of the selected pixels, and the brush mode configured to select pixels in a painting operation.

15. The user interface according to claim 12, wherein the labels identify handwritten text, machine print text, machine print graphics and speckle noise.

16. The user interface according to claim 12, wherein selection in the tentative selection operation, generates a tentative label color which has an augmented appearance of a final label color, wherein the final label color is displayed when the commitment is made to the selection made in the tentative operation.

17. The user interface according to claim 12, further including a bootstrapping facility, which permits other labeling systems to operate with the labeler.

18. The user interface according to claim 12, wherein the groups of pixels may be selected for labeling at a same time by a point and click command.

19. The user interface according to claim 12, wherein groups of pixels are formed through automatic recognition algorithms.

20. The user interface according to claim 12, further including a locking operation wherein pixels already labeled are locked to not be altered by additional labeling operations.

21. The user interface according to claim 12, wherein multiple labeling outputs are compared and differences highlighted.

22. A system comprising:

a memory;

a processor coupled to the memory; and

a plurality of instructions stored in the memory and executed by the processor to present a groundtruth user interface labeler to enable a user to label groundtruths of an image, the user interface comprising;

a labeling mechanism for labeling only foreground pixels in an image stored in a computing environment; and

a grouping mechanism for forming groups of pixels that may be selected for labeling at a same time, wherein a pixel may belong to one or more groups of pixels which are stored in memory as image layers, and wherein the groups of pixels are formed dynamically by user selection actions and/or by automatic recognition algorithms.

23. The system according to claim 22, further including:

a tentative operation mode permitting the assigned label to be changed after pixels have been selected, but not committed to be a user, wherein multiple labeling outputs can be compared and differences highlighted.