PARSING OF DECLARATIONS IN ALL BRANCHES OF PREPROCESSOR CONDITIONALS

• US 20090313604A1
• Filed: 06/17/2008
• Published: 12/17/2009
• Est. Priority Date: 06/17/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Miceli, Thierry
• Granted Patent: US 8,099,721 B2
• Time in Patent Office: Days
• US Class Current: 717/120
• CPC Class Codes: G06F 8/423 Preprocessors
• CPC Class Codes: G06F 8/427 Parsing

Abstract

Declarations from an input source code or tokenized source code are serialized into a stream of tokens produced by following each branch of a preprocessor conditional directive statement that interrupts a declaration. Tokens are labeled with a parsing path indicator corresponding to a parsing path induced by branches of a preprocessor conditional directive. The declarations that are formed along the different parsing paths are serialized by fetching the tokens that belong to the first parsing path in a first pass, and passing the tokens on to a next phase of a compiler. The pointer that marks the next token is repositioned to return to the start of the declaration. The declaration may be serialized again through the second parsing path in a second pass. The operation may be repeated until each of the parsing paths induced by the presence of branches of the preprocessor conditional directives in the source code is exhausted.

Claims

1. A system that parses a plurality of preprocessor conditional branches of a preprocessor conditional directive statement comprising:

a declaration serializer that serializes input into a stream of tokens produced by following each parsing path induced by mutually exclusive branches of a preprocessor conditional directive statement interrupting a declaration,

wherein the declaration serializer labels tokens belonging to a first parsing path with a first parsing path indicator,

wherein the declaration serializer labels tokens belonging to a second parsing path with a second parsing path indicator,

wherein the declaration serializer fetches the tokens that belong to the first parsing path in a first pass and returns the tokens that belong to the first parsing path to a caller and

wherein the declaration serializer fetches the tokens that belong to the second parsing path in a second pass and returns the tokens that belong to the second parsing path to the caller.

2. The system of claim 1, wherein the caller comprises a compiler, a program development tool, a lexer, a syntax analyzer, a preprocessor or a preprocessor/lexer.

3. The system of claim 1, wherein a token buffer is generated by the declaration serializer, the token buffer comprising a plurality of tokens of the declaration, wherein each token of the plurality of tokens of the declaration is labeled with at least one parsing path indicator corresponding to a parsing path to which each token belongs.

4. The system of claim 1, wherein the declaration serializer returns tokens to the caller in the first pass wherein tokens labeled with the first parsing path indicator are processed and the second pass wherein tokens labeled with the second parsing path indicator are processed.

5. The system of claim 4, wherein the declaration serializer detects an interrupted declaration by detecting a beginning of declaration indicator, an end of declaration indicator and a preprocessor conditional directive between the beginning of declaration indicator and the end of declaration indicator.

6. The system of claim 3, wherein in response to detecting an end of declaration indicator, the declaration serializer returns to a beginning of the declaration and serializes tokens of a next parsing path.

7. The system of claim 1, wherein parsing paths induced by mutually exclusive branches of the preprocessor conditional directive statement are detected by matching preprocessor conditional directives of the preprocessor conditional directive statement.

8. A method of parsing a plurality of preprocessor conditional branches of a preprocessor conditional directive statement comprising:

obtaining tokenized input;

in response to determining that the tokenized input represents a declaration, wherein the declaration is interrupted by a preprocessor conditional directive of a preprocessor conditional directive statement, wherein the preprocessor conditional directive statement comprises a plurality of mutually exclusive branches comprising at least a first branch and a second branch;

labeling each token of the interrupted declaration with at least a first parsing path indicator of a plurality of parsing path indicators, the at least first parsing path indicator corresponding to a first parsing path, and wherein a second parsing path indicator corresponds to a second parsing path, wherein each of the plurality of parsing paths is induced by the plurality of mutually exclusive branches of the preprocessor conditional directive statement; and

returning the tokens labeled with the at least first parsing path indicator in a first pass to a caller.

9. The method of claim 8, further comprising:

returning the tokens labeled with the second parsing path indicator in a second pass to the caller.

10. The method of claim 8, further comprising:

storing a location of a beginning of the interrupted declaration and in response to detecting an end of declaration indicator of the first parsing path, serializing the interrupted declaration again by starting at the beginning of the interrupted declaration and processing the tokens belonging to the second parsing path.

11. The method of claim 10, further comprising:

generating a token buffer to keep track of tokens belonging to the first parsing path and tokens belonging to the second parsing path.

12. A computer-readable storage medium comprising computer-executable instructions which when executed cause a computing environment to:

obtain tokenized input representing a declaration;

in response to determining that the tokenized input represents a declaration, wherein the declaration is interrupted by a preprocessor conditional directive of a preprocessor conditional directive statement, wherein the preprocessor conditional directive statement comprises a plurality of mutually exclusive branches comprising at least a first branch and a second branch,

label each token of the interrupted declaration with at least a first parsing path indicator of a plurality of parsing path indicators, the at least first parsing path indicator corresponding to a first parsing path, and wherein a second parsing path indicator corresponds to a second parsing path, wherein each of the plurality of parsing paths is induced by the plurality of mutually exclusive branches of the preprocessor conditional directive statement;

store the labeled tokens in a token buffer; and

return the tokens labeled with the at least first parsing path indicator to a caller in a first pass of the token buffer and return the tokens labeled with the second parsing path indicator to the caller in a second pass of the token buffer.

13. The computer-readable storage medium of claim 12, comprising further computer-executable instructions, which when executed cause the computing environment to:

store a location of a beginning of the interrupted declaration.

14. The computer-readable storage medium of claim 13, comprising further computer-executable instructions, which when executed cause the computing environment to:

in response to detecting an end of declaration indicator of the first parsing path, serialize the declaration again by starting at the location of the beginning of the interrupted declaration and processing the tokens belonging to the second parsing path.

15. The computer-readable storage medium of claim 12, comprising further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a compiler.

16. The computer-readable storage medium of claim 12, comprising further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a program development tool.

17. The computer-readable storage medium of claim 12, comprising further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a lexer.

18. The computer-readable storage medium of claim 12, comprising further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a preprocessor.

19. The computer-readable storage medium of claim 16, comprising further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a preprocessor/lexer.

20. The computer-readable storage medium of claim 14, comprising further computer-executable instructions, which when executed cause the computing environment to:

determine that the first parsing path and the second parsing path are parsing paths induced by mutually exclusive branches of the preprocessor conditional directive statement by matching preprocessor conditional directives of the preprocessor conditional directive statement.