Parsing of declarations in all branches of preprocessor conditionals

• US 8,099,721 B2
• Filed: 06/17/2008
• Issued: 01/17/2012
• Est. Priority Date: 06/17/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Miceli, Thierry
• Primary Examiner(s): Das, Chameli
• Application Number:
  US12/140,267
• Publication Number:
  US 20090313604A1
• Time in Patent Office: 1,309 Days
• Field of Search: None
• US Class Current: 717/141
• CPC Class Codes: G06F 8/423 Preprocessors
• CPC Class Codes: G06F 8/427 Parsing

Abstract

Declarations from an input source code or tokenized source code are serialized into a stream of tokens produced by following each branch of a preprocessor conditional directive statement that interrupts a declaration. Tokens are labeled with a parsing path indicator corresponding to a parsing path induced by branches of a preprocessor conditional directive. The declarations that are formed along the different parsing paths are serialized by fetching the tokens that belong to the first parsing path in a first pass, and passing the tokens on to a next phase of a compiler. The pointer that marks the next token is repositioned to return to the start of the declaration. The declaration may be serialized again through the second parsing path in a second pass. The operation may be repeated until each of the parsing paths induced by the presence of branches of the preprocessor conditional directives in the source code is exhausted.

Claims

1. A system that parses a plurality of preprocessor conditional branches of a preprocessor conditional directive statement comprising:

a computer comprising a processor and a memory, the computer further comprising;

a declaration serializer that;

obtains tokenized input;

in response to determining that the tokenized input represents a declaration, wherein the declaration is interrupted by a preprocessor conditional directive of a preprocessor conditional directive statement, wherein the preprocessor conditional directive statement comprises a plurality of mutually exclusive branches comprising at least a first branch and a second branch;

labels each token of the interrupted declaration with at least a first parsing path indicator of a plurality of parsing path indicators, the at least first parsing path indicator corresponding to a first parsing path, and wherein a second parsing path indicator corresponds to a second parsing path, wherein each of the plurality of parsing paths is induced by the plurality of mutually exclusive branches of the preprocessor conditional directive statement; and

returns the tokens labeled with the at least first parsing path indicator in a first pass to a caller.

2. The system of claim 1, wherein the caller comprises a compiler, a program development tool, a lexer, a syntax analyzer, a preprocessor or a preprocessor/lexer.

3. The system of claim 1, wherein a token buffer is generated by the declaration serializer, the token buffer comprising a plurality of tokens of the declaration, wherein each token of the plurality of tokens of the declaration is labeled with at least one parsing path indicator corresponding to a parsing path to which each token belongs.

4. The system of claim 1, wherein the declaration serializer detects an interrupted declaration by detecting a beginning of declaration indicator, an end of declaration indicator and a preprocessor conditional directive between the beginning of declaration indicator and the end of declaration indicator.

5. The system of claim 4, wherein in response to detecting an end of declaration indicator, the declaration serializer returns to a beginning of the declaration and serializes tokens of a next parsing path.

6. The system of claim 1, wherein parsing paths induced by mutually exclusive branches of the preprocessor conditional directive statement are detected by matching preprocessor conditional directives of the preprocessor conditional directive statement.

7. A method of parsing a plurality of preprocessor conditional branches of a preprocessor conditional directive statement comprising:

obtaining tokenized input;

in response to determining that the tokenized input represents a declaration, wherein the declaration is interrupted by a preprocessor conditional directive of a preprocessor conditional directive statement, wherein the preprocessor conditional directive statement comprises a plurality of mutually exclusive branches comprising at least a first branch and a second branch;

using a processor, labeling each token of the interrupted declaration with at least a first parsing path indicator of a plurality of parsing path indicators, the at least first parsing path indicator corresponding to a first parsing path, and wherein a second parsing path indicator corresponds to a second parsing path, wherein each of the plurality of parsing paths is induced by the plurality of mutually exclusive branches of the preprocessor conditional directive statement; and

returning the tokens labeled with the at least first parsing path indicator in a first pass to a caller.

8. The method of claim 7, further comprising:

returning the tokens labeled with the second parsing path indicator in a second pass to the caller.

9. The method of claim 7, further comprising:

storing a location of a beginning of the interrupted declaration, detecting an end of declaration indicator of the first path, and in response to detecting the end of declaration indicator of the first parsing path, serializing the interrupted declaration again by starting at the beginning of the interrupted declaration and processing the tokens belonging to the second parsing path.

10. The method of claim 9, further comprising:

generating a token buffer to keep track of tokens belonging to the first parsing path and tokens belonging to the second parsing path.

11. A computer-readable storage medium storing computer-executable instructions which when executed cause a computing environment to:

obtain tokenized input representing a declaration;

in response to determining that the tokenized input represents a declaration, wherein the declaration is interrupted by a preprocessor conditional directive of a preprocessor conditional directive statement, wherein the preprocessor conditional directive statement comprises a plurality of mutually exclusive branches comprising at least a first branch and a second branch,

label each token of the interrupted declaration with at least a first parsing path indicator of a plurality of parsing path indicators, the at least first parsing path indicator corresponding to a first parsing path, and wherein a second parsing path indicator corresponds to a second parsing path, wherein each of the plurality of parsing paths is induced by the plurality of mutually exclusive branches of the preprocessor conditional directive statement;

store the labeled tokens in a token buffer; and

return the tokens labeled with the at least first parsing path indicator to a caller in a first pass of the token buffer and return the tokens labeled with the second parsing path indicator to the caller in a second pass of the token buffer.

12. The computer-readable storage medium of claim 11, storing further computer-executable instructions, which when executed cause the computing environment to:

store a location of a beginning of the interrupted declaration.

13. The computer-readable storage medium of claim 12, storing further computer-executable instructions, which when executed cause the computing environment to:

detecting an end of declaration indicator of the first parsing path, and in response to detecting the end of declaration indicator of the first parsing path, serialize the declaration again by starting at the location of the beginning of the interrupted declaration and processing the tokens belonging to the second parsing path.

14. The computer-readable storage medium of claim 13, storing further computer-executable instructions, which when executed cause the computing environment to:

determine that the first parsing path and the second parsing path are parsing paths induced by mutually exclusive branches of the preprocessor conditional directive statement by matching preprocessor conditional directives of the preprocessor conditional directive statement.

15. The computer-readable storage medium of claim 11, storing further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a compiler.

16. The computer-readable storage medium of claim 11, storing further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a program development tool.

17. The computer-readable storage medium of claim 16, storing further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a preprocessor/lexer.

18. The computer-readable storage medium of claim 11, storing further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a lexer.

19. The computer-readable storage medium of claim 11, storing further computer-executable instructions, which when executed cause the computing environment to:

return the tokens to a preprocessor.