Portability | All |
---|---|
Stability | experimental |
Maintainer | Malcolm Wallace <Malcolm.Wallace@cs.york.ac.uk> |
Language.Preprocessor.Cpphs.Tokenise
Description
The purpose of this module is to lex a source file (language unspecified) into tokens such that cpp can recognise a replaceable symbol or macro-use, and do the right thing.
Documentation
linesCpp :: String -> [String]
linesCpp is, broadly speaking, Prelude.lines, except that on a line beginning with a #, line continuation characters are recognised. In a line continuation, the newline character is preserved, but the backslash is not.
tokenise :: Bool -> Bool -> Bool -> Bool -> [(Posn, String)] -> [WordStyle]
tokenise is, broadly-speaking, Prelude.words, except that: * the input is already divided into lines * each word-like token is categorised as one of {Ident,Other,Cmd} * #define's are parsed and returned out-of-band using the Cmd variant * All whitespace is preserved intact as tokens. * C-comments are converted to white-space (depending on first param) * Parens and commas are tokens in their own right. * Any cpp line continuations are respected. No errors can be raised. The inverse of tokenise is (concatMap deWordStyle).
data WordStyle
Each token is classified as one of Ident, Other, or Cmd: * Ident is a word that could potentially match a macro name. * Cmd is a complete cpp directive (#define etc). * Other is anything else.
deWordStyle :: WordStyle -> String