Class ParserElement

Overrides the default whitespace chars

Example:

   # default whitespace chars are space, <TAB> and newline
   OneOrMore(Word(alphas)).parseString("abc def\nghi jkl")  # -> ['abc', 'def', 'ghi', 'jkl']

   # change to just treat newline as significant
   ParserElement.setDefaultWhitespaceChars(" \t")
   OneOrMore(Word(alphas)).parseString("abc def\nghi jkl")  # -> ['abc', 'def']

Set class to be used for inclusion of string literals into a parser.

Example:

   # default literal class used is Literal
   integer = Word(nums)
   date_str = integer("year") + '/' + integer("month") + '/' + integer("day")

   date_str.parseString("1999/12/31")  # -> ['1999', '/', '12', '/', '31']


   # change to Suppress
   ParserElement.inlineLiteralsUsing(Suppress)
   date_str = integer("year") + '/' + integer("month") + '/' + integer("day")

   date_str.parseString("1999/12/31")  # -> ['1999', '12', '31']

x.__init__(...) initializes x; see help(type(x)) for signature

Overrides: object.__init__

(inherited documentation)

Make a copy of this :class:`ParserElement`. Useful for defining different parse actions for the same parsing pattern, using copies of the original parse element.

Example:

   integer = Word(nums).setParseAction(lambda toks: int(toks[0]))
   integerK = integer.copy().addParseAction(lambda toks: toks[0] * 1024) + Suppress("K")
   integerM = integer.copy().addParseAction(lambda toks: toks[0] * 1024 * 1024) + Suppress("M")

   print(OneOrMore(integerK | integerM | integer).parseString("5K 100 640K 256M"))

prints:

   [5120, 100, 655360, 268435456]

Equivalent form of ``expr.copy()`` is just ``expr()``:

   integerM = integer().addParseAction(lambda toks: toks[0] * 1024 * 1024) + Suppress("M")

Define name for this expression, makes debugging and exception messages clearer.

Example:

   Word(nums).parseString("ABC")  # -> Exception: Expected W:(0123...) (at char 0), (line:1, col:1)
   Word(nums).setName("integer").parseString("ABC")  # -> Exception: Expected integer (at char 0), (line:1, col:1)

Define name for referencing matching tokens as a nested attribute
of the returned parse results.
NOTE: this returns a *copy* of the original :class:`ParserElement` object;
this is so that the client can define a basic element, such as an
integer, and reference it in multiple places with different names.

You can also set results names using the abbreviated syntax,
``expr("name")`` in place of ``expr.setResultsName("name")``
- see :class:`__call__`.

Example::

    date_str = (integer.setResultsName("year") + '/'
                + integer.setResultsName("month") + '/'
                + integer.setResultsName("day"))

    # equivalent form:
    date_str = integer("year") + '/' + integer("month") + '/' + integer("day")

Method to invoke the Python pdb debugger when this element is about to be parsed. Set ``breakFlag`` to True to enable, False to disable.

Define one or more actions to perform when successfully matching parse element definition.
Parse action fn is a callable method with 0-3 arguments, called as ``fn(s, loc, toks)`` ,
``fn(loc, toks)`` , ``fn(toks)`` , or just ``fn()`` , where:

- s   = the original string being parsed (see note below)
- loc = the location of the matching substring
- toks = a list of the matched tokens, packaged as a :class:`ParseResults` object

If the functions in fns modify the tokens, they can return them as the return
value from fn, and the modified list of tokens will replace the original.
Otherwise, fn does not need to return any value.

If None is passed as the parse action, all previously added parse actions for this
expression are cleared.

Optional keyword arguments:
- callDuringTry = (default= ``False``) indicate if parse action should be run during lookaheads and alternate testing

Note: the default parsing behavior is to expand tabs in the input string
before starting the parsing process.  See :class:`parseString for more
information on parsing strings containing ``<TAB>`` s, and suggested
methods to maintain a consistent view of the parsed string, the parse
location, and line and column positions within the parsed string.

Example::

    integer = Word(nums)
    date_str = integer + '/' + integer + '/' + integer

    date_str.parseString("1999/12/31")  # -> ['1999', '/', '12', '/', '31']

    # use parse action to convert to ints at parse time
    integer = Word(nums).setParseAction(lambda toks: int(toks[0]))
    date_str = integer + '/' + integer + '/' + integer

    # note that integer fields are now ints, not strings
    date_str.parseString("1999/12/31")  # -> [1999, '/', 12, '/', 31]

Add one or more parse actions to expression's list of parse actions. See :class:`setParseAction`.

See examples in :class:`copy`.

Add a boolean predicate function to expression's list of parse actions. See
:class:`setParseAction` for function call signatures. Unlike ``setParseAction``,
functions passed to ``addCondition`` need to return boolean success/fail of the condition.

Optional keyword arguments:
- message = define a custom message to be used in the raised exception
- fatal   = if True, will raise ParseFatalException to stop parsing immediately; otherwise will raise ParseException

Example::

    integer = Word(nums).setParseAction(lambda toks: int(toks[0]))
    year_int = integer.copy()
    year_int.addCondition(lambda toks: toks[0] >= 2000, message="Only support years 2000 and later")
    date_str = year_int + '/' + integer + '/' + integer

    result = date_str.parseString("1999/12/31")  # -> Exception: Only support years 2000 and later (at char 0), (line:1, col:1)

Define action to perform if parsing fails at this expression.
Fail acton fn is a callable function that takes the arguments
``fn(s, loc, expr, err)`` where:
- s = string being parsed
- loc = location where expression match was attempted and failed
- expr = the parse expression that failed
- err = the exception thrown
The function returns no value.  It may throw :class:`ParseFatalException`
if it is desired to stop parsing immediately.

Enables "packrat" parsing, which adds memoizing to the parsing logic.
Repeated parse attempts at the same string location (which happens
often in many complex grammars) can immediately return a cached value,
instead of re-executing parsing/validating code.  Memoizing is done of
both valid results and parsing exceptions.

Parameters:

- cache_size_limit - (default= ``128``) - if an integer value is provided
  will limit the size of the packrat cache; if None is passed, then
  the cache size will be unbounded; if 0 is passed, the cache will
  be effectively disabled.

This speedup may break existing programs that use parse actions that
have side-effects.  For this reason, packrat parsing is disabled when
you first import pyparsing.  To activate the packrat feature, your
program must call the class method :class:`ParserElement.enablePackrat`.
For best results, call ``enablePackrat()`` immediately after
importing pyparsing.

Example::

    import pyparsing
    pyparsing.ParserElement.enablePackrat()

Execute the parse expression with the given string.
This is the main interface to the client code, once the complete
expression has been built.

Returns the parsed data as a :class:`ParseResults` object, which may be
accessed as a list, or as a dict or object with attributes if the given parser
includes results names.

If you want the grammar to require that the entire input string be
successfully parsed, then set ``parseAll`` to True (equivalent to ending
the grammar with ``StringEnd()``).

Note: ``parseString`` implicitly calls ``expandtabs()`` on the input string,
in order to report proper column numbers in parse actions.
If the input string contains tabs and
the grammar uses parse actions that use the ``loc`` argument to index into the
string being parsed, you can ensure you have a consistent view of the input
string by:

- calling ``parseWithTabs`` on your grammar before calling ``parseString``
  (see :class:`parseWithTabs`)
- define your parse action using the full ``(s, loc, toks)`` signature, and
  reference the input string using the parse action's ``s`` argument
- explictly expand the tabs in your input string before calling
  ``parseString``

Example::

    Word('a').parseString('aaaaabaaa')  # -> ['aaaaa']
    Word('a').parseString('aaaaabaaa', parseAll=True)  # -> Exception: Expected end of text

Scan the input string for expression matches. Each match will return the matching tokens, start location, and end location. May be called with optional ``maxMatches`` argument, to clip scanning after 'n' matches are found. If ``overlap`` is specified, then overlapping matches will be reported.

Note that the start and end locations are reported relative to the string being parsed. See :class:`parseString` for more information on parsing strings with embedded tabs.

Example:

   source = "sldjf123lsdjjkf345sldkjf879lkjsfd987"
   print(source)
   for tokens, start, end in Word(alphas).scanString(source):
       print(' '*start + '^'*(end-start))
       print(' '*start + tokens[0])

prints:

   sldjf123lsdjjkf345sldkjf879lkjsfd987
   ^^^^^
   sldjf
           ^^^^^^^
           lsdjjkf
                     ^^^^^^
                     sldkjf
                              ^^^^^^
                              lkjsfd

Extension to :class:`scanString`, to modify matching text with modified tokens that may be returned from a parse action. To use ``transformString``, define a grammar and attach a parse action to it that modifies the returned token list. Invoking ``transformString()`` on a target string will then scan for matches, and replace the matched text patterns according to the logic in the parse action. ``transformString()`` returns the resulting transformed string.

Example:

   wd = Word(alphas)
   wd.setParseAction(lambda toks: toks[0].title())

   print(wd.transformString("now is the winter of our discontent made glorious summer by this sun of york."))

prints:

   Now Is The Winter Of Our Discontent Made Glorious Summer By This Sun Of York.

Another extension to :class:`scanString`, simplifying the access to the tokens found to match the given parse expression. May be called with optional ``maxMatches`` argument, to clip searching after 'n' matches are found.

Example:

   # a capitalized word starts with an uppercase letter, followed by zero or more lowercase letters
   cap_word = Word(alphas.upper(), alphas.lower())

   print(cap_word.searchString("More than Iron, more than Lead, more than Gold I need Electricity"))

   # the sum() builtin can be used to merge results into a single ParseResults object
   print(sum(cap_word.searchString("More than Iron, more than Lead, more than Gold I need Electricity")))

prints:

   [['More'], ['Iron'], ['Lead'], ['Gold'], ['I'], ['Electricity']]
   ['More', 'Iron', 'Lead', 'Gold', 'I', 'Electricity']

Generator method to split a string using the given expression as a separator. May be called with optional ``maxsplit`` argument, to limit the number of splits; and the optional ``includeSeparators`` argument (default= ``False``), if the separating matching text should be included in the split results.

Example:

   punc = oneOf(list(".,;:/-!?"))
   print(list(punc.split("This, this?, this sentence, is badly punctuated!")))

prints:

   ['This', ' this', '', ' this sentence', ' is badly punctuated', '']

Implementation of + operator - returns :class:`And`. Adding strings to a ParserElement
converts them to :class:`Literal`s by default.

Example::

    greet = Word(alphas) + "," + Word(alphas) + "!"
    hello = "Hello, World!"
    print (hello, "->", greet.parseString(hello))

prints::

    Hello, World! -> ['Hello', ',', 'World', '!']

``...`` may be used as a parse expression as a short form of :class:`SkipTo`.

    Literal('start') + ... + Literal('end')

is equivalent to:

    Literal('start') + SkipTo('end')("_skipped*") + Literal('end')

Note that the skipped text is returned with '_skipped' as a results name,
and to support having multiple skips in the same parser, the value returned is
a list of all skipped text.

Implementation of * operator, allows use of ``expr * 3`` in place of ``expr + expr + expr``. Expressions may also me multiplied by a 2-integer tuple, similar to ``{min, max}`` multipliers in regular expressions. Tuples may also include ``None`` as in:

• ``expr*(n, None)`` or ``expr*(n, )`` is equivalent to ``expr*n + ZeroOrMore(expr)`` (read as "at least n instances of ``expr``")
• ``expr*(None, n)`` is equivalent to ``expr*(0, n)`` (read as "0 to n instances of ``expr``")
• ``expr*(None, None)`` is equivalent to ``ZeroOrMore(expr)``
• ``expr*(1, None)`` is equivalent to ``OneOrMore(expr)``

Note that ``expr*(None, n)`` does not raise an exception if more than n exprs exist in the input stream; that is, ``expr*(None, n)`` does not enforce a maximum number of expr occurrences. If this behavior is desired, then write ``expr*(None, n) + ~expr``

use ``[]`` indexing notation as a short form for expression repetition:

• ``expr[n]`` is equivalent to ``expr*n``
• ``expr[m, n]`` is equivalent to ``expr*(m, n)``
• ``expr[n, ...]`` or ``expr[n,]`` is equivalent to ``expr*n + ZeroOrMore(expr)`` (read as "at least n instances of ``expr``")
• ``expr[..., n]`` is equivalent to ``expr*(0, n)`` (read as "0 to n instances of ``expr``")
• ``expr[...]`` and ``expr[0, ...]`` are equivalent to ``ZeroOrMore(expr)``
• ``expr[1, ...]`` is equivalent to ``OneOrMore(expr)`` ``None`` may be used in place of ``...``.

Note that ``expr[..., n]`` and ``expr[m, n]``do not raise an exception if more than ``n`` ``expr``s exist in the input stream. If this behavior is desired, then write ``expr[..., n] + ~expr``.

Shortcut for :class:`setResultsName`, with ``listAllMatches=False``.

If ``name`` is given with a trailing ``'*'`` character, then ``listAllMatches`` will be passed as ``True``.

If ``name` is omitted, same as calling :class:`copy`.

Example:

   # these are equivalent
   userdata = Word(alphas).setResultsName("name") + Word(nums + "-").setResultsName("socsecno")
   userdata = Word(alphas)("name") + Word(nums + "-")("socsecno")

Disables the skipping of whitespace before matching the characters in the :class:`ParserElement`'s defined pattern. This is normally only used internally by the pyparsing module, but may be needed in some whitespace-sensitive grammars.

Overrides default behavior to expand ``<TAB>``s to spaces before parsing the input string. Must be called before ``parseString`` when the input grammar contains elements that match ``<TAB>`` characters.

Define expression to be ignored (e.g., comments) while doing pattern matching; may be called repeatedly, to define multiple comment or other ignorable patterns.

Example:

   patt = OneOrMore(Word(alphas))
   patt.parseString('ablaj /* comment */ lskjd') # -> ['ablaj']

   patt.ignore(cStyleComment)
   patt.parseString('ablaj /* comment */ lskjd') # -> ['ablaj', 'lskjd']

Enable display of debugging messages while doing pattern matching. Set ``flag`` to True to enable, False to disable.

Example:

   wd = Word(alphas).setName("alphaword")
   integer = Word(nums).setName("numword")
   term = wd | integer

   # turn on debugging for wd
   wd.setDebug()

   OneOrMore(term).parseString("abc 123 xyz 890")

prints:

   Match alphaword at loc 0(1,1)
   Matched alphaword -> ['abc']
   Match alphaword at loc 3(1,4)
   Exception raised:Expected alphaword (at char 4), (line:1, col:5)
   Match alphaword at loc 7(1,8)
   Matched alphaword -> ['xyz']
   Match alphaword at loc 11(1,12)
   Exception raised:Expected alphaword (at char 12), (line:1, col:13)
   Match alphaword at loc 15(1,16)
   Exception raised:Expected alphaword (at char 15), (line:1, col:16)

The output shown is that produced by the default debug actions - custom debug actions can be specified using :class:`setDebugActions`. Prior to attempting to match the ``wd`` expression, the debugging message ``"Match <exprname> at loc <n>(<line>,<col>)"`` is shown. Then if the parse succeeds, a ``"Matched"`` message is shown, or an ``"Exception raised"`` message is shown. Also note the use of :class:`setName` to assign a human-readable name to the expression, which makes debugging and exception messages easier to understand - for instance, the default name created for the :class:`Word` expression without calling ``setName`` is ``"W:(ABCD...)"``.

str(x)

Overrides: object.__str__

(inherited documentation)

repr(x)

Overrides: object.__repr__

(inherited documentation)

Execute the parse expression on the given file or filename. If a filename is specified (instead of a file object), the entire file is opened, read, and closed before parsing.

hash(x)

Overrides: object.__hash__

(inherited documentation)

Method for quick testing of a parser against a test string. Good for simple inline microtests of sub expressions while building up larger parser.

Parameters:

• testString - to test against this expression for a match
• parseAll - (default= ``True``) - flag to pass to :class:`parseString` when running tests

Example:

   expr = Word(nums)
   assert expr.matches("100")

Execute the parse expression on a series of test strings, showing each test, the parsed results or where the parse failed. Quick and easy way to run a parse expression against a list of sample strings.

Parameters:

• tests - a list of separate test strings, or a multiline string of test strings
• parseAll - (default= ``True``) - flag to pass to :class:`parseString` when running tests
• comment - (default= ``'#'``) - expression for indicating embedded comments in the test string; pass None to disable comment filtering
• fullDump - (default= ``True``) - dump results as list followed by results names in nested outline; if False, only dump nested list
• printResults - (default= ``True``) prints test output to stdout
• failureTests - (default= ``False``) indicates if these tests are expected to fail parsing
• postParse - (default= ``None``) optional callback for successful parse results; called as `fn(test_string, parse_results)` and returns a string to be added to the test output
• file - (default=``None``) optional file-like object to which test output will be written; if None, will default to ``sys.stdout``

Returns: a (success, results) tuple, where success indicates that all tests succeeded (or failed if ``failureTests`` is True), and the results contain a list of lines of each test's output

Example:

   number_expr = pyparsing_common.number.copy()

   result = number_expr.runTests('''
       # unsigned integer
       100
       # negative integer
       -100
       # float with scientific notation
       6.02e23
       # integer with scientific notation
       1e-12
       ''')
   print("Success" if result[0] else "Failed!")

   result = number_expr.runTests('''
       # stray character
       100Z
       # missing leading digit before '.'
       -.100
       # too many '.'
       3.14.159
       ''', failureTests=True)
   print("Success" if result[0] else "Failed!")

prints:

   # unsigned integer
   100
   [100]

   # negative integer
   -100
   [-100]

   # float with scientific notation
   6.02e23
   [6.02e+23]

   # integer with scientific notation
   1e-12
   [1e-12]

   Success

   # stray character
   100Z
      ^
   FAIL: Expected end of text (at char 3), (line:1, col:4)

   # missing leading digit before '.'
   -.100
   ^
   FAIL: Expected {real number with scientific notation | real number | signed integer} (at char 0), (line:1, col:1)

   # too many '.'
   3.14.159
       ^
   FAIL: Expected end of text (at char 4), (line:1, col:5)

   Success

Each test string must be on a single line. If you want to test a string that spans multiple lines, create a test like this:

   expr.runTest(r"this is a test\n of strings that spans \n 3 lines")

(Note that this is a raw string literal, you must include the leading 'r'.)