Package org.ek9lang.compiler.tokenizer

Interface IToken

All Superinterfaces:
Serializable
All Known Implementing Classes:
Ek9Token
public interface IToken extends Serializable
The internal model of a parse token - typically created.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    getCharPositionInLine()
    The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1
    int
    getLine()
    The line number on which the 1st character of this token was matched, line=1..n
    String
    getSourceName()
    The name of the source (if known).
    String
    getText()
    The actual text of the token itself.
    int
    getTokenIndex()
    An index from 0..n-1 of the token object in the input stream.
    int
    getType()
    The type of token.

  • Method Details

    • getSourceName

      String getSourceName()
      The name of the source (if known).

    • getText

      String getText()
      The actual text of the token itself.

    • getType

      int getType()
      The type of token.

    • getLine

      int getLine()
      The line number on which the 1st character of this token was matched, line=1..n

    • getCharPositionInLine

      int getCharPositionInLine()
      The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1

    • getTokenIndex

      int getTokenIndex()
      An index from 0..n-1 of the token object in the input stream. This must be valid in order to print token streams and use TokenRewriteStream.
      Return -1 to indicate that this token was conjured up since it doesn't have a valid index.