Token

data class Token<T : TokenKind>(val kind: T, val startOffset: Int, val endOffset: Int)(source)

Represents a token in a sequence of text.

A token is a meaningful unit of text, such as a keyword, identifier, or operator. It is characterized by its kind, start offset, and end offset.

The type of the token kind. Must implement TokenKind.

Constructors

constructor(kind: T, startOffset: Int, endOffset: Int)

The exclusive end offset of the token in the original text.

val kind: T

The kind of the token.

The inclusive start offset of the token in the original text.

open override fun toString(): String