Uses of Class org.netbeans.api.lexer.Token (Lexer) - NetBeans API Javadoc (Current Development Version)

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

org.netbeans.modules.lexer/2 1.19.0 1

PREV NEXT

FRAMES NO FRAMES

Uses of Class
org.netbeans.api.lexer.Token

Packages that use Token
org.netbeans.api.lexer	The entrance point into Lexer API is `TokenHierarchy` class with its static methods that provide its instance for the given input source.
org.netbeans.spi.lexer	The main abstract class in the Lexer SPI that must be implemented is `LanguageHierarchy` that mainly defines set of token ids and token categories for the new language and its `Lexer`.

Uses of Token in org.netbeans.api.lexer

Methods in org.netbeans.api.lexer that return Token

Token<T> TokenSequence.offsetToken()
Similar to TokenSequence.token() but always returns a non-flyweight token with the appropriate offset.

Token<T> TokenSequence.token()
Get token to which this token sequence points to or null if TS is positioned between tokens (TokenSequence.moveNext() or TokenSequence.movePrevious() were not called yet).

Methods in org.netbeans.api.lexer that return Token
`Token<T>`	`TokenSequence.offsetToken()` Similar to `TokenSequence.token()` but always returns a non-flyweight token with the appropriate offset.
`Token<T>`	`TokenSequence.token()` Get token to which this token sequence points to or null if TS is positioned between tokens (`TokenSequence.moveNext()` or `TokenSequence.movePrevious()` were not called yet).

Uses of Token in org.netbeans.spi.lexer

Fields in org.netbeans.spi.lexer declared as Token

static Token TokenFactory.SKIP_TOKEN
Token instance that should be returned by the lexer if there is an active filtering of certain token ids and the just recognized token-id should be skipped.

Fields in org.netbeans.spi.lexer declared as Token
`static Token`	`TokenFactory.SKIP_TOKEN` Token instance that should be returned by the lexer if there is an active filtering of certain token ids and the just recognized token-id should be skipped.

Methods in org.netbeans.spi.lexer that return Token

Token<T> TokenFactory.createCustomTextToken(T id, CharSequence text, int length, PartType partType)
          Create token with a custom text that possibly differs from the text represented by the token in the input text.

Token<T> TokenFactory.createPropertyToken(T id, int length, TokenPropertyProvider propertyProvider, PartType partType)
          Create token with properties.

Token<T> TokenFactory.createToken(T id)
          Create token with token length corresponding to the number of characters read from the lexer input.

Token<T> TokenFactory.createToken(T id, int length)
          Create regular token instance with an explicit length.

Token<T> TokenFactory.createToken(T id, int length, PartType partType)
          Create regular token instance with an explicit length and part type.

Token<T> TokenFactory.getFlyweightToken(T id, String text)
          Get flyweight token for the given arguments.

Token<T> Lexer.nextToken()
          Return a token based on characters of the input and possibly additional input properties.

Token<T> TokenValidator.validateToken(Token<T> token, TokenFactory<T> factory, CharSequence tokenText, int modRelOffset, int removedLength, CharSequence removedText, int insertedLength, CharSequence insertedText)
          This method is invoked in mutable environments prior lexer invocation to check whether token in which the text modification occurred was only slightly modified by the performed modification and the lexer's invocation is not necessary.

Methods in org.netbeans.spi.lexer that return Token
`Token<T>`	`TokenFactory.createCustomTextToken(T id, CharSequence text, int length, PartType partType)` Create token with a custom text that possibly differs from the text represented by the token in the input text.
`Token<T>`	`TokenFactory.createPropertyToken(T id, int length, TokenPropertyProvider propertyProvider, PartType partType)` Create token with properties.
`Token<T>`	`TokenFactory.createToken(T id)` Create token with token length corresponding to the number of characters read from the lexer input.
`Token<T>`	`TokenFactory.createToken(T id, int length)` Create regular token instance with an explicit length.
`Token<T>`	`TokenFactory.createToken(T id, int length, PartType partType)` Create regular token instance with an explicit length and part type.
`Token<T>`	`TokenFactory.getFlyweightToken(T id, String text)` Get flyweight token for the given arguments.
`Token<T>`	`Lexer.nextToken()` Return a token based on characters of the input and possibly additional input properties.
`Token<T>`	`TokenValidator.validateToken(Token<T> token, TokenFactory<T> factory, CharSequence tokenText, int modRelOffset, int removedLength, CharSequence removedText, int insertedLength, CharSequence insertedText)` This method is invoked in mutable environments prior lexer invocation to check whether token in which the text modification occurred was only slightly modified by the performed modification and the lexer's invocation is not necessary.

Methods in org.netbeans.spi.lexer with parameters of type Token

protected LanguageEmbedding<? extends TokenId> LanguageHierarchy.embedding(Token<T> token, LanguagePath languagePath, InputAttributes inputAttributes)
          Get language embedding (if exists) for a particular token of the language at this level of language hierarchy.

abstract LanguageEmbedding<? extends TokenId> LanguageProvider.findLanguageEmbedding(Token<? extends TokenId> token, LanguagePath languagePath, InputAttributes inputAttributes)
          Finds LanguageEmbedding that will define what language is embedded in a given token.

Object TokenPropertyProvider.getValue(Token token, Object key)
          Get value of a token property.

Token<T> TokenValidator.validateToken(Token<T> token, TokenFactory<T> factory, CharSequence tokenText, int modRelOffset, int removedLength, CharSequence removedText, int insertedLength, CharSequence insertedText)
          This method is invoked in mutable environments prior lexer invocation to check whether token in which the text modification occurred was only slightly modified by the performed modification and the lexer's invocation is not necessary.

Methods in org.netbeans.spi.lexer with parameters of type Token
`protected LanguageEmbedding<? extends TokenId>`	`LanguageHierarchy.embedding(Token<T> token, LanguagePath languagePath, InputAttributes inputAttributes)` Get language embedding (if exists) for a particular token of the language at this level of language hierarchy.
`abstract LanguageEmbedding<? extends TokenId>`	`LanguageProvider.findLanguageEmbedding(Token<? extends TokenId> token, LanguagePath languagePath, InputAttributes inputAttributes)` Finds `LanguageEmbedding` that will define what language is embedded in a given token.
`Object`	`TokenPropertyProvider.getValue(Token token, Object key)` Get value of a token property.
`Token<T>`	`TokenValidator.validateToken(Token<T> token, TokenFactory<T> factory, CharSequence tokenText, int modRelOffset, int removedLength, CharSequence removedText, int insertedLength, CharSequence insertedText)` This method is invoked in mutable environments prior lexer invocation to check whether token in which the text modification occurred was only slightly modified by the performed modification and the lexer's invocation is not necessary.