当前页面: 在线文档首页 > NetBeans API Javadoc (Current Development Version)

TokenFactory (Lexer) - NetBeans API Javadoc (Current Development Version)

Overview  Package   Class  Use  Tree  Deprecated  Index  Help		org.netbeans.modules.lexer/2 1.19.0 1
PREV CLASS   NEXT CLASS	FRAMES    NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD	DETAIL: FIELD | CONSTR | METHOD

---

org.netbeans.spi.lexer
Class TokenFactory<T extends TokenId>

java.lang.Object
  org.netbeans.spi.lexer.TokenFactory<T>

---

public final class TokenFactory<T extends TokenId>
extends Object

Lexer should delegate all the token instances creation to this class.
It's not allowed to create empty tokens.

---

Field Summary
static Token	SKIP_TOKEN           Token instance that should be returned by the lexer if there is an active filtering of certain token ids and the just recognized token-id should be skipped.

 

Method Summary
Token<T>	createCustomTextToken(T id, CharSequence text, int length, PartType partType)           Create token with a custom text that possibly differs from the text represented by the token in the input text.
Token<T>	createPropertyToken(T id, int length, TokenPropertyProvider propertyProvider, PartType partType)           Create token with properties.
Token<T>	createToken(T id)           Create token with token length corresponding to the number of characters read from the lexer input.
Token<T>	createToken(T id, int length)           Create regular token instance with an explicit length.
Token<T>	createToken(T id, int length, PartType partType)           Create regular token instance with an explicit length and part type.
Token<T>	getFlyweightToken(T id, String text)           Get flyweight token for the given arguments.

 

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

 

Field Detail

SKIP_TOKEN

public static final Token SKIP_TOKEN

Token instance that should be returned by the lexer if there is an active filtering of certain token ids and the just recognized token-id should be skipped.

Method Detail

createToken

public Token<T> createToken(T id)

Create token with token length corresponding to the number of characters read from the lexer input.

See Also:

createToken(TokenId, int)

---

createToken

public Token<T> createToken(T id,
                            int length)

Create regular token instance with an explicit length.

Parameters:

id - non-null token id recognized by the lexer.

length - >=0 length of the token to be created. The length must not exceed the number of characters read from the lexer input.

Returns:

non-null regular token instance.
If there were any characters preprocessed by CharPreprocessor then a special token instance will be created for it.
SKIP_TOKEN will be returned if tokens for the given token id should be skipped because of token id filter.

---

createToken

public Token<T> createToken(T id,
                            int length,
                            PartType partType)

Create regular token instance with an explicit length and part type.

Parameters:

id - non-null token id recognized by the lexer.

length - >=0 length of the token to be created. The length must not exceed the number of characters read from the lexer input.

partType - whether this token is complete token or a part of a complete token.

Returns:

non-null regular token instance.
If there were any characters preprocessed by CharPreprocessor then a special token instance will be created for it.
SKIP_TOKEN will be returned if tokens for the given token id should be skipped because of token id filter.

---

getFlyweightToken

public Token<T> getFlyweightToken(T id,
                                  String text)

Get flyweight token for the given arguments.
Note: The returned token will not be flyweight under certain conditions - see return value description.

Parameters:

id - non-null token id.

text - non-null text that the flyweight token should carry.

Returns:

non-null flyweight token instance.
If there were any characters preprocessed by CharPreprocessor then a special non-flyweight token instance will be created for it.
For performance reasons there is a limit for number of successive flyweight tokens. If this limit would be exceeded a single non-flyweight token gets created instead of flyweight one.
SKIP_TOKEN will be returned if tokens for the given token id should be skipped because of token id filter.

---

createPropertyToken

public Token<T> createPropertyToken(T id,
                                    int length,
                                    TokenPropertyProvider propertyProvider,
                                    PartType partType)

Create token with properties.

Parameters:

id - non-null token id.

length - >=0 length of the token to be created. The length must not exceed the number of characters read from the lexer input.

propertyProvider - non-null token property provider.

partType - whether this token is complete or just a part of complete token. See TokenPropertyProvider for examples how this parameter may be used.

Returns:

non-null property token instance.
If there were any characters preprocessed by CharPreprocessor then a special token instance will be created for it.
SKIP_TOKEN will be returned if tokens for the given token id should be skipped because of token id filter.

---

createCustomTextToken

public Token<T> createCustomTextToken(T id,
                                      CharSequence text,
                                      int length,
                                      PartType partType)

Create token with a custom text that possibly differs from the text represented by the token in the input text.

---

Overview  Package   Class  Use  Tree  Deprecated  Index  Help		org.netbeans.modules.lexer/2 1.19.0 1
PREV CLASS   NEXT CLASS	FRAMES    NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD	DETAIL: FIELD | CONSTR | METHOD

---

Built on May 28 2007.  |  Portions Copyright 1997-2007 Sun Microsystems, Inc. All rights reserved.