org.apache.lucene.analysis.miscellaneous

Class LimitTokenCountAnalyzer

java.lang.Object

org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.AnalyzerWrapper

org.apache.lucene.analysis.miscellaneous.LimitTokenCountAnalyzer

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public final class LimitTokenCountAnalyzer
extends AnalyzerWrapper

This Analyzer limits the number of tokens while indexing. It is a replacement for the maximum field length setting inside IndexWriter.

Since:

3.1

See Also:

LimitTokenCountFilter

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents

Field Summary

Fields inherited from class org.apache.lucene.analysis.Analyzer

GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY

Constructor Summary

Constructors

Constructor and Description
LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount) Build an analyzer that limits the maximum number of tokens per field.
LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount, boolean consumeAllTokens) Build an analyzer that limits the maximum number of tokens per field.

Method Summary

Modifier and Type	Method and Description
protected Analyzer	getWrappedAnalyzer(java.lang.String fieldName) Retrieves the wrapped Analyzer appropriate for analyzing the field with the given name
java.lang.String	toString()
protected Analyzer.TokenStreamComponents	wrapComponents(java.lang.String fieldName, Analyzer.TokenStreamComponents components) Wraps / alters the given TokenStreamComponents, taken from the wrapped Analyzer, to form new components.

Methods inherited from class org.apache.lucene.analysis.AnalyzerWrapper

attributeFactory, createComponents, getOffsetGap, getPositionIncrementGap, initReader, initReaderForNormalization, normalize, wrapReader, wrapReaderForNormalization, wrapTokenStreamForNormalization

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getReuseStrategy, getVersion, normalize, setVersion, tokenStream, tokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LimitTokenCountAnalyzer

public LimitTokenCountAnalyzer(Analyzer delegate,
                               int maxTokenCount)

Build an analyzer that limits the maximum number of tokens per field. This analyzer will not consume any tokens beyond the maxTokenCount limit

See Also:

LimitTokenCountAnalyzer(Analyzer,int,boolean)

LimitTokenCountAnalyzer

public LimitTokenCountAnalyzer(Analyzer delegate,
                               int maxTokenCount,
                               boolean consumeAllTokens)

Build an analyzer that limits the maximum number of tokens per field.

Parameters:

delegate - the analyzer to wrap

maxTokenCount - max number of tokens to produce

consumeAllTokens - whether all tokens from the delegate should be consumed even if maxTokenCount is reached.

Method Detail

getWrappedAnalyzer

protected Analyzer getWrappedAnalyzer(java.lang.String fieldName)

Description copied from class: AnalyzerWrapper

Retrieves the wrapped Analyzer appropriate for analyzing the field with the given name

Specified by:

getWrappedAnalyzer in class AnalyzerWrapper

Parameters:

fieldName - Name of the field which is to be analyzed

Returns:

Analyzer for the field with the given name. Assumed to be non-null

wrapComponents

protected Analyzer.TokenStreamComponents wrapComponents(java.lang.String fieldName,
                                                        Analyzer.TokenStreamComponents components)

Description copied from class: AnalyzerWrapper

Wraps / alters the given TokenStreamComponents, taken from the wrapped Analyzer, to form new components. It is through this method that new TokenFilters can be added by AnalyzerWrappers. By default, the given components are returned.

Overrides:

wrapComponents in class AnalyzerWrapper

Parameters:

fieldName - Name of the field which is to be analyzed

components - TokenStreamComponents taken from the wrapped Analyzer

Returns:

Wrapped / altered TokenStreamComponents.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object