public final class ShingleTokenFilter extends TokenFilter
Constructor and Description |
---|
ShingleTokenFilter() |
Modifier and Type | Method and Description |
---|---|
String |
getFilterToken()
Get the filterToken property: The string to insert for each position at
which there is no token.
|
Integer |
getMaxShingleSize()
Get the maxShingleSize property: The maximum shingle size.
|
Integer |
getMinShingleSize()
Get the minShingleSize property: The minimum shingle size.
|
String |
getTokenSeparator()
Get the tokenSeparator property: The string to use when joining adjacent
tokens to form a shingle.
|
Boolean |
isOutputUnigrams()
Get the outputUnigrams property: A value indicating whether the output
stream will contain the input tokens (unigrams) as well as shingles.
|
Boolean |
isOutputUnigramsIfNoShingles()
Get the outputUnigramsIfNoShingles property: A value indicating whether
to output unigrams for those times when no shingles are available.
|
ShingleTokenFilter |
setFilterToken(String filterToken)
Set the filterToken property: The string to insert for each position at
which there is no token.
|
ShingleTokenFilter |
setMaxShingleSize(Integer maxShingleSize)
Set the maxShingleSize property: The maximum shingle size.
|
ShingleTokenFilter |
setMinShingleSize(Integer minShingleSize)
Set the minShingleSize property: The minimum shingle size.
|
ShingleTokenFilter |
setOutputUnigrams(Boolean outputUnigrams)
Set the outputUnigrams property: A value indicating whether the output
stream will contain the input tokens (unigrams) as well as shingles.
|
ShingleTokenFilter |
setOutputUnigramsIfNoShingles(Boolean outputUnigramsIfNoShingles)
Set the outputUnigramsIfNoShingles property: A value indicating whether
to output unigrams for those times when no shingles are available.
|
ShingleTokenFilter |
setTokenSeparator(String tokenSeparator)
Set the tokenSeparator property: The string to use when joining adjacent
tokens to form a shingle.
|
getName, setName
public Integer getMaxShingleSize()
public ShingleTokenFilter setMaxShingleSize(Integer maxShingleSize)
maxShingleSize
- the maxShingleSize value to set.public Integer getMinShingleSize()
public ShingleTokenFilter setMinShingleSize(Integer minShingleSize)
minShingleSize
- the minShingleSize value to set.public Boolean isOutputUnigrams()
public ShingleTokenFilter setOutputUnigrams(Boolean outputUnigrams)
outputUnigrams
- the outputUnigrams value to set.public Boolean isOutputUnigramsIfNoShingles()
public ShingleTokenFilter setOutputUnigramsIfNoShingles(Boolean outputUnigramsIfNoShingles)
outputUnigramsIfNoShingles
- the outputUnigramsIfNoShingles value
to set.public String getTokenSeparator()
public ShingleTokenFilter setTokenSeparator(String tokenSeparator)
tokenSeparator
- the tokenSeparator value to set.public String getFilterToken()
public ShingleTokenFilter setFilterToken(String filterToken)
filterToken
- the filterToken value to set.Copyright © 2020 Microsoft Corporation. All rights reserved.