Package opennlp.tools.stopword

# Interface StopwordFilter

public interface StopwordFilter
A pluggable filter that decides whether a token (or a sequence of tokens) is a stopword that should be removed during downstream text processing.

Implementations may be backed by a static bundled list, a user-supplied file, an in-memory data structure, or any other source. Both single-token and multi-token (n-gram) membership tests are supported.

See Also:
  - `LanguageCodeValidator`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `String[]`
  `filter(String[] tokens)`
  Returns a copy of `tokens` with stopword matches removed, preserving the input order.
  `boolean`
  `isCaseSensitive()`
  `boolean`
  `isStopword(CharSequence token)`
  Checks whether the given token is a single-token stopword.
  `boolean`
  `isStopword(String... tokens)`
  Checks whether the given sequence of tokens is a multi-token stopword (n-gram).
  `Set<String>`
  `stopwords()`
  Returns an unmodifiable snapshot of the registered single-token stopwords.

- ## Method Details

  - ### isStopword

    boolean isStopword(CharSequence token)
    Checks whether the given token is a single-token stopword. Equivalent to `isStopword(new String[] { token.toString() })` when `token` is non-`null`.
    Parameters:
      `token` - The token to test. May be `null`, in which case implementations should return `false`.
    Returns:
      `true` if `token` is registered as a single-token stopword, `false` otherwise.

  - ### isStopword

    boolean isStopword(String... tokens)
    Checks whether the given sequence of tokens is a multi-token stopword (n-gram). For a single token this is equivalent to `isStopword(CharSequence)`.
    Parameters:
      `tokens` - The tokens to test as one entry. May be `null` or empty, in which case implementations should return `false`.
    Returns:
      `true` if the sequence is registered as a stopword, `false` otherwise.

  - ### filter

    String[] filter(String[] tokens)
    Returns a copy of `tokens` with stopword matches removed, preserving the input order.

    Implementations should honor both 1-gram and n-gram entries. A recommended strategy is a greedy left-to-right window scan: at each position try the longest registered window first; if it matches, skip those tokens; otherwise advance by one and keep the current token. Implementations that do not support n-gram entries may fall back to 1-gram filtering.

    Parameters:
      `tokens` - The input token array. Must not be `null`. Individual array elements may be `null` and are kept as-is.
    Returns:
      A new array containing the surviving tokens. Never `null`.
    Throws:
      `IllegalArgumentException` - if `tokens` is `null`.

  - ### isCaseSensitive

    boolean isCaseSensitive()
    Returns:
      `true` if this filter performs case-sensitive matching; `false` if matching is case-insensitive.

  - ### stopwords

    Set<String> stopwords()
    Returns an unmodifiable snapshot of the registered single-token stopwords. Multi-token (n-gram) entries are not included in this view and must be tested via `isStopword(String...)`.

    Attempts to mutate the returned `Set` will fail.

    Returns:
      An unmodifiable `Set` of stopwords. Never `null`.
    Throws:
      `UnsupportedOperationException` - if a caller attempts to add to, remove from, or otherwise mutate the returned `Set`.