Package opennlp.tools.tokenize

# Class ThreadSafeTokenizerME

java.lang.Object
opennlp.tools.tokenize.ThreadSafeTokenizerME

All Implemented Interfaces:
  `AutoCloseable`, `opennlp.tools.ml.Probabilistic`, `opennlp.tools.tokenize.Tokenizer`

@Deprecated(since="3.0.0") @ThreadSafe public class ThreadSafeTokenizerME extends Object implements opennlp.tools.tokenize.Tokenizer, opennlp.tools.ml.Probabilistic, AutoCloseable
Deprecated.
As of OPENNLP-1816, `TokenizerME` is itself thread-safe. Use it directly instead.
A thread-safe version of `TokenizerME`. Using it is completely transparent. You can use it in a single-threaded context as well, it only incurs a minimal overhead.

Note: This class is deprecated and now delegates to a single shared `TokenizerME` instance, which is thread-safe as of OPENNLP-1816. Calling `close()` clears the current thread's thread-local state for compatibility.

See Also:
  - `Probabilistic`
  - `Tokenizer`
  - `TokenizerME`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `ThreadSafeTokenizerME(String language)`
  Deprecated.
  Initializes a `ThreadSafeTokenizerME` by downloading a default model for a given `language`.
  `ThreadSafeTokenizerME(TokenizerModel model)`
  Deprecated.
  Initializes a `ThreadSafeTokenizerME` with the specified `model`.
  `ThreadSafeTokenizerME(TokenizerModel model, Dictionary abbDict)`
  Deprecated.
  Instantiates a `ThreadSafeTokenizerME` with an existing `TokenizerModel`.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `close()`
  Deprecated.
  `double[]`
  `getProbabilities()`
  Deprecated, for removal: This API element is subject to removal in a future version.
  Use `probs()` instead.
  `double[]`
  `probs()`
  Deprecated.
  `String[]`
  `tokenize(String s)`
  Deprecated.
  `opennlp.tools.util.Span[]`
  `tokenizePos(String s)`
  Deprecated.

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### ThreadSafeTokenizerME

    public ThreadSafeTokenizerME(String language) throws IOException
    Deprecated.
    Initializes a `ThreadSafeTokenizerME` by downloading a default model for a given `language`.
    Parameters:
      `language` - An ISO conform language code.
    Throws:
      `IOException` - Thrown if the model could not be downloaded or saved.

  - ### ThreadSafeTokenizerME

    public ThreadSafeTokenizerME(TokenizerModel model)
    Deprecated.
    Initializes a `ThreadSafeTokenizerME` with the specified `model`.
    Parameters:
      `model` - A valid `TokenizerModel`.

  - ### ThreadSafeTokenizerME

    public ThreadSafeTokenizerME(TokenizerModel model, Dictionary abbDict)
    Deprecated.
    Instantiates a `ThreadSafeTokenizerME` with an existing `TokenizerModel`.
    Parameters:
      `model` - The `TokenizerModel` to be used.
      `abbDict` - The `Dictionary` to be used. It must fit the language of the `model`.

- ## Method Details

  - ### tokenize

    public String[] tokenize(String s)
    Deprecated.
    Specified by:
      `tokenize` in interface `opennlp.tools.tokenize.Tokenizer`

  - ### tokenizePos

    public opennlp.tools.util.Span[] tokenizePos(String s)
    Deprecated.
    Specified by:
      `tokenizePos` in interface `opennlp.tools.tokenize.Tokenizer`

  - ### probs

    public double[] probs()
    Deprecated.
    Specified by:
      `probs` in interface `opennlp.tools.ml.Probabilistic`

  - ### getProbabilities

    @Deprecated(forRemoval=true, since="2.5.5") public double[] getProbabilities()
    Deprecated, for removal: This API element is subject to removal in a future version.
    Use `probs()` instead.

  - ### close

    public void close()
    Deprecated.
    Specified by:
      `close` in interface `AutoCloseable`