Package opennlp.tools.tokenize

# Class TokenizerStream

java.lang.Object
opennlp.tools.tokenize.TokenizerStream

All Implemented Interfaces:
  `AutoCloseable`, `opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>`

public class TokenizerStream extends Object implements opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>
The `TokenizerStream` uses a `Tokenizer` to tokenize the input string and output `samples`.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `TokenizerStream(opennlp.tools.tokenize.Tokenizer tokenizer, opennlp.tools.util.ObjectStream<String> input)`
  Initializes a `instance`.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `close()`
  `opennlp.tools.tokenize.TokenSample`
  `read()`
  `void`
  `reset()`

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### TokenizerStream

    public TokenizerStream(opennlp.tools.tokenize.Tokenizer tokenizer, opennlp.tools.util.ObjectStream<String> input)
    Initializes a `instance`.
    Parameters:
      `tokenizer` - A working `Tokenizer` instance.
      `input` - A plain text `line stream`.

- ## Method Details

  - ### read

    public opennlp.tools.tokenize.TokenSample read() throws IOException
    Specified by:
      `read` in interface `opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>`
    Throws:
      `IOException`

  - ### close

    public void close() throws IOException
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>`
    Throws:
      `IOException`

  - ### reset

    public void reset() throws IOException, UnsupportedOperationException
    Specified by:
      `reset` in interface `opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>`
    Throws:
      `IOException`
      `UnsupportedOperationException`