Package opennlp.spellcheck.normalizer

# Class SpellCheckingCharSequenceNormalizer

java.lang.Object
opennlp.spellcheck.normalizer.SpellCheckingCharSequenceNormalizer

All Implemented Interfaces:
  `Serializable`, `opennlp.tools.util.normalizer.CharSequenceNormalizer`

public class SpellCheckingCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.CharSequenceNormalizer
A `CharSequenceNormalizer` that corrects spelling in text using a `SpellChecker` (typically a SymSpell engine).

The normalizer works in one of two modes:

- `PER_TOKEN` (default) – the input is split into whitespace-delimited tokens and each token is corrected independently with `SpellChecker.lookup(java.lang.String, opennlp.spellcheck.Verbosity, int)`. The original whitespace runs between tokens are preserved verbatim, so the shape of the line is kept. Tokens the dictionary already contains (best suggestion at edit distance `0`) are left untouched.
- `COMPOUND` – the whole input is passed to `SpellChecker.lookupCompound(java.lang.String, int)`, which additionally repairs wrongly inserted or omitted spaces (word splits and merges). This collapses runs of whitespace to single spaces, as the compound corrector re-tokenizes the input.

Several guards keep the corrector from "fixing" tokens that should be left as they are (configurable through the `SpellCheckingCharSequenceNormalizer.Builder`):

- tokens shorter than `minTokenLength` are skipped;
- numeric tokens are skipped (`skipNumbers`, on by default);
- URL- and email-like tokens are skipped (`skipUrls`, on by default);
- a token whose lower-cased form is already in the dictionary is never changed (the engine returns it at edit distance `0`).

Casing. Dictionaries are normally lower-cased, so lookups are performed on the lower-cased token, and the original casing pattern is re-applied to the correction: an all-upper token yields an all-upper correction, a leading-capital token yields a leading-capital correction, otherwise the suggestion's own casing is used. When no correction applies, the original token (including its casing and any surrounding punctuation) is emitted unchanged.

This normalizer composes cleanly inside an `AggregateCharSequenceNormalizer`; place it after noise-removing normalizers (URL, emoji, shrink) so it sees clean tokens.

Serialization. `CharSequenceNormalizer` is `Serializable`, but the backing `SpellChecker` usually is not; it is therefore held in a `transient` field and is `null` after Java deserialization. A deserialized instance is inert until a checker is re-attached: obtain a working copy with the same settings via `withSpellChecker(SpellChecker)` (this matches how the engine is rebuilt from a model rather than Java-serialized). Calling `normalize(java.lang.CharSequence)` on an instance with no checker throws `IllegalStateException`.

See Also:
  - Serialized Form

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static final class`
  `SpellCheckingCharSequenceNormalizer.Builder`
  A mutable builder for `SpellCheckingCharSequenceNormalizer`.
  `static enum`
  `SpellCheckingCharSequenceNormalizer.Mode`
  The correction mode.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final int`
  `DEFAULT_MIN_TOKEN_LENGTH`
  The default minimum token length below which tokens are left untouched.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `SpellCheckingCharSequenceNormalizer(SymSpellModel model)`
  Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a loaded `SymSpellModel` (uses the model's `engine`).
  `SpellCheckingCharSequenceNormalizer(SpellChecker spellChecker)`
  Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a `SpellChecker`.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static SpellCheckingCharSequenceNormalizer.Builder`
  `builder(SymSpellModel model)`
  `static SpellCheckingCharSequenceNormalizer.Builder`
  `builder(SpellChecker spellChecker)`
  `CharSequence`
  `normalize(CharSequence text)`
  `SpellCheckingCharSequenceNormalizer`
  `withSpellChecker(SpellChecker checker)`
  Returns a copy of this normalizer carrying the same settings but backed by the given checker.

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### DEFAULT_MIN_TOKEN_LENGTH

    public static final int DEFAULT_MIN_TOKEN_LENGTH
    The default minimum token length below which tokens are left untouched.
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### SpellCheckingCharSequenceNormalizer

    public SpellCheckingCharSequenceNormalizer(SpellChecker spellChecker)
    Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a `SpellChecker`.
    Parameters:
      `spellChecker` - the engine used to correct tokens; must not be `null`

  - ### SpellCheckingCharSequenceNormalizer

    public SpellCheckingCharSequenceNormalizer(SymSpellModel model)
    Creates a normalizer in `SpellCheckingCharSequenceNormalizer.Mode.PER_TOKEN` mode with default guards from a loaded `SymSpellModel` (uses the model's `engine`).
    Parameters:
      `model` - the loaded model whose engine is used; must not be `null`

- ## Method Details

  - ### builder

    public static SpellCheckingCharSequenceNormalizer.Builder builder(SpellChecker spellChecker)
    Parameters:
      `spellChecker` - the engine to wrap; must not be `null`
    Returns:
      a new `SpellCheckingCharSequenceNormalizer.Builder` seeded with sensible defaults

  - ### builder

    public static SpellCheckingCharSequenceNormalizer.Builder builder(SymSpellModel model)
    Parameters:
      `model` - the loaded model whose engine to wrap; must not be `null`
    Returns:
      a new `SpellCheckingCharSequenceNormalizer.Builder` seeded with sensible defaults

  - ### withSpellChecker

    public SpellCheckingCharSequenceNormalizer withSpellChecker(SpellChecker checker)
    Returns a copy of this normalizer carrying the same settings but backed by the given checker. This is the supported way to re-attach an engine to an instance restored by Java deserialization (whose `transient` checker is `null`).
    Parameters:
      `checker` - the engine to attach; must not be `null`
    Returns:
      a new, ready-to-use normalizer with this instance's settings

  - ### normalize

    public CharSequence normalize(CharSequence text)
    Specified by:
      `normalize` in interface `opennlp.tools.util.normalizer.CharSequenceNormalizer`