Code example for Splitter

Methods: split

0
   * Tokenizes a string using the simplest method.  This should be over-ridden for more subtle 
   * tokenization. 
   * @see LuceneTextValueEncoder 
   */ 
  protected Iterable<String> tokenize(CharSequence originalForm) {
    return ON_NON_WORD.split(originalForm);
  } 
 
  /** 
   * Converts a value into a form that would help a human understand the internals of how the value 
   * is being interpreted.  For text-like things, this is likely to be a list of the terms found with 
   * associated weights (if any). 
   * 
   * @param originalForm The original form of the value as a string. 
   * @return A string that a human can read. 
   */ 
  @Override 
  public String asString(String originalForm) {
    StringBuilder r = new StringBuilder();
    r.append('[');