STRING TOKENIZER

    The StringTokenizer class provides the first step of the parsing process of a string, often called the lexer(lexical analyzer) or scanner.

     Parsing is the division of text into a set of discrete parts, or tokens, which in a certain sequence can convey a semantic meaning.

     StringTokenizer implements the Enumeration interface.

    Given a input string, one can enumerate the individual tokens contained in it using StringTokenizer.

       To use StringTokenizer,one specifies an input string and a string that contains delimiters.

     Delimiters are character that separate tokens. Each character in the delimiters string is considered a valid delimiter- for example, “,;:” sets the delimiters to a comma,semicolon, and colon.

     The default set of delimiters consists of whitespace characters: space, tab, newline, and carriage return.

     The StringTokenizer contructors are:

o   StringTokenizer(String str)

o   StringTokenizer(String str, String delimiters)

o   StringTokenizer(String str, String delimiters, boolean delimAsToken)

          In all versions, str is the string that will be tokenized.

     In first version, the default delimiters are used.

     In the second and third version, delimiters is a string that specifies the delimiters.

      In the third version, if delimAsToken is True, then the delimiters are also returned as tokens when the string is parsed. Otherwise, the delimiters are not returned. Delimiters are not returned as tokens by the first two forms.

       The methods defined by StringTokenizer are:

Method	Description
int countTokens()	Using the current delimiters, the method determines the number of tokens left to be parsed and returns the result.
boolean hasMoreElements()	Returns true if one or more tokens remain in the string and false if there are none.
boolean hasMoreTokens()	Returns true if one or more tokens remain in the string and false if there are none.
Object nextElement()	Returns the next token as an Object.
String nextToken()	Returns the next token as an String.
String nextToken(String delimiters)	Returns the next token as an String and sets the delimiters string to that specified by delimiters.

    Ex:

import java.util.StringTokenizer;

class Token{

 public static void main(String args[]) {

  String val="Title:My Book ,Name:History ,Price:200";

  StringTokenizer s=new StringTokenizer(val,":,");

  while(s.hasMoreTokens()) {

    String st=s.nextToken();

    String v=s.nextToken();

    System.out.println(st+" : "+v);

  }

 }

}

Output:

Title : My Book

Name : History

Price : 200