Latest Tutorials| Questions and Answers|Ask Questions?|Site Map



Home Java Beginners Making Tokens of a String

 
 

Share on Google+Share on Google+

Making Tokens of a String

Advertisement
Tokens can be used where we want to break an application into tokens.

Making Tokens of a String

     

Tokens can be used where we want to break an application into tokens. We have to break a String  into tokens as well as we will know how many tokens has been generated. That's what we are trying to do in this program. In the program a string is passed into a constructor of StringTokenizer class. StringTokenizer is a class in java.util.package. We are using while loop to generate tokens. The following methods have been used in this program.

countTokens(): It gives the number of tokens remaining in the string.

hasMoreTokens(): It gives true if more tokens are available, else false.

nextToken(): It gives the next token available in the string.

To break a string into tokens what we need first is to create a class named StringTokenizing. Inside this class we will declare our main method. Make an object of class StringTokenizer and pass one string inside the constructor which you want to break into tokens. By using the instance of StringTokenizer call the method countTokens() which gives the number of tokens remaining in the string. It is a method of  StringTokenizer class. If the object have more tokens available then it will call method hasMoreTokens() and print the tokens by using nextToken().

The code of the program is given below:

import java.util.*;

public class StringTokenizing{
  public static void main(String[] args) {
  StringTokenizer stringTokenizer = new 
   StringTokenizer
("You are tokenizing a string");
  System.out.println("The total no. of tokens 
   generated :  " 
+ stringTokenizer.countTokens());
  while(stringTokenizer.hasMoreTokens()){
  System.out.println(stringTokenizer.nextToken());
  }
  }
}

Output of this program is given below:

C:\java>java StringTokenizing
The total no. of tokens generated : 5
You
are
tokenizing
a
string

Download this program:

Advertisement

If you enjoyed this post then why not add us on Google+? Add us to your Circles



Liked it!  Share this Tutorial


Follow us on Twitter, or add us on Facebook or Google Plus to keep you updated with the recent trends of Java and other open source platforms.

Posted on: June 7, 2007

Related Tutorials

Discuss: Making Tokens of a String   View All Comments

Post your Comment


Your Name (*) :
Your Email :
Subject (*):
Your Comment (*):
  Reload Image
 
 
Comments:3
Heidi
September 22, 2011
gRfHgCmAsAhO

I don't know who you wrote this for but you hepeld a brother out.
pooja
July 8, 2012
finding frequency of tokens

after tokanizing,how to find the frequency of the tokens that have appeared and then sort the tokens according to its frequency.......
ARJUN
November 26, 2012
Splitting a string

how can we split 12+12 into three tokens like 12,+ and 12..please reply me if any alternate method available..
DMCA.com