Latest Tutorials| Questions and Answers|Ask Questions?|Site Map



Home Answers Viewqa Java-Beginners How to read a large text file line by line in java?
Login         

View Questions and Answers by Category

Advertisements


 
Have Programming Question? Ask it here!
 
 
 


Java Coder
How to read a large text file line by line in java?
8 Answer(s)      a year and 7 months ago
Posted in : Java Beginners


I have been assigned a work to read big text file and extract the data and save into database. The file size is in many GBs. We have studied the memory requirement of the application and the memory available for the Java program. For Java program max memory assigned is also a limit.

So, we have decided to read the text file line by line and the extract the data line by line. Reading the file line by line will also increase the performance of the application.

Earlier our company was using the text file for saving the data through an application developed in C++. Now we have given a job for extracting the data and save into database. Give use you kind advice and let's know how to read a large text file line by line in java?

Thanks


Advertisement
View Answers

April 28, 2013 at 12:05 PM


Hi,

The only good solution to read big text file is to read the file line by line in your Java program. Reading the file line by line will make your application much faster. The application will use less memory and work even on the computer having less amount of memory.

Since reading the file line by line is the only best method for reading a big file, developers are searching for 'Java read file line by line' on google and other search engines. Many programmers are finding difficult if they starts writing the code to read the file in one go and then process one line at a time. If you try to read a big file in memory then it your application will throw java out of memory error exception. So, it's clear that the best method of reading a big file is to read the text file line by line and then process the data.

Furthermore reading file line by line is the fastest way to read the file. It's also very easy to write code for reading the file line by line.

Here is the code example from our website:

// Open the file
FileInputStream fstream = new FileInputStream("textfile.txt");

// Get the object of DataInputStream
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));

String strLine;

//Read File Line By Line
while ((strLine = br.readLine()) != null)   {
  // Print the content on the console
  System.out.println (strLine);
}

//Close the input stream
in.close();

The above code opens the files and then uses the BufferedReader class to read the file line by line and then prints the data on console. In your real program you can process the line data and then save into database. You can also use the FileReader and BufferedReader classes to read the file line by line. Here is the code example:

import java.io.*;

public class ReadFileFileReader
{
    public static void main(String[] args) 
    {
        System.out.println("Example of reading file line by line!");
        System.out.println("through FileReader and BufferedReader classes");
        try{
        BufferedReader br = new BufferedReader(new FileReader("log.txt"));
        String line;
        while ((line = br.readLine()) != null) {
           // process the line.
           System.out.println(line);
        }
        br.close();

        }catch(Exception e){
            System.out.println("Error:" + e.getMessage());
        }


    }
}

Above code example uses the FileReader and BufferedReader classes to read a text file line by line and then prints the content on console.

Thanks



April 28, 2013 at 12:10 PM




April 28, 2013 at 12:15 PM


Thanks for good explanation.

Code discussed above can the used to read the data from a text file much faster. Above code is very easy to understandable also. After reading one line of data you can use the data and process it according to your business needs. You can also save the data into relational database if you are converting your legacy data.

Thanks



April 28, 2013 at 12:20 PM


Hi, Another very important thing is the proper exception handling of the application. You should update the above code for proper exception handling and closing the resources properly.

You should properly catch all the exception of the application. For example br.close(); throws the IOException, it must be properly handled.

You must also properly handle the FileNotFound exception, this exception is thrown by the FileReader's constructor. So, proper exception handling is very important.

Thanks



April 28, 2013 at 12:22 PM


Hi,

Check this tutorial: How to read big file line by line in java?

Thanks



April 28, 2013 at 12:33 PM


Hello,

The BufferedReader class is very convenient class reading the data from the resource. This helps the programmers to develop the code for reading the data very conveniently. The BufferedReader class provides a method called readLine(), which reads one line at a time from the input stream. The readLine() method returns a String.

Read more at Java Read File Line by Line - Java Tutorial.

Thanks



April 28, 2013 at 12:39 PM


Hi,

There is another way to read the file line by line using the Scanner class of Java API. Read more at Line by Line reading from a file using Scanner Class.

Thanks



August 30, 2013 at 9:50 PM


Hi,

All the above examples are good. You can view more tutorials at Java file examples and tutorials on Rose India website in Files Handling section.

Java API also provides classes and interfaces for reading and writing to the .properties files. Check the tutorial How to read properties file in Java.

Thanks



Related Tutorials/Questions & Answers:
Advertisements
Advertisements
 

 

 

DMCA.com