How to read a large text file line by line in java?

How to read a large text file line by line in java?

I have been assigned a work to read big text file and extract the data and save into database. The file size is in many GBs. We have studied the memory requirement of the application and the memory available for the Java program. For Java program max memory assigned is also a limit.

So, we have decided to read the text file line by line and the extract the data line by line. Reading the file line by line will also increase the performance of the application.

Earlier our company was using the text file for saving the data through an application developed in C++. Now we have given a job for extracting the data and save into database. Give use you kind advice and let's know how to read a large text file line by line in java?

Thanks

View Answers

April 28, 2013 at 12:05 PM

Hi,

The only good solution to read big text file is to read the file line by line in your Java program. Reading the file line by line will make your application much faster. The application will use less memory and work even on the computer having less amount of memory.

Since reading the file line by line is the only best method for reading a big file, developers are searching for 'Java read file line by line' on google and other search engines. Many programmers are finding difficult if they starts writing the code to read the file in one go and then process one line at a time. If you try to read a big file in memory then it your application will throw java out of memory error exception. So, it's clear that the best method of reading a big file is to read the text file line by line and then process the data.

Furthermore reading file line by line is the fastest way to read the file. It's also very easy to write code for reading the file line by line.

Here is the code example from our website:

// Open the file
FileInputStream fstream = new FileInputStream("textfile.txt");

// Get the object of DataInputStream
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));

String strLine;

//Read File Line By Line
while ((strLine = br.readLine()) != null)   {
  // Print the content on the console
  System.out.println (strLine);
}

//Close the input stream
in.close();

The above code opens the files and then uses the BufferedReader class to read the file line by line and then prints the data on console. In your real program you can process the line data and then save into database. You can also use the FileReader and BufferedReader classes to read the file line by line. Here is the code example:

import java.io.*;

public class ReadFileFileReader
{
    public static void main(String[] args) 
    {
        System.out.println("Example of reading file line by line!");
        System.out.println("through FileReader and BufferedReader classes");
        try{
        BufferedReader br = new BufferedReader(new FileReader("log.txt"));
        String line;
        while ((line = br.readLine()) != null) {
           // process the line.
           System.out.println(line);
        }
        br.close();

        }catch(Exception e){
            System.out.println("Error:" + e.getMessage());
        }


    }
}

Above code example uses the FileReader and BufferedReader classes to read a text file line by line and then prints the content on console.

Thanks


April 28, 2013 at 12:10 PM


April 28, 2013 at 12:15 PM

Thanks for good explanation.

Code discussed above can the used to read the data from a text file much faster. Above code is very easy to understandable also. After reading one line of data you can use the data and process it according to your business needs. You can also save the data into relational database if you are converting your legacy data.

Thanks


April 28, 2013 at 12:20 PM

Hi, Another very important thing is the proper exception handling of the application. You should update the above code for proper exception handling and closing the resources properly.

You should properly catch all the exception of the application. For example br.close(); throws the IOException, it must be properly handled.

You must also properly handle the FileNotFound exception, this exception is thrown by the FileReader's constructor. So, proper exception handling is very important.

Thanks


April 28, 2013 at 12:22 PM

Hi,

Check this tutorial: How to read big file line by line in java?

Thanks


April 28, 2013 at 12:33 PM

Hello,

The BufferedReader class is very convenient class reading the data from the resource. This helps the programmers to develop the code for reading the data very conveniently. The BufferedReader class provides a method called readLine(), which reads one line at a time from the input stream. The readLine() method returns a String.

Read more at Java Read File Line by Line - Java Tutorial.

Thanks


April 28, 2013 at 12:39 PM

Hi,

There is another way to read the file line by line using the Scanner class of Java API. Read more at Line by Line reading from a file using Scanner Class.

Thanks


August 30, 2013 at 9:50 PM

Hi,

All the above examples are good. You can view more tutorials at Java file examples and tutorials on Rose India website in Files Handling section.

Java API also provides classes and interfaces for reading and writing to the .properties files. Check the tutorial How to read properties file in Java.

Thanks









Related Tutorials/Questions & Answers:
How to read a large text file line by line in java?
How to Read a file line by line using BufferedReader?
Advertisements
How to read big file line by line in java?
Java Read File Line by Line - Java Tutorial
nio read file line by line
Java read file line by line
how to read file line by line using filereader in java
How to read a line from a doc file
How to write file text In New Line
Read Specific Line from file Using Java
How to Read file line by line in Java program
java - read file line by line in Java
objective c read file line by line
How to read a file line by line?
Java Read File Line By Line, Video Tutorial of Java Read File Line By Line
Java read file line by line - Java Tutorial
Write Text To File In New Line.
Java - How to read a string line per line
How can i read a file from nth line onwards in java
writing a text into text file at particular line number
How to extract a specific line from a text file? - IoC
writing a text into text file at particular line number
writing a text into text file at particular line number
Read file from the Nth line
Read text file in PySpark
How to read file in java
Java file new line
How to write file by line in Java
Java read text file
Read file line by line in Java 8
Java file line reader
how to write to file at the end of the line
Java write to file line by line
How do I read a large file quickly in Java?
Map words to line number in text file and show occurence
Java Write To File By Line
How to read text file in Servlets
how to read a text file with scanner in java
Java Write To File End Of Line
How To Read Integer From Command Line In Java
Example code of reading file line by line in Java with Apache Commons IO library
how to read text file in jtable in netbeans7.0
Line by Line reading from a file using Scanner Class
How to extract the entire line with specific data from a Text in java?
Read text File
java write to a specific line
Layout text along a line using LineMetrics
how to read text file with java 8 stream api
How do I install a .deb file via the command line?
HOW TO PARSE FILE WITH SEVERAL DELIMITERS AND LINE FEEDERS - Java Beginners

Ads