How to read a large text file line by line in java?

Ads

 
 
 

Share on Google+Share on Google+

Java Coder
How to read a large text file line by line in java?
8 Answer(s)      4 years ago
Posted in : Java Beginners

I have been assigned a work to read big text file and extract the data and save into database. The file size is in many GBs. We have studied the memory requirement of the application and the memory available for the Java program. For Java program max memory assigned is also a limit.

So, we have decided to read the text file line by line and the extract the data line by line. Reading the file line by line will also increase the performance of the application.

Earlier our company was using the text file for saving the data through an application developed in C++. Now we have given a job for extracting the data and save into database. Give use you kind advice and let's know how to read a large text file line by line in java?

Thanks

Ads
View Answers

April 28, 2013 at 12:05 PM


Hi,

The only good solution to read big text file is to read the file line by line in your Java program. Reading the file line by line will make your application much faster. The application will use less memory and work even on the computer having less amount of memory.

Since reading the file line by line is the only best method for reading a big file, developers are searching for 'Java read file line by line' on google and other search engines. Many programmers are finding difficult if they starts writing the code to read the file in one go and then process one line at a time. If you try to read a big file in memory then it your application will throw java out of memory error exception. So, it's clear that the best method of reading a big file is to read the text file line by line and then process the data.

Furthermore reading file line by line is the fastest way to read the file. It's also very easy to write code for reading the file line by line.

Here is the code example from our website:

// Open the file
FileInputStream fstream = new FileInputStream("textfile.txt");

// Get the object of DataInputStream
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));

String strLine;

//Read File Line By Line
while ((strLine = br.readLine()) != null)   {
  // Print the content on the console
  System.out.println (strLine);
}

//Close the input stream
in.close();

The above code opens the files and then uses the BufferedReader class to read the file line by line and then prints the data on console. In your real program you can process the line data and then save into database. You can also use the FileReader and BufferedReader classes to read the file line by line. Here is the code example:

import java.io.*;

public class ReadFileFileReader
{
    public static void main(String[] args) 
    {
        System.out.println("Example of reading file line by line!");
        System.out.println("through FileReader and BufferedReader classes");
        try{
        BufferedReader br = new BufferedReader(new FileReader("log.txt"));
        String line;
        while ((line = br.readLine()) != null) {
           // process the line.
           System.out.println(line);
        }
        br.close();

        }catch(Exception e){
            System.out.println("Error:" + e.getMessage());
        }


    }
}

Above code example uses the FileReader and BufferedReader classes to read a text file line by line and then prints the content on console.

Thanks



April 28, 2013 at 12:10 PM




April 28, 2013 at 12:15 PM


Thanks for good explanation.

Code discussed above can the used to read the data from a text file much faster. Above code is very easy to understandable also. After reading one line of data you can use the data and process it according to your business needs. You can also save the data into relational database if you are converting your legacy data.

Thanks



April 28, 2013 at 12:20 PM


Hi, Another very important thing is the proper exception handling of the application. You should update the above code for proper exception handling and closing the resources properly.

You should properly catch all the exception of the application. For example br.close(); throws the IOException, it must be properly handled.

You must also properly handle the FileNotFound exception, this exception is thrown by the FileReader's constructor. So, proper exception handling is very important.

Thanks



April 28, 2013 at 12:22 PM


Hi,

Check this tutorial: How to read big file line by line in java?

Thanks



April 28, 2013 at 12:33 PM


Hello,

The BufferedReader class is very convenient class reading the data from the resource. This helps the programmers to develop the code for reading the data very conveniently. The BufferedReader class provides a method called readLine(), which reads one line at a time from the input stream. The readLine() method returns a String.

Read more at Java Read File Line by Line - Java Tutorial.

Thanks



April 28, 2013 at 12:39 PM


Hi,

There is another way to read the file line by line using the Scanner class of Java API. Read more at Line by Line reading from a file using Scanner Class.

Thanks



August 30, 2013 at 9:50 PM


Hi,

All the above examples are good. You can view more tutorials at Java file examples and tutorials on Rose India website in Files Handling section.

Java API also provides classes and interfaces for reading and writing to the .properties files. Check the tutorial How to read properties file in Java.

Thanks


Ads









Tutorials   
Java Spring Hibernate Struts Training java.lang.NoClassDefFoundError: org/apache/http/client/HttpClient How do I resolve this Java Class not found exception? httpclient java.lang.NoClassDefFoundError Apache Commons ioutils maven dependency Read/Convert an inputStream to a String What is the meaning of Java Platform? Why Java is a platform independent language? What is the benefits of learning Core Java? Which technology should I learn after Java? What is array in java with example? How to Convert ArrayList to Array? How to substring in Java? How to format number in Java? What is instance variable in Java? How to download MySQL JDBC driver? What is Calendar class in Java? Which is the best Java tutorials for beginners? How to rename a file in Java? How to delete file in Java code? How to get day from date in Java using Calendar? How to get day of week in Java? How to calculate Date Difference in Java? How to compare date in Java? How to declare array in Java? How to calculate average of array in Java? What is Array in Java? write a java program to find the summation of all the integers entered on command line Sum of two numbers using command line arguments in Java How to create and use Array in Java? How to pass command line arguments in Java? How to create Applet Hello World? Appending String efficiently in Java How to append String in Java? How to list even numbers between 1 and 100? How to add BigDecimal in Java? What is Abstraction In Java? Which is best Beginners Java Tutorial? What is java.util package? Create list from array in Java Filter collection in Java 8 What is the best way to filter a Java Collection? Easy way to transform Collection to Array? How to convert Collection to Array in Java? What are Basic Java Language Elements? Advanced Java Tutorials in 2017 Java brief history Best Reasons to learn Java Java Example Codes and Tutorials in 2017 How do I read a large file quickly in Java? Is learning Java worthwhile?

Ads

 
Advertisement null

Ads