Latest Tutorials| Questions and Answers|Ask Questions?|Site Map



Home Answers Viewqa Java-Beginners search engine build by lucene and eclipse
Login         

View Questions and Answers by Category

Advertisements


 
Have Programming Question? Ask it here!
 
 
 


baya
search engine build by lucene and eclipse
2 Answer(s)      4 years and a month ago
Posted in : Java Beginners


Hi,

here is the code:

package org.apache.lucene;

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.search.Collector;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.Scorer;
import org.apache.lucene.search.Searcher;
import org.apache.lucene.search.TopScoreDocCollector;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.util.Version;

import java.io.BufferedInputStream;
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.text.ParseException;
import java.util.Date;

public class SearchEngine extends JFrame {

    String filePath = "C:/Users/yOuRs/Desktop/files";
    String indexPath = "C:/Users/yOuRs/Desktop/index";
    static JLabel jlblNoResult = new JLabel("");
    static JLabel[][] jlblResult = new JLabel[20][2];

    JButton btnsrc;
    JButton btnidx;
    JTextField qry;
    JLabel label;
    JLabel result;
    JRadioButton poor;
    JRadioButton average;
    JRadioButton good;

    static final File INDEX_DIR = new File("index");

    public static void main (String []args)
    {
        SearchEngine frame = new SearchEngine();
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setTitle("SEARCH ENGINE");
        frame.setSize(500, 400);
        frame. setVisible(true);
    }



public SearchEngine () {
        Container pane = getContentPane();
        pane.setLayout(new FlowLayout());

        JPanel panel1 = new JPanel();
        panel1.setLayout(new FlowLayout());

        btnsrc = new JButton("Search");
        label = new JLabel("Query:");
        qry = new JTextField(20);
        btnidx = new JButton("Index");

        panel1.add(label);
        panel1.add(qry);
        panel1.add(btnsrc);
        panel1.add(btnidx);

        JPanel panel2 = new JPanel();

        result = new JLabel("Search Result:");
        panel2.add(result);

        ButtonGroup grp = new ButtonGroup();
        poor = new JRadioButton("Poor");
        average = new JRadioButton("Average");
        good = new JRadioButton("Good");

        panel2.add(poor);
        panel2.add(average);
        panel2.add(good);


        grp.add(poor);
        grp.add(average);
        grp.add(good);

        JPanel panel3 = new JPanel(new GridLayout(41,0));
        panel3.add(jlblNoResult);
        for (int i=0;i<20;i++){
            for(int k=0;k<2;k++){
            jlblResult[i][k] = new JLabel("");
            panel3.add(jlblResult[i][k]);
            }
        }

        pane.add(panel1);
        pane.add(panel2);
        pane.add(panel3);

        btnidx.addActionListener(new ActionListener() {
            private boolean deleteDir1;

            public void actionPerformed(ActionEvent e) {
                File INDEX_DIR = new File(indexPath);

                if (INDEX_DIR.exists()) {
                  try {
                    deleteDir1 = deleteDir1(INDEX_DIR);
                } catch (IOException e1) {
                    // TODO Auto-generated catch block
                    e1.printStackTrace();
                }
                }

                final File docDir = new File(filePath);
                if (!docDir.exists() || !docDir.canRead()) {
                  System.out.println("Document directory '" +docDir.getAbsolutePath()+ "' does not exist or is not readable, please check the path");
                  System.exit(1);
                }

                Date start = new Date();
                try {
                  IndexWriter writer = new IndexWriter(FSDirectory.open(INDEX_DIR), new StandardAnalyzer(Version.LUCENE_CURRENT), true, IndexWriter.MaxFieldLength.LIMITED);
                  System.out.println("Indexing to directory '" +INDEX_DIR+ "'...");
                  indexDocs(writer, docDir);
                  System.out.println("Optimizing...");
                  writer.optimize();
                  writer.close();

                  Date end = new Date();
                  System.out.println(end.getTime() - start.getTime() + " total milliseconds");

                } catch (IOException event) {
                  System.out.println(" caught a " + event.getClass() +
                   "\n with message: " + event.getMessage()); 
                }
            }

            private void indexDocs(IndexWriter writer, File docDir) {
                // TODO Auto-generated method stub

            }
        });

            btnsrc.addActionListener(new ActionListener() {
                public void actionPerformed(ActionEvent e) {

                        String field = "contents";
                        String queries = null;
                        int repeat = 0;
                        boolean raw = false;
                        String normsField = null;
                        boolean paging = true;
                        int hitsPerPage = 10;
                        int control = 0;

                        jlblNoResult.setText("");
                        for (int i=0;i<20;i++){
                            for(int k=0;k<2;k++)
                            jlblResult[i][k].setText("");
                        }

                        IndexReader reader = null;
                        try {
                            reader = IndexReader.open(FSDirectory.open(new File(indexPath)), true);
                        } catch (CorruptIndexException e2) {
                            // TODO Auto-generated catch block
                            e2.printStackTrace();
                        } catch (IOException e2) {
                            // TODO Auto-generated catch block
                            e2.printStackTrace();
                        }

                        if (normsField != null)
                          reader = new OneNormsReader(reader, normsField);

                        Searcher searcher = new IndexSearcher(reader);
                        Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_CURRENT);

                        BufferedReader in = null;
                        if (queries != null) {
                          try {
                            in = new BufferedReader(new FileReader(queries));
                        } catch (FileNotFoundException e1) {
                            // TODO Auto-generated catch block
                            e1.printStackTrace();
                        }
                        } else {
                          try {
                            in = new BufferedReader(new InputStreamReader(System.in, "UTF-8"));
                        } catch (UnsupportedEncodingException e1) {
                            // TODO Auto-generated catch block
                            e1.printStackTrace();
                        }
                        }
                        QueryParser parser = new QueryParser(Version.LUCENE_CURRENT, field, analyzer);
                        while (control ==0) {
                          JLabel jtfInput = null;
                        String line = jtfInput.getText();

                          if (line == null || line.length() == -1)
                            break;

                          line = line.trim();
                          if (line.length() == 0)
                            break;

                          Query query = null;
                        try {
                            query = parser.parse(line);
                        } catch (org.apache.lucene.queryParser.ParseException e1) {
                            // TODO Auto-generated catch block
                            e1.printStackTrace();
                        }


                          if (repeat > 0) {                           // repeat & time as benchmark
                            Date start = new Date();
                            for (int i = 0; i < repeat; i++) {
                              try {
                                searcher.search(query, null, 100);
                            } catch (IOException e1) {
                                // TODO Auto-generated catch block
                                e1.printStackTrace();
                            }
                            }
                            Date end = new Date();
                            System.out.println("Time: "+(end.getTime()-start.getTime())+"ms");
                          }

                          if (paging) {
                            try {
                                doPagingSearch(in, searcher, query, hitsPerPage, raw, queries == null);
                              } catch (IOException e1) {
                                // TODO Auto-generated catch block
                                e1.printStackTrace();
                            }
                          } 
                          else {
                            try {
                                doStreamingSearch(searcher, query);
                            } catch (IOException e1) {
                                // TODO Auto-generated catch block
                                e1.printStackTrace();
                            }
                          }

                          control = 1;
                        }
                        try {
                            reader.close();
                        } catch (IOException e1) {
                            // TODO Auto-generated catch block
                            e1.printStackTrace();
                        }
                }
            });
}



        public static void indexDocs(IndexWriter writer, File file)
        throws IOException {
        // do not try to index files that cannot be read
        if (file.canRead()) {
          if (file.isDirectory()) {
            String[] files = file.list();
            // an IO error could occur
            if (files != null) {
              for (int i = 0; i < files.length; i++) {
                indexDocs(writer, new File(file, files[i]));
              }
            }
          } else {
            System.out.println("adding " + file);
            try {
              writer.addDocument(FileDocument.Document(file));
            }
            // at least on windows, some temporary files raise this exception with an "access denied" message
            // checking if the file can be read doesn't help
            catch (FileNotFoundException fnfe) {
              ;
            }
          }
        }
      }
        public static boolean deleteDir1(File path) throws IOException {
            if( path.exists() ) {
                  File[] files = path.listFiles();
                  for(int i=0; i<files.length; i++) {
                     if(files[i].isDirectory()) {
                       deleteDir1(files[i]);
                     }
                     else {
                       files[i].delete();
                     }
                  }
                }
                return( path.delete() );
        }
        public static void doStreamingSearch(final Searcher searcher, Query query) throws IOException {
            Collector streamingHitCollector = new Collector() {
              private Scorer scorer;
              private int docBase;

              // simply print docId and score of every matching document
              @Override
              public void collect(int doc) throws IOException {
                System.out.println("doc=" + doc + docBase + " score=" + scorer.score());
              }

              @Override
              public boolean acceptsDocsOutOfOrder() {
                return true;
              }

              @Override
              public void setNextReader(IndexReader reader, int docBase)
                  throws IOException {
                this.docBase = docBase;
              }

              @Override
              public void setScorer(Scorer scorer) throws IOException {
                this.scorer = scorer;
              }

            };

            searcher.search(query, streamingHitCollector);
          }
        public static void doPagingSearch(BufferedReader in, Searcher searcher, Query query, 
                int hitsPerPage, boolean raw, boolean interactive) throws IOException {

            // Collect enough docs to show 5 pages
            TopScoreDocCollector collector = TopScoreDocCollector.create(
                5 * hitsPerPage, false);
            searcher.search(query, collector);
            ScoreDoc[] hits = collector.topDocs().scoreDocs;

            int numTotalHits = collector.getTotalHits();
            jlblNoResult.setText(numTotalHits + " total matching documents");

            int manipulation = 0;
            int start = 0;
            //int end = Math.min(numTotalHits, hitsPerPage);
            int end = numTotalHits;
            FileInputStream fis = null;
            BufferedInputStream bis = null;
            DataInputStream dis = null;

    while (manipulation == 0) {
        if (end > hits.length) {
            System.out.println("Only results 1 - " + hits.length +" of " + numTotalHits + " total matching documents collected.");
            System.out.println("Collect more (y/n) ?");
            String line = in.readLine();
            if (line.length() == 0 || line.charAt(0) == 'n') {
              break;
            }

            collector = TopScoreDocCollector.create(numTotalHits, false);
            searcher.search(query, collector);
            hits = collector.topDocs().scoreDocs;
          }

        //end = Math.min(hits.length, start + hitsPerPage);

    for (int i = start; i < end; i++) {


    if (raw) {                              // output raw format
    System.out.println("doc="+hits[i].doc+" score="+hits[i].score);
    continue;
    }

    Highlighter highlighter = new Highlighter(new QueryScorer(query));
    highlighter.setTextFragmenter(new SimpleFragmenter(400));
    Document doc = searcher.doc(hits[i].doc);
    String path = doc.get("path");
    if (path != null) {
        jlblResult[i][0].setText((i+1) + ". " + path);
        jlblResult[i][1].setText("    Score = " + hits[i].score);
    String title = doc.get("title");
    if (title != null) {
        jlblResult[i][0].setText("   Title: " + doc.get("title"));
        jlblResult[i][1].setText("    Score = " + hits[i].score);
    }
    } else {
        jlblResult[i][0].setText((i+1) + ". " + "No path for this document");
        //JOptionPane.showMessageDialog(frame, "Eggs are not supposed to be green.");
    }
    File file = new File(path);
    try {
        fis = new FileInputStream(file);

        // Here BufferedInputStream is added for fast reading.
        bis = new BufferedInputStream(fis);
        dis = new DataInputStream(bis);

        // dis.available() returns 0 if the file does not have more lines.
        while (dis.available() != 0) {

        // this statement reads the line from the file and print it to
          // the console.
            System.out.println(dis.readLine());
        }

        // dispose all the resources after using them.
        fis.close();
        bis.close();
        dis.close();

      } catch (FileNotFoundException e) {
        e.printStackTrace();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
        manipulation = 1;
    }

    }
protected void deleteDir(File iNDEX_DIR2) {
    // TODO Auto-generated method stub

}
}

this is my coding to build a search engine by using eclipse n lucene.. however there is error and when i click the search button it has no output..i want to display the document that matching with my query that i have inserted. can you help me to fix these errors ?? i just want to index and search text files in my local data..how can i make these search engine function like google, but not for web..just for text files in my laptop..help me pleaseee...


Advertisement
View Answers

October 18, 2010 at 6:27 PM


Hi, I will look into it tomorrow and send you the corrected code. Thanks


October 19, 2010 at 12:22 PM


Hi Friend,

There is one error which occurs at the following line:

reader = new OneNormsReader(reader, normsField);

Actually we haven't get the OneNormsReader class. So please explain the above line and provide this class.

Thanks



Related Tutorials/Questions & Answers:
search engine build by lucene and eclipse
search engine build by lucene and eclipse  Hi, here is the code...); frame.setTitle("SEARCH ENGINE"); frame.setSize(500, 400); frame...()); btnsrc = new JButton("Search"); label = new JLabel("Query
Lucene Search in JSP - JSP-Servlet
Lucene Search in JSP  Hello Sir, I want to Develop lucene - Search engine in Jsp page which can Search full text from XML file..through XML parser.. Example: in Search interface there are one textbox after putting some value
Advertisements
Search Engine
Search Engine  I have gone through the below mentioned link document to create search engine and fetch the data from mysql database.But i am getting errors. http://www.roseindia.net/sql/mysql-table/sql-search.shtml Error: Notice
Hibernate Search
full text index using apache Lucene engine. Then the index data can be used... Persistence API and Apache Lucene. Apache Lucene has high performance search engine capabilities. In Hibernate Search, both the object-relational mapping capacity
Search engine for email
Search engine for email  Build a search engine which will look at your email inbox and sort your email into 5 or more bins according to some criteria (e.g. date, email address, text of the email
Search Engine Interface
Search Engine Interface   ... interface of our search engine. For searching and displaying the result we... Search will search the data from Lucene Index and display on the page as shown below
Search Engine - Java Beginners
Search Engine  Hello Sir, I am new jsp, also i need search engine coding from the local computer or database only.pls send the jsp with mysql codes.I dont know how to create a search engine(like google) in local computer
Java lucene
Java lucene  Build your own search engine.how can i implement lucene code in my GUI interface of search engine so that my search engine interface can... search engine.i am using eclipse to do this task.package org.apache.lucene;import
simple java search engine
. ABSTRACT Title : Simple Search Engine System Specification: The system on which.... This is needed for every one. The project is simple search engine, which searches...simple java search engine  i have already downloaded the project
Developing Search Engine in Java
Developing Search Engine in Java       In this section we will discuss about the search engine, and then show you how you can develop your own search engine for your website
Finding searching phrase of a search engine
Finding searching phrase of a search engine  how to find out searching phrase of a search engine..? like, if visitors enter the keyword to google, is is any possible to get that keyword
JSP search engine - JSP-Servlet
JSP search engine  Hi! In my project i have a concept of search engine which has to search in google and display the results in my page. My PM told me use GOOGLE API for search engine. I am developing applicatin in JSP. Can
Eclipse helios "Build Before Launch"
Eclipse helios "Build Before Launch"  Hi The IDE Eclipse Helios present having an issue related to ?Build before Launch?. it still tries to build... look into this issue . i have tried these settings for Eclipse Helios
Eclipse helios "Build Before Launch"
Eclipse helios "Build Before Launch"  Hi The IDE Eclipse Helios present having an issue related to ?Build before Launch?. it still tries to build... into this issue . i have tried these settings for Eclipse Helios Disable
Submitting Web site to search engine
to the search engines, the search engine crawls will visit your site and will index your pages to the search engine index. Once your web site appears... their search engine placement. How to submit the web site to Major
SEO and Search Engines,Best Search Engines on Web,What is Search Engine
SEO and Search Engine   ... of the search engine while doing SEO work. Since in SEO we are using search engine to promote our website, so understanding the search engine details like
Java Search Engine - Java Beginners
Java Search Engine  Hello Sir,The Code Provided by u is very Useful for me,Thank u for Help, but I have one Question is How to Install and Run Mysql ,can I use Sql Server insted of MySql. and How to Install Servlet and Run
Building Search Engine Applications Using Servlets !
Building Search Engine Applications Using Servlets Introduction This tutorial takes you through the process of building search engine... these days. This search engine shows you how to use Java Servlets
save links clicked in search engine results
save links clicked in search engine results  hello i need to access search engine results in my program(any search engine).ie suppose i give a keyword,i need the results returned by that search engine.from the returned results
Writing to be located through the Search Engine
Writing to be located through the Search Engine       The majority of web users depend upon search... a search he should immediately know how the page that has opened up connects to his
Writing to be located through the Search Engine
Writing to be located through the Search Engine  ... search engines of all kind to locate and navigate web pages. While writing text... a search he should immediately know how the page that has opened up connects
Download Search Engine Code its free and Search engine is developed in Servlets
web server. To test your search engine key http://localhost:serverport/servlet/search in your browser. By now your search engine should work.   Download Search
regarding designing of web search engine - Development process
regarding designing of web search engine  we want to design a web search engine in java. so, how to get started with our coding...can i get sample code for web crawlers or similar requirements... help us
How to code a Product id search engine?
How to code a Product id search engine?  I always wonder how people... a search box that let users search products with id. Suppose, "If an user enters id:1234 (and hits search button) go to product page url: http://#" if user
SEO Guide,Search Engine Optimization,Basic of SEO,Search Engine Optimization Information
Search Engine Optimization What is Search Engine Optimization? Search... ranking within search engines. Search Engine Optimization are done to attend the highest ranking in the search engine results for some targeted keywords
Top 10 SEO Mistakes,Common Mistakes in Search Engine Optimization,Search Engine Marketing Mistakes
Search Engine Mistakes - Search Engine Optimization Mistakes... content but they do not rank well on the search engine results. These low placements.... People uses the different Search Engine Optimization techniques to boost web
Welcome to Free search engine secrets: webmaster's guide to search engine registration!
engines, so each search engine submission is your most cost effective form... their search engine placement.      If you... and experience the Search Engine World.    Here I am assuming
Google Desktop Search
plug-in for Eclipse. Instead of using the default (and kind-of slow) File search capabilities, this plug-in uses the Google Desktop Search Engine instead... Google Desktop Search     
Source Code for Search Engine Project in java - Java Beginners
Source Code for Search Engine Project in java  Hello Sir ,I want Java Project for Search Engine(like google),How I can Make it,Plz Give Me Complete Source Code Of Search Engine Project in Java.  Hi Friend, Please
What is SEO,What is Search Engine Optimization,Definition of SEO
; In this article you learn about SEO (Search Engine Optimization) and basics behind the Search Engine Optimization. These days search engine optimization... Engine Optimization? Search Engine Optimization or SEO for short is modification
Eclipse Plunging-Build and Deploy
Eclipse Plunging-Build and Deploy   ...;   Fat Jar-Build and Deploy The Fat Jar Eclipse Plug-In allows building... builder Pluginbuilder supports the build automation of Eclipse plug-ins
Advertisements
 

 

 

DMCA.com