|
if(documentsIndexed.size() > NO_OF_DOCUMENT_TO_COMMIT) { |
|
try { |
|
solr.add(doSolrInputDocument); |
|
|
|
solr.commit(true, true); |
|
} catch(Exception e) { |
|
System.out.println(e.getMessage()); |
|
e.printStackTrace(); |
|
} |
|
} |
Here you don't add the documentsIndexed but only the doSolrInputDocument to solr.
I would suggest something like this:
// Declare an atomic counter in the class
private AtomicInteger cnt = new AtomicInteger(0);
// Instead of the quoted lines do
try {
solr.add(doSolrInputDocument);
if (cnt.incrementAndGet() % NO_OF_DOCUMENT_TO_COMMIT == 0)
solr.commit(true, true);
} catch (SolrServerException | IOException e) {
System.out.println(e.getMessage());
e.printStackTrace();
}
Solr-Sample/src/main/java/Unit_testing/MyCrawler.java
Lines 105 to 114 in f5c9c1a
Here you don't add the
documentsIndexedbut only thedoSolrInputDocumentto solr.I would suggest something like this: