4 messages in org.apache.lucene.solr-devRE: OutofMemory on Highlightling| From | Sent On | Attachments |
|---|---|---|
| Gargate, Siddharth | Apr 27, 2009 1:32 am | |
| Gargate, Siddharth | Apr 28, 2009 4:04 am | |
| Gargate, Siddharth | May 3, 2009 9:36 pm | |
| Gargate, Siddharth | May 7, 2009 4:19 am |
| Subject: | RE: OutofMemory on Highlightling | |
|---|---|---|
| From: | Gargate, Siddharth (sgar...@ptc.com) | |
| Date: | Apr 27, 2009 1:32:55 am | |
| List: | org.apache.lucene.solr-dev | |
Also adding solr dev group to this mail.
http://www.nabble.com/OutofMemory-on-Highlightling-td23075331.html
I am not able to figure out the issue
-----Original Message----- From: Gargate, Siddharth [mailto:sgar...@ptc.com] Sent: Friday, April 24, 2009 10:46 AM To: solr...@lucene.apache.org Subject: RE: OutofMemory on Highlightling
I am not sure whether lazy loading should help solve this problem. I have set enableLazyFieldLoading to true but it is not helping.
I went through the code and observed that DefaultSolrHighlighter.doHighlighting is reading all the documents and the fields for highlighting (In my case, 1 MB stored field is read for all documents).
Also I am confused over the following code in SolrIndexSearcher.doc() method
if(!enableLazyFieldLoading || fields == null) { d = searcher.getIndexReader().document(i); } else { d = searcher.getIndexReader().document(i, new SetNonLazyFieldSelector(fields)); }
Are we setting the fields as NonLazy even if lazy loading is enabled?
Thanks, Siddharth
-----Original Message----- From: Gargate, Siddharth [mailto:sgar...@ptc.com] Sent: Wednesday, April 22, 2009 11:12 AM To: solr...@lucene.apache.org Subject: RE: OutofMemory on Highlightling
Here is the stack trace
SEVERE: java.lang.OutOfMemoryError: Java heap space at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:133) at java.lang.StringCoding.decode(StringCoding.java:173) at java.lang.String.<init>(String.java:444) at org.apache.lucene.store.IndexInput.readString(IndexInput.java:125) at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:390) at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:230) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:892) at org.apache.lucene.index.MultiSegmentReader.document(MultiSegmentReader.j ava:277) at org.apache.solr.search.SolrIndexReader.document(SolrIndexReader.java:176 ) at org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:457) at org.apache.solr.search.SolrIndexSearcher.readDocs(SolrIndexSearcher.java :482) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultS olrHighlighter.java:253) at org.apache.solr.handler.component.HighlightComponent.process(HighlightCo mponent.java:84) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(Search Handler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerB ase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.ja va:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j ava:232) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica tionFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt erChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv e.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv e.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java :128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java :102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve. java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:2 86) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:84 5) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process( Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619)
-----Original Message----- From: Gargate, Siddharth [mailto:sgar...@ptc.com] Sent: Wednesday, April 22, 2009 9:29 AM To: solr...@lucene.apache.org Subject: RE: OutofMemory on Highlightling
I tried disabling the documentCache but still the same issue.
<documentCache class="solr.LRUCache" size="0" initialSize="0" autowarmCount="0"/>
-----Original Message----- From: Koji Sekiguchi [mailto:ko...@r.email.ne.jp] Sent: Monday, April 20, 2009 4:38 PM To: solr...@lucene.apache.org Subject: Re: OutofMemory on Highlightling
Gargate, Siddharth wrote:
Anybody facing the same issue? Following is my configuration ... <field name="content" type="text" indexed="true" stored="false" multiValued="true"/> <field name="teaser" type="text" indexed="false" stored="true"/> <copyField source="content" dest="teaser" maxChars="1000000" /> ...
... <requestHandler name="standard" class="solr.SearchHandler" default="true"> <lst name="defaults"> <str name="echoParams">explicit</str>
<int name="rows">500</int> <str name="hl">true</str> <str name="fl">id,score</str> <str name="hl.fl">teaser</str> <str name="hl.alternateField">teaser</str> <int name="hl.fragsize">200</int> <int name="hl.maxAlternateFieldLength">200</int> <int name="hl.maxAnalyzedChars">500</int> </lst> </requestHandler> ...
Search works fine if I disable highlighting and it brings 500 results. But if I enable hightlighting and set the no. of rows to just 20 I get OOME.
How about switching documentCache off?
Koji