Re: Searching on mulit-core Solr - Fergus McMenemie - org.apache.lucene.solr-user

10 messages in org.apache.lucene.solr-userRe: Searching on mulit-core Solr

From	Sent On	Attachments
vivek sar	Apr 3, 2009 1:01 pm
vivek sar	Apr 6, 2009 12:32 pm
Fergus McMenemie	Apr 6, 2009 2:40 pm
vivek sar	Apr 8, 2009 1:57 am
vivek sar	Apr 8, 2009 11:07 am
vivek sar	Apr 9, 2009 12:00 am
Erik Hatcher	Apr 9, 2009 2:56 am
Fergus McMenemie	Apr 9, 2009 4:06 am
vivek sar	Apr 9, 2009 9:49 am
vivek sar	Apr 9, 2009 10:01 am	.xml

Subject:	Re: Searching on mulit-core Solr
From:	Fergus McMenemie (fer...@twig.me.uk)
Date:	Apr 9, 2009 4:06:44 am
List:	org.apache.lucene.solr-user

Any help on this issue? Would distributed search on multi-core on same Solr instance even work? Does it has to be different Solr instances altogether (separate shards)?

As best I can tell this works fine for me. Multiple cores on the one machine. Very different schema and solrconfig.xml for each of the cores. Distributed searching using shards works fine. But I am using the trunk version.

Perhaps you should post your solr.xml file.

I'm kind of stuck at this point right now. Keep getting one of the two errors (when running distributed search - single searches work fine) as mentioned in this thread earlier.

Thanks, -vivek

Thanks Fergus. I'm still having problem with multicore search.

I tried the following with two cores (they both share the same schema and solrconfig.xml) on the same box on same solr instance,

1) http://10.4.x.x:8080/solr/core0/admin/ - works fine, shows all the cores in admin interface 2) http://10.4.x.x:8080/solr/admin/cores - works fine, see all the cores in xml 3) http://10.4.x.x:8080/solr/20090407_2/select?q=japan - works fine, gives me top 10 records 4) http://10.4.x.x:8080/solr/20090408_3/select?q=japan - works fine, gives me top 10 records 5)
http://10.4.x.x:8080/solr/20090407_2/select?shards=10.4.x.x:8080/solr/20090407_2,10.4.x.x:8080/solr/20090408_3&indent=true&q=japan - this FAILS. I've seen two problems with this.

a) When index are being committed I see,

SEVERE: org.apache.solr.common.SolrException: org.apache.solr.client.solrj.SolrServerException: java.net.SocketException: Connection reset at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:282) at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845) at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:637)

b) Other times I see this,

SEVERE: java.lang.NullPointerException at
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:432) at
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:276) at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:290) at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845) at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:637)

Any tips on how can I search on multicore on same solr instance?

Thanks, -vivek

On Mon, Apr 6, 2009 at 2:40 PM, Fergus McMenemie <fer...@twig.me.uk> wrote:

vivek,

404 from the URL you provided in the message! Similar URLs work OK for me.

hmm try http://localhost:8080/solr/admin/cores?action=status and see if that gives a 404.

Also are you running a nightly build or a svn checkout? Using tomcat? Perhaps it should be

http://localhost:8080/apache-solr-1.4-dev/admin/cores?action=status

Fergus.

Hi,

Any help on this. I've looked at DistributedSearch on Wiki, but that doesn't seem to be working for me on multi-core and multiple Solr instances on the same box.

Scenario,

1) Two boxes (localhost, 10.4.x.x) 2) Two Solr instances on each box (8080 and 8085 ports) 3) Two cores on each instance (core0, core1)

I'm not sure how to construct my search on the above setup if I need to search across all the cores on all the boxes. Here is what I'm trying,

http://localhost:8080/solr/core0/select?shards=localhost:8080/solr/core0,localhost:8085/solr/core0,localhost:8080/solr/core1,localhost:8085/solr/core1,10.4.x.x:8080/solr/core0,10.4.x.x:8085/solr/core0,10.4.x.x:8080/solr/core1,10.4.x.x:8085/solr/core1&indent=true&q=vivek+japan

I get 404 error. Is this the right URL construction for my setup? How else can I do this?

Thanks, -vivek

Hi,

I've a multi-core system (one core per day), so there would be around 30 cores in a month on a box running one Solr instance. We have two boxes running the Solr instance and input data is feeded to them in round-robin fashion. Each box can have up to 30 cores in a month. Here are questions,

1) How would I search for a term in multiple cores on same box?

Single core I'm able to search like, http://localhost:8080/solr/20090402/select?q=*:*

2) How would I search for a term in multiple cores on both boxes at the same time?

3) Is it possible to have two Solr instances on one box with one doing the indexing and other perform only searches on that index? The idea is have two JVMs with each doing its own task - I'm not sure whether the indexer process needs to know about searcher process - like do they need to have the same solr.xml (for multicore etc). We don't want to replicate the indexes also (we got very light search traffic, but very high indexing traffic) so they need to use the same index.

Thanks, -vivek

=============================================================== Fergus McMenemie Email:fer...@twig.me.uk Techmore Ltd Phone:(UK) 07721 376021

Unix/Mac/Intranets Analyst Programmer ===============================================================

	Start a set with this search
	Include this search in one of my sets
	Exclude this search from one of my sets