|Robert Muir||Apr 26, 2011 8:07 pm|
|Yonik Seeley||Apr 26, 2011 8:34 pm|
|Michael McCandless||Apr 27, 2011 3:27 am|
|Michael McCandless||Apr 27, 2011 3:36 am|
|Mark Miller||Apr 27, 2011 5:12 am|
|Robert Muir||Apr 27, 2011 5:33 am|
|Steven A Rowe||Apr 27, 2011 6:18 am|
|Steven A Rowe||Apr 27, 2011 6:48 am|
|Michael McCandless||Apr 27, 2011 8:49 am|
|Grant Ingersoll||Apr 27, 2011 5:50 pm|
|Yonik Seeley||Apr 27, 2011 6:23 pm|
|Greg Stein||Apr 27, 2011 8:44 pm|
|Grant Ingersoll||May 2, 2011 3:11 pm|
|Michael McCandless||May 3, 2011 9:49 am|
|Shai Erera||May 3, 2011 10:29 am|
|Mark Miller||May 3, 2011 10:46 am|
|Robert Muir||May 4, 2011 6:29 am|
|Uwe Schindler||May 4, 2011 6:41 am|
|Mark Miller||May 4, 2011 6:49 am|
|Simon Willnauer||May 4, 2011 7:01 am|
|Grant Ingersoll||May 5, 2011 7:25 am|
|Simon Willnauer||May 5, 2011 8:03 am|
|Grant Ingersoll||May 5, 2011 8:34 am|
|Jason Rutherglen||May 5, 2011 9:58 am|
|Chris Hostetter||May 6, 2011 1:35 pm|
|Mark Miller||May 6, 2011 1:44 pm|
|Simon Willnauer||May 7, 2011 3:34 am|
|Simon Willnauer||May 7, 2011 4:10 am|
|Grant Ingersoll||May 7, 2011 5:20 am|
|Subject:||Re: modularization discussion|
|From:||Chris Hostetter (hoss...@fucit.org)|
|Date:||May 6, 2011 1:35:25 pm|
: To me, the third camp is just saying the proof is in the pudding. If : you want to refactor, then go for it. Just make sure everything still : works, which of course I know people will (but part of that means : actually running Solr, IMO). Perhaps, more importantly don't get mad : that if I have only one day a week to work on Lucene/Solr that I spend : it putting a specific feature in a specific place. Just because : something can/should be modularized, doesn't mean that a person working : in that area must do it before they add whatever they were working on. : For instance, if and when function queries are a module, I will add to : them there and be happy to do so. In the meantime, I will likely add to : them in Solr if that is something I happen to be interested in at that : time b/c I can certainly add a new function in a day, but I can't : refactor the whole module _and_ add my new function in a day.
I want to get that printed on a t-shirt
the corrolarry issue in my mind...
I am happily in favor of code reuse and modularization in the abstract, and when it works in practice i'm plesantly delighted.
But when people talk about modularization as a goal, and make a laundry list things in solr that people think should be refactored into modules (w/o showing specifics of what that module would look like) then i have a hard time buying into some of these ideas panning out in a way that: a) is a useful module to people in and of itself b) doesn't hamstring the evolution/performance in solr.
To look at "faceting" as a concrete example, there are big the reasons faceting works so well in Solr: Solr has total control over the index, knows exactly when the index has changed to rebuild caches, has a strict schema so it can make sense of field types and pick faceting algos accordingly, has multi-phase distributed search approach to get exact counts efficiently across multiple shards, etc... (and there are still a lot of additional enhancements and improvements that can be made to take even more advantage of knowledge solr has because it "owns" the index that we no one has had time to tackle)
I find it really hard to picture a way that this code could be refactored into a reusable module in such a way that it could have an API that would be easily usable outside of Solr -- and when i do get a glimmer of an inkling of what that might look like, that vision scares me because of how that API might then "hobble" Solr's ability to leverage it's total control of the underlying index to add additional performance/features.
To be crystal clear: I recognize that this is *my* hangup -- I am not suggesting that "I am short sighted and have little imagination therefore this code should never be modularized."
I'm trying to explain why i *personally* am hesitant and sceptical of how well modularizations of features like like this might actually work in practice, and why i'm not eager to jump in and contribute on a goal whose end result is something that i can't fully picture (and when i can picture it, i'm a little scared by what i see)
That doesn't mean i'm opposed to it happening -- i would love to live in the land of candy where houses are made of ginger bread and sugar plums grow on trees, I'm just too skeptical that such a land exists (or is as great as legend describes) to go slogging along on an epic journey to try and reach it -- i'm too old for that shit.
I'm certainly not going to stop anyone else fro going on that quest -- but i am entitled to voice my skepticism and concerns, just as adventursome folks are entitled to ignore me.