| * g8tor_ leaves | 00:40 | |
| * dcage joins | 09:35 | |
| * rsinger leaves | 10:16 | |
| * rsinger joins | 10:19 | |
| * dcage leaves | 10:23 | |
| <cbeer> | BillDueber: that edismax thing was too easy.. guess i'll have to start on implementing designs now | 10:31 |
| <BillDueber> | cbeer: Glad it came together for you! I'm actually having some problems with it right now....dammit. | |
| <cbeer> | oh? what should i watch out for? | 10:33 |
| <erikhatcher> | BillDueber: what kinds of problems? | 10:36 |
| <BillDueber> | erikhatcher: Dude, you're *everywhere*! :-) | |
| erikhatcher: I have a field type that I call "Exactmatcher" -- it takes out spaces, punctuation, lowercases everything, downgrades diacritics, etc. For index & query | 10:37 | |
| erikhatcher: But it doesn't seem to work as a pf field, which threw me off. | ||
| erikhatcher: Actually, contines to throw me off. | ||
| <erikhatcher> | can you show a simple debugQuery output? | 10:38 |
| <BillDueber> | erikhatcher: If...you tell me how to do that. Then, yes! | |
| * BillDueber has giant, gaping hole in his knowledge about pretty much everything. | 10:39 | |
| <erikhatcher> | &debugQuery=true | 10:40 |
| look at the response and see what it parses the query to | ||
| <BillDueber> | erikhatcher: OK. I'm on it. | |
| erikhatcher: OK, so I guess I had a fundamental misundersatnding about dismax | 10:52 | |
| I'm seeing from the debug output that the "phrase query" is really doing bigrams. Not the whole thing. | ||
| (this is with the edismax) | ||
| <erikhatcher> | BillDueber: do you have a bigramming field setup? | 10:53 |
| what do you get if you switch to pure dismax? | 10:54 | |
| <BillDueber> | erikhatcher: Not to the best of my knowledge. Certainly not on purpose. | |
| erikhatcher: Lemme restart it and check. | ||
| <erikhatcher> | BillDueber: you should be able to do &debugQuery&defType=dismax or defType=edismax to compare the two | 10:55 |
| no restart needed to toggle query parsers | 10:56 | |
| <BillDueber> | erikhatcher: With straight-up dismax, I'm *not* seeing the bigrams - -just the whole phrase, as expected. | 10:59 |
| erikhatcher: I'm using the edismax patch in jira against 1.4 stable | 11:00 | |
| <erikhatcher> | BillDueber: and you're using exactly the same config except for the defType parameter? | |
| <BillDueber> | erikhatcher: Yup. | 11:01 |
| <erikhatcher> | BillDueber: now it's example time... show me a parsed query from each | |
| <BillDueber> | erikhatcher: OK. | |
| @help lisppaste | ||
| erikhatcher: http://paste.lisp.org/+1YNY | 11:12 | |
| <erikhatcher> | BillDueber: ok, i'm looking into edismax now.... | 11:16 |
| BillDueber: looks like an issue you'll want to bring up on solr-user | ||
| <BillDueber> | erikhatcher: Yeah. Dammit | |
| <erikhatcher> | BillDueber: are you removing stop words? | |
| <BillDueber> | erikhatcher: No | |
| erikhatcher: Not in the data this one is hitting, anyway. | ||
| <erikhatcher> | BillDueber: your results are the same amount.... i'm not sure what's not working right for you | 11:17 |
| <BillDueber> | erikhatcher: The ranking | |
| erikhatcher: If someone searches for "gone with the wind", an item with the title "The wind" shouldn't appear on the first page. | 11:18 | |
| erikhatcher: It toally screws up my ability to boost complete title matches | ||
| <erikhatcher> | BillDueber: yeah, i see what you mean | 11:19 |
| BillDueber: best to post this very example to solr-user... yonik will surely respond | ||
| <BillDueber> | erikhatcher: Writing now -- thanks! | |
| erikhatcher++ yet again | ||
| <erikhatcher> | nah, no increment for me on this one... i'm scratching my head! | |
| <BillDueber> | erikhatcher: Yeah, but do you have any idea how long I would have screwed with this trying to figure out what was going on without you? The answer is "too damn". Too damn long. | 11:20 |
| <erikhatcher> | query parsing - no trivial job to do "right" | 11:21 |
| * tachyonwill_ joins | ||
| * jkeck joins | 12:35 | |
| <BillDueber> | cbeer, erikhatcher: I patched to allow a pf1 arugment (turn query into a single phrase, not bigrams). Now it works as I'd expect. Wish I understood the code I was writing, though... | 13:40 |
| <erikhatcher> | BillDueber: you and me both, man | 13:42 |
| BillDueber++ | ||
| <ndushay> | BillDueber: holy @$!#$ - you already fixed that problem???? | 14:14 |
| will your fix be avail? | ||
| I need to use that patch - our advanced search isn't recognizing phrase searches. | ||
| <BillDueber> | ndushay: Well, part of the issue is that not everyone agrees it's a problem. I imagine if you're not working with something as complex as bib data, what they're doing is smart. | |
| I'll build a patch against 1.4 | 14:15 | |
| <ndushay> | i'm not sure I fully understand your special field | |
| <BillDueber> | ndushay: My confidence that I'm not doing something stupid is ....low. | |
| <ndushay> | no wait, i do understand | |
| you're saying "if this is an EXACT match with the words in the exact-match field, then boost the heck out of it" | ||
| i got around that by | 14:16 | |
| having unstemmed and stemmed versions of fields | ||
| boosting the unstemmed higher | ||
| and really boosting in pf | ||
| not sure if that's better or not. | ||
| <BillDueber> | ndushay: It should be the same, depending on how aggressive you are about eliminating punctuation and diacritics and such. | 14:17 |
| <ndushay> | right. we get rid of that stuff. | |
| i can hardly wait to tackle CJK, though. NOT | ||
| <BillDueber> | ndushay: Yeah, I'm not sure how much of the magic I'm doing is necessary with a move to dismax. | |
| Need to think hard about it, and my head just isn't in the game today. | 14:18 | |
| <ndushay> | BillDueber: let dismax magic work for you | |
| <BillDueber> | ndushay: Right. I just need to make sure I'm not losing anything. | 14:19 |
| <ndushay> | is this vufind? | |
| <BillDueber> | ndushay: Only kindof. | |
| :-) | ||
| <ndushay> | is there a way to do relevancy testing, the way we use cucumber for our blacklight relevancy testing? | |
| 'cause that would be the best way to tell. | ||
| <BillDueber> | ndushay: Sure. Get two people who agree on relevancy for a given search and get back to me. | |
| <ndushay> | i have tests like this: | |
| "results should have these 3 resources in top 5 hits" | 14:20 | |
| I get the specifics from feedback emails - every time someone complains about a particular search | ||
| <BillDueber> | ndushay: Except sometimes they're wrong. | |
| <ndushay> | and also when i am trying to figure out certain things and have to find my own searches. | |
| sure - but I let OTHER librarians catch that | ||
| or I figure it out and we negotiate. | ||
| <BillDueber> | ndushay: I've got one prof who keeps insisting that a search on 'mind' should have the journal show up first -- except that's not the full name of the jouranl, and there are plenty of items whose entire title is 'Mind'. | 14:21 |
| <ndushay> | ah | |
| that's a query time boost fix. | ||
| <BillDueber> | ndushay: But, of course, that's an outlier. | |
| ndushay: And only if he's logged in :-) | ||
| <ndushay> | right! "if jerkface is searching for X, then use a function(?) query to manipulate the results this way" | |
| right to his IP address. | 14:22 | |
| heh. | ||
| a new standard for library discovery software. | ||
| actually, it would be kinda cool if you could adjust query time boost in your personal profile ... | ||
| <BillDueber> | ndushay: Yeah. Esp. for subjects or our High Level Browse stuff -- general stuff like that. | 14:23 |
| * ndushay leaves | 14:39 | |
| * ndushay joins | ||
| <cbeer> | not sure if any of you guys are doing this, but with the solr DIH, is it possible to run an xsl transform using the FileDataSource? I see some stuff about xsl + urldatasource or the xpath processor, but.. | 14:59 |
| * g8tor leaves | 16:35 | |
| * tachyonwill_ leaves | 17:00 | |
| * BillDueber leaves | 17:02 | |
| * jkeck leaves | 17:56 | |
| * jkeck joins | 17:57 | |
| * jkeck leaves | 19:28 | |
| * cbeer_ joins | 19:31 | |
| * wlayton joins | 20:37 | |
| * cbeer_ leaves | 21:35 | |
| * ndushay leaves | 21:44 | |
| * rsinger leaves | 22:05 | |
| * wlayton leaves | 23:21 | |
Generated by Sualtam