| * bess leaves | 01:30 | |
| * erikhatcher joins | 04:09 | |
| * erikhatcher leaves | 04:24 | |
| * rduplain joins | 08:15 | |
| * erikhatcher joins | 08:28 | |
| * rsinger leaves | 08:59 | |
| * BillDueber joins | 09:13 | |
| * rsinger joins | ||
| * jaron_ joins | 10:55 | |
| * rduplain leaves | 11:36 | |
| * bess joins | 11:46 | |
| * jrochkind joins | 11:52 | |
| <cbeer_> | hm.. they apparently mentioned blacklight on the fedora committers call.. is anyone here on the call to know what was said? | 12:12 |
| <bess> | no, but now I'm really curious | 12:13 |
| hang on, Paul Pound @ UPEI might be on that call… let me ask him | ||
| <cbeer_> | i should get better at getting on these calls | 12:14 |
| * jkeck joins | 12:24 | |
| * mattmitchell joins | 12:40 | |
| jrochkind: you around? | ||
| your gems problem, might be something in rsolr-ext i can help you with. | 12:41 | |
| <jkeck> | mattmitchell: about the modularization of AppHelper etc. | |
| <mattmitchell> | jkeck: oh yeah, how are things going? | |
| <jkeck> | mattmitchell: I realized that I can't actually test it accurately because the problem mainly exists in our install of BL that is operating SearchWorks. | 12:42 |
| mattmitchell: fresh install of BL doesn't have the problem for me. | ||
| <mattmitchell> | jkeck: well thats good news. | |
| jkeck: you could still try the technique though if you wanted | 12:43 | |
| jkeck: on your searchworks copy | ||
| <jkeck> | mattmitchell: so I'm gonna have to play around with updating SW from that branch. | |
| <mattmitchell> | jkeck: oh i see. sounds good, let me know if i can help. | 12:44 |
| <jkeck> | mattmitchell: will do. Don't know when I'll do that exactly, you know, with code4lib coming up fast. | |
| <erikhatcher> | i'm getting ready to build a new index with solrmarc.... should i use solrmarc trunk as the best starting point? | 12:45 |
| * jkeck leaves | 12:46 | |
| <mattmitchell> | erikhatcher: we're about ready to pull in 2.1 to the BL codebase. Not sure how diff the trunk is though. | |
| erikhatcher: hey, the way that SolrMarc talks to Solr... is it similar to SolrDirectConnection ? | 12:47 | |
| <erikhatcher> | it depends.... it can use EmbeddedSolrServer or over HTTP | |
| <mattmitchell> | erikhatcher: OK, so is EmbeddedSolrServer that different from SolrDirectConnection? | 12:48 |
| <erikhatcher> | i think embedded and direct are mostly the same | 12:49 |
| just different in how you make requests | ||
| with direct being for HTTP-like requests | ||
| <mattmitchell> | erikhatcher: OK that's what I kinda though too. Like SolrDirectConnection was a wrapper around Embedded, to make "http" requests easier. | |
| erikhatcher: so rsolr-direct, combined with the marc gem and your mapper technique could compete with SolrMarc :) | 12:51 | |
| <erikhatcher> | it's not quite a wrapper, different code, but does use SolrCore and friends directly, like Embedded | |
| yes, indeed | ||
| <mattmitchell> | erikhatcher: ok nice | |
| <erikhatcher> | however, i'm still not down with non-http indexing, mainly because of the shuffle games that need to be played to push an index to production and being able to offload indexing to different/several machines, and being able to do sharded indexing, etc | 12:52 |
| <mattmitchell> | erikhatcher: right, i guess it would depend on your setup/app needs. | 12:57 |
| <erikhatcher> | i'd be curious to see some real performance numbers between direct and streaming update server over http | 12:59 |
| <BillDueber> | mattmitchell: I'm gonna post about using streaming update solr server via jruby in the next few hours. I'm not sure how much faster it is, partially because I'm threading it and I don't think solrmarc threads. But I'm getting a 3-4 times speed increase. | 13:01 |
| mattmitchell: But I haven't reproduced all the custom functions I need yet, either. | ||
| <mattmitchell> | BillDueber: awesome! i've been messing around with that too. i'd love to see what you come up with. | 13:02 |
| <BillDueber> | mattmitchell: I'll ping you when I'm done posting. Gems are all there, though, on gemcutter -- jruby_streaming_update_solr_server, jruby_producer_consumer, and marc4j4r | |
| <erikhatcher> | marc4j4r - beautiful! | 13:03 |
| <mattmitchell> | BillDueber: That's really great. Can't wait to see it. | |
| <BillDueber> | erikhatcher: Just a little syntactic sugar. Nothing fancy. I *did* go against my personal desires and made [] work like it does for ruby-marc, where record['035'] gives you just the *first* 035. Drives me crazy, but there's value to following the existing API in the ruby world. | 13:04 |
| <erikhatcher> | #[] - nice. | 13:05 |
| but yeah, an array versus a single value, no clear winner there | ||
| <BillDueber> | Well, the other way I could always do rec['035'][0]. Oh well. Spilt milk and all. | 13:06 |
| * ndushay leaves | 13:27 | |
| * erikhatcher leaves | 13:39 | |
| * mattmitchell leaves | 13:44 | |
| * mattmitchell joins | 13:50 | |
| * jkeck joins | 14:04 | |
| mattmitchell: are you around? | ||
| <mattmitchell> | jkeck: hey yeah | 14:05 |
| <jkeck> | mattmitchell: I'm trying to write a test for the solr_helper to fix that bug, but am having trouble mimicing the qt parameter being set in the URL. | 14:06 |
| mattmitchell: Any thoughts? | ||
| <mattmitchell> | jkeck: hmm, you mean in your tests? can you post a code example? | 14:07 |
| jkeck: let me find that email... | ||
| <jkeck> | mattmitchell: solr_helper_spec.rb | |
| describe "overriding of qt parameter" do | 14:09 | |
| it "should return the correct parameter" do | ||
| params[:qt] = "override" | ||
| @solr_helper.solr_search_params.should == "override" | ||
| end | ||
| end | ||
| <mattmitchell> | jkeck: what happens with that? | 14:10 |
| <jkeck> | It fails. | |
| mattmitchell: it fails | 14:11 | |
| <mattmitchell> | jkeck: wait... @solr_helper.solr_search_params would be a hash right? | |
| <jkeck> | mattmitchell: sorry, that was just debugging code. It prints out the entire hash when I do that, which is why I have it that way. Here is another way that it fails, which will better illustrate. | 14:12 |
| <mattmitchell> | jkeck: ok, i'll try your code | 14:13 |
| <jkeck> | describe "overriding of qt parameter" do | |
| it "should return the correct parameter" do | ||
| params[:qt] = "override" | ||
| @solr_helper.solr_search_params[:qt].should == "override" | ||
| end | ||
| end | ||
| mattmitchell: and that is after making the fix to solr_helper.rb | ||
| <mattmitchell> | jkeck: looks like a scope thing with "params". Try this: @solr_helper.params[:qt] = "override" | 14:14 |
| <jkeck> | mattmitchell: sweet. You're the best! | 14:15 |
| <mattmitchell> | jkeck: awesome! | |
| <jrochkind> | I can look at all this stuff later this afternoon, since it's my code, probably my fault. Have a meeting soon though. | 14:17 |
| Oh, wait, did you guys figure it out? :) I'm still confused about what's going on with your problem. | 14:18 | |
| It remains astounding to me that the tests I wrote didn't cover that test, I could swear I wrote test that did. Oh well, iterative development indeed. | ||
| It may be because in general you aren't expected to send a :qt in the request params anymore, that's really just a backwards compatibility thing, the request params normally includes a search_field now instead. | 14:20 | |
| jkeck: I'm curious about your use case where you want to send a qt in the request params? | ||
| <jkeck> | jrochkind: Advanced Search | 14:27 |
| jrochkind: from a separate plugin | ||
| <jrochkind> | aha. I wonder if it's better to try to use the new method where you instead of passing a raw qt, you refer to a search_field defined elsewhere? | |
| But up to you. | ||
| of course. | ||
| <jkeck> | jrochkind: either way, it needs to be possible to pass the qt, and there is code intended for that, but there was a typo. It needs to be fixed no matter what. | 14:28 |
| <jrochkind> | okeydoke, sounds good, thanks for figuring it out. | |
| in the code i wrote, the explicit passing of the qt was more of a legacy thing, I hadn't realized it still needed to be possible going forward, but agreed it was obviously a bug that it didn't work. | ||
| <jkeck> | jrochkind: what I was worried about was it showing up in the drop down. To be honest, I haven't had enough time to look diligently through your search_fields code, but I got that impression through my skimming. | |
| <jrochkind> | True, right now it would show up in the drop-down if you just included it in the search_fields list. | 14:29 |
| <jkeck> | jrochkind: and we don't want that. | |
| <jrochkind> | There could be another list of advanced_search_fields that is available for lookup but doesn't automatically show up in the drop down. Or the things listed in the search_fields could have a possible "show_in_drop_down: false" or something. But just brainstorming, what ever makes sense to you, go for it. | |
| Can always be further enhanced later if there's a reason for it. | 14:30 | |
| <jkeck> | Exactly. | |
| <jrochkind> | jkeck: thinking about a plugin to do this functionality, I've been curious about how you're going to have a plugin make BL generate it's solr_params differently. But I guess I'll see the code when it's there! | 14:31 |
| <jkeck> | jrochkind: magic! Just kidding. It's actually mostly going to be done in solr_config I think. That's why I need to pass the new qt. | 14:32 |
| jrochkind: the plugin is really more responsible for supplying some config, the UI, and the query parser. | ||
| <jrochkind> | mix-in over-riding a part of core Blacklight to do it? Or what? I was wondering if maybe the core blacklight could profitably be rewritten to provide clearer hooks for a plug-in to provide custom logic. | 14:33 |
| Or wait, you're going to try to have the form submit with all the solr params in the request to blacklight? | ||
| <jkeck> | jrochkind: no. | |
| <jrochkind> | yeah, it's how you inject your own query parser into BL via a plugin I'm curious about. | |
| i think. if query parser means what I think it does. But just curious, I can wait and see if this conversation grows confusing, heh. | 14:34 | |
| <jkeck> | jrochkind: have you seen the Advanced Search in SearchWorks? http://searchworks.stanford.edu/advanced ? | 14:36 |
| <jrochkind> | I have, yeah. | |
| <jkeck> | Ok, so the plugin will provide the form and a small advanced controller to get the URL easily. Then it will send the query to CatalogController#index which is overriden by the plugin. | 14:38 |
| To pass what the query parser interprets from the form submission to the solr_helper method that returns results. | ||
| <jrochkind> | aha, override CatalogController#index, that's what I was curious about. Cool. | 14:39 |
| if there were another plugin that ALSO wanted to over-ride CatalogController#index... then we'd have to start thinking about architecting that a little bit more intentionally, provide a hook for plugins or something. | 14:40 | |
| I mean, if you wanted em both at once. | ||
| <jkeck> | Yeah, if people are already overriding that at the app level already then they'll need to look closely at what the plugin is doing. For now. | |
| <jrochkind> | cool. good luck! :) | |
| <jkeck> | Thanks. | 14:41 |
| <BillDueber> | Just posted about using jruby to push stuff to solr: http://robotlibrarian.billdueber.com/indexing-data-into-solr-via-jruby-with-threads | 15:04 |
| Feedback, please! | ||
| <mattmitchell> | BillDueber: wow. need some time to grok all that. can't wait to dig in though! | 15:13 |
| BillDueber: looks really useful, and simple to use | ||
| <BillDueber> | mattmitchell: Focus on the first ....counting...errr | |
| mattmitchell: Just up to the first suss.commit. The rest is gravy. | 15:14 | |
| mattmitchell: It's really ridiculously easy. | ||
| <mattmitchell> | BillDueber: ok good to know, i'm sure i'll learn a lot by going through it. | 15:15 |
| * BillDueber leaves | 15:21 | |
| * erikhatcher joins | 15:22 | |
| <jrochkind> | I still don't entirely understand how to run jruby code, heh. | 16:02 |
| * wlayton joins | 16:07 | |
| * ndushay joins | 16:25 | |
| erikhatcher: I'm late to the party, but use solrmarc 2.1 for your codebase | 16:26 | |
| <cbeer_> | bess: does hydra have anything public facing yet? | 16:37 |
| <bess> | cbeer: not really. You can look at UVA's virgo beta, or at Hull's repository | |
| and there's a page on the fedora commons wiki | ||
| that's about it | ||
| there isn't a separate codebase | ||
| it's just blacklight pointed at fedora right now | 16:38 | |
| <cbeer_> | ok; i'm just trying to come up with a list of cool digital libraries projects to help convince some folk that universities are doing cool things | |
| <MrDys> | bess: is there going to be a separate codebase? | 16:39 |
| <bess> | MrDys: not clear at this point | |
| MrDys: we're not forking BL, that's for sure | ||
| MrDys: but part of the puzzle is ActiveFedora, and it's a separate codebase | ||
| not sure what *hydra* is right now, except a loose affiliation of sevreral OSS projects | 16:40 | |
| <MrDys> | maybe that's all it should be. tie it together with a nice installer and call it a project | |
| <bess> | yup | 16:41 |
| that's my thinking too | ||
| <mattmitchell> | ... detangling sirsi commands is a form of punishment. oh, it's 5:00! | 17:12 |
| <jrochkind> | mattmitchell: at least you aren't trying to install a new version of SirsiDynix Horizon Information Portal. Oh my god. | 17:13 |
| <mattmitchell> | jrochkind: oh i'm so sorry. i can only imagine. | 17:14 |
| <jrochkind> | hell on earth, seriously. | 17:17 |
| Nothing works, and apparently help with installing a new version is somehow not covered by our support contract, it's an extra billable service. HUH? | ||
| They actually promissed us that Horizon had been end-of-lifed nad there'd never be a new version again! I LIKED that promise. But they changed their mind and started releasing new versions anyway. | 17:18 | |
| So, okay, mattmitchell if you're still there. I'm slowly installing the new gem dependencies for the newest BL's test stuff. | 17:19 | |
| Once I do.... are there any special instructions I need (written somewhere) on how to prepare the test solr index so I can run the tests? Or if I've done the submodule update, I'm done? | 17:20 | |
| <mattmitchell> | jrochkind: hmm, well i wrote a simple task that indexes the index, which we can re-evaluate once i merge in your solr marc patch goodness... | |
| jrochkind: the command is "rake solr:marc:index_test_data" | 17:21 | |
| <jrochkind> | mattmitchell: okay, so if I have the submodules updated, then I can just run "rake solr:marc:index_test_data", and then run the tests, and all should be well? Sweet. | |
| We should probably document that somewhere. | ||
| <mattmitchell> | jrochkind: yes, that should do it. | |
| jrochkind: it is let me see... | ||
| <jrochkind> | mattmitchell: PS: is the mislav-hanna thing yours? | |
| <mattmitchell> | jrochkind: oh now, that's something created by the guy who built will_paginate | 17:22 |
| <jrochkind> | Huh, okay. | |
| <mattmitchell> | jrochkind: here you go, bottom of the page: http://github.com/projectblacklight/blacklight/blob/master/doc/README_INSTALLATION.rdoc | |
| <jrochkind> | I'm getting annoyed at how every time I start up anything that uses that gem, I get the lines cluttering my stderr (it's not just me, right? you're all seeing this too?) ruby/gems/1.8/gems/mislav-hanna-0.1.11/lib/hanna/version.rb:3: warning: already initialized constant VERSION | |
| mattmitchell++ | 17:23 | |
| <mattmitchell> | jrochkind: yeah, that's pretty annoying. i don't know why he hsan't fixed that. | |
| <jrochkind> | In my own git repo that uses submodules, I'm _constantly_ forgetting to run 'git submodule update' and confusing myself. I predict you all will start getting bit by that too. | 17:24 |
| Every time you do a git pull, you have to also do a 'git submodule update', basically. Otherwise, if something in a submodule changed, you won't see it. And how do you know if something in a submodule changed? You don't. So you kind of have to run submodule update every time. | ||
| In other news, it is is odd that we have both 'hanna' and 'mislav-hanna' as dependencies? | 17:25 | |
| <mattmitchell> | jrochkind: interesting, i've never updated the submodules because no one ever changes anything, but still a valid point. | 17:26 |
| jrochkind: yeah, those deps should not be a requirement | ||
| <jrochkind> | It's not really a point, it's just a gotcha I'm warning you about, heh. | |
| <mattmitchell> | jrochkind: thanks, good to know! | |
| <jrochkind> | oh, and those are not required to run the app. They're just listed as unmet dependencies if I try to run tests. I just thought it was odd because I'd figure mislav-hanna was a forked version of hanna, yet I need both? | 17:27 |
| they are required for running tests, not for starting up the app. | ||
| <mattmitchell> | jrochkind: oh i see. the mislav- thing is old. that's when github hosted gems. | |
| jrochkind: we can now just specify "hanna" - which is hosted by gemcutter | ||
| <jrochkind> | mattmitchell: The mislav thing is the one that outputs that garbage to stderr too. Maybe we don't really need it anymore? | |
| mattmitchell: So maybe remove mislav-hanna from the config.gem list? | 17:28 | |
| <mattmitchell> | jrochkind: they should be required when generating docs, that's it really | |
| jrochkind: maybe, but if an rdoc task explicitly uses it then we should be careful. heck, just throw a require in the rdoc tasks? | ||
| jrochkind: then remove the dep | 17:29 | |
| <jrochkind> | I think that's what I'd do, I dunno. Right now it's listed only for TEST environment anyway, so you'll have the same problem if you're running that rdoc task in anything but test environnment. | |
| mainly I'm just annoyed by the stderr garbage on every app or task startup. | ||
| <mattmitchell> | jrochkind: right, annoying it is. | 17:30 |
| <jrochkind> | and also that every time I up blacklight, I end up needing to spend 30 minutes installing gem dependencies, heh. If some of those aren't really required.... it would save us all time. Like hanna seems to be listed TWICE, once as 'hanna' and once as 'mislav-hanna'. And I had to install it twice to satisfy the dependencies. | 17:31 |
| "rake solr:marc:index_test_data" just worked. hooray! | 17:32 | |
| <mattmitchell> | jrochkind: yeah mislav-hanna shouldn't be there. | |
| jrochkind: nice! :) | ||
| <jrochkind> | and 'rake test' seems to be working. I like how someone somewhere over-rode the default 'rake test' so it'll start the test solr server, so I didn't need to figure out a special BL way to run tests, I could just use 'rake test'. Thanks! | 17:33 |
| * bess leaves | ||
| <mattmitchell> | jrochkind: oh really? try turning off solr and then running it. i didn't think that worked, but that would be nice. | 17:34 |
| <jrochkind> | it definitely works. | |
| I have no idea why. But I've been doing it for a while. | ||
| <mattmitchell> | jrochkind: great | |
| <jrochkind> | It even outputs lines like "starting up the test solr server" in the stdout. | |
| It does start it up once for spec, then stop it, then start it up again for cucumber, then stop it. Which is a bit slower than if it just left it running, but oh well. | 17:35 | |
| okay, and the tests run and pass. Phew! | 17:36 | |
| <mattmitchell> | jrochkind: ahh yeah, the Rakefile - http://github.com/projectblacklight/blacklight/blob/master/Rakefile | |
| <jrochkind> | So, NOW I'm ready to start developing and try to finish the 'more facet values link' stuff. Matt, maybe tomorrow you'll have some time to consult on it? | |
| <mattmitchell> | jrochkind: i'll do my best yeah, couple of meetings throughout the day but shouldn't be a problem. | |
| <jrochkind> | cool, hope to catch ya. | |
| Or if you're planning on staying at work for a while, we could chat now. But if you're not as unlucky as me, hopefully you're about to go home. | 17:37 | |
| <mattmitchell> | jrochkind: yeah, well i think my wife is well and ready for me to stop now :) working at home today. | 17:38 |
| <jrochkind> | cool cool, catch you later. | |
| <mattmitchell> | jrochkind: ok , see ya! | |
| * mattmitchell leaves | ||
| * ndushay leaves | 17:56 | |
| * bess joins | ||
| * ndushay joins | 17:57 | |
| * jkeck leaves | 17:58 | |
| * ndushay leaves | 18:02 | |
| * jaron_ leaves | 18:08 | |
| * ndushay joins | 18:23 | |
| * erikhatcher leaves | 18:28 | |
| * bess leaves | 18:49 | |
| * ndushay leaves | 18:51 | |
| * ndushay joins | 18:52 | |
| * jkeck joins | 19:36 | |
| * bess joins | 19:40 | |
| * cbeer leaves | 20:02 | |
| * wlayton leaves | 20:15 | |
| * bess leaves | 20:39 | |
| * ndushay leaves | 20:48 | |
| * cbeer joins | 21:12 | |
| * jkeck leaves | 21:14 | |
| * bess joins | 21:22 | |
| * jvenner joins | 21:25 | |
| * erikhatcher joins | 21:28 | |
| * ndushay joins | 22:01 | |
| * bess leaves | 22:09 | |
| * bess joins | 22:20 | |
| * bess leaves | 22:23 | |
| * Naomi joins | 22:32 | |
| * ndushay leaves | 22:36 | |
| <jvenner> | Is anyone working on the solr 1.4 style range facets? | 22:59 |
| I am running 2.4 | ||
Generated by Sualtam