| * erikhatcher joins | 03:57 | |
| * cbeer joins | 07:51 | |
| * erikhatcher leaves | 07:53 | |
| * cbeer leaves | 08:11 | |
| * rsinger joins | 08:20 | |
| * erikhatcher joins | 08:21 | |
| * cbeer joins | 08:38 | |
| * ndushay joins | 08:44 | |
| * cbeer leaves | ||
| * rsinger leaves | 08:48 | |
| * bess joins | 08:56 | |
| * cbeer joins | 08:57 | |
| * rsinger joins | 08:58 | |
| * BillDueber joins | 09:07 | |
| * jkeck joins | 09:14 | |
| * bess leaves | 09:23 | |
| * jkeck leaves | 09:30 | |
| * jamieorc joins | 09:45 | |
| * rduplain joins | 10:03 | |
| * tachyonwill_ joins | 10:05 | |
| * jrochkind joins | 10:36 | |
| * rsinger leaves | ||
| * rsinger joins | 10:39 | |
| * bess joins | 10:53 | |
| * jkeck joins | ||
| * cbeer leaves | 11:01 | |
| * cbeer joins | ||
| <rduplain> | wow, deploying multiple Blacklights at sub-urls on a single Apache httpd vhost was *really easy*... much easier than I expected. using mod_rails/passenger. | 11:11 |
| <bess> | rduplain++ # awesome! | |
| <rduplain> | deployed multiple Solr instances under Tomcat | 11:15 |
| been in a mode lately of finding a deployment configuration the sysadmins can appreciate around here | ||
| <bess> | :) | 11:16 |
| <rduplain> | we have some Tomcat devs at another site, and one of our main web admins happily supports Tomcat | 11:20 |
| and we use Apache httpd everywhere | ||
| had a little panic over nginx the other day | ||
| that puts us at: Apache httpd + mod_rails + Tomcat = win | 11:21 | |
| <erikhatcher> | rduplain: you rock | 11:22 |
| <rduplain> | erikhatcher: haha, aw shucks | 11:23 |
| config ended up being pretty simple... I was thinking about putting together a simple appliance to provide an example deployment | 11:24 | |
| in other news, I'll be spreading the good word on Blacklight at an upcoming astronomy conference (one of the bigger software for astronomy conferences), http://bit.ly/9bQJUc | 11:31 | |
| do a find: "blacklight" on that page, and you'll see the talk | 11:32 | |
| <bess> | sweet! | 11:33 |
| <jamieorc> | rduplain: Apache httpd + mod_rails + Tomcat is a good combo. I've had great success with all | |
| Passenger is just incredible | 11:34 | |
| <rduplain> | jamieorc: enjoying the stack so far | |
| <jamieorc> | I use it for development on the mac too | |
| Passenger Pane is great. It means my dev apps are always up and running | ||
| but it's mac only, aren't you developing on linux? | 11:35 | |
| <rduplain> | jamieorc: yes, use Ubuntu or Debian in most cases... deploying on RedHat | |
| <erikhatcher> | rduplain: nice talk you're doing at that conference, way cool | 11:36 |
| bess - can we get things like this listed on the blacklight site? | ||
| <rduplain> | erikhatcher: thanks, looking forward to it. should be one of my actually useful conference talks/papers | |
| <bess> | erikhatcher: sure! | 11:37 |
| <jamieorc> | rduplain: you might checkout the RailsMachine capistrano gem. Automates some of the apache/passenger config from capistrano | 11:40 |
| <rduplain> | jamieorc: http://railsmachine.com/projects/ <-- ?. Cool. Setting up gems was probably the "weird" part of deployment for me. Was looking for something like Python's virtualenv. | 11:41 |
| * ndushay leaves | 11:52 | |
| * cbeer leaves | ||
| * bess leaves | 11:53 | |
| * bess joins | ||
| * rsinger leaves | 11:57 | |
| * bess leaves | 11:58 | |
| <BillDueber> | Hey, has anyone here done any profiling of solrmarc? I'm doing similar stuff via jruby and finding out that marc4j object -> marc-xml string is ridiculously expensive -- like 20% of my processing time. Can this be right? | 12:02 |
| <erikhatcher> | BillDueber: why are you going to marc-xml? | 12:29 |
| <BillDueber> | erikhatcher: We pull stuff out of it for display. Easier than having to decide ahead-of-time everything that we might want to display and storing it all as separate entities. | 12:30 |
| <erikhatcher> | right, but why XML versus binary marc? | |
| <BillDueber> | erikhatcher: My JRuby-based indexer also keeps dying on me after I get a bunch of records in -- 4.2M or so with my current setup. No errors or anything, it all just stops dead. No idea what's going on. | 12:31 |
| erikhatcher: Binary marc has length limitations. We have some big reocrds :-) | ||
| <erikhatcher> | for example, UVa stores binary marc | |
| been working with bob on fixing that a bit though... using base64 enoding | ||
| <BillDueber> | erikhatcher: I'm pretty sure they use MARC-XML | |
| <erikhatcher> | hmm.... i don't think so, at least not in the tinkerings he and i have been working on today together | |
| <BillDueber> | erikhatcher: The problem isn't the encoding -- it's that the structure of a marc binary record is limited to 10K, and some records have more than that. The length of the record needs to be stored in the "directory" portion of the marc record, and if it's more than four digits, everything breaks. | 12:32 |
| (by 10K I mean '9999 bytes') | ||
| <erikhatcher> | ah, ok | |
| i'll ask Bob when he returns from lunch | 12:33 | |
| <BillDueber> | erikhatcher: yeah, fundamental limitation of binary marc. | |
| erikhatcher: Big records are a pain. | ||
| erikhatcher: Any hints on how to track down my sudden stoppage? I'm assuming it's either a thread deadlock or a memory leak, but wouldn't one or the other eventually throw an error of some sort? | 12:34 | |
| (and thread deadlock seems unlikley since it happens at pretty much teh same point in the index cycle each time) | ||
| * cbeer joins | 12:50 | |
| * bess joins | 13:03 | |
| * rsinger joins | 13:06 | |
| <erikhatcher> | BillDueber: using streaming update server? | 13:07 |
| <BillDueber> | erikhatcher: Yup. Committing every 100K documents | |
| * cbeer leaves | 13:08 | |
| <erikhatcher> | probably this: https://issues.apache.org/jira/browse/SOLR-1711 | |
| try a nightly build of SolrJ | ||
| * cbeer joins | ||
| * jkeck leaves | 13:09 | |
| * cbeer leaves | ||
| * cbeer joins | ||
| * ndushay joins | 13:24 | |
| * jkeck joins | ||
| * ndushay leaves | 13:30 | |
| * ndushay joins | 13:31 | |
| * ndushay leaves | 13:34 | |
| * ndushay joins | ||
| * ndushay leaves | 13:54 | |
| <BillDueber> | erikhatcher++ # Thanks -- will give it a try! | 13:55 |
| * ndushay joins | 14:30 | |
| * rsinger leaves | 14:34 | |
| * bess leaves | 14:37 | |
| * rsinger joins | 14:48 | |
| * jrochkind leaves | 14:51 | |
| * jrochkind joins | ||
| * rduplain leaves | 14:52 | |
| * bess joins | 15:24 | |
| * jrochkind leaves | 15:43 | |
| * jrochkind joins | ||
| * jrochkind leaves | 15:44 | |
| * jrochkind joins | 15:45 | |
| * tachyonwill_ leaves | 15:50 | |
| * cbeer leaves | 16:15 | |
| * bess leaves | ||
| * ndushay leaves | 16:16 | |
| * rsinger leaves | ||
| * erikhatcher leaves | 16:17 | |
| * jrochkind leaves | 16:18 | |
| * jkeck leaves | 16:20 | |
| * jkeck joins | 16:28 | |
| * jkeck leaves | 16:33 | |
| * jkeck joins | 16:35 | |
| * BillDueber leaves | 16:51 | |
| * BillDueber joins | ||
| * jkeck leaves | 17:38 | |
| * bess joins | 17:42 | |
| * rsinger joins | 17:48 | |
| * bess leaves | 17:49 | |
| * jamieorc leaves | 18:23 | |
| * erikhatcher joins | 20:12 | |
| * jrochkind joins | 21:55 | |
| * ndushay joins | 22:14 | |
| * erikhatcher leaves | 22:29 | |
Generated by Sualtam