Log of the #blacklight channel on chat.freenode.net

Using timezone: GMT-05:00
* ndushay leaves00:44
* jaron leaves
* rsinger leaves
* BillDueber leaves
* ndushay joins00:46
* jaron joins
* BillDueber joins
* rsinger joins
* ndushay leaves02:04
* cbeer joins07:01
* jaron_ joins07:25
* jaron_ leaves07:29
* cbeer leaves08:48
* bess joins09:04
* jamieorc joins
* bess leaves09:14
* tachyonwill_ joins09:17
* rsinger leaves09:19
* bess joins10:07
* g8tor joins10:17
Hello World!
<bess>hi, g8tor10:21
I've got Will working on some changes to make the docs more obvious
but I think we're ready to release
what do you think?
<g8tor>yeah I think so10:22
I'm just concerned that we have not setteled ona dev cycle
<bess>oh we haven't?
<g8tor>the debate is still going on and we need to decide on branches vs tags vs a combo of the two etc
maybe I missed it10:23
jamieorc: I just saw your message to the list and I agree. I think I may have been thinking that branches were more expensive than they actually are10:24
<bess>maybe that's something we can tackle in the friday call?
<g8tor>I'm goint to miss the call for the funeral
<bess>oh
I'm sorry. I don't have any idea what happened, but a funeral is never good news.
<g8tor>Thanks, I'll try to make the call I think things start at 2 or so on Friday in Miami10:25
In any event, I thing Matt's suggestion that branches be temorary i.e used for a feture like the advanced search and once complete and merged into master and tagged as version 2.X.X the branch is deleted10:27
I think that works best10:28
<bess>ok10:29
that seems fine to me
<g8tor>Is there anything else you need me to do to release today?10:32
<bess>g8tor: I don't think so
<g8tor>good I feel like we all did some serious work this last two weeks this is good10:33
* bess leaves
* darthmolly joins10:53
<g8tor>hi darthmolly (have I told you I just love that nick?)10:56
* rsinger joins11:12
<darthmolly>g8tor: sorry, was away for a bit. thanks!11:19
<g8tor>np
* ndushay joins11:27
* ndushay leaves11:39
* cbeer joins11:44
* cbeer leaves12:03
* ndushay joins12:18
g8tor: i saw your question after you were gone yesterday12:40
what is the current email address for github projectblacklight account?12:41
* tachyonwill_ leaves13:07
* bess joins13:50
* darthmolly leaves13:51
* ndushay leaves13:56
* darthmolly joins14:00
* ndushay joins14:06
* bess_away leaves14:11
* cbeer joins14:15
* cbeer leaves
* cbeer joins14:16
* g8tor leaves14:26
* darthmolly_ joins15:30
* darthmolly leaves15:31
* darthmolly joins15:33
* bess joins15:37
* darthmolly_ leaves15:51
* ndushay leaves16:38
* bess leaves17:28
* darthmolly leaves17:30
* jamieorc leaves17:44
* ndushay joins17:54
* bess joins18:51
* ndushay leaves18:59
* ndushay joins20:38
* g8tor joins20:47
Hello All
<bess>hi, g8tor
<g8tor>so????? we released?
<bess>g8tor: not yet! working on it!20:48
* cbeer leaves
<g8tor>oh ok
* alxp joins20:49
<bess>ndushay, meet alxp
<alxp>hi
<ndushay>hi
<bess>g8tor: we're going to talk about solrmarc for a bit okay? but I'm working on the 2.4 release, I promise.
<g8tor>I've been reading the list emails on my phone so I'm not sure if I got it all but the consensus is we will do branches for major releases with tags for bug fixes?
<ndushay>g8tor: as for the email for github - is it you personally?20:50
i think bl-committers is a good idea.
g8tor: yes, that's my take on branches/tags
<g8tor>ndushay: no i made a gmail account for it
<ndushay>i have to learn how to access the internet via cat -- i'm just not cool enough to hang out with you kids.
<g8tor>i can chane it if you and bess think its best
<bess>g8tor: no it's fine the way it is20:51
ndushay: alxp is trying to index some records he has exported from evergreen
<ndushay>really? what about the proverbial "run over by a truck" test?
ok.
<bess>and he's having trouble getting started
<alxp>yeah, I am trying to just use solrmarc on its own
<ndushay>ok.
<alxp>and am lost on how to configure it
<bess>ndushay: I think this is a good opportunity to whip the docs into shape20:52
<ndushay>(mutter mutter bob's complex auto-build mutter mutter)
<bess>ndushay: I know. This is how we fix things, by bringing in other people from other institutions.
<alxp>I have a fresh working solr installation running on tomcat
with the example schema and nothing changed
<ndushay>alxp: ok.
"example schema" --> schema that comes with vanilla solr?20:53
<alxp>and then I have the source solrmarc downloaded and am at the install questions part of the gettingstarted doc
ndushay: yes
<ndushay>alxp: ok
so you're here: http://code.google.com/p/solrmarc/wiki/GettingStarted
and you're running ant build and getting prompted?20:54
<alxp>that's right
<ndushay>ok.
fire away.
<alxp>ok ant init says "enter name of example configuration"20:55
<ndushay>bess: i was thinking having a link on the projectblacklight.org sidebar to irc archives would be grand.
<bess>ndushay: good call. I'm on it.
<ndushay>alxp: there are 4 examples to choose from
it may be that none of them will suit you20:56
<alxp>yeah.. now do those examples assume things about the structure of the marc records?
<ndushay>genericBlacklight may be the simplest.
well, marc records have structure
<alxp>because my marc records are exports from evergreen
I assume they're standard
<ndushay>they assume you have valid marc bib records
<alxp>but they have holdings info attached
<ndushay>how is the holdings info attached?
<alxp>as 901 type fields
<ndushay>for example, we currently have our holdings shoved in 999 fields.20:57
<alxp>so that should be fine I think
<ndushay>yes, should be fine.
<alxp>ok
I'll go genericBlacklight
<ndushay>ok
<alxp>i'll go with no site-specific code this go-round...20:58
<ndushay>yep
(all of this is easy to change later)
<alxp>ok now it's asking for the url of my solr server, here do I need to make a new solr instance or can I use the vanilla one?20:59
it's empty right now
<ndushay>use the vanilla one
<alxp>ok
<ndushay>it can even be http://localhost:8983/solr
<alxp>I have marc records in a couple of different formats hoping one would work, can I just go with BESTGUESS?21:00
sorry if you have to answer this kind of question over and over21:01
<ndushay>BESTGUESS is fine
<alxp>great
<ndushay>i am less familiar with that part of the code, but I think Bob set it up to make good guesses.21:02
<alxp> Properties files should be copied to the one-jar jar file that is produced (true, [false])
<ndushay>i said false - that allows you to fiddle the properties files later without having to monkey about with the jar
<alxp>ok, that's what I want, that makes sense now that you explain it21:03
<ndushay>did that prompt come after the one for marc encoding?21:04
i think we're missing it in the docs
<alxp>yes
ok build successful21:05
<ndushay>yay
* bess says yay from the peanut gallery
<g8tor>yay21:06
<ndushay>alxp: i'll bet you actually want to index something now.21:07
<alxp>haha yes
I can try the examples first
<ndushay>ok.
<alxp>can I just go ahead and run 'indexfile' with one of the test data?21:08
<ndushay>try running ant test and see what it says21:09
<alxp>(I think what was messing me up was because I want to try and use solrmarc without tying it to blacklight or vufind I was selecting 'none' at first and of course then the index config files were just empty)
<ndushay>i expect it will fail.
<alxp>ok
<ndushay>alxp: good point!
<alxp>UILD FAILED
./Users/al/dev/solrmarc/build.xml:1041: /Users/al/dev/solrmarc/eg/extra_data not found.21:10
'eg' was just the site.dir
<ndushay>make a folder "extra_data" manually within the eg directory21:11
i already reported that one to bob21:12
<alxp>ok
<ndushay>alxp: what is your full name and project
<alxp>ok successful
<ndushay>and may i write this up as a posting for the solrmarc list?
yay!
<alxp>sure
<ndushay>it ran a test?
<alxp>ndushay: Alexander O'Neill , VRE project with UPEI21:13
University of Prince Edward Island
oh sorry the ant was successful
but reports test failed
<ndushay>yeah
ok.
well, half of the battle, anyway ...
<alxp>haha21:14
the test is "test"
Fatal error: Unable to find specified properties file: demo_config.properties
<ndushay>ew. ok21:15
hang on21:16
<alxp>ok
<ndushay>alxp: hang on - distraction21:18
<alxp>no problem
* darthmolly joins21:24
<bess>alxp: when you say you're using the vanilla solr schema, do you mean the one that comes distibuted w/ solr? or the one that comes distributed with solrmarc? 21:28
<alxp>bess: I haven't touched it so at the moment it's the one that comes with solr
<bess>ndushay: doesn't he need to replace that with the one that comes with solrmarc? 21:29
I guess she's not back yet
alxp: try running the indexer
if it tells you it doesn't recognize certain field names it's b/c you're using the wrong schema file
<alxp>got an error, I'll pastebin it
<bess>ok
<alxp>http://pastebin.com/m230fda1d21:30
<bess>alxp: try running the indexer from inside the dist directory21:32
<ndushay>alxp: you still here?
<alxp>then it reports it can't find the config file
ndushay: yes I'm here
<ndushay>do you have an eg_config.properties file?21:33
<alxp>yes in the solrmarc/eg directory
which was what was in site.dir.properties
<ndushay>ok, look in there and see if it has
solr.indexer.properties = demo_index.properties
if so, change it to
solr.indexer.properties = eg_index.properties21:34
<alxp>yes it says demo_index.properties
<ndushay>chage it
<alxp>the solr.path is also pointing to some jetty directory, so I guess I have to change that to my tomcat?21:35
<ndushay>yes21:36
<alxp>yeah it says "solr.path = /Users/naomi/NGDE/blacklight/bl-demo/jetty/solr
"
<ndushay>yeah, that's my cruft.
<alxp>probably a bug
ok
ok changes done21:38
<ndushay>no, not a bug, just a side effect of how I work + bob's build script + what was checked into svn
<alxp>ahh ok
<ndushay>(crossing fingers)
<alxp>new error http://pastebin.com/d3fcd6acf21:39
that's trying an index, not running tests21:40
<bess>hey, progress!
<ndushay>ok.
can you pastie in:
1. eg_config.properties
2. eg/build_override.properties21:41
<alxp>ok
eg_config.properties http://pastebin.com/d8e1201a21:42
build_override.properties: http://pastebin.com/d3dc0af7c21:43
<ndushay>thanks21:44
what files are in your eg/translation_maps directory?
<alxp>callnumber_map.properties format_map.properties21:45
<ndushay>ok, good.
ok, try this21:47
uh ... hang on
<alxp>ok
<ndushay>in top level build.properties, is there a line:
trans.map.path = translation_maps
<alxp>yes21:48
<ndushay>hmm.21:49
ok
i'm trying to remember how it does this
<alxp>alright21:50
<ndushay>it's either something with how the ant auto-build sets a property21:52
or it's a bug i've encountered before
<alxp>ok
<ndushay>oh - wait - what did you do to index?21:53
<alxp>index where?
<ndushay>what did you do to get this error?21:54
ant XXX ?
<alxp>what are you looking at right now?
<ndushay>http://pastebin.com/d3fcd6acf21:55
<alxp>ahh, I just went ahead and tried to fun indexfile on test/data/selectedRecs.mrc
to see what would happen
<ndushay>what did you type at command line?
oh
i get it
bob wrote those scripts and they're handy ... but i don't use 'em.21:56
ok
<alxp>../dist/indexfile ../test/data/selectedRecs.mrc
<ndushay>ok
i *think* that b/c the properties files aren't bundled in
they need to be in the classpath21:57
<alxp>yeah I had to switch to the eg directory to run the scripts
<ndushay>ok.
another thought
run
ant dist
<alxp>ok
BUILD FAILED
Target "dist" does not exist in the project "SolrMarc Custom".
<ndushay>ant build21:58
sorry.
<alxp>ok that was successful
<ndushay>just for giggles, try indexfile again.
<alxp>ok
ERROR [main] (SolrIndexer.java:248) - Unable to find file containing specified translation map (callnumber_map.properties)21:59
<ndushay>same error
ok
i think the indexfile script will need to have the current directory on the classpath (?)22:00
to be honest, bob added the option to include or exclude the properties files in the jar recently22:01
and the default was to have them included.
so i don't think i've had to work with them excluded before.
alxp: are you comfortable mucking about with java classpaths?22:02
<alxp>I haven't done it in a while
but sure
<ndushay>ok.
actually, for giggles
try copying the callnumber_map.properties into the top level directory
i mean, it finds the eg_index.properties okay ...22:03
<alxp>ok
<ndushay>i'm not sure it would be a good thing if that fixes it ...
is it 11pm where you are?
<alxp>yes22:04
ok now it's looking for "language_map.properties)"
<bess>yay! progress!22:05
<alxp>which I see there's one in lib/solrmarc/translation_maps
<ndushay>ok, copy that one up
<alxp>ok
ok doing same for format_map.properties22:06
<ndushay>i've been writing an email with all of these hiccups to send to solrmarc list.
you should probably join the solrmarc list22:07
<alxp>Indexed 0 at a rate of about 0.0 per se
ok I'll do that
<ndushay>well, it tried
<alxp>NFO [main] (MarcImporter.java:585) - Using Solr data dir null22:08
do I need to put that in the config file?
<ndushay>does your solr dir have a /data subdir?
<alxp>yes
<ndushay>ok.22:09
hang on
try putting this in eg_config.properties
<alxp>ok
* darthmolly_ joins
<ndushay>solr.data.dir = /yer/solr/data22:10
<alxp>ok
<ndushay>i wouldn't expect it to need that, but ...
* darthmolly leaves22:11
<alxp>now re-run ant build?
<ndushay>i think you can re-run indexfile
if not, try ant build first22:12
<alxp>ok
getting errors22:13
<ndushay>new errors
?
<alxp>I think bess mentioned above that I need to replace solar's schema.xml, is this right?
<ndushay>probably
you could use the schema.xml and solrconfig.xml from genericBlacklight/solrConf
<alxp>they're 'unknown field' type errors so I'm guessing that's it
<bess>yes!
those are the errors I was talking about
that's a good sign
it's trying to index but solr is choking22:14
<alxp>I see in test/solr/conf' there are some files
inside eg
<ndushay>right
right - use those.
but keep the orig around w rename
<alxp>ok
<ndushay>just in case.
i know schema.xml and solrconfig.xml stuff pretty well.22:15
<alxp>ok, so now do I need to restart tomcat or anything?22:16
<ndushay>i'm not sure
i think there's a way to get solr to reload, but i haven't used tomcat in a while
<alxp>hopefully it won't hurt to do it
hang on
<g8tor>hey didn't someone say something on the commiters call (a while back) about moving to tomcat?
<ndushay>yes.
<g8tor>just checking22:17
<ndushay>jetty fooches some utf8 in the urls themselves, i believe (not in http responses)
somehow it just wasn't at the top of my priorities
:-)
ok - just sent email to solrmarc list22:18
<alxp>d'oh, tomcat got an error trying to deploy solr
<bess>d'oh!
what's the error? or are you already on top of it? 22:19
<alxp>looking at catalina.out right now
SEVERE: org.apache.solr.common.SolrException: Unknown Search Component: tvCompon22:20
ent
<bess>what?
<ndushay>what version of solr do you have?
actually
<alxp>is this expecting solr 1.4 or something?
1.3
<ndushay>for now
okay, genericblacklight is 1.422:21
but don't worry
go back to original solrconfig.xml
but keep new schema.xml
<alxp>ok
<ndushay>alxp: any progress?22:26
actually, i think my non-internet life may be calling.22:27
i think i have to take off.
<bess>ndushay: you have a non-internet life?
;)
* alxp leaves22:28
* g8tor leaves
* rsinger leaves
<ndushay>you can get help on east coast time from Bob H. if he's around tomorrow
bess: only occasionally
* alxp joins
* g8tor joins
* rsinger joins22:29
* ndushay leaves
<alxp>WARNING: No queryConverter defined, using default converter
Error: Problem invoking addField in SolrCoreProxy
ERROR [main] (MarcImporter.java:278) - ERROR:unknown field 'pub_date' at record count = 1
<g8tor>wwhat happened?Everyone just disconnected @ once?
<bess>irc glitch
alxp: okay, unknown field pub_date is a good error22:30
did you restart tomcat after you replaced the schema.xml file?
<alxp>yes it sounds like solr is at least looking at stuff
<bess>alxp: do you know how to look at your schema file via the solr web admin page? 22:31
just to make sure it's running the one you think it is?
<alxp>sec
<bess>g8tor: am I missing anything major from this page? http://projectblacklight.org/
<alxp>bess: <schema name="example" version="1.1">22:32
looks like it's not the one in my conf dire
<bess>ok
what does solr think your solr home is?
<alxp>where do I find that?22:33
<bess>alxp: the main admin page should tell you what it thinks solr home is
<alxp>ahh sorry yes , cwd=/usr/local/fedora/tomcat/bin SolrHome=/opt/solr/example/
<bess>great
<alxp>that's correct as far as what I've configured
<bess>so is /opt/solr/example where you put the new schema.xml file? 22:34
<alxp>yes
<bess>and did you restart tomcat?
<alxp>yes22:35
<bess>hmm
<alxp>looking at this solr file it looks just like the generic one at the top
any identifiers I can search for in it?
<g8tor>bess: maybe a link to the twitter page?
<bess>ours looks like this:
http://solrpowr.lib.virginia.edu:8984/solr/admin/file/?file=schema.xml
g8tor: good call
alxp: see if it has something like "*_date" in it22:36
that's what's missing
<field name="*_date" type="date" indexed="true" stored="true" multiValued="false"/>
<alxp>yeah grepping I don't see any _date
<g8tor>bess: while were @ it probably a facebook account . (LOL)22:37
)
<alxp>shall I just replace it with the one in your link?
<bess>alxp: well, you should just replace it with the one you got from solrmarc22:38
the one I showed you is customized for UVA and I don't know what problems that would cause
what worries me it that you're telling me you've already replaced the schema.xml file and solr doesn't seem to be loading it
<alxp>the one in solrmarc/eg/test/solr/conf/schema.xml doesn't have any _date fields22:39
<bess>it should say <schema name="Blacklight Demo Index" version="2.4">
it doesn't????
bleah
no tests
yeah add this line22:40
<field name="*_date" type="date" indexed="true" stored="true" multiValued="false"/>
<alxp>grep Blalight gives nothing
ok
<bess>hang on
does it have this line?
<field name="pub_date" type="string" indexed="true" stored="true" multiValued="true"/>
<alxp>no22:41
nothing with "date" in it
<bess>so, hang on
where did you get this version of solrmarc?
checkout from trunk?
<alxp>svn, yes
<bess>hmm
and you copied the file from solrmarc/examples/GenericBlacklight/solrConf22:42
?
<alxp>no I copied from the directory structure ant created
where it put the eg_config.properties files etc
so I should go get the schema from examples/GenericBlacklight?\
<bess>yes22:43
http://code.google.com/p/solrmarc/source/browse/#svn/trunk/examples/GenericBlacklight/solrConf
<alxp>ok
restarting tomcat22:44
* bess crosses fingers
<alxp>ok solr won't start now22:48
SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.component.TermsComponent'
<bess>d'oh
aaargh
this should not be this hard
I wonder if that's a solr 1.3 vs 1.4 problem
<alxp>should I go download solr 1.4?22:49
<bess>yup it's a solr 1.4 thing
how much more time can you devote to this tonight?
I'm starting to feel guilty about keeping you awake.22:50
but I'm game if you are.
<alxp>no it's fine
the phillies are still losing
<bess>yeah, download solr 1.4
install those two files into your solrhome/config22:51
and let's try again
I have to go get PJs on and do a couple of things but I'll be back soon
<alxp>haha the solr wiki 1.4 page looks pretty dire http://wiki.apache.org/solr/Solr1.4
ok
<bess>ha ha22:52
we run on 1.4 in production and haven't had any problems
<alxp>ok
<bess>alxp: any luck? 23:05
<alxp>installing solr now
hoping I can mostly drop in the new war and keep most of the configured paths the same
<bess>yeah, you should be able to
alxp: if you're still installing I'll give you a little more time
brb
<alxp>k
<bess>how about now?23:11
<alxp>starting up tomcat
with vanilla solr 1.4 23:12
* bess crosses fingers again
alxp: you've really been thrown in the deep end with solr23:14
<alxp>ok solr is back up
<bess>great
try indexing again?
<alxp>now copying the solrmarc conf files
not there yet :)
<bess>ok
<alxp>ok copied the solr files into my solr conf dir and solr won't start again23:16
checking catalina.out
<bess>d'oh!
<alxp>SEVERE: Could not start SOLR. Check solr/home property23:17
org.apache.solr.common.SolrException: Unknown Search Component: tvComponent
<bess>what?
googling
oh where is erikhatcher when you need him?23:18
oh right, he's at apachecon right now
<alxp>hahaha
solr seems to initialize ok before that point
INFO: Adding component:org.apache.solr.handler.component.HighlightComponent@4eb5ad68
<bess>try commenting it out23:19
+ <searchComponent name="tvComponent" class="org.apache.solr.handler.component.TermVectorComponent"/>
I don't think we're using TermVectors 23:20
<alxp>there are other errors like that one SEVERE: org.apache.solr.common.SolrException: Error loading class 'solr.DoubleMetaphoneFilterFactory
wow this thing has more verbose logging than fedora, that's saying something
<bess>heh
aargh23:21
where is it even calling TermVectorComponent from?
it isn't in our config files
<alxp>not sure23:22
<bess>ah it's in solrconfig.xml
but.. but... 23:23
it isn't in the solrmarc solrconfig.xml
it's only in the out of the box solr example one
wtf?
<alxp>hm23:24
<bess>so solr started okay out of the box, right?
<alxp>yes
<bess>let's replace one thing at a time
<alxp>ok
<bess>restore the old config files and make sure it still runs
<alxp>the only things i changed from default were the two files in solrConfig in GenericBlacklight23:25
so I'll take those out
<bess>just to be really clear
you're talking about which files?
full paths please
<alxp>solrmarc/example/GenericBlacklight/solrConf/schema.xml23:26
and same dir solrconfig.xml
<bess>and you were putting them where?
<alxp>. /opt/solr/example/conf23:27
<bess>okay
try only replacing schema.xml
but not solrconfig.xml
and restart solr
<alxp>ok solr's back
<bess>and it started okay?
<alxp>replacing schema.xml
restarting again23:29
ok solr won't start now23:30
<bess>what's the error?
<alxp>same error, can't find components
<bess>okay23:31
I think we've made a valiant effort
time to put it down and try again tomorrow
I've got a little downtime, I'll try a fresh install so I know the procedure better
<alxp>sounds good
thanks for all your help bess and naomi
<bess>:)23:32
* bess leaves23:33
* alxp leaves
* darthmolly_ leaves00:20
* g8tor leaves00:40

Generated by Sualtam