| * erikhatcher joins | 03:01 | |
| * cbeer leaves | 09:42 | |
| * cbeer joins | 10:08 | |
| * ndushay joins | 10:41 | |
| * Naomi joins | 10:52 | |
| * ndushay leaves | 11:00 | |
| * Naomi leaves | 11:09 | |
| * ndushay joins | 11:34 | |
| * bess joins | 12:17 | |
| * bess leaves | 12:21 | |
| * bess joins | 12:23 | |
| * bess leaves | 12:26 | |
| * bess joins | 12:33 | |
| * bess leaves | 12:58 | |
| * bess joins | 13:13 | |
| * bess leaves | 13:23 | |
| * bess1 joins | 13:44 | |
| * bess1 leaves | 14:14 | |
| * bess joins | 14:21 | |
| * bess leaves | 14:50 | |
| * erikhatcher leaves | 15:02 | |
| * bess joins | 15:18 | |
| * bess leaves | 15:55 | |
| <jrochkind> | now my solrmarc has slowed down to 1 record per second for some reason. okay, time to try something else. | 16:07 |
| * erikhatcher joins | ||
| <jrochkind> | heh, glad erik missed that message talking to myself, or he'd be telling me "why are you still using solrmarc." | |
| I said, fwiw, that my solrmarc is for some reason now down to 1 marc record per second. | ||
| <erikhatcher> | path-e-tech | |
| <jrochkind> | okay, a ruby try. must do. too many things i want to do, not enough time. | 16:08 |
| <erikhatcher> | jrochkind: i know the feeling well | |
| <jrochkind> | Huh, and also it keeps giving me java outfomemory heap excpetions. Which explains the slowness, it was running out of memory. | 16:09 |
| But even for a ginormous file, there's no reason it ought to need a lot of memory. Marc files can easily be read one file at a time, streamed from disk, no need to load the whole thing into memory. But apparentlyit maybe does? | ||
| s/one file/one record/ | ||
| I hope ruby-marc is capable of reading a marc file streamed from disk one record at a time, not loading the whole thing into memory? If not, I guess it can be patched. | 16:10 | |
| <erikhatcher> | what runs out of memory? solr-marc? or solr? | 16:13 |
| yeah, ruby-marc does fine streaming | ||
| <jrochkind> | It's unclear what is running out of memory. The error is reported in the solrmarc log. But I guess I should go look in the solr log for it too. Maybe that would explain my solr instability issues. | 16:17 |
| <erikhatcher> | that's why i asked about OOM for Solr | |
| <jrochkind> | yeah,I gotcha. | |
| <erikhatcher> | Solr has never been unstable, given enough RAM for the task at hand | |
| <jrochkind> | What log would I look at in Jetty/Solr for that? jetty/logs/stderr.log? | |
| <erikhatcher> | if solrmarc is logging it, i'd guess it's on the solrmarc side of things though | |
| how'd you launch Solr? | 16:18 | |
| by default it logs to console | ||
| you have to tweak logging config to have it do otherwise | ||
| jetty logs requests, but it won't log the detail solr itself will | ||
| <jrochkind> | Yep, it's indeed in that log. However jetty is setup, it's throwing things there. Launched solr with jetty. And indeed there is an OOM in there. | 16:19 |
| Okay, now to figure out how to seriously bump up solr memory in jetty. | ||
| <erikhatcher> | java -Xmx2048m -jar start.jar ;) | 16:20 |
| -server is a good option too | ||
| how much RAM do you have? | ||
| shouldn't need a lot for indexing | ||
| <jrochkind> | It's being run inside a jetty, I'm not executing the command line manually. so not sure how to do it. I guess I need to bump up jetty's own ram? | 16:21 |
| <MrDys> | jrochkind: change it in the jetty init.d script | |
| <jrochkind> | MrDys: Sweet, thanks. I'll go see what it's got now, and how much mem on th server. It _shouldn't_ need a lot for indexing, but apparently needs more than it's got. | |
| <erikhatcher> | how is jetty launched? | 16:22 |
| <MrDys> | jrochkind: heh. /proc/meminfo is saying that VM only has 1gb of physical ram | |
| <erikhatcher> | oh, init.d | 16:23 |
| <jrochkind> | MrDys: You think we need to ask Tony for more? Bah. So many JIRA tickets. | |
| Any idea how much I _ought_ to give it, and how much the VM needs for that? | ||
| <MrDys> | jrochkind: we're going to need more than a gig if it's going to handle even a minimal amount of users | |
| <jrochkind> | You think 2G is enough for now? | |
| MrDys: What should I ask Tony for? I don't care, this is not something I know about. | 16:24 | |
| <MrDys> | I don't know what the performance aspects of most of the ruby/solr stack are | |
| 2 would be a start, probably more would be even better | ||
| <jrochkind> | I can ask Tony for whatever we want. I don't know the performance aspects either. | 16:25 |
| You think I shoudl ask for 3? | ||
| 4? | ||
| 10? | ||
| :) | ||
| I have no idea either, but you seem to have some more idea than me, MrDys. Since I didn't even know 1 wasn't enough. :) | ||
| <MrDys> | I'd ask for 4 | 16:26 |
| <jrochkind> | Okay, I'll ask for 4, see if we can get it. | |
| <MrDys> | not sure how divvied up that machine is | |
| <erikhatcher> | i think the default for java without specifying is like 64meg | |
| so even 512m might do the trick | 16:27 | |
| <jrochkind> | right now the init.d doesn't seem to specify it all, as far as i can tell. So no surprise that 64m isn't enough. :) | |
| But 1G on the host OS isn't enough even to bump it to 512M. | ||
| Of course, we've got TWO cores running in solr, which is the only thing running in jetty. So if 512M is enough... does that mean for two cores, double it to 1G? | ||
| <erikhatcher> | i'd give 512 a try | 16:28 |
| <MrDys> | unless those two cores are going to be hit with the same amount of requests, I wouldn't think so | |
| <erikhatcher> | you really don't need a lot of RAM for indexing | |
| <MrDys> | but I really think that the solrmarc process is just eating the machine | |
| <erikhatcher> | on the searching side, for facets and caching, you'll need more, but for fiddling around indexing should be fine | |
| well, run solrmarc on another machine ;) | ||
| that's the idea for posting over HTTP anyway | ||
| <jrochkind> | I don't have another machien to run it on at the moment. But, yeah, ultimately, I understand that's the goal. | 16:29 |
| <erikhatcher> | run 5 machines, posting in marc all separately, pegging each machine with the mapping crap | |
| <jrochkind> | I'll try giving it 512m, we probably have enough OS mem for even that now. | |
| * bess joins | 16:34 | |
| * bess leaves | 16:37 | |
| * cbeer_ joins | 16:46 | |
| * erikhatcher leaves | 17:32 | |
| * bess joins | 18:10 | |
| * [LorD-VipsS] joins | 18:12 | |
| * [LorD-VipsS] leaves | 18:14 | |
| * bess leaves | 18:26 | |
| * bess joins | 20:14 | |
| * cbeer_ leaves | 20:35 | |
| * bess leaves | 21:06 | |
| * cbeer_ joins | 21:45 | |
| * cbeer_ leaves | 21:47 | |
| * erikhatcher joins | 21:51 | |
| * rsinger_ joins | 22:16 | |
| * ndushay leaves | 22:20 | |
| * rsinger leaves | 22:28 | |
| * rduplain joins | 22:33 | |
| * rduplain leaves | 22:51 | |
| * erikhatcher leaves | 23:09 | |
| * ndushay joins | 00:00 | |
| * ndushay leaves | 00:03 | |
| * erikhatcher joins | 00:24 | |
| * erikhatcher leaves | 00:30 | |
Generated by Sualtam