Thanks Rich. Any idea on when you guys might produce a fix? Processing a gig of log data takes several hours of compute time and I have several gigs per day to process. I will never get caught up on producing log summaries without throwing multiple VMs at it and that is hard to justify. I'd be happy to take a whack at it if you like but if you guys are close to producing something then I'd rather not throw away my time. If someone has ideas on how it should be solved I'd be happy to go that path as well.
/mrg
On Sep 25, 2013, at 11:36 AM, Rich Megginson <rmeggins@xxxxxxxxxx> wrote:
> On 09/25/2013 09:34 AM, Michael Gettes wrote:
>> Rich,
>>
>> thanks again for pointing out where to get the latest logconv.pl.
>>
>> I made the following minor changes to deal with my gzipped log files and the use of an initialized variable $ip
>>
>> 421a422
>>> $files[$count] = "/bin/zcat $files[$count] |" if ($files[$count] =~ /.gz$/);
>> 2511c2512,2513
>> < return $hashes->{conn_hash}->{$connid};
>> ---
>>> return $hashes->{conn_hash}->{$connid} if defined($hashes->{conn_hash}->{$connid});
>>> return "";
>> the Unindexed Components report takes a VERY long time. I am going to try and figure that one out at some point. Is this a known problem?
> Yes.
>>
>> FYI on my experience thus far… I used the --data switch to get the db files into tmpfs and I ended up having to mount tmpfs over top of /var/tmp as dbopen wants to create files in /var/tmp - so now I have everything in memory. Some 80-90% of my time is spent spitting out the unindexed stuff which is critical information to fix apps or provide additional indices.
> Right. We are working on this.
>>
>> /mrg
>>
>> On Sep 16, 2013, at 4:01 PM, Rich Megginson <rmeggins@xxxxxxxxxx> wrote:
>>
>>> On 09/16/2013 01:58 PM, Michael Gettes wrote:
>>>> Hi, I am currently on 389-ds-base 1.2.11.15-22.el6_4 and I am running logconv.pl on 5.5M line log file. At the end it hangs up - in a loop forever and doesn't finish generating the report. What I am wondering is if I installed 1.3 latest on another system and got the log file over there, will it be able to process the "older" style log file?
>>> log file format hasn't changed in quite some time
>>>
>>> even easier, just grab the latest version of logconv.pl from the git repo https://git.fedorahosted.org/cgit/389/ds.git/tree/ldap/admin/src/logconv.pl
>>>
>>> No need to install 1.3.1, etc.
>>>> I know I can try this but I am wondering if anyone has insight into this sort of thing.
>>>>
>>>> thanks!
>>>>
>>>> /mrg
>>>>
>>>> --
>>>> 389 users mailing list
>>>> 389-users@xxxxxxxxxxxxxxxxxxxxxxx
>>>> https://admin.fedoraproject.org/mailman/listinfo/389-users
>
--
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users
