RE: Custom Log Format -> Adding milliseconds to timestamp -> %{format}t

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



> Well, for sequence and order any quantity that only increases (or only
> decreases, for that matter) each time it is sampled, would serve.
> Such as a gadget that just hands back the next integer in series every
> time it is queried (properly interlocked across threads/processes).

Agreed, good point and if such a beast existed, we would use it.

> Come to think of it, if you are serving requests fast enough,
> milliseconds won't be sufficient.

If we were looking for a UUID for each log entry, mod_serial would provide
that (a hash across IP address, requested object, user agent, etc is pretty
good too).  That's not the problem.  The problem is that when we're doing 30
to 60 requests per second, the log entries get difficult to separate as to
which *order* the events are happening, etc.

> But the available
> precision will depend on what APR can get from the OS, so you may only
> be getting the illusion of microsecond precision while in fact the
> value returned jumps by units of 1,000,000 (that is, seconds) or so.

Agreed but that's not a problem here - most current *NIX systems (we're on
RHEL 5/CentOS 5) perform actual millisecond or better precision.
 
> Time is far more complex.

Agreed, but as Consumer's bandwidth increases and processors get faster
we're probably going to want to start logging things with millisecond
precision.  But that's speculation and this isn't the forum for that.

We thought of another solution that might meet the need: use the line number
of the log entry.  So the first entry in the log is #1, second log entry is
#2, etc.  But as with many busy web sites, we don't wait till the end of the
day to look at web logs.  We have a job that ETL's the log into a database
every x minutes.  Each time it runs, it picks up where we left off.  So for
us, the line number starts over at "1" each time we start the ETL job.
Thus, the incrementing number would be unique only for each ETL job.  This
is ok because, again, we're not looking for a UUID for each log line as much
as we want to recognize the sequence of events.  But this leads to another
question.

What order does Apache write the log entries?  I'd guess that as soon as the
response the given to the client, Apache generates the log entry?  If Apache
serves 100 responses all in the same time second window, I suppose it's
going to write them in the order the response happened and not scramble them
up.  In other words, although there's 100 log entries all stamped
"21/Feb/2010:20:24:25", the order they are written in is hopefully the rough
order in which they were served?

Thanks List,

http://www.t1shopper.com/

Attachment: smime.p7s
Description: S/MIME cryptographic signature


[Index of Archives]     [Open SSH Users]     [Linux ACPI]     [Linux Kernel]     [Linux Laptop]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Squid]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux