NNTPC: overview.fmt & Xref:full

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, 24 Mar 1997 proff@suburbia.net wrote:

> > 1. 
> > Change the hostname to it's own hostname.
> > Remove all references to newsgroups it gets from a different server than
> > the server it got this message from.
> 
> [...]
> 
> The xref issue is a difficult one. The reason it hasn't been
> addressed is (a) for a typical configuration it is a non-issue and
> (b) there is a performence hit in re-writing the xrefs.

But there are no performance hit in deleting in after you have used it,
which some of the newsreader will do anyway since it is malformed (wrong
hostname).

> Multiple servers chop up the group space according to group rules. i.e
> there is a 1:1 correlation between particular groups and particular servers.
> 
> In the case where you are running local groups, or pulling in specialist
> groups, it is rare to find crossposts between the two servers, because
> the organisational and subject domains are typically very different.


Not if you pull in specialist groups. In *that* case many of the articles
might well be cross-posted outside the small subset you pull in to get
better performance on.

At least that's our experience, and it's easy to show why this is so for
the kind of newsgroups that may be involved. Let's take an example:

You have a SGI server, so you pull in a local copy of comp.sys.sgi.*
You will then get frequent cross-posts between comp.sys.sgi.* and lots of
other nwesgroups, like comp.arch.*, comp.unix.*, comp.security.*,
comp.windows.x and the rest of comp.sys.*

Pulling in those other groups are a false solution since that would only
generate even *MORE* collisions...


> For those that don't, when a collision event occurs, the worst-case
> outcome is that there is chance (only a chance, because it is likely
> that the article number is either a long way in the past, refers
> to a message already read, or is a long way into the future (although
> the future will eventully become the present, for a moment at least
> according to Descate') the newsreader will mistakingly think the
> user has already read the article in that group, but not in the
> other group's it may have been cross-posted to.

Nope, this isn't the worst-case outcome, by *FAR*. You have *MUCH* to
little imagination if you seriously think that.

*Worst-case* outcome is that the newsreader either:

1. The number is to high, above the current high-water mark for that
group, and the newsreader marks everything up to that number as read!

2. The number is to high, below the current low-water mark for that
group, and the newsreader resets the articles read!


The explanations of why it happens are guesses, but the effects are
real, both 'no new articles' and 'read articles reseted'...

Yes, strn/trn4 is probably the newsreader that have most problems with
this, which might be because it might be one of the more advanced
newsreaders, falling short of only ding-GNUS, which I suspect works
because it ignores the Xref lines because of bogus hostname...

Other newsreader had other quirks with nntpcache, including GNUS. I
don't think I heard any of the Netscape3 users complain that much, but
then it has been debated (on news.software.nntp) if it *IS* a
newsreader :-)


> > I'm not exactly sure which of these strn/trn4 uses to list the newsgroups
> > and an approximate number of articles, but this operation could easily
> > take 3-5 seconds for a screenful of newsgroups, 40-50 or so... With the
> > INN 1.5.1 server now in place this takes an almost unmeasurable
> > time-period, we're talking about something like less than 0.1
> > seconds in most cases! (OK, so page 5 sometimes take 0.1-0.2 sec, unknown
> > why, but then that page sometimes took 5+ seconds with nntpcache)
> 
> On the nntpcache development servers (pentium-100 class machines), a
> 15,000 group list active takes around 5 seconds. If you have listSecurity
> on, it will take longer due to authentication checks on every group.
> At 5 seconds a page however, I can only guess that you have logToClient
> debugging switched on on, or a truely sick machine.


Note that it does 'list active <newsgroup>' on each individual newsgroup,
not a big 'list active', see a previous message for details. I suspect
this is so that it will be usable on a slow server, where 'list active'
might take very long time.

Also noteworthy is that 3 seconds / 50 lines is 60 ms per 'list active
<newsgroup>'. The machine in question is Pentium-Pro 200 with 128 MB of
RAM running Linux 2.0.27 (now 2.0.30-pre), not exactly a machine you would
expect to be slow. It certainly flies under INN.

We still have the configuration file, and here's some relevant excerpts:
listSecurity	off
logFromClient	yes
logToClient	no
logFromServer	yes
logToServer	yes


I believe these are the default settings...


You could try fetching a recent version of trn4 and try if you get the
same result. Trn4 is still in public beta testing, and is available from:
ftp://ftp.clari.net/private/trn/

The version I used back then was probably trn4-beta53, but I suspect
beta56 behaves the same. Note that strn also have the same kind of delays.


> It is possible to get this figure down from 5 seconds to around
> 1.5-2, using a secondary pre-formatted in-memory cache, at the
> expensive of around 2Mb of memory and some code-complexity. I had
> thoughts about this previously (as careful readers of nntpcache.config
> will note), but until now no one has ever mentioned list active
> speed as an issue, and until people do, I'm not about to introduce this
> memory overhead.

I'm not sure this would help with 50 'list active <newsgroups>'.


-- 
Torbjörn Lindgren
E-mail: tl@funcom.com
If Santa ever DID deliver presents on Christmas Eve, he's dead now.


[Index of Archives]     [Yosemite]     [Yosemite Campsites]     [Bugtraq]     [Linux]     [Trn]

Powered by Linux