Hi,
Thanks again fro the reply. This time i stopped using the nocem feature. Yet
again I cant get nntpcache to work. Please note that I have no arrangement
with the newsgroups provider for downloading news. I am under the impression
that I can pull news from any free news server for replication at my end and
post and read remote newsgroups at my local servre (am i right??? )
I have included my nntpcache.servers, nntpcache.config, . access here. the
error message i get is:
503 initial server rebuild in progress (10 groups complete), please try
again la
ter
there are 10 newsgroups on my localhost
there 57 newsgroups on newsgroups.bea.com
Can you please help?
My NNTPCACHE.ACCESS:
--begin--
* * read,post
--end--
My NNTPCACHE.SERVERS
--begin--
#$Id: nntpcache.servers,v 1.12 1996/07/26 11:55:31 proff Exp $
#---------------------------------------------------------------------------
--
# Example: leaf machine, slow link, one NNTP server, no local groups
# /* timeouts */
# host:port Interface Active Act.tim Newsgrp Group Xover Arts
newsgroups.bea.com DEFAULT 4h 2d 2d 30m 60d
6
0d
localhost DEFAULT 4h 2d 2d 30m 60d 60d
%BeginGroups
# Group pattern Host
* newsgroups.bea.com
* localhost
#---------------------------------------------------------------------------
--
--end--
My NNTPCACHE.CONFIG
--begin--
# $Id: nnconf.cf.in,v 1.3 1998/08/04 20:18:43 proff Exp $
# Note: you can't set compile-time defaults for stringl (not string)
# types. The specified stringl defaults will be automatically migrated
# over to the run-time configuration file however, so this shouldn't
# pose any problems.
# location of configuration directory, the config files below
# are relative to this path. kinda important
configDir /usr/local/etc/nntpcache
# location of configuration file (hey, thats me, here)
configFile nntpcache.config
# server configuration file
serversFile nntpcache.servers
# access control configuration file
accessFile nntpcache.access
# the root directory for nntpcached all cache files are created under.
# if the chroot option is on, this this path is relative to chrootDir
# (but still needs a leading slash)
cacheDir /usr/local/var/nntpcache
# who should receive email complaints / reports?
adminEmail news@zero.planetasia.com
# file/directory creation umask
umask 022
# nice value for master server
niceMaster 0
# nice value for clients
niceClient 10
# nice value for web-server
niceHTTP 2
# nice value for update daemon
niceUpdate 4
# nice value for nocem task
niceNoCem 15
# nice value for expiry
niceExpire 15
# for high security, it is possible to run nntpcache in a minimal chrooted
# environment. the only files/directories nntpcached should need in the
# chroot hierarchy are cacheDir and those associated with the resolving
# of hostnames (typically /etc/host.conf, /etc/resolve.conf and /etc/hosts)
# and syslog (/dev/log) (you will also need to run a syslogd on the
# chrooted /dev/log (e.g syslogd -p /usr/local/nntpcache_root/dev/log))
#
# on systems (such as solaris) which wrap connect() and other tcp functions
# in library calls which open /dev/tcp* you may also need to mknod (or
# tar copy) the required devices.
#
# note that the configuration files do not need to be, and should not be,
# in the chroot hierarchy
#
# nb. nntpcached must be invoked as root for chroot() to succeed
chrootDir /usr/local/nntpcache_root
chroot off
# change the format below to your most frequently used cached server.
# nntpcached is smart and doesn't translate xover information if the
# remote server is using the same overview scheme as nntpcached
# (xover format translation is a relatively expensive operation). You can
dump
# the xover format, by telneting to the nntp.server, port 119 and typing
#
# list overview.fmt
#
# note that if you change the format, you will need to purge the xover
# cache. the following command will do just that:
#
# find /usr/local/var/nntpcache -type f -name \*_xover -print | xargs rm
#
# order IS important. The below is the default for INN1.4+ servers.
overviewFmtInternal Subject: From: Date: Message-ID: References: Bytes:
Lines: Xref:full
# the following is used for a few brain-dead servers that xover but refuse
# to provide an overview.fmt. You really need to get this right if you
# expect to talk to such a delinquent bozo server.
overviewFmtBozo Subject: From: Date: Message-ID: References: Bytes: Lines:
# minimum active entries. We hold off serving clients till we have
# (a) performed at least one server update pass (normally initiated on
# the first invocation) and (b) have at least minActive entries in
# our collated active file. This is designed to avoid handing clients
# an access file that has been debilitated by upstream server failure
# at start up.
minActive 30
# IHAVE servers. If an IHAVE command is issued, then it will be passed
# onto the servers specified here. This is for situations where nntpcache
# is acting as an intermediary between two news-servers attempting to
# feed each other news. (watch out for recursion). Note that you
# shouldn't need to specify multiple servers here unless they are
# are unconnected insofar as usenet network topology is concerned.
ihaveServers localhost:2222
# speedup IHAVE handshakes. If you have multiple IHAVE servers, then
# this will cause a NNTP_TOOKIT message to be returned to the IHAVE client
# as soon as the first NNTP_TOOK message comes in from anyone of the
# IHAVE servers listed above, rather than waiting until all servers
# have accepted or rejected the article.
ihaveSpeedHandshake on
# Maximum number of servers to consult for <msgid> cache-misses. When
# an article (etc) is required by <msgid> that is not in the cache,
# nntpcache has no information on which servers have it. It deals with
# this by first asking the current server (i.e the last server that
# found a <msgid> style request, or is serving the current group), then
# checking all the other servers in the order they are mentioned in
# the nntpcache.servers configuration file. There are two issues here.
#
# 1) Recursive servers. If both servers are nntpcache servers,
# and each has the other listed as one of its servers, then
# if there is a cache miss on a <msgid> style article request,
# and the current server didn't have the article (perhaps
# simply because the article expired), then first server
# will ask the second server for the <msgid> which may
# forward the request to the first server and so on. We
# can't just filter out requests going back to the host that
# the client came in on, as some of these maybe quite
# legitimate, although we could perhaps create a failed
# message id request cache, so when the second forward
# comes back, the request can be denied without consulting
# other servers that potentially have it. this sort of
# solution does however have race conditions, in the unlikely
# event that two clients request the same message id at the
# same time.
#
# 2) Many servers scenario. Fortunately this case isn't nearly
# as pathological as the above. When there are many servers
# listed in nntpcache.servers it can take quite some time
# to progress though them all, only to find that the <msgid>
# in question has expired. This can lead to user hair-pulling.
# The other issue here, is that if your reader population is high
# and you are pulling in groups from external servers (i.e that
# don't belong to you directly), the news admins who run them
# may become annoyed with the constant requests for expired
# <msgid>'s.
#
# the default of 2 (below) will search the current server and one
# other (the first server, or the second server if the first server is
# the current server).
maxMsgIDsearch 2
# maximum number of concurrent readers
# this must be smaller than MAX_CHILDREN in cf/nconf.h
maxReaders 200
# permissible characters in a group name. Anything not matching is simply
# stripped out. You may want to modify this if your news feed uses kanji
# or iso high bit characters et al in regional group names.
# nb. two of the microsoft.* groups from msnews.microsoft.com have
# ampersands ('&') in them. I know I'm wearing black.
safeGroup
.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890_-+=':
# group security. if activated, nntpcache performs group level security
# checks on all messages and group oriented commands, not just connection
# security checks. group-level POST checks are always on, regardless
# (because POSTs are so rare as to not be a performance issue).
groupSecurity off
# list security. if activated, nntpcache performs group level security
# checks on every group involved in a group listing (i.e a LIST of
# active, active.times, newsgroups or a NEWGROUPS). This will remove
# all groups from the listing which the client does not have read
# access to. If you do not mind users seeing groups that exist and
# which they can not access, then leave this option off. The checking
# is an intensive operation, because every group in the list
# (and that's normally ALL groups for a LIST active or LIST newsgroups
# i.e around 15,000 groups as of this writing) have to be matched
# against the group security entries in the access file.
# If you want to strip out particular groups for all clients, then
# use the strip access control, as it comes into effect during
# nntpcache group collation (rare) rather than during group listings
# (frequent)
listSecurity off
# some servers return newsgroups in 'newsgroups' and 'active.times'
# that have no corresponding entry in 'active'. Do we permit this?
# note that the ListActiveThreshold won't have any effect unless
# listPermitLonelyness is false
listPermitLonelyness no
# some servers state they carry a group, but on closer inspection
# the group appears to be permanently barren. On the other hand,
# the server may carry the group, but the group never sees any
# articles (other than spam). What is the minimum article high
# count before we will believe that a group is active?
listActiveThreshold 0
# filter articles by scanning article and/or header content
# according to the filters specified in the access file.
# our content filtering routines are as greased as
# a lawyer-turned bail bondsman on Prince William Sound beech
# demonstrating Reef Tan, but you may find yourself getting
# wet if a lot of traffic comes in. You need to have groupSecurity
# on for this to have proper effect.
contentFilters on
# as above, but for XOVER and XHDR commands. You can only
# filter these commands based on header patterns, not article
# content. filtering is transparent as XOVERs and XHDRs
# are used to build the list of messages in a group.
xoverFilters on
# NoCem - auto-spam killer
# see http://www.nocem.org for more information about nocem
nocem off
# initial scan - what is the maximum number of articles
# we will read in one session?
nocemInitialScan 8192
# what is the minimum time period between scans for new
# nocem articles?
nocemInterval 3m
# groups to monitor for nocem messages
nocemGroups news.lists.filters
# cache nocem requests. By default we don't cache nocem
# fetched articles, but if other nntpcache's are pulling
# nocem groups from you, you may wish to.
nocemCache false
# Process NoCem messages from (regex)
# The default (.) is everyone. However provided PGP
# authentication of NoCem messages is enabled (see
# further below) only the NoCem messages we have public
# keys for will be accepted.
nocemFrom .
# nocem type regex
nocemType spam|spew|MMF
# nocem action regex
nocemAction hide
# require PGP signed nocem messages this is turned off if nntpcache's
# configure script couldn't find PGP on the file-system. If turned off
# net kooks(tm) can forge nocem advisories. No worse than news-server
# that accepts cancel control messages, but ideally you should have
# signature verification turned on.
nocemPGP no
# start of PGP signed nocem message. we only
# handle clear signed messages. for performance reasons
# we don't use the PGP generated data, except for messages
# sent to stderr, which we use to check for signature validity
# All nocem messages are currently clear-signed,
# but we may need to review this policy if (say) PGP compressed
# nocem messages are seen on the wire.
nocemPGPbegin -----BEGIN PGP SIGNED MESSAGE-----
# end of PGP signed nocem message. necessary to prevent
# out of window attacks (pgp in stream mode will normaly
# process all messages in the stream, not just the first)
nocemPGPend -----END PGP SIGNATURE-----
# command used for PGP signature verification - returns with exit
# status zero if the signature is valid. %s expands to a temp file in
# which nntpcache expects the stderr messages from pgp. if chroot is
# on, then you will need /bin/sh and a statically linked version of
# PGP (or ld.so/rtld and the relevant libraries) within the chrooted
# region - and the path below will be relative to it. These flags have
# only been tested with PGP2.6
nocemPGPcommand pgp -f +batchmode >/dev/null 2>%s
# nocemPGPgood (regex)
nocemPGPgood ^Good signature
# PGPPATH
# By default pubring.pgp is stored in /usr/local/etc/nntpcache. If
# chroot is on, you will need to copy pubring.pgp into
/usr/local/var/nntpcache
# and change this path to be relative to /usr/local/var/nntpcache (or
whatever
# you have set cacheDir to be)
nocemPGPPATH /usr/local/etc/nntpcache
# relay unknown commands to the most recently used news-server.
# may bypass group (and other) security features, depending on
# what commands the remote news-server supports.
relayUnknowns no
# nntpcached sets it's uid to that of the username specified below
# before it listens for connections. you *can* run nntpcache even
# though you do not have root access, by setting the user and
# group variables to those your own and bindAddr (see
# below) to a high port (i.e >1024)
user news
# what group to run as
group news
# organization (Yankee spelling)
# used when the reader posting doesn't specify one, or
# when replaceOrganization is on
Organization Ye 'Ol Disorganized NNTPCache groupie
# force replacement of already present Organization header in posts.
# (Suuuure you're from the Julian Assange Memorial Society)
replaceOrganization no
# strip post header regex
# remove headers matching the following regex
postStripHeader ^X-foo:
# expire if there are under this number of blocks in the cache partition
minBlocksFree 1000
# address:port to listen for connections on (can be changed with
# -b at run time). nb. DEFAULT:119 = all interfaces, port 119.
# if you do not have root, you will need to make the port a
# number greater than PORT_RESERVED (usually 1024) e.g
# bindAddr DEFAULT:2119.` You will also need to change the
# NNTPSERVER and/or NNTPPORT environmental variables (or
# whatever configuration your news reader uses for such
# things) accordingly
bindAddr DEFAULT:2222
# max mmap data size. a 15,000 group newsfeed takes around 4Mb for our
# internal cache structures. we set the data space here to 16Mb
# as there is almost no overhead for the unused portion.
# if this figure runs out, BAD juju will happen.
maxMmap 16M
# if set, we don't use a memory mapped file for our cache structures,
# but rather store the lot in an anonymous memory region. this means
# modifications by nntpcached to the mmaped region are not saved
# between restarts of nntpcached. On the other hand, such information
# tends to expire relatively quickly. anonymous regions have superior
# performance characteristics (i.e it's all in memory) -- depending on
# how effectively your OS cache's mmaped disk blocks. this value has
# no effect one way or the other if mmap_tests determined that shared,
# inherited, anonymous memory maps or file maps were non-existent or
# buggy -- nntpcached will just choose whatever works :)
anonMmap yes
# run expire if there are under this number of inodes in the cache partition
minFilesFree 1000
# don't let the history file grow larger than this (bytes)
hisHighWater 60M
# when it does, trim it back to this (bytes)
hisLowWater 40M
# as above, but for the newsgroups list. usually
# not as important, as it is fetched with less
# frequency
secondaryNewsgroupsCache false
# When we do an expire, kill articles older than this
maxArtAge 2w
# check disk space, etc for automatic expiry this often
# (but only after a client connects or disconnects)
expireCheckPeriod 5m
# how often do we update overview.fmt's?
# the master nntpcache will holdoff incoming connections
# during this (brief) process. this data should never
# change, so the refetch is set to be quite infrequent
overviewFmtTimeout 7d
# perform identd lookups
rfc931 no
# hide the name of the current group from ps(1)
taskInfoPrivacy no
# nnrpd compatible log messages (i.e innreport fodder)
logInn yes
# log network input/output
logFromClient yes
logToClient no
logFromServer no
logToServer yes
# log {active,newsgroups,active.times} merger messages. these can be
# quite voluminous if you have a lot of group deny rules.
# also log ignored groups due to group <-> server pattern correlations
# both of these are logged at syslog level LOG_DEBUG
logListMerge yes
logListMergeCorrelation yes
# verbose nocem logging
logNocem no
# you probably do not want to turn any of these syslog messages
# off execpt for logDebug, and maybe logInfo on very high-volume
# installations
logDebug yes
logInfo yes
logWarnings yes
logErrors yes
# collect statistics. involves scanning some strings for length, if you
# are paranoid about performance, turn it off.
# note that some statistics will still be collected (i.e in many cases
# it is actually faster to collect the statistics than decide whether
# statistics are collected or not), but the values will not be wholly
# accurate.
statistics yes
# a magic article id is used to serve up a magic article containing
# nntpcache statistics. if you don't want users reading your cache
# statistics then set this to something obscure
statsArticleID stats@nntpcache
httpServer yes
# address to bind the nntpcache web-server to
httpBindAddr DEFAULT:9119
# If chroot is in effect, this needs to be relative to
/usr/local/var/nntpcache
httpFiles /usr/local/etc/nntpcache/http
# don't refresh list cache unless at least this amount of time has passed
# since the last check, despite lower timeout values given anywhere
# else
minUpdateDelay 10m
# some news servers (particularly those running on microsloth boxes)
# don't support all the list types that nntpcache does. Most often
# attempts by nntpcache to view unavailable lists generate a 501
# syntax error. However it is possible that there was some other cause
# of the error. It's also possible that the news server may be
# upgraded in the future and the list will become available. How long
# do we flag a server/list combination as unavailable when we
# receive an error in response to a list request?
minUpdateRefusedDelay 24h
# if we are refreshing the list cache for a particular list/server
# combination and the attempt is unsuccessful (excluding the above),
# what is the minimum delay before the next check?
minUpdateFailDelay 10m
# period after which a network connection is considered dead
networkTimeout 5m
# exit if client is idle for longer than this period
idleTimeout 25m
# shutdown the remote connection if it idle for longer than this period.
# nntpcache will open a connection to the remote server for each
# cache miss. for very high volume sites, this means the remote
# server can start to build up a lot of rarely active NNTP sockets
remoteIdleTimeout 10m
# If a server has gone down, don't try and reconnect until this
# time period has elapsed (we run out of cache till then).
serverDownRecheck 5m
# output buffer size (to client newsreader)
# large buffers gain efficiency, but increase the
# time it takes for a client to see the first part of a
# long XOVER or ARTICLE. We use the size below for
# the internal stdio-buffer and the socket layer buffer
outputBufferSize 8k
# in each nntpcache sub-process, file descriptor's
# and the index into each xover file are cached during
# xover/xhdr scans, at one per 512 articles.
# this variable sets how many of these we can cache.
# do not set to less than two or bad juju will find
# its way into your life
maxXoverNodes 20
# you probably do not need to change anything here on in
statsFile nntpcache.stats
# history database file name
historyFile cache.history
# file in which to store the nntpcached master server process id. PID_FILE
# has the addr:port number appended, incase multiple nntpcached's are
running
pidFile nntpcache.pid
# overview.fmt file, no point in changing this
overviewFmtFile overview.fmt
# shared mmap file, no need to change this.
# if USE_MMAP_ANON is defined then the following isn't used
mmapFile cache.mmap
# shared mmap base pointer file, no need to change this.
# if USE_MMAP_ANON is defined then the following isn't used
mmapBaseFile cache.base
--end--
Regards
Sachin Dole
-----Original Message-----
From: Christian Krackowizer [mailto:ckrackowiz@std.schuler-ag.com]
Sent: Wednesday, October 11, 2000 8:45 AM
To: nntpcache-users@suburbia.net
Cc: 'linux-india-help@lists.linux-india.org'
Subject: NNTPC: Newbie
At 18:49 11.10.2000 +0530, Sachin Dole wrote:
>Hi,
>
>Thanks for the reply. But I have only 57 newsgroups on the site i am trying
>to get and the message is
>
>
>the message i get in the /var/log/news/news.err is
>Oct 11 18:52:10 zero nntpcache-nocem[15668]: nocem.c:325:Bad file
>descriptor: gr
>oup change to 'news.lists.filters' failed
hmm, no really. Try 'nocem off' in nntpcache.config
with best regards
Christian Krackowizer
schuler technodat GmbH
Jakob-Haringer-Strasse 6
A-5020 Salzburg
Phone: +43(0)662/2282-0
FAX: +43(0)662/2282-9
e-Mail: ckrackowiz@std.schuler-ag.com
