giovanni_re posted on Sat, 26 Jun 2010 00:34:08 -0700 as excerpted:
> About every 10 minutes, lasting about 1.5 minutes, virtuoso-t is running
> (maybe also "kdeinit4: plasma-desktop"), doing about 100 reads per
> second, & slowing things down right then, & I think also in the minutes
> following.
>
> I think this might be the source of my system running so slow at times.
Very possibly, particularly if you're running < 1 gig memory, which would
force the system to decide between swapping apps out to cache data, or
keeping apps and having less swap.
I see below that you're running kubuntu (and note -- after writing the
above, that you're running 4 gig RAM. That should be enough not to be
doing the swap thing, unless you have something strange going on... I'm
actually running 6 gig RAM here, amd64, but four would be plenty). That
means Linux. There's a kernel swappiness parameter that you can tweak
that could help the system better decide between swap and cache, but let's
deal with your questions, first.
> I think it might be causing other applications (ex, firefox, or a switch
> on the desktop to a different window, when I click to some other
> window), to have to do many disk reads to read in cached information,
> which has been lost from RAM when virtuoso-t just ran.
>
>
> == What is v-t doing?
> In the service of what other sw? Where can this be learned about? I
> didn't find a man page for v-t.
>
> Where can settings for v-t, or the other sw, be viewed, & perhaps
> modified? In the "System Settings" program someplace?
In general, kde apps (tho virtuoso is kde related but not direct kde)
don't have traditional manpages, unfortunately. I've often wished they
did. Most of them do offer the usual short description and command line
parameter info if run from konsole or the like, with the --help parameter,
and many of them have HTML based GUI help (kde handbook, see khelpcenter),
but that doesn't replace a decent manpage!
Also, there's user oriented help at userbase on the kde website, and
similarly, much more technical mostly dev oriented help, at techbase.
There's also the sysadmin section in the kde handbook, available both
online and in khelpcenter, but as with too much of the rest of the
khelpcenter documentation, some of it hasn't been updated /well/ since the
kde3 era (and not late in the kde3 era at that!). Never-the-less, I've
found the sysadmin section, particularly the info on default directories
and environmental variables, quite helpful indeed while adminning my own
systems. A lot of that info transferred over directly from kde3, and much
of what didn't is pretty close and/or has had at least some updates done
to it.
In the case of virtuoso, most of the info you'll find will be on the user-
and techbases, since it's quite new. But I'll post some basics.
Virtuoso is database software, the backend for strigi/nepomuk's semantic
desktop stuff. Only with kde 4.4 did it get good enough to be the default
strigi/nepomuk backend. Before that, the default was the Java based
Sesame2, with a waayyy slooowww fallback to something else, I forgot what).
What going on is that nepomuk/strigi is setup to index the files on your
disk, thus the reads you're seeing, to enable better semantic searches,
etc. However, in practice, kde4.4 doesn't yet have the UI to make
particularly good use of those indexes, so a lot of folks simply turn off
that indexing, at least for now.
Perhaps with 4.5, but more likely it'll be 4.6 or 4.7 (so next year), the
front-ends will actually give you usable access to all this indexed data
giving you all sorts of new ways to access your stuff. For instance, you
could do a search listing all the images you have of a particular
resolution, or if you're a photographer with a number of digital cameras,
with a particular camera, or taken on specific dates, or using tagging, of
a particular location, or perhaps all those you've edited with a
particular photo editor. Combining those could give you some really
interesting flexibility in image search. Text documents, email, instant-
messaging-logs, all sorts of stuff can be indexed, and correlated so that
you can pull up say the instant message log discussing the photo someone
sent you in email, along with the email, and the photo, all together.
But, as I said, the front-ends really aren't there to do much with the
indexes at this point.
And really, truth be told, a lot of technically minded folks probably
won't find such things /that/ interesting anyway, at least compared to the
cost of the indexing. Consider how long MS Office has had its indexer,
for instance, and that all that time, one of the biggest speed-up hints
for it and one of the first things many tech inclined people do is disable
it. Why? Because tech oriented people intuitively understand the
computer directory structure and generally have a good idea where they put
something (or where a package is likely to install it) in the first place,
so can either go directly to it, or find it with a minimal real-time grep/
file-search. The cost of running that indexer doesn't give them enough
benefit to be worth it. OTOH, the folks who benefit the most from such
tools are the non-tech oriented folks who have thousands of files dumped
on their desktop, the folks who download stuff, and then can't find where
they saved it, even tho they did the save just 30 seconds ago. These
folks simply have no clue about these black- boxes (umm, beige boxes)
called computers, or how they work, nor do they really care, they just
want to do whatever it is they're doing and not worry about it, and they
need all the help they can get. It's this type of folks that really find
such file indexing and search tools worthwhile, even if it does slow down
their system, because slow and being able to find that file they just
downloaded 30 seconds ago, beats fast and can't find it to use, any day.
So depending on how much you need fancy searches, etc, perhaps with kde
4.6 or so, you may want indexing on. But meanwhile, chances are you
aren't going to use it that much in kde 4.4, because the tools to really
make use of it simply aren't there or aren't mature enough to do so, yet,
so you might as well turn it off, for now.
Or alternatively, confine it to searching only part of your disk. Perhaps
you only care about your homedir, or the documents dir in your homedir,
and can tell it to turn off indexing for the rest of the system, thereby
dramatically reducing the time it takes doing that indexing.
As you suspected, system settings (which really should be called kde or
user settings, as few of them apply to the system as a whole, only to the
specific user while using kde, kde3's name, kcontrol, was *FAR* more
accurate in that regard, and *FAR* more effectively googlable as well!),
Advanced User Settings, Desktop Search. You control what it indexes on
the File Indexing tab, or simply turn off indexing entirely, on the Basic
Settings tab. (This applies to kde 4.4. I don't track what versions
various distributions ship with their various releases, as I run Gentoo,
with its rolling releases, and update kde from the gentoo/kde overlay
generally the day, certainly within the week, a new kde release comes out.)
> Perhaps one thing that might improve system response times is to run v-t
> less often - maybe once per hour, or per day. Is this possible? Useful?
> A good idea? What would need to be done/modified to achieve that?
>
> Is this KDE specific sw? Does GNOME also use this?
Nepomuk/strigi/virtuoso are pretty new, and generally kde-only at this
point. However, they're designed to be reasonably generic, usable by
other software as it's developed to do so. Also note that gnome, Apple,
and MS, have similar technologies at various stages of advancement. But
these happen to be what KDE's using and being new, pretty specific to kde,
ATM.
> == What is "kdeinit4: plasma-desktop" doing? In the service of what
> other sw?
>
> Where can settings for "k4 p-d", or the other sw, be viewed, & perhaps
> modified? In the "System Settings" program someplace?
Plasma-desktop is the desktop "shell", including the wallpaper and widgets
on the desktop, any panels (one by default) you have configured, the
kickoff menu (by default, the lancelot menu widget can replace kickoff,
and there's also the classic kmenu), etc.
It can be noted that while krunner (the kde run dialog) shares many of the
same base libraries, it's deliberately a separate app, so if either krunner
or plasma-desktop crash, you can still work (and restart the crashed bit)
using the other one. Similarly, khotkeys operates independent of plasma-
desktop, so hot-key launching still works when plasma-desktop is
crashed. Thus, it's quite possible to operate kde without plasma-
desktop, but as it's the shell that most folks will know as the "face" of
kde, it's not something most folks would find pleasant at all.
The kdeinit4 bit is somewhat of a technical trick. By starting all the kde
base apps (plasma-desktop, krunner, kwin, ksycoca4 (which stands for Kde
SYstem COnfig CAche 4) from kdeinit4, it allows them to share memory
resources and startup faster than they would otherwise. If an app crashes
or you shut it down manually, like I do with plasmadesktop occasionally
(issuing the usual SIGTERM with killall, to let it exit gracefully), you
can restart it, and it won't show the kdeinit4: bit in front, but it'll be
duplicating certain memory resources that were shared with the other kde
apps, when it was launched with kdeinit4.
So obviously, since plasma-desktop /is/ the desktop, changing the desktop
settings, desktop background, number of activities, adding and deleting
panels and widgets, all the stuff that you do to configure the desktop and
panels, is the config for plasma-desktop. =:^)
> == System tray "Search Service" - does what? Note: In the system tray,
> when this disk IO had been going on (starting at 1146 PM) for several
> minutes, I left clicked on the "Search Service" icon, selected "suspend
> file indexing", but the IO kept going for several minutes more, & from
> iotop it was v-t & k4 p-d running.
Disabling file indexing as above, should take care of that.
FWIW, the plasma-desktop activity you noted was due to updating the systray
status info, because the systray is one of the widgets running in plasma-
desktop. If there's nothing to update... But you'll almost certainly
always have /some/ plasma-desktop activity, particularly if you have any
system-monitor or other dynamically updating plasmoids running.
> Does that suspend mean "don't start it again", or "immediately"? Because
> it didn't stop immediately, but continued for several more minutes.
That I'm not actually sure. I would have guessed immediately, but
obviously, that's not what you're seeing. Given that, I'd say it may be
delayed, but that it should disable it for the rest of the session (or
until unsuspended, not permanently, like disabling it thru kcontrol does),
once it stops.
> == Where is the "file search" function? In KDE 3, IIRC, there was an
> item in the KDE start button to do a file search.
>
> I don't see a file search function anywhere under the K-Start button
> menu now.
If you're using the kde4 default kickoff menu, there should be a "Find
Files and Folders" menu item at the bottom of the Applications tab.
Lancelot also has that menu item, again on the applications tab, and so
does the classic menu (note that on it, the applications menu can be
disabled, and you'd lose the find dialog item on it as well). I just
checked all three. Of course, it's also possible kubuntu uses its own
customized launcher widget, in which case I wouldn't know.
Dolphin also has a menu item to activate the find dialog, and of course,
typing "find" in the run dialog pops up the find files and folder option,
likely among others.
Actually, one of krunner's runner applets (configurable by clicking on the
wrench icon) is a nepomuk based find, one of the places it *IS* already
enabled. So for instance, when I just typed "find" in krunner here, to
check that it did offer the find dialog, it also popped up a whole list of
various pdfs, shell scripts, etc, that it indexed, that happen to have the
word "find" in them. =:^)
So that's one real-world use, right there. Whether it's actually worth
the hassle of the indexing is of course your decision, but that's one
use. Why use the find files and folders dialog when typing the content in
question brings it up as an option in the run dialog? =:^)
> If v-t & "Search Service" are doing an indexing, what sw _uses_ that
> index to let the user do a search of the files on the disk?
> ===================================================================== ==
> System: KUbuntu 10.4 64 bit version, 4 core processor, 4GB RAM, TB hard
> disk
FWIW, as I mentioned above, 6 gigs here (main system, I also have a
netbook). I'm running a now aging but still very capable dual Opteron 290
(so dual-dual-core, 2.8 GHz), and at one point had two, two-gig sticks,
hanging off each processor. But one stick went bad and I've not replaced
it yet, so it's 6 gigs RAM ATM. But if I had it to do over, especially at
what those sticks of registered/ecc RAM that the Opterons of that era
required cost, I'd have stuck with four 1-gig sticks, 4 gig total. I do
run Gentoo, so build from source, and have the temporary build dir setup
as a tmpfs so it's in RAM, which makes having the 8 (now 6) gig nice, but
4 gig is really the sweet spot. At what RAM costs now, sure, 8 gig, but
IIRC I paid nearly $1000 a stick, and while it's nice to have the extra
memory, at that price, there were better ways to spend my money.
Meanwhile, even accounting for the effect of that extra memory, I've
noticed that I seem to have less problems with disk activity than most,
even under heavy I/O. And I simply don't have the slowdowns others seem
to have with Virtuoso, etc, even with full indexing enabled.
I've come to the conclusion that there's several factors involved in this.
1) This is the latest one I discovered. As I mentioned my system's now
aging. It still has PCI buses, no PCI-E (tho Opteron being a server CPU,
and with dual-socket especially, I'm running on a server board, so they're
64-bit PCI-X not the old 32-bit PCI, but it's still bus architecture, not
the point-to-point of PCI-E). As such, I'm not sure if this applies to
newer PCI-E systems or not, but it certainly made a difference on my PCI/
PCI-X system: In the BIOS, there's a PCI Latency Timer setting. This is
a characteristic latency vs. thruput tradeoff, with higher values meaning
less overhead and thus better thruput, by allowing each device to use the
bus for a longer period. But with multiple devices sharing the bus, the
longer any one gets to use it, the longer other devices have to wait to
use it.
Under high I/O, higher values allow latency to get high enough to cause
sound stuttering and even UI freezes. I recently discovered that by
setting the PCI Latency Timer to the lowest possible value, I could avoid
audio skips even under extreme I/O pressure. At a notch higher, I get
music stuttering, but the UI's fine. As I go higher still, the UI can lag
out as well.
What was interesting to me, is that setting PCI Latency Timer low enough,
I could lower my kernel tick rate to server-class 100 Hz and similarly set
server class no preemptivity, and STILL have better responsiveness, than I
did with even a single notch higher PCI Latency, with standard kernel
desktop preemptivity and 300 Hz clocking.
2) I've been running 4-disk md/kernel RAID for a number of years now.
I've tried both RAID-6 and 4-way RAID-1 for the main system. The Linux
kernel I/O scheduler is VERY impressive in its ability to handle multiple
concurrent I/O tasks with md/RAID-1 (which works far better than RAID-6
for my purposes). I've been EXTREMELY pleased with md/RAID-1, and believe
the kernel's ability to efficiently multi-task I/O on it is a major factor
in why I literally don't notice disk access issues that are major problems
for a more typical single-disk install.
So yes, it's as a longer term recommendation -- I don't expect people to
jump up and go RAID right now -- but having seen the benefits of RAID-1 in
system interactivity (not to mention its primary purpose, data protection
thru redundant data storage), I wouldn't want to go back, personally, and
IMO it's a cryin' shame that at the cost of disks these days, 2-way RAID-1
minimum isn't standard.
3) I mentioned swappiness, above. The swappiness knob is located at
/proc/sys/vm/swappiness. This controls how the kernel balances
applications vs. disk cache. To change the value temporarily (as for
testing), write (as root) a value between 0 and 100 to the file. To make
your change permanent, either setup a script to trigger such a write at
boot, or set it up via the sysctl service that most distributions run at
boot. Here on Gentoo, that means adding a vm.swappiness = <number> line
to /etc/sysctl.conf. (FWIW, if you compare the files under /proc/sys/ to
the entries in your sysctl config, you should quickly see the
relationship. Now you have a whole list of other knobs to google on and
experiment with! =:^)
A swappiness value of 0 tells the kernel to ALWAYS favor running
applications over cache -- the only time an app gets swapped out is if
there's no disk cache left to dump in its place. A value of 100 is the
reverse -- the kernel will always try to keep cache if at all possible,
dumping apps into swap in ordered to do so. The kernel default (ubuntu
may change it) is 60, slightly favoring cache over apps.
Now I mentioned that I have four disks and am striping swap over all four
(using equal swap priority settings), so in theory, fetching applications
back from swap should be four times as fast as reading them back in off
the RAID-1 main system filesystem. As such, I'm a bit of a special case,
and have swappiness set to 100.
But from what I've read, many people with only one disk, or multiple disks
but not in RAID, find that a a swappiness value of around 20 works far
better for them than the default 60. A value of 20 will reasonably
strongly favor keeping apps in memory, at the expense of dumping disk
cache when necessary, but will start swapping out apps before it dumps all
cache, unlike the extreme value of 0, mentioned above.
So if you suspect you're having responsiveness issues due to cached data
forcing apps out to swap, do experiment with swappiness. You may well
find, as have many others, that a value rather lower than the default 60
works better for you.
Hope at least some of that's useful! =:^)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
___________________________________________________
This message is from the kde mailing list.
Account management: https://mail.kde.org/mailman/listinfo/kde.
Archives: http://lists.kde.org/.
More info: http://www.kde.org/faq.html.
