Re: debugging hang on headless server

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi

So access to the server is required, if there are other servers in the network, then netconsole will send to the remote box over same syslog idea logs that get printed to the console

syslog for logs that go to the normal log files

and kdump will be able to generate a memory dump and some logs if there is any crash

with that you should be able to have ll the information outside this server when there is a hang

othertools that can be user, are sar/sysstat you can copy the files out for inspection and get an idea about performance and what was doing what before a hang

good luck
alvaro

> On 5 Jul 2017, at 03:54, Alexandr <sss@xxxxxxxxxxxxxxx> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> В Вт, 04/07/2017 в 07:48 -0500, Billy Crook пишет:
>> You can set up syslog to forward of to an external system.  That
>> helps but
>> sometimes it can miss things if the crash is severe enough.
>> 
>> Serial console can capture much more if there's another server near
>> by that
>> you can connect and leave minicom logging.  Even better if you have a
>> nice
>> physical "serial console server" like an OpenGear.
>> 
>> Most server motherboards these days contain some sort of management
>> chip
>> implementing ipmi and additional remote access features which often
>> include
>> remote KVM over IP.
>> 
>> Even the most basic ipmi controllers will include serial over LAN
>> which
>> let's do you use serial console without a server nearby to connect a
>> physical serial cable.
>> 
>> If the hang is caused by a hardware problem you may find indication
>> of it
>> in the ipmi system event list.  'ipmitool sel list'
>> 
>> There's also a kernel module, netconsole, that forwards kernel
>> messages off
>> host in a syslog compatible format. It implements IP and UDP and its
>> own
>> code so it works independent of the system's TCP stack which is
>> important
>> during some crashes which might otherwise break networking before the
>> messages get transmitted.
>> 
>> That's just a few ways.  There's probably others. Hope it helps
>> 
>>> On Jul 4, 2017 07:32, "Alexandr" <sss@xxxxxxxxxxxxxxx> wrote:
>>> 
>>> Good day to all.
>>> please suggest methods of system hang debugging on headless server
>>> ?
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-
>>> admin" in
>>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> 
> 
> thx for reply.
> it's self built server on regular desktop hardware, so have not ipmi.
> it's possible to have phisical access to it, so i guess serial console
> is my solution.
> i am also have few questions about serial console setup.
> as i understand simplest way to achieve it is default serial com port,
> i am also have read something about serial via usb (looks like it
> require some sort of special adaptor)
> 
> also netconsole looks interesting too, i will setup it right now.
> -----BEGIN PGP SIGNATURE-----
> 
> iQIzBAEBCgAdFiEEl/sA7WQg6czXWI/dsEApXVthX7wFAllcRvEACgkQsEApXVth
> X7w5DA//VnlRsWDxy0RifqMrmBtiL1/fxdl3Wz/PhH/Fdu+AVgM2XdaITYHfI0uU
> 7l6bYPPnMnXlmiqtJ0DkYEV+Xgq7Gkuw3JK8eppLHehgMfaIEmnXWPMoxVVZalmn
> u1HVGBdEGfRtiLisrNvZvGgeCwXdGIfMeZbnBFwFbG3vbxzHzaGd7mPTO9CvhPyj
> FArTHF+i3DtVpteXbww78JS0qEKLTh83cW1FcCqAWjZBvrnz35i7Egp4grajhhzQ
> ki8wFUqwwVWY+JVg6UcpGr5CH4b11b4qv3Vol/ideooiTHWPv1xKXMjT3+DIggzc
> gh0g6ZwK3IDcR2rtJ2lGD/UOi4jJNti8A6O/48sXIsgruePuDIPNJOIbBbRR5EDv
> cDlb6vVHA0QO6XilUcTKFPEjg9yG++JIyYHmGrGxJGo28KpF+GbStA1xgm98Mbqj
> AJP/mZIdLCoKuRGEGkPCvHAX4BfMm4zdPmBcj3XAiw9xGJCFuts0xowX9uSqnzT6
> ITlVA9VHWlm030TFYp0h2HYjuzpVrKNw06OQ0AqHy7E932DlPr3us2y7ugWGXUgE
> Sxewc2+QXR2RGEHXVWfbymGr6uKtqwmwuyl8q7QQ/oqYGFK99WblRMT+6+K0kJI6
> f/YzGyAqHRtN2gBZvRjX2BYVPdPKcGbNKqp27B9wOkz58NNRpF8=
> =PmkQ
> -----END PGP SIGNATURE-----
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-admin" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-admin" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Newbie]     [Audio]     [Hams]     [Kernel Newbies]     [Util Linux NG]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux Device Drivers]     [Samba]     [Video 4 Linux]     [Git]     [Fedora Users]

  Powered by Linux