Re: (EXT) CentOS Digest, Vol 162, Issue 29

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



Send CentOS mailing list submissions to
	centos@xxxxxxxxxx

-----Original Message-----
From: CentOS <centos-bounces@xxxxxxxxxx> On Behalf Of centos-request@xxxxxxxxxx
Sent: Tuesday, July 31, 2018 5:30 PM
To: centos@xxxxxxxxxx
Subject: (EXT) CentOS Digest, Vol 162, Issue 29

Send CentOS mailing list submissions to
	centos@xxxxxxxxxx

To subscribe or unsubscribe via the World Wide Web, visit
	https://lists.centos.org/mailman/listinfo/centos
or, via email, send a message with subject or body 'help' to
	centos-request@xxxxxxxxxx

You can reach the person managing the list at
	centos-owner@xxxxxxxxxx

When replying, please edit your Subject line so it is more specific than "Re: Contents of CentOS digest..."


Today's Topics:

   1. Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (Yannis Milios)
   2. Re: Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (Simon Matter)
   3. Re: Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (Yannis Milios)
   4. Re: Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (mark)
   5. Re: Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (Yannis Milios)
   6. Re: Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (mark)
   7. Re: Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (Yannis Milios)
   8. Re: Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (Simon Matter)
   9. Issues booting centos7 [dracut is failing to enable
      centos/root, centos/swap LVs] (Yannis Milios)
  10. Increase Disk Space with LVM (Felix K?lzow)


----------------------------------------------------------------------

Message: 1
Date: Mon, 30 Jul 2018 16:26:43 +0100
From: Yannis Milios <yannis.milios@xxxxxxxxx>
To: centos@xxxxxxxxxx
Subject:  Issues booting centos7 [dracut is failing to enable
	centos/root, centos/swap LVs]
Message-ID:
	<CAFiF2OpdpWuVLyZq7_4n1MxtMr2ypJkgKhLt+03ayapexgNuqA@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="UTF-8"

Hello,

I'm having a strange problem booting a new centos7 installation. Below some background on this. [I have attached the tech details at the bottom of this message]

I started a new CentOS7 installation on a VM, so far all good, o/s boots fine. Then I decided to increase VM disk size (initially was 10G) to 13G.
Powered off the VM, increased the vhd via the hypervisor, booted from CentOS livecd, selected "recover my centos installation". Then I used the following sequence of commands to make the new vhd size "visible" to the o/s ...

- ran fdisk /dev/sda and deleted partition 2. (This is the LVM partition where the o/s is stored. The first partition is /boot [xfs] partition *non
lvm*.)
- created a new partition with same starting sector as before deleting it, but a different ending sector (to reflect to the increased space).
- set partition type to 8e (lvm), saved settings and exit.
- ran pvresize and lvresize to make new space visible to the o/s.All good, I can see the space increase on 'lvdisplay centos/root'.
- ran 'mount /dev/mapper/centos-root /mnt/root' to temporarily mount the o/s partition.
- ran 'xfs_growfs /mnt/root' to resize the o/s fs to the new size. It was successful and I could actually chroot to the o/s and verify new disk size.
- rebooted  and tried to boot from hdd this time. Grub menu shows up and loads default kernel (3.10.0-862.9.1.el7.x86_64).
- after initial kernel boot up process, booting continues to initrd and then it's where the problem starts...
- looks like dracut has issues locating/enabling /dev/mapper/centos-root
(lv) and as a result it cannot boot to the 'real' root fs (/).
- while in dracut shell, I execute the following command sequence:
      1. lvm lvchange -ay /dev/centos/root
      2. lvm lvchange -ay /dev/centos/swap
      3. ln -s /dev/mapper/centos-root /dev/root
      4. exit
...and the o/s boots fine...so looks like the pv,vg,lv is detected properly while in initrd, but somehow dracut has difficulties enabling the root,swap LVs ?

- While in o/s, I rebuild initrd by using: 'dracut -f -v -a lvm'
Note, I have to use '-a lvm' as for some reason, if don't, the lvm utils
(lvm,lvm_scan) are not being included to the initrd, not sure why this happens.
After rebooting, same thing happens, I have to manually boot the system via dracut shell.

I'm a bit stuck at this point, any clues what I'm doing wrong in here ?

As I previously said, this is a new installation, so I could simply reinstall the whole thing, but I would rather try to find out some answer to why is this happening ... :-) [curiosity]

Some additional details:
==================
I've attached /run/initramfs/rdsosreport.txt file [generated while in inird] here:
https://privatebin.net/?fdc4052c0c402884#gdB/QYR3IeR55SxUbjfrkZPQfJ7jMxiUxq5DWhTVbWc=

o/s ver=CentOS Linux release 7.5.1804 (Core)

[root@localhost ~]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   13G  0 disk
??sda1            8:1    0  500M  0 part /boot
??sda2            8:2    0 12.5G  0 part
  ??centos-root 253:0    0 11.5G  0 lvm  /
  ??centos-swap 253:1    0    1G  0 lvm  [SWAP]

[root@localhost ~]# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rd.shell rhgb"
GRUB_DISABLE_RECOVERY="true"


------------------------------

Message: 2
Date: Mon, 30 Jul 2018 17:53:56 +0200
From: "Simon Matter" <simon.matter@xxxxxxxxx>
To: "CentOS mailing list" <centos@xxxxxxxxxx>
Subject: Re:  Issues booting centos7 [dracut is failing to
	enable centos/root, centos/swap LVs]
Message-ID:
	<09679ddf131cfabaaa9a38ee3a874541.squirrel@xxxxxxxxxxxxxxxxxxxx>
Content-Type: text/plain;charset=utf-8

Hi,

> Hello,
>
> I'm having a strange problem booting a new centos7 installation. Below 
> some background on this. [I have attached the tech details at the 
> bottom of this message]
>
> I started a new CentOS7 installation on a VM, so far all good, o/s 
> boots fine. Then I decided to increase VM disk size (initially was 10G) to 13G.
> Powered off the VM, increased the vhd via the hypervisor, booted from 
> CentOS livecd, selected "recover my centos installation". Then I used 
> the following sequence of commands to make the new vhd size "visible" 
> to the o/s ...
>
> - ran fdisk /dev/sda and deleted partition 2. (This is the LVM 
> partition where the o/s is stored. The first partition is /boot [xfs] 
> partition *non
> lvm*.)
> - created a new partition with same starting sector as before deleting 
> it, but a different ending sector (to reflect to the increased space).
> - set partition type to 8e (lvm), saved settings and exit.

At this point I usually reboot and do the following steps while the system is running. I don't say how you do it is wrong, it's just easier/faster to do things while the system is online if downtime matters.

I don't really know what causes your issues. The reason may be that I have more and more problems understanding the whole boot process and all the variants we used to have over more than two decades...

But, what happens if you let the kernel post install scripts do the work or setting up the initrd things? If you reinstall the latest kernel, does it change anything?

Regards,
Simon

> - ran pvresize and lvresize to make new space visible to the o/s.All 
> good, I can see the space increase on 'lvdisplay centos/root'.
> - ran 'mount /dev/mapper/centos-root /mnt/root' to temporarily mount 
> the o/s partition.
> - ran 'xfs_growfs /mnt/root' to resize the o/s fs to the new size. It 
> was successful and I could actually chroot to the o/s and verify new 
> disk size.
> - rebooted  and tried to boot from hdd this time. Grub menu shows up 
> and loads default kernel (3.10.0-862.9.1.el7.x86_64).
> - after initial kernel boot up process, booting continues to initrd 
> and then it's where the problem starts...
> - looks like dracut has issues locating/enabling 
> /dev/mapper/centos-root
> (lv) and as a result it cannot boot to the 'real' root fs (/).
> - while in dracut shell, I execute the following command sequence:
>       1. lvm lvchange -ay /dev/centos/root
>       2. lvm lvchange -ay /dev/centos/swap
>       3. ln -s /dev/mapper/centos-root /dev/root
>       4. exit
> ...and the o/s boots fine...so looks like the pv,vg,lv is detected 
> properly while in initrd, but somehow dracut has difficulties enabling 
> the root,swap LVs ?
>
> - While in o/s, I rebuild initrd by using: 'dracut -f -v -a lvm'
> Note, I have to use '-a lvm' as for some reason, if don't, the lvm 
> utils
> (lvm,lvm_scan) are not being included to the initrd, not sure why this 
> happens.
> After rebooting, same thing happens, I have to manually boot the 
> system via dracut shell.
>
> I'm a bit stuck at this point, any clues what I'm doing wrong in here ?
>
> As I previously said, this is a new installation, so I could simply 
> reinstall the whole thing, but I would rather try to find out some 
> answer to why is this happening ... :-) [curiosity]
>
> Some additional details:
> ==================
> I've attached /run/initramfs/rdsosreport.txt file [generated while in 
> inird] here:
> https://privatebin.net/?fdc4052c0c402884#gdB/QYR3IeR55SxUbjfrkZPQfJ7jM
> xiUxq5DWhTVbWc=
>
> o/s ver=CentOS Linux release 7.5.1804 (Core)
>
> [root@localhost ~]# lsblk
> NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
> sda               8:0    0   13G  0 disk
> ??sda1            8:1    0  500M  0 part /boot
> ??sda2            8:2    0 12.5G  0 part
>   ??centos-root 253:0    0 11.5G  0 lvm  /
>   ??centos-swap 253:1    0    1G  0 lvm  [SWAP]
>
> [root@localhost ~]# cat /etc/default/grub
> GRUB_TIMEOUT=5
> GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
> GRUB_DEFAULT=saved
> GRUB_DISABLE_SUBMENU=true
> GRUB_TERMINAL_OUTPUT="console"
> GRUB_CMDLINE_LINUX="rd.lvm.lv=centos/root rd.lvm.lv=centos/swap 
> rd.shell rhgb"
> GRUB_DISABLE_RECOVERY="true"
> _______________________________________________
> CentOS mailing list
> CentOS@xxxxxxxxxx
> https://lists.centos.org/mailman/listinfo/centos
>




------------------------------

Message: 3
Date: Mon, 30 Jul 2018 17:32:53 +0100
From: Yannis Milios <yannis.milios@xxxxxxxxx>
To: centos@xxxxxxxxxx
Subject: Re:  Issues booting centos7 [dracut is failing to
	enable centos/root, centos/swap LVs]
Message-ID:
	<CAFiF2Or=a_2b+e+jj9AeUufnZg4N+rHvS8KJHF+LoBDJ+QcRNg@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="UTF-8"

>
> But, what happens if you let the kernel post install scripts do the 
> work or setting up the initrd things?


Initially, I just rebooted from LiveCD and left Grub,kernel and dracut do their job with the defaults, but unfortunately, boot process stops at initrd (dracut) stage with the following error:

[  197.768159] localhost.localdomain dracut-initqueue[252]: Warning: Could
> not boot.
> [  197.821409] localhost.localdomain systemd[1]: Received SIGRTMIN+20 
> from PID 254 (plymouthd).
> [  197.822328] localhost.localdomain dracut-initqueue[252]: Warning:
> /dev/centos/root does not exist
> [  197.823044] localhost.localdomain dracut-initqueue[252]: Warning:
> /dev/centos/swap does not exist
> [  197.846807] localhost.localdomain systemd[1]: Starting Dracut 
> Emergency Shell...
> [  197.870983] localhost.localdomain systemd[1]: Received SIGRTMIN+21 
> from PID 254 (plymouthd).


Then, I'm dropped to dracut shell, where I boot the system by using the lvm commands as described to my first post.
Once in O/S, I re-create initrd by using 'dracut -f -v -a lvm' command and reboot again the system.
But still I get the same error during boot ...

If you reinstall the latest kernel, does
> it change anything?
>

 This is actually the latest kernel (3.10.0-862.9.1.el7.x86_64) but I tried to boot with the second (older) kernel (vmlinuz-3.10.0-327.36.3.el7.x86_64), where I got exactly the same error during boot.

Clearly, this is not a kernel issue, perhaps not even a dracut (initrd) issue, since everything was working fine, before resizing the PV/LV. I suspect that something went wrong during that step, but what? As I can successfully boot the system manually via dracut, normally everything should be ok? no?

What can prevent dracut from activating the 2 LVs (centos/root and
centos/swap) during initial boot phase ?

Thanks,
Yannis


------------------------------

Message: 4
Date: Mon, 30 Jul 2018 15:13:08 -0400
From: "mark" <m.roth@xxxxxxxxx>
To: "CentOS mailing list" <centos@xxxxxxxxxx>
Subject: Re:  Issues booting centos7 [dracut is failing to
	enable centos/root, centos/swap LVs]
Message-ID:
	<e9796451bea645de4bc4fbe210591182.squirrel@xxxxxxxxxxxxxxxxxxxxxxx>
Content-Type: text/plain;charset=utf-8

Yannis Milios wrote:
>> But, what happens if you let the kernel post install scripts do the 
>> work or setting up the initrd things?
>
> Initially, I just rebooted from LiveCD and left Grub,kernel and dracut 
> do  their job with the defaults, but unfortunately, boot process stops 
> at initrd (dracut) stage with the following error:
>
> [  197.768159] localhost.localdomain dracut-initqueue[252]: Warning:
> Could
>
>> not boot. [  197.821409] localhost.localdomain systemd[1]: Received
>> SIGRTMIN+20 from
>> PID 254 (plymouthd).
>> [  197.822328] localhost.localdomain dracut-initqueue[252]: Warning:
>> /dev/centos/root does not exist
>> [  197.823044] localhost.localdomain dracut-initqueue[252]: Warning:
>> /dev/centos/swap does not exist
>> [  197.846807] localhost.localdomain systemd[1]: Starting Dracut 
>> Emergency Shell...
>> [  197.870983] localhost.localdomain systemd[1]: Received SIGRTMIN+21 
>> from PID 254 (plymouthd).
>>
>
>
> Then, I'm dropped to dracut shell, where I boot the system by using 
> the lvm commands as described to my first post. Once in O/S, I 
> re-create initrd by using 'dracut -f -v -a lvm' command and reboot 
> again the system. But still I get the same error during boot ...
>
> If you reinstall the latest kernel, does
>
>> it change anything?
>>
>
> This is actually the latest kernel (3.10.0-862.9.1.el7.x86_64) but I 
> tried to boot with the second (older) kernel 
> (vmlinuz-3.10.0-327.36.3.el7.x86_64), where I got exactly the same 
> error during boot.
>
> Clearly, this is not a kernel issue, perhaps not even a dracut 
> (initrd) issue, since everything was working fine, before resizing the 
> PV/LV. I suspect that something went wrong during that step, but what? 
> As I can successfully boot the system manually via dracut, normally 
> everything should be ok? no?
>
> What can prevent dracut from activating the 2 LVs (centos/root and
> centos/swap) during initial boot phase ?
>
Suggestion: once it's up, rebuild the initramfs.

      mark



------------------------------

Message: 5
Date: Mon, 30 Jul 2018 22:14:44 +0100
From: Yannis Milios <yannis.milios@xxxxxxxxx>
To: CentOS mailing list <centos@xxxxxxxxxx>
Subject: Re:  Issues booting centos7 [dracut is failing to
	enable centos/root, centos/swap LVs]
Message-ID:
	<CAFiF2OrJUiNh35M6PBxe=CQrtF1dRYb14b5H4pewTX-EsSm-TA@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="UTF-8"

Suggestion: once it's up, rebuild the initramfs.
>
>
I tried that already, but still the same problem.
Aparently dracut does not want to activate the LVs required to boot to the root filesystem, for some reason ...

Yannis


--
Sent from Gmail Mobile


------------------------------

Message: 6
Date: Mon, 30 Jul 2018 17:16:51 -0400
From: "mark" <m.roth@xxxxxxxxx>
To: "CentOS mailing list" <centos@xxxxxxxxxx>
Subject: Re:  Issues booting centos7 [dracut is failing to
	enable centos/root, centos/swap LVs]
Message-ID:
	<ce2b2fbed114eda6c7159b25098d48d1.squirrel@xxxxxxxxxxxxxxxxxxxxxxx>
Content-Type: text/plain;charset=utf-8

Yannis Milios wrote:
> Suggestion: once it's up, rebuild the initramfs.
>>
> I tried that already, but still the same problem.
> Aparently dracut does not want to activate the LVs required to boot to the
>  root filesystem, for some reason ...
>
At this point, I'd start wondering about the grub2 defaults, and the
kernel command line.

     mark



------------------------------

Message: 7
Date: Mon, 30 Jul 2018 23:06:15 +0100
From: Yannis Milios <yannis.milios@xxxxxxxxx>
To: CentOS mailing list <centos@xxxxxxxxxx>
Subject: Re:  Issues booting centos7 [dracut is failing to
	enable centos/root, centos/swap LVs]
Message-ID:
	<CAFiF2Or5Ave-rhSCMmcD-h7X-sbhGsAGRG6psc0=jxdma7rN5g@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="UTF-8"

On Monday, July 30, 2018, mark <m.roth@xxxxxxxxx>
>
> At this point, I'd start wondering about the grub2 defaults, and the
> kernel command line.
>
>
This is Grub config..

[root@localhost ~]# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rd.shell
rhgb"
GRUB_DISABLE_RECOVERY="true"



-- 
Sent from Gmail Mobile


------------------------------

Message: 8
Date: Tue, 31 Jul 2018 08:22:12 +0200
From: "Simon Matter" <simon.matter@xxxxxxxxx>
To: "CentOS mailing list" <centos@xxxxxxxxxx>
Subject: Re:  Issues booting centos7 [dracut is failing to
	enable centos/root, centos/swap LVs]
Message-ID:
	<970e648731344f68e1a1837a94b71d41.squirrel@xxxxxxxxxxxxxxxxxxxx>
Content-Type: text/plain;charset=utf-8

> Yannis Milios wrote:
>>> But, what happens if you let the kernel post install scripts do the
>>> work or setting up the initrd things?
>>
>> Initially, I just rebooted from LiveCD and left Grub,kernel and dracut
>> do
>>  their job with the defaults, but unfortunately, boot process stops at
>> initrd (dracut) stage with the following error:
>>
>> [  197.768159] localhost.localdomain dracut-initqueue[252]: Warning:
>> Could
>>
>>> not boot. [  197.821409] localhost.localdomain systemd[1]: Received
>>> SIGRTMIN+20 from
>>> PID 254 (plymouthd).
>>> [  197.822328] localhost.localdomain dracut-initqueue[252]: Warning:
>>> /dev/centos/root does not exist
>>> [  197.823044] localhost.localdomain dracut-initqueue[252]: Warning:
>>> /dev/centos/swap does not exist
>>> [  197.846807] localhost.localdomain systemd[1]: Starting Dracut
>>> Emergency
>>> Shell...
>>> [  197.870983] localhost.localdomain systemd[1]: Received SIGRTMIN+21
>>> from PID 254 (plymouthd).
>>>
>>
>>
>> Then, I'm dropped to dracut shell, where I boot the system by using the
>> lvm commands as described to my first post. Once in O/S, I re-create
>> initrd
>> by using 'dracut -f -v -a lvm' command and reboot again the system. But
>> still I get the same error during boot ...
>>
>> If you reinstall the latest kernel, does
>>
>>> it change anything?
>>>
>>
>> This is actually the latest kernel (3.10.0-862.9.1.el7.x86_64) but I
>> tried to boot with the second (older) kernel
>> (vmlinuz-3.10.0-327.36.3.el7.x86_64), where I got exactly the same error
>> during boot.
>>
>> Clearly, this is not a kernel issue, perhaps not even a dracut (initrd)
>> issue, since everything was working fine, before resizing the PV/LV. I
>> suspect that something went wrong during that step, but what? As I can
>> successfully boot the system manually via dracut, normally everything
>> should be ok? no?
>>
>> What can prevent dracut from activating the 2 LVs (centos/root and
>> centos/swap) during initial boot phase ?
>>
> Suggestion: once it's up, rebuild the initramfs.

And when you did so, did you remove the "rd.shell" from GRUB_CMDLINE_LINUX
before?

Regards,
Simon



------------------------------

Message: 9
Date: Tue, 31 Jul 2018 08:40:52 +0100
From: Yannis Milios <yannis.milios@xxxxxxxxx>
To: CentOS mailing list <centos@xxxxxxxxxx>
Subject:  Issues booting centos7 [dracut is failing to enable
	centos/root, centos/swap LVs]
Message-ID:
	<CAFiF2Or3nxaJcY44mWjosiTbwb==w2hU95iMpMC4=sTY7Wvn2A@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="UTF-8"

And when you did so, did you remove the "rd.shell" from GRUB_CMDLINE_LINUX
> before?
>
>
Yes, basically that option was not there by default, I added it during the
troubleshootig steps. Normally, rd.lvm.lv=centos/root and rd.lvm.lv=centos/swap
are the kernel parameters which are parsed to dracut for enablig the two
LVs, but somehow dracut does not.

I've copied the log from dracut in here, can this be helpful to locate the
problem?

https://privatebin.net/?fdc4052c0c402884#gdB/QYR3IeR55SxUbjfrkZPQfJ7jMxiUxq5DWhTVbWc=




-- 
Sent from Gmail Mobile


------------------------------

Message: 10
Date: Tue, 31 Jul 2018 13:31:15 +0200
From: Felix K?lzow <felix.koelzow@xxxxxx>
To: centos@xxxxxxxxxx
Subject:  Increase Disk Space with LVM
Message-ID: <1dd31e6b-ee9e-4a00-1bf2-22c0c2f0e699@xxxxxx>
Content-Type: text/plain; charset=utf-8; format=flowed

Dear CentOS-Community,

we have a server with four hard drives that are configured as raid10 
(/dev/sda).

Now, /home and /root are almost full. Therefore, we decided to buy four

additional hard drives that should configured also as raid 10 (/dev/sdb).


I want to use LVM to extend disk space for root and home.

My (successful) test procedure in a virtual environment looks like this:


 1. devide /dev/sdb into /dev/sdb1 for root and /dev/sdb2 for home using
    parted
 2. Convert disk to physical volume: pvcreate /dev/sdb1
 3. add physical volume to volume group (called centos): vgextend centos
    /dev/sdb1
 4. Allocate physical volume to a logical volume:lvextend -l +100$FREE
    /dev/centos/root
 5. resize2fs /dev/centos/root or xfs_grows /dev/centos/root depending
    on file system used
 6. repeat steps 2-6 for /home and sdb2


The mentioned instruction I've got from this page:

https://askubuntu.com/questions/458476/adding-disks-with-lvm#459176


Now my question:

Is this procedure really safe or is something missing or are there some 
steps

which I overlooked actually?


Kind Regards

Felix K?lzow



------------------------------

Subject: Digest Footer

_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos


------------------------------

End of CentOS Digest, Vol 162, Issue 29
***************************************
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos



[Index of Archives]     [CentOS]     [CentOS Announce]     [CentOS Development]     [CentOS ARM Devel]     [CentOS Docs]     [CentOS Virtualization]     [Carrier Grade Linux]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [Xorg]     [Linux USB]


  Powered by Linux