Re: ansible stalls because the inventory is too big

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On 03/28/2016 12:33 AM, Loic Dachary wrote:
[resuming the previous mail, sent too early by mistake :-]

On 28/03/2016 00:29, Loic Dachary wrote:
Hi Thierry,

When teuthology runs ansible on a newly created host as:

/home/ubuntu/teuthology/virtualenv/bin/python /home/ubuntu/teuthology/virtualenv/bin/ansible-playbook -v --extra-vars {"ansible_ssh_user": "ubuntu"} -i /etc/ansible/hosts --limit target010011010028.teuthology /home/ubuntu/src/ceph-cm-ansible_master/cephlab.yml

it stalls because there are too many hosts in the inventory (65537) and

newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203193.teuthology", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203193.teuthology.yml", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203193.teuthology.yaml", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203193.teuthology.json", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/etc/ansible/hosts/", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/etc/ansible/hosts", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203194.teuthology", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203194.teuthology.yml", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203194.teuthology.yaml", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/etc/ansible/hosts/host_vars/target010011203194.teuthology.json", 0x7fc8f1a4f0, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)

takes forever. This is running on an ARMv8 virtual machine:

(virtualenv)ubuntu@teuthology:~/teuthology$ cat /proc/cpuinfo
Processor	: AArch64 Processor rev 1 (aarch64)
processor	: 0
Features	: fp asimd evtstrm
CPU implementer	: 0x50
CPU architecture: AArch64
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 1

Hardware	: linux,dummy-virt

which also explains why it's not very fast. Do you know of a way to ask ansible to only consider one host instead of what appears to be a systematic exploration of all possible hosts in the inventory, only to finally select the one from the --limit argument ?
Salut loic,

I didn't test, I don't examine how exactly ansible implement inventory construction and I can't presently duplicate your testing environment. This said, You could try to change ceph-cm-ansible/teuthology.yml to allow host restriction via play hosts directive rather than limit parameter.

echo -e "/hosts: teuthology/s/teuthology/\"{{ targets | default('teuthology') }}\"/\nwq" | ed ceph-cm-ansible/teuthology.yml

And then use a extra-vars param as usual.

The idea being that when using limit parameter you may need to access other hosts facts to achieve your play on limited target (or for delegated actions) and thus need to construct the full set of hosts facts.

If I guessed right, when limiting via hosts directive you explicitly choose a hosts subset (usually a group) from inventory before running the play rather than applying a filter at a later time.

If this works, some tasks may fail because you really need some other hosts from the inventory (say to enable delegated actions), but you could refine your targets variables by adding more hosts.



--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux