Hi Casey,
On Fri, Mar 16, 2018 at 3:52 PM, Casey Schaufler <casey@xxxxxxxxxxxxxxxx> wrote:
> On 3/16/2018 7:49 AM, Mimi Zohar wrote:
>> On Fri, 2018-03-16 at 14:34 +0000, Martin Townsend wrote:
>>> Hi Mimi,
>>>
>>> On Fri, Mar 16, 2018 at 1:25 PM, Mimi Zohar <zohar@xxxxxxxxxxxxxxxxxx> wrote:
>>>> On Fri, 2018-03-16 at 09:32 +0000, Martin Townsend wrote:
>>>>> [Resending to new integrity mailing list]
>>>>>
>>>>> Hi,
>>>>>
>>>>> I have a system with a pre-signed UBI root filesystem image with both
>>>>> IMA/EVM signatures on all files. The Root CA Cert is compiled into
>>>>> the kernel and the public keys is in the rootfs. All SMACK labels
>>>>> have also been applied although at this early stage there aren't many
>>>>> (just a few application specific ones) so it's mainly the defaults.
>
> If you can apply a label to all of the files on the filesystem
> ( find $FSROOT -exec chsmack -a "_" {} \; ) and then set the
> specific file labels you want you will ensure that you won't
> go through the label defaulting mechanism. I don't think that
> is your problem, but it's worth a try.
>
I tried this but no joy.
>>>>> This image is then flashed to the on board NAND.
>>>>>
>>>>> The kernel bootargs for IMA are
>>>>>
>>>>> "ima_audit=1 ima_template=ima-ng ima_hash=sha1 ima_tcb
>>>>> ima_appraise_tcb rootflags=i_version"
>>>>>
>>>>> and I'm enabling SMACK by using the kernel bootarg
>>>>>
>>>>> "security=smack"
>>>>>
>>>>> now if I boot without the "security=smack" it boots fine and I can
>>>>> check the IMA/EVM signatures and can see that measurements are being
>>>>> taken, but if I enable SMACK using the above kernel bootarg it fails
>>>>> to boot and it looks like some problem early in systemd where it
>>>>> mounts the required filesystems in mount-setup.c (log provided below).
>>>>> Now if I flash an image that hasn't been signed and enable SMACK it
>>>>> boots fine and I can use SMACK to enforce access control. So there
>>>>> seems to some interaction between the two when mounting the early
>>>>> filesystems.
>>>>>
>>>>> Before I delve into this I would appreciate any pointers to where to
>>>>> start looking, any printk's to put in SMACK/IMA/mount code to help
>>>>> diagnose this would be really appreciated.
>>>>>
>>>>> The Kernel is 4.9 LTSI, systemd is v229
>
> Are you using a base distribution? Fedora, Ubuntu, Debian, Alpine?
> That information may make the issue easier to reproduce.
It's a linux-yocto distribution for an i.MX6UL board. I'm trying to
get QEMU working to make it easier to debug and reproduce but alas
this may prove harder to achieve than debugging this issue with
printk's
>
>>>>>
>>>>> Apologies if I have the wrong mailing list for SMACK, I couldn't find
>>>>> one on vger.kernel.org.
>
> SMACK-discuss@xxxxxxxxxxxx is the most Smack directed list.
> linux-security-module@xxxxxxxxxxxxxxx is the "official" place.
> casey@xxxxxxxxxxxxxxxx is me directly.
Thank you I will use these in future.
>
>>>>>
>>>>>
>>>>> Boot log.
>>>>> ...
>>>>> Security Framework initialized
>>>>> Smack: Initializing.
>>>>> Smack: IPv6 port labeling enabled.
>>>>> Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
>>>>> Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
>>>>> CPU: Testing write buffer coherency: ok
>>>>> Setting up static identity map for 0x80100000 - 0x80100058
>>>>> devtmpfs: initialized
>>>>> evm: security.SMACK64
>>>>> evm: security.SMACK64EXEC
>>>>> evm: security.SMACK64TRANSMUTE
>>>>> evm: security.SMACK64MMAP
>>>>> evm: security.ima
>>>>> evm: security.capability
>>>>> ...
>>>>> Loading compiled-in X.509 certificates
>>>>> Loaded X.509 cert 'IMA-EVM Root CA: cc972d25acf7c1efaa5329a48104efa303f0833a'
>>>>> ...
>>>>> UBIFS (ubi0:0): FS size: 201764864 bytes (192 MiB, 1589 LEBs), journal
>>>>> size 9023488 bytes (8 MiB, 72 LEBs)
>>>>> UBIFS (ubi0:0): reserved for root: 0 bytes (0 KiB)
>>>>> UBIFS (ubi0:0): media format: w4/r0 (latest is w4/r0), UUID
>>>>> F6EA70A5-1931-4049-89CB-93B82F37F6A4, small LPT model
>>>>> VFS: Mounted root (ubifs filesystem) readonly on device 0:16.
>>>>> devtmpfs: mounted
>>>>> integrity: Loaded X.509 cert 'IMA Certificate Authority:
>>>>> e2c191a6e31fd02d6beba0c7c7847720a35fd9c6': /etc/keys/ima-x509.der
>>>>> Freeing unused kernel memory: 1024K
>>>>> systemd[1]: Successfully loaded Smack policies.
>
> Is your list of Smack rules large? What is it based on?
At the moment there are 2 rules :) It's not based on anything, the
next step is to start creating rules for the system but I don't
envisage that there will be many.
>
>>>>> systemd[1]: Successfully loaded Smack/CIPSO policies.
>>>>> systemd[1]: System time before build time, advancing clock.
>>>>> systemd[1]: Failed to mount tmpfs at /dev/shm: No such file or directory
>>>>> systemd[1]: Failed to mount tmpfs at /dev/shm: No such file or directory
>>>>> systemd[1]: Failed to mount cgroup at /sys/fs/cgroup/systemd: No such
>>>>> file or directory
>
> This is where the audit trail could be very helpful. It can tell us
> what action failed.
I thought I found the problem. systemd was changing the SMACK label
on /dev to '*' and I had put IMA/EVM signature on /dev in the rootfs
so I was thinking the new SMACK label would break IMA/EVM for the dev
folder and as the first message were about /dev/shm not being found I
thought this was it. But after removing the IMA/EVM signing of
directories, labelling /dev with '*' during the image signing and even
replacing all calls to lsetxattr in systemd with a log message the
problem still exists.
I put a printk at the end of common_lsm_audit and it never appeared so
I'm assuming there are no audit messages. I put a prink on
process_measurements in IMA and it always returned 0 except for one
place where systemd was trying to load rules into load2
process_measurement: process_measurement: [smackfs magic:43415d53] /load2
process_measurements: ret -EACCESS rc:4
Not sure this is the problem though as there are 2 rules which are
application specific. I load them later anyway with a systemd
service. Put a printk on various SMACK functions including
smack_sb_kern_mount and they all return 0.
It looks like this is something particular to my system so I don't
want to take anymore of your time unless I find something that tells
me otherwise. I'm going to start by investigating what's special
about mounting /dev/shm and /sys/fs/cgroup/systemd as these are the
two that are failing, it seems to mounting other things by looking at
the systemd mount table.
If anyone can suggest a good place to start looking that would be
appreciated otherwise I'll start with mounting code and shm code.
>
>>>>> [!!!!!!] Failed to mount API filesystems, freezing.
>>>>> systemd[1]: Freezing execution.
>>>> [Cc'ing Sascha]
>>>>
>>>> Are there any additional messages in /var/log/audit/audit.log?
>>>>
>>>> Mimi
>>>>
>>> Sadly I can't see this file, I don't even think the relevant
>>> filesystems have been mounted as this point. I tried the emergency
>>> shell but no joy. Is there a way of patching the kernel to show audit
>>> messages to the console? If you point me at the relevant code I'll
>>> hack something in. I'm currently putting printk's everywhere I can
>>> think of to see what's going on. If you can think of anywhere in the
>>> IMA that would be good to see a debug print let me know, currently I
>>> have something in process_measurements and a few other places in
>>> ima_main.c.
>> I would think the audit messages would go to the console. With
>> dracut, I use a couple of boot command line options for debugging (eg.
>> rd.debug rd.break=pre-mount, rd.shell).
>>
>> Mimi
>>
>>
>
