Re: [Lsf-pc] [LSF/MM/BPF ATTEND][LSF/MM/BPF TOPIC] Meta/Integrity/PI improvements

[Kanchan Joshi <joshi.k@xxxxxxxxxxx>]

On 3/28/2024 6:00 AM, Martin K. Petersen wrote:
> 
> Hi Kanchan!
> 
>> Not sure how far it is from the requirements you may have. Feedback
>> will help. Perhaps the interface needs the ability to tell what kind
>> of checks (guard, apptag, reftag) are desired. Doable, but that will
>> require the introduction of three new RWF_* flags.
> 
> I'm working on getting my test tooling working with your series.

If it helps somehow, here is a simple application for the interface [*].
It is devoid of guard (and stuff); for that fio is better.

> But
> yes, I'll definitely need a way to set the bip flags.

Just to be clear, I was thinking of three new flags that userspace can 
pass: RWF_CHK_GUARD, RWF_CHK_APPTAG, RWF_CHK_REFTAG.
And corresponding bip flags will need to be introduced (I don't see 
anything existing). Driver will see those and convert to protocol 
specific flags.
Does this match with you what you have in mind.


>> Right. This can work for the case when host does not need to pass the
>> buffer (meta-size is equal to pi-size). But when meta-size is greater
>> than pi-size, the meta-buffer needs to be allocated. Some changes are
>> required so that Block-integrity does that allocation, without having
>> to do read_verify/write_generate.
> 
> Not sure I follow. Do you want the non-PI metadata to be passed in from
> userland but the kernel or controller to generate the PI?
> 

No, this has no connection with userland. Seems concurrent discussion 
(with the user-interface topic) will cause the confusion. Maybe this is 
better to be discussed (after some time) along with its own RFC.

[*]

#define _GNU_SOURCE

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <linux/io_uring.h>
#include "liburing.h"

/* write data/meta. read data/meta. compare data/meta buffers */
/* prerequisite: format namespace with 4KB + 8b, pi_type = 0 */

#define DATA_LEN 4096
#define META_LEN 8

int main(int argc, char *argv[])
{
         struct io_uring ring;
         struct io_uring_sqe *sqe = NULL;
         struct io_uring_cqe *cqe = NULL;
         void *wdb,*rdb;
         char wmb[META_LEN], rmb[META_LEN];
         char *data_str = "data buffer";
         char *meta_str = "meta";
         int fd, ret, blksize;
         struct stat fstat;
         unsigned long long offset = 0;

         if (argc != 2) {
                 fprintf(stderr, "Usage: %s <block-device>", argv[0]);
                 return 1;
         };

         if (stat(argv[1], &fstat) == 0) {
                 blksize = (int)fstat.st_blksize;
         } else {
                 perror("stat");
                 return 1;
         }

         if (posix_memalign(&wdb, blksize, DATA_LEN)) {
                 perror("posix_memalign failed");
                 return 1;
         }
         if (posix_memalign(&rdb, blksize, DATA_LEN)) {
                 perror("posix_memalign failed");
                 return 1;
         }

         strcpy(wdb, data_str);
         strcpy(wmb, meta_str);

         fd = open(argv[1], O_RDWR | O_DIRECT);
         if (fd < 0) {
                 printf("Error in opening device\n");
                 return 0;
         }

         ret = io_uring_queue_init(8, &ring, 0);
         if (ret) {
                 fprintf(stderr, "ring setup failed: %d\n", ret);
                 return 1;
         }

         /* write data + meta-buffer to device */
         sqe = io_uring_get_sqe(&ring);
         if (!sqe) {
                 fprintf(stderr, "get sqe failed\n");
                 return 1;
         }

         io_uring_prep_write(sqe, fd, wdb, DATA_LEN, offset);
         sqe->opcode = IORING_OP_WRITE_META;
         sqe->meta_addr = (__u64)wmb;
         sqe->meta_len = META_LEN;
         /*
          * TBD: Flags to ask for guard/apptag/reftag checks
          * sqe->rw_flags = RWF_CHK_GUARD | RWF_CHK_APPTAG | RWF_CHK_REFTAG;
          */
         ret = io_uring_submit(&ring);
         if (ret <= 0) {
                 fprintf(stderr, "sqe submit failed: %d\n", ret);
                 return 1;
         }

         ret = io_uring_wait_cqe(&ring, &cqe);
         if (!cqe) {
                 fprintf(stderr, "cqe is NULL :%d\n", ret);
                 return 1;
         }
         if (cqe->res < 0) {
                 fprintf(stderr, "write cqe failure: %d", cqe->res);
                 return 1;
         }

         io_uring_cqe_seen(&ring, cqe);

         /* read dat + meta-buffer back from device */
         sqe = io_uring_get_sqe(&ring);
         if (!sqe) {
                 fprintf(stderr, "get sqe failed\n");
                 return 1;
         }

         io_uring_prep_read(sqe, fd, rdb, DATA_LEN, offset);
         sqe->opcode = IORING_OP_READ_META;
         sqe->meta_addr = (__u64)rmb;
         sqe->meta_len = META_LEN;

         ret = io_uring_submit(&ring);
         if (ret <= 0) {
                 fprintf(stderr, "sqe submit failed: %d\n", ret);
                 return 1;
         }

         ret = io_uring_wait_cqe(&ring, &cqe);
         if (!cqe) {
                 fprintf(stderr, "cqe is NULL :%d\n", ret);
                 return 1;
         }

         if (cqe->res < 0) {
                 fprintf(stderr, "read cqe failure: %d", cqe->res);
                 return 1;
         }
         io_uring_cqe_seen(&ring, cqe);

         if (strncmp(wmb, rmb, META_LEN))
                 printf("Failure: meta mismatch!, wmb=%s, rmb=%s\n", 
wmb, rmb);

         if (strncmp(wdb, rdb, DATA_LEN))
                 printf("Failure: data mismatch!\n");

         io_uring_queue_exit(&ring);
         free(rdb);
         free(wdb);
         return 0;
}