Re: [PATCH] Make the backing-store modules shared objects that are loaded at runtime

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Tomo, ping ?

On Tue, Aug 13, 2013 at 8:10 AM, ronnie sahlberg
<ronniesahlberg@xxxxxxxxx> wrote:
> Tomo ?
>
> On Sat, Aug 10, 2013 at 2:37 AM, ronnie sahlberg
> <ronniesahlberg@xxxxxxxxx> wrote:
>> On Fri, Aug 9, 2013 at 11:09 PM, Dan Mick <dan.mick@xxxxxxxxxxx> wrote:
>>> I can confirm that, with all instances of 'rdb' changed to 'rbd', and the
>>> obvious change to bs_rbd.c, this works for bs_rbd as well.  If I'd been more
>>> intelligent I'd have committed your patch locally before hacking on it.
>>> It's late. :)
>>
>> Thanks. I have updated the patch.
>>>
>>> +1 on the idea and overall structure; it would be interesting in particular
>>> to hear others' opinions about the choice of /usr/lib/tgtd/backing-store,
>>> although it seems fine to me.
>>
>> I have no strong feelings about where to store the modules so if
>> anyone has a better suggestion for the directory, please let me know
>> and I'll change it.
>>
>>
>>>
>>> Ceph users would benefit greatly from the ability for the official released
>>> packages (assuming the package maintainers will do it) to contain the rbd
>>> backend without requiring compilation.
>>
>> Yes. It would also be really neat if at a later stage one could
>> unload/reload a bs module at runtime. This I think would greatly
>> improve the experience for people that are developing and testing
>> backends.
>>
>>>
>>>
>>> On 08/09/2013 10:47 PM, Dan Mick wrote:
>>>>
>>>> This would be super-cool, as it would allow building and distributing
>>>> rbd support without requiring users of the package to install Ceph just
>>>> to use stgt without Ceph.
>>>>
>>>> Some initial comments below:
>>>>
>>>> On 08/09/2013 04:02 PM, Ronnie Sahlberg wrote:
>>>>>
>>>>> Turn most of TGTD into a shared library libtgt.so and install it under
>>>>> $(PREFIX)/lib/tgtd
>>>>>
>>>>> Change the six backing stores  bs_aio/null/rdb/rdwr/sg/ssc into shared
>>>>> objects
>>>>> and install them under $(PREFIX)/lib/tgtd/backing-store
>>>>>
>>>>> When tgtd is starting, have it traverse the directory for backing
>>>>> stores and
>>>>> automatically load and initialize all backing stores files that are
>>>>> found.
>>>>>
>>>>> This allows for example to distribute bs_aio.so as a separate package
>>>>> since it has additional dependencies (libaio) that tgtd itself does
>>>>> not have
>>>>> Similarly for bs_rdb.so.
>>>>> This means that core TGTD can be distributed with minimal dependencies
>>>>> and backends that add additional dependencies can be distributed
>>>>> separately.
>>>>>
>>>>> Once we have this basics for a modularized TGTD later patches can
>>>>> build ontop
>>>>> of this and add features such as :
>>>>> * list all modules and which luns are using them
>>>>> * unload module if unused
>>>>> * re-load module
>>>>> But that can come in later patches. Lets get the basics in first.
>>>>>
>>>>> Signed-off-by: Ronnie Sahlberg <ronniesahlberg@xxxxxxxxx>
>>>>> ---
>>>>>   usr/Makefile  |   42 ++++++---
>>>>>   usr/bs.c      |   46 +++++++++
>>>>>   usr/bs_aio.c  |    2 +-
>>>>>   usr/bs_null.c |    2 +-
>>>>>   usr/bs_rdwr.c |    2 +-
>>>>>   usr/bs_sg.c   |    2 +-
>>>>>   usr/bs_ssc.c  |    2 +-
>>>>
>>>>
>>>> No 2-line change to bs_rbd.c?
>>>>
>>>>>   usr/target.c  |  285
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
>>>>>   usr/tgtd.c    |  276
>>>>> +-------------------------------------------------------
>>>>>   usr/tgtd.h    |    2 +
>>>>>   10 files changed, 360 insertions(+), 301 deletions(-)
>>>>>
>>>>> diff --git a/usr/Makefile b/usr/Makefile
>>>>> index 453eb1a..5c76f43 100644
>>>>> --- a/usr/Makefile
>>>>> +++ b/usr/Makefile
>>>>> @@ -1,4 +1,5 @@
>>>>>   sbindir ?= $(PREFIX)/sbin
>>>>> +libdir ?= $(PREFIX)/lib/tgtd
>>>>>
>>>>>   ifneq ($(shell test -e /usr/include/linux/signalfd.h && echo 1),)
>>>>>   CFLAGS += -DUSE_SIGNALFD
>>>>> @@ -11,17 +12,14 @@ endif
>>>>>   TGTD_OBJS += $(addprefix iscsi/, conn.o param.o session.o \
>>>>>           iscsid.o target.o chap.o sha1.o md5.o transport.o iscsi_tcp.o \
>>>>>           isns.o)
>>>>> -TGTD_OBJS += bs_rdwr.o
>>>>>
>>>>>   ifneq ($(CEPH_RBD),)
>>>>> -TGTD_OBJS += bs_rbd.o
>>>>> -LIBS += -lrados -lrbd
>>>>> +BS_OBJS += bs_rdb.so
>>>>
>>>>
>>>> bs_rbd.so, not rdb
>>>>
>>>> It may seem weird to compile, but it *should* be enough to add package
>>>> 'librbd1' (which will also pull in librados2) to allow the build with
>>>> CEPH_RBD enabled.  If Tomo agrees this patch is worthwhile, I'd sure
>>>> like to see it go in with all bsdrvs supported.
>>>>
>>>>>   endif
>>>>>
>>>>>   ifneq ($(shell test -e /usr/include/sys/eventfd.h && test -e
>>>>> /usr/include/libaio.h && echo 1),)
>>>>>   CFLAGS += -DUSE_EVENTFD
>>>>> -TGTD_OBJS += bs_aio.o
>>>>> -LIBS += -laio
>>>>> +BS_OBJS += bs_aio.so
>>>>>   endif
>>>>>
>>>>>   ifneq ($(ISCSI_RDMA),)
>>>>> @@ -40,25 +38,32 @@ CFLAGS += -g -O2 -fno-strict-aliasing
>>>>>   endif
>>>>>   CFLAGS += -Wall -Wstrict-prototypes -fPIC
>>>>>   CFLAGS += -DTGT_VERSION=\"$(VERSION)$(EXTRAVERSION)\"
>>>>> +CFLAGS += -DBSDIR=\"$(DESTDIR)$(libdir)/backing-store\"
>>>>>
>>>>>   LIBS += -lpthread
>>>>>
>>>>>   PROGRAMS += tgtd tgtadm tgtimg
>>>>> -TGTD_OBJS += tgtd.o mgmt.o target.o scsi.o log.o driver.o util.o
>>>>> work.o \
>>>>> -        concat_buf.o parser.o spc.o sbc.o mmc.o osd.o scc.o smc.o \
>>>>> -        ssc.o bs_ssc.o libssc.o \
>>>>> -        bs_null.o bs_sg.o bs.o libcrc32c.o
>>>>> +TGTD_OBJS += tgtd.o
>>>>> +LIBTGT_OBJS = bs.o \
>>>>> +    concat_buf.o driver.o libcrc32c.o libssc.o log.o mgmt.o mmc.o \
>>>>> +    osd.o parser.o sbc.o scc.o scsi.o smc.o spc.o ssc.o \
>>>>> +    target.o util.o work.o
>>>>> +
>>>>> +BS_OBJS += bs_null.so bs_rdwr.so bs_sg.so bs_ssc.so
>>>>>
>>>>>   TGTD_DEP = $(TGTD_OBJS:.o=.d)
>>>>>
>>>>>   .PHONY:all
>>>>> -all: $(PROGRAMS)
>>>>> +all: libtgt.so $(PROGRAMS) $(BS_OBJS)
>>>>>
>>>>>   tgtd: $(TGTD_OBJS)
>>>>> -    $(CC) $^ -o $@ $(LIBS)
>>>>> +    $(CC) $^ -o $@ $(LIBS) libtgt.so
>>>>>
>>>>>   -include $(TGTD_DEP)
>>>>>
>>>>> +libtgt.so: $(LIBTGT_OBJS)
>>>>> +    $(CC) -shared -fPIC -DPIC $^ -o $@  -ldl
>>>>> +
>>>>>   TGTADM_OBJS = tgtadm.o concat_buf.o
>>>>>   TGTADM_DEP = $(TGTADM_OBJS:.o=.d)
>>>>>
>>>>> @@ -79,11 +84,24 @@ tgtimg: $(TGTIMG_OBJS)
>>>>>       $(CC) -c $(CFLAGS) $*.c -o $*.o
>>>>>       @$(CC) -MM $(CFLAGS) -MF $*.d -MT $*.o $*.c
>>>>>
>>>>> +%.so: %.c
>>>>> +    $(CC) -shared $(CFLAGS) $*.c -o $*.so
>>>>> +
>>>>> +bs_aio.so: bs_aio.c
>>>>> +    $(CC) -shared $(CFLAGS) bs_aio.c -o bs_aio.so -laio
>>>>> +
>>>>> +bs_rdb.so: bs_rdb.c
>>>>> +    $(CC) -shared $(CFLAGS) bs_rdb.c -o bs_rdb.so -lrados -lrbd
>>>>> +
>>>>>   .PHONY: install
>>>>>   install: $(PROGRAMS)
>>>>>       install -d -m 755 $(DESTDIR)$(sbindir)
>>>>>       install -m 755 $(PROGRAMS) $(DESTDIR)$(sbindir)
>>>>> +    install -d -m 755 $(DESTDIR)$(libdir)/backing-store
>>>>> +    install -m 755 $(BS_OBJS) $(DESTDIR)$(libdir)/backing-store
>>>>> +    install -m 755 libtgt.so $(DESTDIR)$(libdir)
>>>>> +    ldconfig $(DESTDIR)$(libdir)
>>>>>
>>>>>   .PHONY: clean
>>>>>   clean:
>>>>> -    rm -f *.[od] $(PROGRAMS) iscsi/*.[od] ibmvio/*.[od] fc/*.[od]
>>>>> +    rm -f *.[od] *.so $(PROGRAMS) iscsi/*.[od] ibmvio/*.[od] fc/*.[od]
>>>>> diff --git a/usr/bs.c b/usr/bs.c
>>>>> index 65c332e..7974e3a 100644
>>>>> --- a/usr/bs.c
>>>>> +++ b/usr/bs.c
>>>>> @@ -19,6 +19,8 @@
>>>>>    * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
>>>>>    * 02110-1301 USA
>>>>>    */
>>>>> +#include <dirent.h>
>>>>> +#include <dlfcn.h>
>>>>>   #include <errno.h>
>>>>>   #include <string.h>
>>>>>   #include <inttypes.h>
>>>>> @@ -307,10 +309,54 @@ destroy_cond_mutex:
>>>>>       return 1;
>>>>>   }
>>>>>
>>>>> +void update_lbppbe(struct scsi_lu *lu, int blksize)
>>>>> +{
>>>>> +    lu->attrs.lbppbe = 0;
>>>>> +    while (blksize > (1U << lu->blk_shift)) {
>>>>> +        lu->attrs.lbppbe++;
>>>>> +        blksize >>= 1;
>>>>> +    }
>>>>> +}
>>>>> +
>>>>>   int bs_init(void)
>>>>>   {
>>>>> +    DIR *dir;
>>>>>       int ret;
>>>>>
>>>>> +    dir = opendir(BSDIR);
>>>>> +    if (dir == NULL) {
>>>>> +        eprintf("could not open backing-store module directory %s\n",
>>>>> +            BSDIR);
>>>>> +    } else {
>>>>> +        struct dirent *dirent;
>>>>> +        void *handle;
>>>>> +        while ((dirent = readdir(dir))) {
>>>>> +            char *soname;
>>>>> +            void (*register_bs_module)(void);
>>>>> +
>>>>> +            if (dirent->d_name[0] == '.') {
>>>>> +                continue;
>>>>> +            }
>>>>> +
>>>>> +            asprintf(&soname, "%s/%s", BSDIR, dirent->d_name);
>>>>> +            handle = dlopen(soname, RTLD_NOW|RTLD_LOCAL);
>>>>> +            if (handle == NULL) {
>>>>> +                eprintf("failed to dlopen backing-store "
>>>>> +                    "module %s error %s \n",
>>>>> +                    soname, dlerror());
>>>>> +                continue;
>>>>> +            }
>>>>> +            register_bs_module = dlsym(handle, "register_bs_module");
>>>>> +            if (register_bs_module == NULL) {
>>>>> +                eprintf("could not find register_bs_module "
>>>>> +                    "symbol in module %s\n",
>>>>> +                    soname);
>>>>> +                continue;
>>>>> +            }
>>>>> +            register_bs_module();
>>>>> +        }
>>>>> +        closedir(dir);
>>>>> +    }
>>>>>       ret = bs_init_signalfd();
>>>>>       if (!ret) {
>>>>>           eprintf("use signalfd notification\n");
>>>>> diff --git a/usr/bs_aio.c b/usr/bs_aio.c
>>>>> index c0cbadd..cc59cf6 100644
>>>>> --- a/usr/bs_aio.c
>>>>> +++ b/usr/bs_aio.c
>>>>> @@ -414,7 +414,7 @@ static struct backingstore_template aio_bst = {
>>>>>       .bs_cmd_submit      = bs_aio_cmd_submit,
>>>>>   };
>>>>>
>>>>> -__attribute__((constructor)) static void bs_rdwr_constructor(void)
>>>>> +void register_bs_module(void)
>>>>>   {
>>>>>       register_backingstore_template(&aio_bst);
>>>>>   }
>>>>> diff --git a/usr/bs_null.c b/usr/bs_null.c
>>>>> index d463f18..4dbe144 100644
>>>>> --- a/usr/bs_null.c
>>>>> +++ b/usr/bs_null.c
>>>>> @@ -56,7 +56,7 @@ static struct backingstore_template null_bst = {
>>>>>       .bs_cmd_submit        = bs_null_cmd_submit,
>>>>>   };
>>>>>
>>>>> -__attribute__((constructor)) static void bs_null_constructor(void)
>>>>> +void register_bs_module(void)
>>>>>   {
>>>>>       register_backingstore_template(&null_bst);
>>>>>   }
>>>>> diff --git a/usr/bs_rdwr.c b/usr/bs_rdwr.c
>>>>> index 47d2d99..82807d1 100644
>>>>> --- a/usr/bs_rdwr.c
>>>>> +++ b/usr/bs_rdwr.c
>>>>> @@ -423,7 +423,7 @@ static struct backingstore_template rdwr_bst = {
>>>>>       .bs_oflags_supported    = O_SYNC | O_DIRECT,
>>>>>   };
>>>>>
>>>>> -__attribute__((constructor)) static void bs_rdwr_constructor(void)
>>>>> +void register_bs_module(void)
>>>>>   {
>>>>>       register_backingstore_template(&rdwr_bst);
>>>>>   }
>>>>> diff --git a/usr/bs_sg.c b/usr/bs_sg.c
>>>>> index 5f1e687..43dc5f3 100644
>>>>> --- a/usr/bs_sg.c
>>>>> +++ b/usr/bs_sg.c
>>>>> @@ -517,7 +517,7 @@ static struct device_type_template sg_template = {
>>>>>       .cmd_passthrough    = bs_sg_rw,
>>>>>   };
>>>>>
>>>>> -__attribute__((constructor)) static void bs_sg_constructor(void)
>>>>> +void register_bs_module(void)
>>>>>   {
>>>>>       register_backingstore_template(&sg_bst);
>>>>>       register_backingstore_template(&bsg_bst);
>>>>> diff --git a/usr/bs_ssc.c b/usr/bs_ssc.c
>>>>> index 117e274..98b84b5 100644
>>>>> --- a/usr/bs_ssc.c
>>>>> +++ b/usr/bs_ssc.c
>>>>> @@ -702,7 +702,7 @@ static struct backingstore_template ssc_bst = {
>>>>>       .bs_cmd_submit        = bs_thread_cmd_submit,
>>>>>   };
>>>>>
>>>>> -__attribute__((constructor)) static void bs_ssc_constructor(void)
>>>>> +void register_bs_module(void)
>>>>>   {
>>>>>       register_backingstore_template(&ssc_bst);
>>>>>   }
>>>>> diff --git a/usr/target.c b/usr/target.c
>>>>> index b1729b3..b7872fb 100644
>>>>> --- a/usr/target.c
>>>>> +++ b/usr/target.c
>>>>> @@ -19,6 +19,7 @@
>>>>>    * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
>>>>>    * 02110-1301 USA
>>>>>    */
>>>>> +#include <ctype.h>
>>>>>   #include <errno.h>
>>>>>   #include <fcntl.h>
>>>>>   #include <inttypes.h>
>>>>> @@ -27,8 +28,10 @@
>>>>>   #include <string.h>
>>>>>   #include <time.h>
>>>>>   #include <unistd.h>
>>>>> +#include <sys/epoll.h>
>>>>>   #include <sys/socket.h>
>>>>>   #include <sys/time.h>
>>>>> +#include <sys/wait.h>
>>>>>
>>>>>   #include "list.h"
>>>>>   #include "util.h"
>>>>> @@ -42,8 +45,11 @@
>>>>>   #include "spc.h"
>>>>>
>>>>>   static LIST_HEAD(device_type_list);
>>>>> +static LIST_HEAD(tgt_events_list);
>>>>> +static LIST_HEAD(tgt_sched_events_list);
>>>>>
>>>>>   static struct target global_target;
>>>>> +int ep_fd;
>>>>>
>>>>>   int device_type_register(struct device_type_template *t)
>>>>>   {
>>>>> @@ -2411,15 +2417,6 @@ tgtadm_err lld_show(struct concat_buf *b)
>>>>>       return TGTADM_SUCCESS;
>>>>>   }
>>>>>
>>>>> -void update_lbppbe(struct scsi_lu *lu, int blksize)
>>>>> -{
>>>>> -    lu->attrs.lbppbe = 0;
>>>>> -    while (blksize > (1U << lu->blk_shift)) {
>>>>> -        lu->attrs.lbppbe++;
>>>>> -        blksize >>= 1;
>>>>> -    }
>>>>> -}
>>>>> -
>>>>>   int is_system_available(void)
>>>>>   {
>>>>>       return (sys_state == TGT_SYSTEM_READY);
>>>>> @@ -2430,6 +2427,276 @@ int is_system_inactive(void)
>>>>>       return list_empty(&target_list);
>>>>>   }
>>>>>
>>>>> +int tgt_event_add(int fd, int events, event_handler_t handler, void
>>>>> *data)
>>>>> +{
>>>>> +    struct epoll_event ev;
>>>>> +    struct event_data *tev;
>>>>> +    int err;
>>>>> +
>>>>> +    tev = zalloc(sizeof(*tev));
>>>>> +    if (!tev)
>>>>> +        return -ENOMEM;
>>>>> +
>>>>> +    tev->data = data;
>>>>> +    tev->handler = handler;
>>>>> +    tev->fd = fd;
>>>>> +
>>>>> +    memset(&ev, 0, sizeof(ev));
>>>>> +    ev.events = events;
>>>>> +    ev.data.ptr = tev;
>>>>> +    err = epoll_ctl(ep_fd, EPOLL_CTL_ADD, fd, &ev);
>>>>> +    if (err) {
>>>>> +        eprintf("Cannot add fd, %m\n");
>>>>> +        free(tev);
>>>>> +    } else
>>>>> +        list_add(&tev->e_list, &tgt_events_list);
>>>>> +
>>>>> +    return err;
>>>>> +}
>>>>> +
>>>>> +static struct event_data *tgt_event_lookup(int fd)
>>>>> +{
>>>>> +    struct event_data *tev;
>>>>> +
>>>>> +    list_for_each_entry(tev, &tgt_events_list, e_list) {
>>>>> +        if (tev->fd == fd)
>>>>> +            return tev;
>>>>> +    }
>>>>> +    return NULL;
>>>>> +}
>>>>> +
>>>>> +void tgt_event_del(int fd)
>>>>> +{
>>>>> +    struct event_data *tev;
>>>>> +    int ret;
>>>>> +
>>>>> +    tev = tgt_event_lookup(fd);
>>>>> +    if (!tev) {
>>>>> +        eprintf("Cannot find event %d\n", fd);
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    ret = epoll_ctl(ep_fd, EPOLL_CTL_DEL, fd, NULL);
>>>>> +    if (ret < 0)
>>>>> +        eprintf("fail to remove epoll event, %s\n", strerror(errno));
>>>>> +
>>>>> +    list_del(&tev->e_list);
>>>>> +    free(tev);
>>>>> +}
>>>>> +
>>>>> +int tgt_event_modify(int fd, int events)
>>>>> +{
>>>>> +    struct epoll_event ev;
>>>>> +    struct event_data *tev;
>>>>> +
>>>>> +    tev = tgt_event_lookup(fd);
>>>>> +    if (!tev) {
>>>>> +        eprintf("Cannot find event %d\n", fd);
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    memset(&ev, 0, sizeof(ev));
>>>>> +    ev.events = events;
>>>>> +    ev.data.ptr = tev;
>>>>> +
>>>>> +    return epoll_ctl(ep_fd, EPOLL_CTL_MOD, fd, &ev);
>>>>> +}
>>>>> +
>>>>> +void tgt_init_sched_event(struct event_data *evt,
>>>>> +              sched_event_handler_t sched_handler, void *data)
>>>>> +{
>>>>> +    evt->sched_handler = sched_handler;
>>>>> +    evt->scheduled = 0;
>>>>> +    evt->data = data;
>>>>> +    INIT_LIST_HEAD(&evt->e_list);
>>>>> +}
>>>>> +
>>>>> +void tgt_add_sched_event(struct event_data *evt)
>>>>> +{
>>>>> +    if (!evt->scheduled) {
>>>>> +        evt->scheduled = 1;
>>>>> +        list_add_tail(&evt->e_list, &tgt_sched_events_list);
>>>>> +    }
>>>>> +}
>>>>> +
>>>>> +void tgt_remove_sched_event(struct event_data *evt)
>>>>> +{
>>>>> +    if (evt->scheduled) {
>>>>> +        evt->scheduled = 0;
>>>>> +        list_del_init(&evt->e_list);
>>>>> +    }
>>>>> +}
>>>>> +
>>>>> +int tgt_exec_scheduled(void)
>>>>> +{
>>>>> +    struct list_head *last_sched;
>>>>> +    struct event_data *tev, *tevn;
>>>>> +    int work_remains = 0;
>>>>> +
>>>>> +    if (!list_empty(&tgt_sched_events_list)) {
>>>>> +        /* execute only work scheduled till now */
>>>>> +        last_sched = tgt_sched_events_list.prev;
>>>>> +        list_for_each_entry_safe(tev, tevn, &tgt_sched_events_list,
>>>>> +                     e_list) {
>>>>> +            tgt_remove_sched_event(tev);
>>>>> +            tev->sched_handler(tev);
>>>>> +            if (&tev->e_list == last_sched)
>>>>> +                break;
>>>>> +        }
>>>>> +        if (!list_empty(&tgt_sched_events_list))
>>>>> +            work_remains = 1;
>>>>> +    }
>>>>> +    return work_remains;
>>>>> +}
>>>>> +
>>>>> +/* strcpy, while eating multiple white spaces */
>>>>> +static void str_spacecpy(char **dest, const char *src)
>>>>> +{
>>>>> +    const char *s = src;
>>>>> +    char *d = *dest;
>>>>> +
>>>>> +    while (*s) {
>>>>> +        if (isspace(*s)) {
>>>>> +            if (!*(s+1))
>>>>> +                break;
>>>>> +            if (isspace(*(s+1))) {
>>>>> +                s++;
>>>>> +                continue;
>>>>> +            }
>>>>> +        }
>>>>> +        *d++ = *s++;
>>>>> +    }
>>>>> +    *d = '\0';
>>>>> +}
>>>>> +
>>>>> +int call_program(const char *cmd, void (*callback)(void *data, int
>>>>> result),
>>>>> +        void *data, char *output, int op_len, int flags)
>>>>> +{
>>>>> +    pid_t pid;
>>>>> +    int fds[2], ret, i;
>>>>> +    char *pos, arg[256];
>>>>> +    char *argv[sizeof(arg) / 2];
>>>>> +
>>>>> +    i = 0;
>>>>> +    pos = arg;
>>>>> +    str_spacecpy(&pos, cmd);
>>>>> +    if (strchr(cmd, ' ')) {
>>>>> +        while (pos != '\0')
>>>>> +            argv[i++] = strsep(&pos, " ");
>>>>> +    } else
>>>>> +        argv[i++] = arg;
>>>>> +    argv[i] =  NULL;
>>>>> +
>>>>> +    ret = pipe(fds);
>>>>> +    if (ret < 0) {
>>>>> +        eprintf("pipe create failed for %s, %m\n", cmd);
>>>>> +        return ret;
>>>>> +    }
>>>>> +
>>>>> +    dprintf("%s, pipe: %d %d\n", cmd, fds[0], fds[1]);
>>>>> +
>>>>> +    pid = fork();
>>>>> +    if (pid < 0) {
>>>>> +        eprintf("fork failed for: %s, %m\n", cmd);
>>>>> +        close(fds[0]);
>>>>> +        close(fds[1]);
>>>>> +        return pid;
>>>>> +    }
>>>>> +
>>>>> +    if (!pid) {
>>>>> +        close(1);
>>>>> +        ret = dup(fds[1]);
>>>>> +        if (ret < 0) {
>>>>> +            eprintf("dup failed for: %s, %m\n", cmd);
>>>>> +            exit(-1);
>>>>> +        }
>>>>> +        close(fds[0]);
>>>>> +        execv(argv[0], argv);
>>>>> +
>>>>> +        eprintf("execv failed for: %s, %m\n", cmd);
>>>>> +        exit(-1);
>>>>> +    } else {
>>>>> +        struct timeval tv;
>>>>> +        fd_set rfds;
>>>>> +        int ret_sel;
>>>>> +
>>>>> +        close(fds[1]);
>>>>> +        /* 0.1 second is okay, as the initiator will retry anyway */
>>>>> +        do {
>>>>> +            FD_ZERO(&rfds);
>>>>> +            FD_SET(fds[0], &rfds);
>>>>> +            tv.tv_sec = 0;
>>>>> +            tv.tv_usec = 100000;
>>>>> +            ret_sel = select(fds[0]+1, &rfds, NULL, NULL, &tv);
>>>>> +        } while (ret_sel < 0 && errno == EINTR);
>>>>> +        if (ret_sel <= 0) { /* error or timeout */
>>>>> +            eprintf("timeout on redirect callback, terminating "
>>>>> +                "child pid %d\n", pid);
>>>>> +            kill(pid, SIGTERM);
>>>>> +        }
>>>>> +        do {
>>>>> +            ret = waitpid(pid, &i, 0);
>>>>> +        } while (ret < 0 && errno == EINTR);
>>>>> +        if (ret < 0) {
>>>>> +            eprintf("waitpid failed for: %s, %m\n", cmd);
>>>>> +            close(fds[0]);
>>>>> +            return ret;
>>>>> +        }
>>>>> +        if (ret_sel > 0) {
>>>>> +            ret = read(fds[0], output, op_len);
>>>>> +            if (ret < 0) {
>>>>> +                eprintf("failed to get output from: %s\n", cmd);
>>>>> +                close(fds[0]);
>>>>> +                return ret;
>>>>> +            }
>>>>> +        }
>>>>> +
>>>>> +        if (callback)
>>>>> +            callback(data, WEXITSTATUS(i));
>>>>> +        close(fds[0]);
>>>>> +    }
>>>>> +
>>>>> +    return 0;
>>>>> +}
>>>>> +
>>>>> +struct tgt_param {
>>>>> +    int (*parse_func)(char *);
>>>>> +    char *name;
>>>>> +};
>>>>> +
>>>>> +static struct tgt_param params[64];
>>>>> +
>>>>> +int setup_param(char *name, int (*parser)(char *))
>>>>> +{
>>>>> +    int i;
>>>>> +
>>>>> +    for (i = 0; i < ARRAY_SIZE(params); i++)
>>>>> +        if (!params[i].name)
>>>>> +            break;
>>>>> +
>>>>> +    if (i < ARRAY_SIZE(params)) {
>>>>> +        params[i].name = name;
>>>>> +        params[i].parse_func = parser;
>>>>> +
>>>>> +        return 0;
>>>>> +    } else
>>>>> +        return -1;
>>>>> +}
>>>>> +
>>>>> +int parse_params(char *name, char *p)
>>>>> +{
>>>>> +    int i;
>>>>> +
>>>>> +    for (i = 0; i < ARRAY_SIZE(params) && params[i].name; i++) {
>>>>> +        if (!strcmp(name, params[i].name))
>>>>> +            return params[i].parse_func(p);
>>>>> +    }
>>>>> +
>>>>> +    fprintf(stderr, "'%s' is an unknown option\n", name);
>>>>> +
>>>>> +    return -1;
>>>>> +}
>>>>> +
>>>>>   static void __attribute__((constructor)) target_constructor(void)
>>>>>   {
>>>>>       static int global_target_aids[DEFAULT_NR_ACCOUNT];
>>>>> diff --git a/usr/tgtd.c b/usr/tgtd.c
>>>>> index f985510..3ef7220 100644
>>>>> --- a/usr/tgtd.c
>>>>> +++ b/usr/tgtd.c
>>>>> @@ -30,11 +30,9 @@
>>>>>   #include <stdlib.h>
>>>>>   #include <string.h>
>>>>>   #include <unistd.h>
>>>>> -#include <ctype.h>
>>>>>   #include <sys/resource.h>
>>>>>   #include <sys/epoll.h>
>>>>>   #include <sys/types.h>
>>>>> -#include <sys/wait.h>
>>>>>   #include <sys/stat.h>
>>>>>
>>>>>   #include "list.h"
>>>>> @@ -44,12 +42,10 @@
>>>>>   #include "util.h"
>>>>>
>>>>>   unsigned long pagesize, pageshift;
>>>>> +extern int ep_fd;
>>>>>
>>>>>   int system_active = 1;
>>>>> -static int ep_fd;
>>>>>   static char program_name[] = "tgtd";
>>>>> -static LIST_HEAD(tgt_events_list);
>>>>> -static LIST_HEAD(tgt_sched_events_list);
>>>>>
>>>>>   static struct option const long_options[] = {
>>>>>       {"foreground", no_argument, 0, 'f'},
>>>>> @@ -174,238 +170,6 @@ set_rlimit:
>>>>>       return 0;
>>>>>   }
>>>>>
>>>>> -int tgt_event_add(int fd, int events, event_handler_t handler, void
>>>>> *data)
>>>>> -{
>>>>> -    struct epoll_event ev;
>>>>> -    struct event_data *tev;
>>>>> -    int err;
>>>>> -
>>>>> -    tev = zalloc(sizeof(*tev));
>>>>> -    if (!tev)
>>>>> -        return -ENOMEM;
>>>>> -
>>>>> -    tev->data = data;
>>>>> -    tev->handler = handler;
>>>>> -    tev->fd = fd;
>>>>> -
>>>>> -    memset(&ev, 0, sizeof(ev));
>>>>> -    ev.events = events;
>>>>> -    ev.data.ptr = tev;
>>>>> -    err = epoll_ctl(ep_fd, EPOLL_CTL_ADD, fd, &ev);
>>>>> -    if (err) {
>>>>> -        eprintf("Cannot add fd, %m\n");
>>>>> -        free(tev);
>>>>> -    } else
>>>>> -        list_add(&tev->e_list, &tgt_events_list);
>>>>> -
>>>>> -    return err;
>>>>> -}
>>>>> -
>>>>> -static struct event_data *tgt_event_lookup(int fd)
>>>>> -{
>>>>> -    struct event_data *tev;
>>>>> -
>>>>> -    list_for_each_entry(tev, &tgt_events_list, e_list) {
>>>>> -        if (tev->fd == fd)
>>>>> -            return tev;
>>>>> -    }
>>>>> -    return NULL;
>>>>> -}
>>>>> -
>>>>> -void tgt_event_del(int fd)
>>>>> -{
>>>>> -    struct event_data *tev;
>>>>> -    int ret;
>>>>> -
>>>>> -    tev = tgt_event_lookup(fd);
>>>>> -    if (!tev) {
>>>>> -        eprintf("Cannot find event %d\n", fd);
>>>>> -        return;
>>>>> -    }
>>>>> -
>>>>> -    ret = epoll_ctl(ep_fd, EPOLL_CTL_DEL, fd, NULL);
>>>>> -    if (ret < 0)
>>>>> -        eprintf("fail to remove epoll event, %s\n", strerror(errno));
>>>>> -
>>>>> -    list_del(&tev->e_list);
>>>>> -    free(tev);
>>>>> -}
>>>>> -
>>>>> -int tgt_event_modify(int fd, int events)
>>>>> -{
>>>>> -    struct epoll_event ev;
>>>>> -    struct event_data *tev;
>>>>> -
>>>>> -    tev = tgt_event_lookup(fd);
>>>>> -    if (!tev) {
>>>>> -        eprintf("Cannot find event %d\n", fd);
>>>>> -        return -EINVAL;
>>>>> -    }
>>>>> -
>>>>> -    memset(&ev, 0, sizeof(ev));
>>>>> -    ev.events = events;
>>>>> -    ev.data.ptr = tev;
>>>>> -
>>>>> -    return epoll_ctl(ep_fd, EPOLL_CTL_MOD, fd, &ev);
>>>>> -}
>>>>> -
>>>>> -void tgt_init_sched_event(struct event_data *evt,
>>>>> -              sched_event_handler_t sched_handler, void *data)
>>>>> -{
>>>>> -    evt->sched_handler = sched_handler;
>>>>> -    evt->scheduled = 0;
>>>>> -    evt->data = data;
>>>>> -    INIT_LIST_HEAD(&evt->e_list);
>>>>> -}
>>>>> -
>>>>> -void tgt_add_sched_event(struct event_data *evt)
>>>>> -{
>>>>> -    if (!evt->scheduled) {
>>>>> -        evt->scheduled = 1;
>>>>> -        list_add_tail(&evt->e_list, &tgt_sched_events_list);
>>>>> -    }
>>>>> -}
>>>>> -
>>>>> -void tgt_remove_sched_event(struct event_data *evt)
>>>>> -{
>>>>> -    if (evt->scheduled) {
>>>>> -        evt->scheduled = 0;
>>>>> -        list_del_init(&evt->e_list);
>>>>> -    }
>>>>> -}
>>>>> -
>>>>> -/* strcpy, while eating multiple white spaces */
>>>>> -void str_spacecpy(char **dest, const char *src)
>>>>> -{
>>>>> -    const char *s = src;
>>>>> -    char *d = *dest;
>>>>> -
>>>>> -    while (*s) {
>>>>> -        if (isspace(*s)) {
>>>>> -            if (!*(s+1))
>>>>> -                break;
>>>>> -            if (isspace(*(s+1))) {
>>>>> -                s++;
>>>>> -                continue;
>>>>> -            }
>>>>> -        }
>>>>> -        *d++ = *s++;
>>>>> -    }
>>>>> -    *d = '\0';
>>>>> -}
>>>>> -
>>>>> -int call_program(const char *cmd, void (*callback)(void *data, int
>>>>> result),
>>>>> -        void *data, char *output, int op_len, int flags)
>>>>> -{
>>>>> -    pid_t pid;
>>>>> -    int fds[2], ret, i;
>>>>> -    char *pos, arg[256];
>>>>> -    char *argv[sizeof(arg) / 2];
>>>>> -
>>>>> -    i = 0;
>>>>> -    pos = arg;
>>>>> -    str_spacecpy(&pos, cmd);
>>>>> -    if (strchr(cmd, ' ')) {
>>>>> -        while (pos != '\0')
>>>>> -            argv[i++] = strsep(&pos, " ");
>>>>> -    } else
>>>>> -        argv[i++] = arg;
>>>>> -    argv[i] =  NULL;
>>>>> -
>>>>> -    ret = pipe(fds);
>>>>> -    if (ret < 0) {
>>>>> -        eprintf("pipe create failed for %s, %m\n", cmd);
>>>>> -        return ret;
>>>>> -    }
>>>>> -
>>>>> -    dprintf("%s, pipe: %d %d\n", cmd, fds[0], fds[1]);
>>>>> -
>>>>> -    pid = fork();
>>>>> -    if (pid < 0) {
>>>>> -        eprintf("fork failed for: %s, %m\n", cmd);
>>>>> -        close(fds[0]);
>>>>> -        close(fds[1]);
>>>>> -        return pid;
>>>>> -    }
>>>>> -
>>>>> -    if (!pid) {
>>>>> -        close(1);
>>>>> -        ret = dup(fds[1]);
>>>>> -        if (ret < 0) {
>>>>> -            eprintf("dup failed for: %s, %m\n", cmd);
>>>>> -            exit(-1);
>>>>> -        }
>>>>> -        close(fds[0]);
>>>>> -        execv(argv[0], argv);
>>>>> -
>>>>> -        eprintf("execv failed for: %s, %m\n", cmd);
>>>>> -        exit(-1);
>>>>> -    } else {
>>>>> -        struct timeval tv;
>>>>> -        fd_set rfds;
>>>>> -        int ret_sel;
>>>>> -
>>>>> -        close(fds[1]);
>>>>> -        /* 0.1 second is okay, as the initiator will retry anyway */
>>>>> -        do {
>>>>> -            FD_ZERO(&rfds);
>>>>> -            FD_SET(fds[0], &rfds);
>>>>> -            tv.tv_sec = 0;
>>>>> -            tv.tv_usec = 100000;
>>>>> -            ret_sel = select(fds[0]+1, &rfds, NULL, NULL, &tv);
>>>>> -        } while (ret_sel < 0 && errno == EINTR);
>>>>> -        if (ret_sel <= 0) { /* error or timeout */
>>>>> -            eprintf("timeout on redirect callback, terminating "
>>>>> -                "child pid %d\n", pid);
>>>>> -            kill(pid, SIGTERM);
>>>>> -        }
>>>>> -        do {
>>>>> -            ret = waitpid(pid, &i, 0);
>>>>> -        } while (ret < 0 && errno == EINTR);
>>>>> -        if (ret < 0) {
>>>>> -            eprintf("waitpid failed for: %s, %m\n", cmd);
>>>>> -            close(fds[0]);
>>>>> -            return ret;
>>>>> -        }
>>>>> -        if (ret_sel > 0) {
>>>>> -            ret = read(fds[0], output, op_len);
>>>>> -            if (ret < 0) {
>>>>> -                eprintf("failed to get output from: %s\n", cmd);
>>>>> -                close(fds[0]);
>>>>> -                return ret;
>>>>> -            }
>>>>> -        }
>>>>> -
>>>>> -        if (callback)
>>>>> -            callback(data, WEXITSTATUS(i));
>>>>> -        close(fds[0]);
>>>>> -    }
>>>>> -
>>>>> -    return 0;
>>>>> -}
>>>>> -
>>>>> -static int tgt_exec_scheduled(void)
>>>>> -{
>>>>> -    struct list_head *last_sched;
>>>>> -    struct event_data *tev, *tevn;
>>>>> -    int work_remains = 0;
>>>>> -
>>>>> -    if (!list_empty(&tgt_sched_events_list)) {
>>>>> -        /* execute only work scheduled till now */
>>>>> -        last_sched = tgt_sched_events_list.prev;
>>>>> -        list_for_each_entry_safe(tev, tevn, &tgt_sched_events_list,
>>>>> -                     e_list) {
>>>>> -            tgt_remove_sched_event(tev);
>>>>> -            tev->sched_handler(tev);
>>>>> -            if (&tev->e_list == last_sched)
>>>>> -                break;
>>>>> -        }
>>>>> -        if (!list_empty(&tgt_sched_events_list))
>>>>> -            work_remains = 1;
>>>>> -    }
>>>>> -    return work_remains;
>>>>> -}
>>>>> -
>>>>>   static void event_loop(void)
>>>>>   {
>>>>>       int nevent, i, sched_remains, timeout;
>>>>> @@ -471,44 +235,6 @@ static void lld_exit(void)
>>>>>       }
>>>>>   }
>>>>>
>>>>> -struct tgt_param {
>>>>> -    int (*parse_func)(char *);
>>>>> -    char *name;
>>>>> -};
>>>>> -
>>>>> -static struct tgt_param params[64];
>>>>> -
>>>>> -int setup_param(char *name, int (*parser)(char *))
>>>>> -{
>>>>> -    int i;
>>>>> -
>>>>> -    for (i = 0; i < ARRAY_SIZE(params); i++)
>>>>> -        if (!params[i].name)
>>>>> -            break;
>>>>> -
>>>>> -    if (i < ARRAY_SIZE(params)) {
>>>>> -        params[i].name = name;
>>>>> -        params[i].parse_func = parser;
>>>>> -
>>>>> -        return 0;
>>>>> -    } else
>>>>> -        return -1;
>>>>> -}
>>>>> -
>>>>> -static int parse_params(char *name, char *p)
>>>>> -{
>>>>> -    int i;
>>>>> -
>>>>> -    for (i = 0; i < ARRAY_SIZE(params) && params[i].name; i++) {
>>>>> -        if (!strcmp(name, params[i].name))
>>>>> -            return params[i].parse_func(p);
>>>>> -    }
>>>>> -
>>>>> -    fprintf(stderr, "'%s' is an unknown option\n", name);
>>>>> -
>>>>> -    return -1;
>>>>> -}
>>>>> -
>>>>>   int main(int argc, char **argv)
>>>>>   {
>>>>>       struct sigaction sa_old;
>>>>> diff --git a/usr/tgtd.h b/usr/tgtd.h
>>>>> index 484e6e9..0d4c6ae 100644
>>>>> --- a/usr/tgtd.h
>>>>> +++ b/usr/tgtd.h
>>>>> @@ -295,6 +295,7 @@ extern void tgt_event_del(int fd);
>>>>>
>>>>>   extern void tgt_add_sched_event(struct event_data *evt);
>>>>>   extern void tgt_remove_sched_event(struct event_data *evt);
>>>>> +extern int tgt_exec_scheduled(void);
>>>>>
>>>>>   extern int tgt_event_modify(int fd, int events);
>>>>>   extern int target_cmd_queue(int tid, struct scsi_cmd *cmd);
>>>>> @@ -373,6 +374,7 @@ extern struct backingstore_template
>>>>> *get_backingstore_template(const char *name)
>>>>>   extern int lld_init_one(int lld_index);
>>>>>
>>>>>   extern int setup_param(char *name, int (*parser)(char *));
>>>>> +extern int parse_params(char *name, char *p);
>>>>>
>>>>>   extern int bs_init(void);
>>>>>
>>>>>
>>>
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux SCSI]     [Linux RAID]     [Linux Clusters]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]

  Powered by Linux