Hi Ben, Yes, this patch's main goal is to work around doing a reconfigure. So no paths will be eliminated before storage paths come back. I agree with your way to improve reconfigure procedure ,since this patch looks too tricky. I also did a little more work to test if we could fix this issue like your way but not digged deeply. Good is that this trial proves that your way is feasible and it fits current DM architecture, however, due to customer's emergency, no extra efforts into fixing reconfigure. But I will check it ASAP I am free. BR Chauncy. -----邮件原件----- 发件人: Benjamin Marzinski [mailto:bmarzins@xxxxxxxxxx] 发送时间: 2016年6月9日 6:43 收件人: gechangwei 12382 (CCPL) 抄送: dm-devel@xxxxxxxxxx; christophe.varoqui@xxxxxxxxx 主题: Re: [PATCH]RE: multipath-tools : A way needless to restart multipathd damon while attaching new device to host which met a path failure On Sun, May 29, 2016 at 06:13:08AM +0000, Gechangwei wrote: > Hi Ben, > > It is inconvenient to add blacklist exception to conf without restart multipath service or perform reconfigure command. > Suffering storage path failure, either performing reconfigure or restarting multipathd daemon will flush the map with path failure. > > With this patch, blacklist exception will be added dynamically without restarting multipathd daemon, hence, no map will be flushed. > > I am looking forward for your comments I don't see anything broken in this patch, but it's not the way that I'd like to go about fixing this, if at all possible. Your goal in this patch was to work around doing a reconfigure to change your blacklists and update your devices because reconfiguring can remove paths and whole maps, simply because the path is down when you do the reconfigure, correct? I'd rather fix reconfigure so that it can be safely used in these cases, instead of trying to duplicate some of its functionality by piecemeal additions to the interactive commands. Did you try going this route, and find that it wasn't possible, becasuse I'm looking into it right now. -Ben > > Thanks. > > Br. > > > Date: Tue, 17 May 2016 14:13:25 +0800 > Subject: [PATCH] Dynamic blacklist exception addition > > Signed-off-by: gechangwei <ge.changwei@xxxxxxx> > --- > multipath-tools-0.5.0/libmultipath/structs.c | 19 +++ > multipath-tools-0.5.0/libmultipath/structs.h | 1 + > multipath-tools-0.5.0/multipathd/cli.c | 8 + > multipath-tools-0.5.0/multipathd/cli.h | 10 +- > multipath-tools-0.5.0/multipathd/cli_handlers.c | 195 ++++++++++++++++++++++++ > multipath-tools-0.5.0/multipathd/cli_handlers.h | 5 + > multipath-tools-0.5.0/multipathd/main.c | 4 + > 7 files changed, 241 insertions(+), 1 deletion(-) > > diff --git a/multipath-tools-0.5.0/libmultipath/structs.c > b/multipath-tools-0.5.0/libmultipath/structs.c > index 0538327..6d3235a 100644 > --- a/multipath-tools-0.5.0/libmultipath/structs.c > +++ b/multipath-tools-0.5.0/libmultipath/structs.c > @@ -424,6 +424,25 @@ find_path_by_devt (vector pathvec, char * dev_t) > return NULL; > } > > +struct blentry * > +find_wwid_in_elist (vector wwid_vector, char * wwid) { > + int i; > + struct blentry *ele = NULL; > + > + if (!wwid_vector) { > + condlog(0, "wwid vector is NULL while finding wwid"); > + return NULL; > + } > + > + vector_foreach_slot (wwid_vector, ele, i) > + if (!strncmp(ele->str, wwid, WWID_SIZE)) > + return ele; > + > + condlog(3, "not found in wwid vecotr"); > + return NULL; > +} > + > extern int > pathcountgr (struct pathgroup * pgp, int state) { diff --git > a/multipath-tools-0.5.0/libmultipath/structs.h > b/multipath-tools-0.5.0/libmultipath/structs.h > index ab7dc25..75a39cd 100644 > --- a/multipath-tools-0.5.0/libmultipath/structs.h > +++ b/multipath-tools-0.5.0/libmultipath/structs.h > @@ -332,6 +332,7 @@ struct multipath * find_mp_by_minor (vector mp, > int minor); struct path * find_path_by_devt (vector pathvec, char * > devt); struct path * find_path_by_dev (vector pathvec, char * dev); > struct path * first_path (struct multipath * mpp); > +struct blentry * find_wwid_in_elist (vector wwid_vector, char * > +wwid); > > int pathcountgr (struct pathgroup *, int); int pathcount (struct > multipath *, int); diff --git a/multipath-tools-0.5.0/multipathd/cli.c > b/multipath-tools-0.5.0/multipathd/cli.c > index 6a5c6db..75389d7 100644 > --- a/multipath-tools-0.5.0/multipathd/cli.c > +++ b/multipath-tools-0.5.0/multipathd/cli.c > @@ -188,6 +188,10 @@ load_keys (void) > r += add_key(keys, "getprstatus", GETPRSTATUS, 0); > r += add_key(keys, "setprstatus", SETPRSTATUS, 0); > r += add_key(keys, "unsetprstatus", UNSETPRSTATUS, 0); > + r += add_key(keys, "elist", ELIST, 1); > + r += add_key(keys, "discover", DISCOVER, 0); > + r += add_key(keys, "newpath", NEWPATH, 0); > + r += add_key(keys, "mappath", MAPPATH, 1); > r += add_key(keys, "format", FMT, 1); > > if (r) { > @@ -504,6 +508,10 @@ cli_init (void) { > add_handler(UNSETPRSTATUS+MAP, NULL); > add_handler(FORCEQ+DAEMON, NULL); > add_handler(RESTOREQ+DAEMON, NULL); > + add_handler(ADD+ELIST, NULL); > + add_handler(DEL+ELIST, NULL); > + add_handler(DISCOVER+NEWPATH, NULL); > + add_handler(DEL+MAPPATH, NULL); > > return 0; > } > diff --git a/multipath-tools-0.5.0/multipathd/cli.h > b/multipath-tools-0.5.0/multipathd/cli.h > index f6d2726..6a0ffd8 100644 > --- a/multipath-tools-0.5.0/multipathd/cli.h > +++ b/multipath-tools-0.5.0/multipathd/cli.h > @@ -33,6 +33,10 @@ enum { > __GETPRSTATUS, > __SETPRSTATUS, > __UNSETPRSTATUS, > + __ELIST, > + __DISCOVER, > + __NEWPATH, > + __MAPPATH, > __FMT, > }; > > @@ -71,7 +75,11 @@ enum { > #define GETPRSTATUS (1UL << __GETPRSTATUS) > #define SETPRSTATUS (1UL << __SETPRSTATUS) > #define UNSETPRSTATUS (1UL << __UNSETPRSTATUS) > -#define FMT (1UL << __FMT) > +#define ELIST (1UL << __ELIST) > +#define DISCOVER (1UL << __DISCOVER) > +#define NEWPATH (1UL << __NEWPATH) > +#define MAPPATH (1UL << __MAPPATH) > +#define FMT (1UL << __FMT) > > #define INITIAL_REPLY_LEN 1200 > > diff --git a/multipath-tools-0.5.0/multipathd/cli_handlers.c > b/multipath-tools-0.5.0/multipathd/cli_handlers.c > index 3d39acf..26d0a77 100644 > --- a/multipath-tools-0.5.0/multipathd/cli_handlers.c > +++ b/multipath-tools-0.5.0/multipathd/cli_handlers.c > @@ -516,6 +516,110 @@ cli_del_path (void * v, char ** reply, int * len, void * data) > return ev_remove_path(pp, vecs); > } > > +int > +cli_add_elist(void * v, char ** reply, int * len, void * data) { > + char *param = get_keyparam(v, ELIST); > + struct blentry *ble = NULL; > + char *elist_wwid = NULL; > + > + condlog(2, "%s: add elist (operator)", param); > + > + if(find_wwid_in_elist(conf->elist_wwid, param)) { > + condlog(0, "elist already contains wwid(%s)", param); > + goto out; > + } > + > + elist_wwid = MALLOC(WWID_SIZE); > + if (!elist_wwid) { > + condlog(0, "malloc temporary wwid space failed."); > + goto out; > + } > + > + strncpy(elist_wwid, param, WWID_SIZE); > + elist_wwid[WWID_SIZE - 1] = '\0'; > + > + ble = MALLOC(sizeof(struct blentry)); > + if (!ble) { > + condlog(0, "malloc black list exception entry failed."); > + goto out; > + } > + > + if (regcomp(&ble->regex, param, REG_EXTENDED|REG_NOSUB)) > + goto out; > + > + ble->origin = ORIGIN_CONFIG; > + ble->str = elist_wwid; > + > + if (!vector_alloc_slot(conf->elist_wwid)) { > + condlog(0, "malloc black list exception entry failed."); > + goto out; > + } > + vector_set_slot(conf->elist_wwid, ble); > + return 0;; > +out: > + *len = 0; > + FREE(elist_wwid); > + FREE(ble); > + return 1; > +} > + > +int > +cli_del_elist(void * v, char ** reply, int * len, void * data) { > + int i; > + struct blentry *ble = NULL; > + char *param = get_keyparam(v, ELIST); > + > + *len = 0; > + > + condlog(2, "%s: del elist (operator)", param); > + > + vector_foreach_slot (conf->elist_wwid, ble, i) { > + if (ble->str && !strncmp(ble->str, param, WWID_SIZE)) { > + FREE(ble->str); > + regfree(&ble->regex); > + FREE(ble); > + vector_del_slot(conf->elist_wwid, i); > + i--; > + return 0; > + } > + } > + > + condlog(0, "specific wwid(%s) is not found!", param); > + return 1; > +} > + > +int > +cli_discover_new_path(void * v, char ** reply, int * len, void * > +data) { > + int ret = 0; > + struct vectors *vecs = (struct vectors *)data; > + vector pathvec = vecs->pathvec; > + > + condlog(2, "discover new paths, this could update current paths' state (operator)"); > + if (path_discovery(pathvec, conf, DI_ALL) < 0) { > + condlog(0, "discover new paths failed."); > + ret = 1; > + } > + > + struct path *pp; > + int i; > + > + vector_foreach_slot(pathvec, pp, i) { > + if(0 == pp->checkint) { > + pp->checkint = conf->checkint; > + condlog(2, "%s: device is found.", pp->dev); > + } > + } > + > + *len = 0; > + return ret; > +} > + > int > cli_add_map (void * v, char ** reply, int * len, void * data) { @@ > -618,6 +722,97 @@ cli_del_map (void * v, char ** reply, int * len, void * data) > return rc; > } > > +int > +cli_del_map_and_paths(void * v, char ** reply, int * len, void * > +data) { > + int rc; > + int ret = 1; > + struct vectors *vecs = (struct vectors *)data; > + char *param = get_keyparam(v, MAPPATH); /* SCSI ID */ > + char *refwwid = NULL; > + char *wwid_tmp = NULL; > + vector pathvec = NULL; > + vector cmdvec = NULL; > + struct key *kw = NULL; > + char *dev_name = NULL; > + > + condlog(2, "%s: del map and paths (operator)", param); > + pathvec = vecs->pathvec; > + if (pathvec == NULL) > + goto out; > + > + param = convert_dev(param, 0); > + wwid_tmp = MALLOC(WWID_SIZE); > + if(!wwid_tmp) { > + condlog(0, "malloc temporary wwid space failed."); > + goto out; > + } > + > + rc = get_refwwid(param, DEV_DEVMAP, pathvec, &refwwid); > + if (rc) { > + condlog(0, "get ref wwid of map(%s) failed.", param); > + goto out; > + } > + > + condlog(3, "ref wwid(%s) is gotten.", refwwid); > + strncpy(wwid_tmp, refwwid, WWID_SIZE); > + wwid_tmp[WWID_SIZE - 1] = '\0'; > + > + /* prepare path deletion parameter vector */ > + /* this is triky since we fake a command vector */ > + cmdvec = vector_alloc(); > + dev_name = MALLOC(FILE_NAME_SIZE); > + kw = (struct key *)MALLOC(sizeof(struct key)); > + if(NULL == cmdvec || NULL == dev_name || NULL == kw || !vector_alloc_slot(cmdvec)) { > + condlog(0, "prepare command vector failed."); > + goto out; > + } > + vector_set_slot(cmdvec, kw); > + > + kw->code = MAP; > + kw->has_param = 1; > + kw->param = wwid_tmp; > + > + rc = cli_del_map(cmdvec, NULL, NULL, data); > + if(0 != rc) { > + condlog(0, "del map(%s) faild before del all its paths", param); > + goto out; > + } > + > + kw->code = PATH; > + kw->has_param = 1; > + kw->param = dev_name; > + > + int i; > + int del_ret = 0; > + int del_ret_once = 0; > + struct path *pp = NULL; > + > + vector_foreach_slot(pathvec, pp, i) { > + if(!strncmp(pp->wwid, wwid_tmp, WWID_SIZE)) { > + condlog(2, "path(%s) of wwid(%s) found, try to remove it.", pp->dev, wwid_tmp); > + strncpy(dev_name, pp->dev, FILE_NAME_SIZE); > + if((del_ret_once = cli_del_path(cmdvec, NULL, NULL, data)) != 0) > + condlog(0, "del path(%s) failed.", dev_name); > + else > + i--; > + del_ret |= del_ret_once; > + } > + } > + if(del_ret) > + goto out; > + > + ret = 0; > +out: > + FREE(wwid_tmp); > + FREE(dev_name); > + vector_free(cmdvec); > + FREE(kw); > + *len = 0; > + return ret; > +} > + > int > cli_reload(void *v, char **reply, int *len, void *data) { diff --git > a/multipath-tools-0.5.0/multipathd/cli_handlers.h > b/multipath-tools-0.5.0/multipathd/cli_handlers.h > index 799f8da..b44d528 100644 > --- a/multipath-tools-0.5.0/multipathd/cli_handlers.h > +++ b/multipath-tools-0.5.0/multipathd/cli_handlers.h > @@ -39,4 +39,9 @@ int cli_reassign (void * v, char ** reply, int * > len, void * data); int cli_getprstatus(void * v, char ** reply, int * > len, void * data); int cli_setprstatus(void * v, char ** reply, int * > len, void * data); int cli_unsetprstatus(void * v, char ** reply, int > * len, void * data); > +int cli_add_elist(void * v, char ** reply, int * len, void * data); > +int cli_del_elist(void * v, char ** reply, int * len, void * data); > +int cli_discover_new_path(void * v, char ** reply, int * len, void * > +data); int cli_del_map_and_paths(void * v, char ** reply, int * len, > +void * data); > + > > diff --git a/multipath-tools-0.5.0/multipathd/main.c > b/multipath-tools-0.5.0/multipathd/main.c > index 626f703..d27b95b 100644 > --- a/multipath-tools-0.5.0/multipathd/main.c > +++ b/multipath-tools-0.5.0/multipathd/main.c > @@ -1013,6 +1013,10 @@ uxlsnrloop (void * ap) > set_handler_callback(UNSETPRSTATUS+MAP, cli_unsetprstatus); > set_handler_callback(FORCEQ+DAEMON, cli_force_no_daemon_q); > set_handler_callback(RESTOREQ+DAEMON, cli_restore_no_daemon_q); > + set_handler_callback(ADD+ELIST, cli_add_elist); > + set_handler_callback(DEL+ELIST, cli_del_elist); > + set_handler_callback(DISCOVER+NEWPATH, cli_discover_new_path); > + set_handler_callback(DEL+MAPPATH, cli_del_map_and_paths); > > umask(077); > uxsock_listen(&uxsock_trigger, ap); > > > > > -----邮件原件----- > 发件人: Benjamin Marzinski [mailto:bmarzins@xxxxxxxxxx] > 发送时间: 2016年5月26日 1:46 > 收件人: gechangwei 12382 (CCPL) > 抄送: dm-devel@xxxxxxxxxx; christophe.varoqui@xxxxxxxxx; zhongjinming > 02800 (CCPL); guozhonghua 02084 (CCPL); shichangkuo 09727 (CCPL); > zhangguanghui 10102 (CCPL) > 主题: Re: multipath-tools : A way needless to restart multipathd damon > while attaching new device to host which met a path failure > > While having multipathd retain failed devices when it restarts may be a little tricky, since it won't know anything about those devices, having it retain these devices during a reload (which is enough to be able to change the configuration) should be doable. I agree that multipathd should not be removing devices if they haven't been removed from the system since, like you say, that will keep multipathd from reacting to the device's later recovery. > > I'll take a look into doing this. > > -Ben > > On Mon, May 23, 2016 at 03:38:35AM +0000, Gechangwei wrote: > > Hi, > > > > > > > > I encountered an issue that devmapper disappeared while the multipathd > > dameon was restarted. > > > > > > > > Digging into the root cause of preceding issue, I found storage path > > failure while restarting the multipathd daemon. It seems unreasonable, > > because the storage path may recover in the future. > > > > > > > > When using multipath-tools to create device mapper, I configured > > “blacklist section” and “blacklist exception section”, which meets my > > requirement that only a small part of SCSI disk can be used underneath a > > device mapper. > > > > > > > > After restarting the multipathd daemon, the device mapper of path failure > > disappears and it won’t comeback even the storage paths recover. So the > > file system on that devmapper can’t be available. > > > > > > > > I wonder that if there is a solution that a device mapper of path failure > > can come back to active even I restart the multipathd daemon? > > > > > > > > If such a solution exists, it will be marvelous > > > > > > > > If not, I propose a userland interface to add blacklist exception > > dynamically. After the blacklist exception is added to “conf”, “multipathd > > add map $map” and “multipathd add path” can be used to add new map and > > path. > > > > > > > > In a nutshell, we can dynamically add elist, map and paths respectively. > > And multipathd damon is not necessary to restart due to preceding > > solution. > > > > > > > > Any experts of DM have any comments on my solution and the issue I faced > > with? > > > > > > > > > > > > Your reply is very important to me. > > > > > > > > Thanks. > > > > > > > > BR. > > > > > > > > Chauncey Ge > > > > ------------------------------------------------------------------------------------------------------------------------------------- > > 本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址 > > 中列出 > > 的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、 > > 复制、 > > 或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并 > > 删除本 > > 邮件！ > > This e-mail and its attachments contain confidential information from H3C, > > which is > > intended only for the person or entity whose address is listed above. Any > > use of the > > information contained herein in any way (including, but not limited to, > > total or partial > > disclosure, reproduction, or dissemination) by persons other than the > > intended > > recipient(s) is prohibited. If you receive this e-mail in error, please > > notify the sender > > by phone or email immediately and delete it! -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
