Hi,
I have attached some DHT files to demonstrate the 2 logging approaches. (*_1 is the original approach, *_2 is the proposed approach).I personally think the 2 approach leads to better code readability and propose that we follow approach 2. Please let me know of any concerns with this.
To consolidate all the points raised in the earlier discussions:
What are we trying to solve?
Improving gluster logs to make end user debugging easier by providing a sufficient information and a consistent logging mechanism and message format .
The new logging framework already logs the function name and line, msgid and strerror, which improves the log messages and debug-ability. However, there are some potential issues with the way it is getting used. Please note - there are no changes being proposed to the underlying logging framework.
Current approach (approach 1):
Define message_ids for each log message (except Trace and Debug) and associate both id and string with a msg_id macro
Replace all calls to gf_log with gf_msg passing in the message_id for the message. This message_id will be printed as part of the log message.
Document each log string with details of what caused it/how to fix it.
Issues:
1. Code readability - It becomes difficult to figure out what the following is actually printing and can cause issues with incorrect params being passed or params being passed in the wrong order:
gf_msg ("dht", GF_LOG_ERROR, 0, dht_msg_23, param1, param2, param3);
2.Code Redundancy -multiple messages for the same thing differing in small details can potentially use up a large chunk of allocated ids as well as making it difficult for end users - they will need to search for multiple string formats/msgids as they could all refer to more or less the same thing. For example:
dht_msg_1 123, "Failed to get cached subvol for %s"
dht_msg_2 124, "Failed to get cached subvol for %s on %s"
3. Documentation redundancy -
The proposed format for documenting these messages is as follows:
Msg ID
Message format string
Cause
Recommended action
This could potentially lead to documentation like:
Msg ID : 123
Message format string : Failed to get cached subvol for <path>
Cause : The subvolume might not be reachable etc etc
Recommended action : Check network connection etc etc
Msg ID : 124
Message format string : Failed to get cached subvol for <path> on <path2>
Cause : The subvolume might not be reachable etc etc
Recommended action : Check network connection etc etc
The end user now has to search for multiple msgids and string formats to find all instances of this error.
NOTE: It may be possible to consolidate all these strings into a single one, say, "Failed to get cached subvol for %s on %s" and mandate that it be used in all calls which are currently using variations of the string. However, this might not be possible in all scenarios - some params might not be available or might not be meaningful in a particular case or a developer might want to provide additional info in a particular scenario.
Proposed approach (approach 2):
Define meaningful macros for message_ids for a class of message (except Trace and Debug) without associating them to a message string. For example
#define DHT_CACHED_SUBVOL_GET_FAILED 123
#define DHT_MEM_ALLOC_FAILED 124
Replace all calls to gf_log with gf_msg but pass in the msg id and string separately. The string is defined by the developer based on an agreed upon format.
Define a log message format policy that all developers need to follow. This will need to be enforced by reviews. For example, we could mandate that all log messages must start with the name of the file on which the operation is performed and end with the strerror if it exists.This can also include rules as to sentence structure and wording - whether to use "failed", "unable to", "could not" etc.
Consolidate existing messages and reword them if necessary to make them more meaningful. If a single message will work in multiple instances, use that one everywhere.
Add your documentation writer as a reviewer for all patches. S/he will be responsible for ensuring that all newly introduced log messages are meaningful, consistent and follow the agreed upon format.
Devs will define new message classes ids as and when required.
Ideally, common message classes like dict-set-failed or memory-alloc-failed should be defined in a common file and included by others - no point having each component define a memory_alloc_failed id.
With the proposed approach:
#define DHT_CACHED_SUBVOL_GET_FAILED 123
#define DHT_HASHED_SUBVOL_GET_FAILED 124
Calls would then look like:
gf_msg ("dht", GF_LOG_ERROR, DHT_CACHED_SUBVOL_GET_FAILED, "Failed to get cached subvolume for path %s", param1);
...
gf_msg ("dht", GF_LOG_ERROR, DHT_CACHED_SUBVOL_GET_FAILED, "Failed to get cached subvolume for path %s on %s", param1, param2);
Documentation would be as follows:
Msg ID : 123
Description : Failed to get the cached subvolume for the specified path
Cause : The subvolume might not be reachable etc etc
Recommended action : Check network connection etc etc
Admins could just search for [MSGID 123] and find all instances of where an operation failed to get a cached sub volume.
Issues raised with proposed approach:
1. Internationalization: Having the strings in a single file is required to make L10N easy.
While i18n is very important for user tools, utils etc, log messages are usually targeted at developers and sys admins who usually know English. Plus it seems unlikely that log messages will be localized in the near future. However, the document describing the msgid can be localized so the msg id mapping information can still be used.
2. Having the strings in a header file makes it easier to change the format later.
This is a valid point. However, IMHO, the code readability is more important especially in the case where we pass arguments to the format string.
3. Having a string defined in a single header file can make it easier for a dev to reuse it if necessary
I would suggest searching on the message id instead and copy the string from elsewhere if it suits his purpose as those will already have been reviewed by the doc writer.
Regards,
Nithya/*
Copyright (c) 2008-2012 Red Hat, Inc. <http://www.redhat.com>
This file is part of GlusterFS.
This file is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3 or
later), or the GNU General Public License, version 2 (GPLv2), in all
cases as published by the Free Software Foundation.
*/
#ifndef _CONFIG_H
#define _CONFIG_H
#include "config.h"
#endif
/* TODO: add NS locking */
#include "glusterfs.h"
#include "xlator.h"
#include "dht-common.h"
#include "dht-messages.h"
#include "defaults.h"
#include <sys/time.h>
int
dht_du_info_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
int op_ret, int op_errno, struct statvfs *statvfs,
dict_t *xdata)
{
dht_conf_t *conf = NULL;
call_frame_t *prev = NULL;
int this_call_cnt = 0;
int i = 0;
double percent = 0;
double percent_inodes = 0;
uint64_t bytes = 0;
conf = this->private;
prev = cookie;
if (op_ret == -1) {
gf_msg (this->name, GF_LOG_WARNING, 0, dht_msg_43,
prev->this->name);
goto out;
}
if (statvfs && statvfs->f_blocks) {
percent = (statvfs->f_bavail * 100) / statvfs->f_blocks;
bytes = (statvfs->f_bavail * statvfs->f_frsize);
}
if (statvfs && statvfs->f_files) {
percent_inodes = (statvfs->f_ffree * 100) / statvfs->f_files;
} else {
/* set percent inodes to 100 for dynamically allocated inode filesystems
this logic holds good so that, distribute has nothing to worry about
total inodes rather let the 'create()' to be scheduled on the hashed
subvol regardless of the total inodes. since we have no awareness on
loosing inodes this logic fits well
*/
percent_inodes = 100;
}
LOCK (&conf->subvolume_lock);
{
for (i = 0; i < conf->subvolume_cnt; i++)
if (prev->this == conf->subvolumes[i]) {
conf->du_stats[i].avail_percent = percent;
conf->du_stats[i].avail_space = bytes;
conf->du_stats[i].avail_inodes = percent_inodes;
gf_log (this->name, GF_LOG_DEBUG,
"on subvolume '%s': avail_percent is: "
"%.2f and avail_space is: %"PRIu64" "
"and avail_inodes is: %.2f",
prev->this->name,
conf->du_stats[i].avail_percent,
conf->du_stats[i].avail_space,
conf->du_stats[i].avail_inodes);
}
}
UNLOCK (&conf->subvolume_lock);
out:
this_call_cnt = dht_frame_return (frame);
if (is_last_call (this_call_cnt))
DHT_STACK_DESTROY (frame);
return 0;
}
int
dht_get_du_info_for_subvol (xlator_t *this, int subvol_idx)
{
dht_conf_t *conf = NULL;
call_frame_t *statfs_frame = NULL;
dht_local_t *statfs_local = NULL;
call_pool_t *pool = NULL;
loc_t tmp_loc = {0,};
conf = this->private;
pool = this->ctx->pool;
statfs_frame = create_frame (this, pool);
if (!statfs_frame) {
goto err;
}
/* local->fop value is not used in this case */
statfs_local = dht_local_init (statfs_frame, NULL, NULL,
GF_FOP_MAXVALUE);
if (!statfs_local) {
goto err;
}
/* make it root gfid, should be enough to get the proper info back */
tmp_loc.gfid[15] = 1;
statfs_local->call_cnt = 1;
STACK_WIND (statfs_frame, dht_du_info_cbk,
conf->subvolumes[subvol_idx],
conf->subvolumes[subvol_idx]->fops->statfs,
&tmp_loc, NULL);
return 0;
err:
if (statfs_frame)
DHT_STACK_DESTROY (statfs_frame);
return -1;
}
int
dht_get_du_info (call_frame_t *frame, xlator_t *this, loc_t *loc)
{
int i = 0;
dht_conf_t *conf = NULL;
call_frame_t *statfs_frame = NULL;
dht_local_t *statfs_local = NULL;
struct timeval tv = {0,};
loc_t tmp_loc = {0,};
conf = this->private;
gettimeofday (&tv, NULL);
/* make it root gfid, should be enough to get the proper
info back */
tmp_loc.gfid[15] = 1;
if (tv.tv_sec > (conf->refresh_interval
+ conf->last_stat_fetch.tv_sec)) {
statfs_frame = copy_frame (frame);
if (!statfs_frame) {
goto err;
}
/* In this case, 'local->fop' is not used */
statfs_local = dht_local_init (statfs_frame, loc, NULL,
GF_FOP_MAXVALUE);
if (!statfs_local) {
goto err;
}
statfs_local->call_cnt = conf->subvolume_cnt;
for (i = 0; i < conf->subvolume_cnt; i++) {
STACK_WIND (statfs_frame, dht_du_info_cbk,
conf->subvolumes[i],
conf->subvolumes[i]->fops->statfs,
&tmp_loc, NULL);
}
conf->last_stat_fetch.tv_sec = tv.tv_sec;
}
return 0;
err:
if (statfs_frame)
DHT_STACK_DESTROY (statfs_frame);
return -1;
}
gf_boolean_t
dht_is_subvol_filled (xlator_t *this, xlator_t *subvol)
{
int i = 0;
dht_conf_t *conf = NULL;
gf_boolean_t subvol_filled_inodes = _gf_false;
gf_boolean_t subvol_filled_space = _gf_false;
gf_boolean_t is_subvol_filled = _gf_false;
conf = this->private;
/* Check for values above specified percent or free disk */
LOCK (&conf->subvolume_lock);
{
for (i = 0; i < conf->subvolume_cnt; i++) {
if (subvol == conf->subvolumes[i]) {
if (conf->disk_unit == 'p') {
if (conf->du_stats[i].avail_percent <
conf->min_free_disk) {
subvol_filled_space = _gf_true;
break;
}
} else {
if (conf->du_stats[i].avail_space <
conf->min_free_disk) {
subvol_filled_space = _gf_true;
break;
}
}
if (conf->du_stats[i].avail_inodes <
conf->min_free_inodes) {
subvol_filled_inodes = _gf_true;
break;
}
}
}
}
UNLOCK (&conf->subvolume_lock);
if (subvol_filled_space && conf->subvolume_status[i]) {
if (!(conf->du_stats[i].log++ % (GF_UNIVERSAL_ANSWER * 10))) {
gf_msg (this->name, GF_LOG_WARNING, 0, dht_msg_30,
subvol->name,
(100 - conf->du_stats[i].avail_percent));
}
}
if (subvol_filled_inodes && conf->subvolume_status[i]) {
if (!(conf->du_stats[i].log++ % (GF_UNIVERSAL_ANSWER * 10))) {
gf_msg (this->name, GF_LOG_CRITICAL, 0, dht_msg_94,
subvol->name,
(100 - conf->du_stats[i].avail_inodes));
}
}
is_subvol_filled = (subvol_filled_space || subvol_filled_inodes);
return is_subvol_filled;
}
/*Get the best subvolume to create the file in*/
xlator_t *
dht_free_disk_available_subvol (xlator_t *this, xlator_t *subvol,
dht_local_t *local)
{
xlator_t *avail_subvol = NULL;
dht_conf_t *conf = NULL;
dht_layout_t *layout = NULL;
loc_t *loc = NULL;
conf = this->private;
if (!local)
goto out;
loc = &local->loc;
if (!local->layout) {
layout = dht_layout_get (this, loc->parent);
if (!layout) {
gf_log (this->name, GF_LOG_DEBUG,
"layout missing path=%s parent=%s",
loc->path, uuid_utoa (loc->parent->gfid));
goto out;
}
} else {
layout = dht_layout_ref (this, local->layout);
}
LOCK (&conf->subvolume_lock);
{
avail_subvol = dht_subvol_with_free_space_inodes(this, subvol,
layout);
if(!avail_subvol)
{
avail_subvol = dht_subvol_maxspace_nonzeroinode(this,
subvol,
layout);
}
}
UNLOCK (&conf->subvolume_lock);
out:
if (!avail_subvol) {
gf_log (this->name,
GF_LOG_DEBUG,
"no subvolume has enough free space and/or inodes\
to create");
avail_subvol = subvol;
}
if (layout)
dht_layout_unref (this, layout);
return avail_subvol;
}
static inline
int32_t dht_subvol_has_err (xlator_t *this, dht_layout_t *layout)
{
int ret = -1;
int i = 0;
if (!this || !layout)
goto out;
/* check if subvol has layout errors, before selecting it */
for (i = 0; i < layout->cnt; i++) {
if (!strcmp (layout->list[i].xlator->name, this->name) &&
(layout->list[i].err != 0)) {
ret = -1;
goto out;
}
}
ret = 0;
out:
return ret;
}
/*Get subvolume which has both space and inodes more than the min criteria*/
xlator_t *
dht_subvol_with_free_space_inodes(xlator_t *this, xlator_t *subvol,
dht_layout_t *layout)
{
int i = 0;
double max = 0;
double max_inodes = 0;
int ignore_subvol = 0;
xlator_t *avail_subvol = NULL;
dht_conf_t *conf = NULL;
conf = this->private;
for(i=0; i < conf->subvolume_cnt; i++) {
/* check if subvol has layout errors, before selecting it */
ignore_subvol = dht_subvol_has_err (conf->subvolumes[i],
layout);
if (ignore_subvol)
continue;
if ((conf->disk_unit == 'p') &&
(conf->du_stats[i].avail_percent > conf->min_free_disk) &&
(conf->du_stats[i].avail_inodes > conf->min_free_inodes)) {
if ((conf->du_stats[i].avail_inodes > max_inodes) ||
(conf->du_stats[i].avail_percent > max)) {
max = conf->du_stats[i].avail_percent;
max_inodes = conf->du_stats[i].avail_inodes;
avail_subvol = conf->subvolumes[i];
}
}
if ((conf->disk_unit != 'p') &&
(conf->du_stats[i].avail_space > conf->min_free_disk) &&
(conf->du_stats[i].avail_inodes > conf->min_free_inodes)) {
if ((conf->du_stats[i].avail_inodes > max_inodes) ||
(conf->du_stats[i].avail_space > max)) {
max = conf->du_stats[i].avail_space;
max_inodes = conf->du_stats[i].avail_inodes;
avail_subvol = conf->subvolumes[i];
}
}
}
return avail_subvol;
}
/* Get subvol which has atleast one inode and maximum space */
xlator_t *
dht_subvol_maxspace_nonzeroinode (xlator_t *this, xlator_t *subvol,
dht_layout_t *layout)
{
int i = 0;
double max = 0;
int ignore_subvol = 0;
xlator_t *avail_subvol = NULL;
dht_conf_t *conf = NULL;
conf = this->private;
for (i = 0; i < conf->subvolume_cnt; i++) {
/* check if subvol has layout errors, before selecting it */
ignore_subvol = dht_subvol_has_err (conf->subvolumes[i],
layout);
if (ignore_subvol)
continue;
if (conf->disk_unit == 'p') {
if ((conf->du_stats[i].avail_percent > max)
&& (conf->du_stats[i].avail_inodes > 0 )) {
max = conf->du_stats[i].avail_percent;
avail_subvol = conf->subvolumes[i];
}
} else {
if ((conf->du_stats[i].avail_space > max)
&& (conf->du_stats[i].avail_inodes > 0)) {
max = conf->du_stats[i].avail_space;
avail_subvol = conf->subvolumes[i];
}
}
}
return avail_subvol;
}
/*
Copyright (c) 2013 Red Hat, Inc. <http://www.redhat.com>
This file is part of GlusterFS.
This file is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3 or
later), or the GNU General Public License, version 2 (GPLv2), in all
cases as published by the Free Software Foundation.
*/
#ifndef _DHT_MESSAGES_H_
#define _DHT_MESSAGES_H_
#ifndef _CONFIG_H
#define _CONFIG_H
#include "config.h"
#endif
#include "glfs-message-id.h"
/* NOTE: Rules for message additions
* 1) Each instance of a message is _better_ left with a unique message ID, even
* if the message format is the same. Reasoning is that, if the message
* format needs to change in one instance, the other instances are not
* impacted or the new change does not change the ID of the instance being
* modified.
* 2) Addition of a message,
* - Should increment the GLFS_NUM_MESSAGES
* - Append to the list of messages defined, towards the end
* - Retain macro naming as glfs_msg_X (for redability across developers)
* NOTE: Rules for message format modifications
* 3) Check across the code if the message ID macro in question is reused
* anywhere. If reused then then the modifications should ensure correctness
* everywhere, or needs a new message ID as (1) above was not adhered to. If
* not used anywhere, proceed with the required modification.
* NOTE: Rules for message deletion
* 4) Check (3) and if used anywhere else, then cannot be deleted. If not used
* anywhere, then can be deleted, but will leave a hole by design, as
* addition rules specify modification to the end of the list and not filling
* holes.
*/
#define GLFS_COMP_BASE GLFS_MSGID_COMP_DHT
#define GLFS_NUM_MESSAGES 218
#define GLFS_MSGID_END (GLFS_COMP_BASE + GLFS_NUM_MESSAGES + 1)
/* Messages with message IDs */
#define dht_msg_start_x GLFS_COMP_BASE, "Invalid: Start of messages"
#define dht_msg_1 (GLFS_COMP_BASE + 1), "Alloc or fill failure"
#define dht_msg_2 (GLFS_COMP_BASE + 2), "Attempting to migrate hardlink %s with gfid %s from %s -> %s"
#define dht_msg_3 (GLFS_COMP_BASE + 3), "Cannot create linkfile file for %s on %s: hashed subvolume cannot be found."
#define dht_msg_4 (GLFS_COMP_BASE + 4), "Cannot parse glusterd node uuid"
#define dht_msg_5 (GLFS_COMP_BASE + 5), "Cannot parse rebalance-filter (%s)"
#define dht_msg_6 (GLFS_COMP_BASE + 6), "Catastrophic error layout with unknown type found %d"
#define dht_msg_7 (GLFS_COMP_BASE + 7), "Check \"scheduler.switch.case\" option in unify volume. Exiting."
#define dht_msg_8 (GLFS_COMP_BASE + 8), "Child loc build failed"
#define dht_msg_9 (GLFS_COMP_BASE + 9), "chown failed for %s on %s (%s)"
#define dht_msg_10 (GLFS_COMP_BASE + 10), "Compiling regex %s failed"
#define dht_msg_11 (GLFS_COMP_BASE + 11), "Completed migration of %s from subvolume %s to %s"
#define dht_msg_12 (GLFS_COMP_BASE + 12), "Could not create task for rebalance"
#define dht_msg_13 (GLFS_COMP_BASE + 13), "Could not find hostname (%s)"
#define dht_msg_14 (GLFS_COMP_BASE + 14), "Could not set ACL_ACCESS xattr"
#define dht_msg_15 (GLFS_COMP_BASE + 15), "Could not set ACL_DEFAULT xattr"
#define dht_msg_16 (GLFS_COMP_BASE + 16), "Could not set pre-set layout for subvolume %s"
#define dht_msg_17 (GLFS_COMP_BASE + 17), "Couldn't find a local brick"
#define dht_msg_18 (GLFS_COMP_BASE + 18), "Creating %s on %s (got create on %s)"
#define dht_msg_19 (GLFS_COMP_BASE + 19), "Dangling volume. check volfile"
#define dht_msg_20 (GLFS_COMP_BASE + 20), "Data movement attempted from node (%s) with higher disk space to a node (%s) with lesser disk space (%s)"
#define dht_msg_21 (GLFS_COMP_BASE + 21), "Data movement attempted from node (%s) with to node (%s) which does not have required free space for %s"
#define dht_msg_22 (GLFS_COMP_BASE + 22), "Data to bin failed"
#define dht_msg_23 (GLFS_COMP_BASE + 23), "Decommissioning subvolume %s"
#define dht_msg_24 (GLFS_COMP_BASE + 24), "Deleting stale linkfile %s on %s"
#define dht_msg_25 (GLFS_COMP_BASE + 25), "dht aggregate dict set failed"
#define dht_msg_26 (GLFS_COMP_BASE + 26), "Dict is NULL, need to make sure gfids are same"
#define dht_msg_27 (GLFS_COMP_BASE + 27), "dict_new failed"
#define dht_msg_28 (GLFS_COMP_BASE + 28), "Dict set failed"
#define dht_msg_29 (GLFS_COMP_BASE + 29), "Disk layout has invalid count %d"
#define dht_msg_30 (GLFS_COMP_BASE + 30), "Disk space on subvolume '%s' is getting full (%.2f %%), consider adding more nodes"
#define dht_msg_31 (GLFS_COMP_BASE + 31), "Distribute needs more than one subvolume"
#define dht_msg_32 (GLFS_COMP_BASE + 32), "%d subvolumes down -- not fixing"
#define dht_msg_33 (GLFS_COMP_BASE + 33), "%d subvolumes have unrecoverable errors"
#define dht_msg_34 (GLFS_COMP_BASE + 34), "Error no layout on disk for merge"
#define dht_msg_35 (GLFS_COMP_BASE + 35), "Error while finding anomalies in %s -- not good news"
#define dht_msg_36 (GLFS_COMP_BASE + 36), "Failed lookup %s on %s (%s)"
#define dht_msg_37 (GLFS_COMP_BASE + 37), "Failed to create fd"
#define dht_msg_38 (GLFS_COMP_BASE + 38), "Failed to create local_t's memory pool"
#define dht_msg_39 (GLFS_COMP_BASE + 39), "Failed to create %s on %s (%s)"
#define dht_msg_40 (GLFS_COMP_BASE + 40), "Failed to fstat file %s on %s (%s)"
#define dht_msg_41 (GLFS_COMP_BASE + 41), "Failed to get cached subvol for %s"
#define dht_msg_42 (GLFS_COMP_BASE + 42), "Failed to get cached subvol for %s on %s"
#define dht_msg_43 (GLFS_COMP_BASE + 43), "Failed to get disk info from %s"
#define dht_msg_44 (GLFS_COMP_BASE + 44), "Failed to get hashed subvol for %s"
#define dht_msg_45 (GLFS_COMP_BASE + 45), "Failed to get hashed subvol for %s on %s"
#define dht_msg_46 (GLFS_COMP_BASE + 46), "Failed to get linkto subvol for %s"
#define dht_msg_47 (GLFS_COMP_BASE + 47), "Failed to get node-uuid for %s"
#define dht_msg_48 (GLFS_COMP_BASE + 48), "Failed to get node-uuid from dict for %s"
#define dht_msg_49 (GLFS_COMP_BASE + 49), "Failed to get statfs of %s on %s (%s)"
#define dht_msg_50 (GLFS_COMP_BASE + 50), "Failed to link the layout in inode"
#define dht_msg_51 (GLFS_COMP_BASE + 51), "Failed to lookup %s on %s (%s)"
#define dht_msg_52 (GLFS_COMP_BASE + 52), "Failed to open dir %s"
#define dht_msg_53 (GLFS_COMP_BASE + 53), "Failed to open file %s on %s (%s)"
#define dht_msg_54 (GLFS_COMP_BASE + 54), "Failed to open %s on %s"
#define dht_msg_55 (GLFS_COMP_BASE + 55), "Failed to open the fd (%p, flags=0%o) on file %s @ %s"
#define dht_msg_56 (GLFS_COMP_BASE + 56), "Failed to send open the fd (%p, flags=0%o) on file %s @ %s"
#define dht_msg_56 (GLFS_COMP_BASE + 56), "Failed to send open the fd (%p, flags=0%o) on file %s @ %s"
#define dht_msg_57 (GLFS_COMP_BASE + 57), "Failed to set dict str"
#define dht_msg_58 (GLFS_COMP_BASE + 58), "Failed to set dict value."
#define dht_msg_59 (GLFS_COMP_BASE + 59), "Failed to set dict value for %s"
#define dht_msg_60 (GLFS_COMP_BASE + 60), "Failed to set failure count"
#define dht_msg_61 (GLFS_COMP_BASE + 61), "Failed to set file count"
#define dht_msg_62 (GLFS_COMP_BASE + 62), "Failed to set internal dict key for %s"
#define dht_msg_63 (GLFS_COMP_BASE + 63), "Failed to set key %s"
#define dht_msg_64 (GLFS_COMP_BASE + 64), "Failed to set layout for subvol %s"
#define dht_msg_65 (GLFS_COMP_BASE + 65), "Failed to set layout for subvolume %s"
#define dht_msg_66 (GLFS_COMP_BASE + 66), "Failed to set linkto key"
#define dht_msg_67 (GLFS_COMP_BASE + 67), "Failed to set linkto key in dict"
#define dht_msg_68 (GLFS_COMP_BASE + 68), "Failed to set linkto xattr in dict for %s"
#define dht_msg_69 (GLFS_COMP_BASE + 69), "Failed to set lookedup file count"
#define dht_msg_70 (GLFS_COMP_BASE + 70), "Failed to set marker dont account key for %s"
#define dht_msg_71 (GLFS_COMP_BASE + 71), "Failed to set mode on %s in %s (%s)"
#define dht_msg_72 (GLFS_COMP_BASE + 72), "Failed to set quota limit key on %s"
#define dht_msg_73 (GLFS_COMP_BASE + 73), "Failed to set run-time"
#define dht_msg_74 (GLFS_COMP_BASE + 74), "Failed to set size of xfer"
#define dht_msg_75 (GLFS_COMP_BASE + 75), "Failed to set '%s' key"
#define dht_msg_76 (GLFS_COMP_BASE + 76), "Failed to set skipped file count"
#define dht_msg_77 (GLFS_COMP_BASE + 77), "Failed to set status"
#define dht_msg_78 (GLFS_COMP_BASE + 78), "Failed to set uid/gid on %s on %s subvol (%s)"
#define dht_msg_79 (GLFS_COMP_BASE + 79), "Failed to set xattr in dict for %s (linkto:%s)"
#define dht_msg_80 (GLFS_COMP_BASE + 80), "Failed to set xattr on %s in %s (%s)"
#define dht_msg_81 (GLFS_COMP_BASE + 81), "Failed to unlink listener socket %s, error: %s"
#define dht_msg_82 (GLFS_COMP_BASE + 82), "Failed to write (%s)"
#define dht_msg_83 (GLFS_COMP_BASE + 83), "File %s exists in %s with different gfid"
#define dht_msg_84 (GLFS_COMP_BASE + 84), "Files migrated: %"PRIu64", size: %" PRIu64", lookups: %"PRIu64", failures: %"PRIu64", skipped: %"PRIu64
#define dht_msg_85 (GLFS_COMP_BASE + 85), "Fixing the layout of %s"
#define dht_msg_86 (GLFS_COMP_BASE + 86), "Fix layout failed for %s"
#define dht_msg_87 (GLFS_COMP_BASE + 87), "Fix layout on %s failed"
#define dht_msg_88 (GLFS_COMP_BASE + 88), "Found anomalies in %s. holes=%d overlaps=%d"
#define dht_msg_89 (GLFS_COMP_BASE + 89), "Found local subvol, %s"
#define dht_msg_90 (GLFS_COMP_BASE + 90), "ftruncate failed for %s on %s (%s)"
#define dht_msg_91 (GLFS_COMP_BASE + 91), "getxattr err (%s) for dir"
#define dht_msg_92 (GLFS_COMP_BASE + 92), "Got non-linkfile %s:%s"
#define dht_msg_93 (GLFS_COMP_BASE + 93), "Hash computation failed for type=%d name=%s"
#define dht_msg_94 (GLFS_COMP_BASE + 94), "inodes on subvolume '%s' are at (%.2f %%), consider adding more nodes"
#define dht_msg_95 (GLFS_COMP_BASE + 95), "Invalid number format \"%s\""
#define dht_msg_96 (GLFS_COMP_BASE + 96), "Layout is NULL"
#define dht_msg_97 (GLFS_COMP_BASE + 97), "Layout merge from subvolume %s failed"
#define dht_msg_98 (GLFS_COMP_BASE + 98), "Linkfile found in revalidate for %s"
#define dht_msg_99 (GLFS_COMP_BASE + 99), "Linkfile not having link subvolume. path=%s"
#define dht_msg_100 (GLFS_COMP_BASE + 100), "Link/file %s on %s failed (%s)"
#define dht_msg_101 (GLFS_COMP_BASE + 101), "link of %s -> %s failed on subvol %s (%s)"
#define dht_msg_102 (GLFS_COMP_BASE + 102), "Linkto setxattr failed %s -> %s (%s)"
#define dht_msg_103 (GLFS_COMP_BASE + 103), "!local, should not happen"
#define dht_msg_104 (GLFS_COMP_BASE + 104), "loc->gfid is NULL for %s"
#define dht_msg_105 (GLFS_COMP_BASE + 105), "loc->pargfid is NULL for %s"
#define dht_msg_106 (GLFS_COMP_BASE + 106), "Lookup failed on %s"
#define dht_msg_107 (GLFS_COMP_BASE + 107), "Lookup of %s on %s (following linkfile) failed (%s)"
#define dht_msg_108 (GLFS_COMP_BASE + 108), "Lookup of %s on %s (following linkfile) reached dir"
#define dht_msg_109 (GLFS_COMP_BASE + 109), "Lookup of %s on %s (following linkfile) reached link"
#define dht_msg_110 (GLFS_COMP_BASE + 110), "Look up on / failed"
#define dht_msg_111 (GLFS_COMP_BASE + 111), "Memory accounting init failed"
#define dht_msg_112 (GLFS_COMP_BASE + 112), "Memory allocation failed"
#define dht_msg_113 (GLFS_COMP_BASE + 113), "migrate data called on %s"
#define dht_msg_114 (GLFS_COMP_BASE + 114), "migrate-data failed for %s"
#define dht_msg_115 (GLFS_COMP_BASE + 115), "migrate-data on %s failed: %s"
#define dht_msg_116 (GLFS_COMP_BASE + 116), "Migration of file:%s size:%"PRIu64" bytes took %.2f secs"
#define dht_msg_117 (GLFS_COMP_BASE + 117), "Migration operation on dir %s took %.2f secs"
#define dht_msg_118 (GLFS_COMP_BASE + 118), "Mismatching filetypes 0%o v/s 0%o for %s"
#define dht_msg_119 (GLFS_COMP_BASE + 119), "Mismatching layouts for %s"
#define dht_msg_120 (GLFS_COMP_BASE + 120), "Multiple subvolumes (%s and %s) have file %s (preferably rename the file in the backend, and do a fresh lookup)"
#define dht_msg_121 (GLFS_COMP_BASE + 121), "No cached subvolume for path=%s"
#define dht_msg_122 (GLFS_COMP_BASE + 122), "node-uuid not specified"
#define dht_msg_123 (GLFS_COMP_BASE + 123), "No nodes left for pattern '*'. Exiting"
#define dht_msg_124 (GLFS_COMP_BASE + 124), "No pre-set layout for subvolume %s"
#define dht_msg_125 (GLFS_COMP_BASE + 125), "No subvolume for hash (value) = %u"
#define dht_msg_126 (GLFS_COMP_BASE + 126), "No subvolume in layout for path=%s"
#define dht_msg_127 (GLFS_COMP_BASE + 127), "Not able to form layout for the directory"
#define dht_msg_128 (GLFS_COMP_BASE + 128), "One of the subvolumes down (%s)"
#define dht_msg_129 (GLFS_COMP_BASE + 129), "opendir on %s for %s failed (%s)"
#define dht_msg_130 (GLFS_COMP_BASE + 130), "Path %s exists as a file on one subvolume and directory on another."
#define dht_msg_131 (GLFS_COMP_BASE + 131), "'*' pattern will be taken by default for all the unconfigured child nodes hence neglecting current option"
#define dht_msg_132 (GLFS_COMP_BASE + 132), "Readdir returned %s. Aborting fix-layout"
#define dht_msg_133 (GLFS_COMP_BASE + 133), "Readdir returned %s. Aborting migrate-data"
#define dht_msg_134 (GLFS_COMP_BASE + 134), "Readdir returned %s. Aborting migrate-data"
#define dht_msg_135 (GLFS_COMP_BASE + 135), "Rebalance is %s. Time taken is %.2f secs"
#define dht_msg_136 (GLFS_COMP_BASE + 136), "Received CHILD_DOWN. Exiting"
#define dht_msg_137 (GLFS_COMP_BASE + 137), "Received stop command on rebalance"
#define dht_msg_138 (GLFS_COMP_BASE + 138), "Reconfigure: lookup-unhashed should be boolean, not (%s), defaulting to (%d)"
#define dht_msg_139 (GLFS_COMP_BASE + 139), "Rename %s -> %s on %s failed (%s)"
#define dht_msg_140 (GLFS_COMP_BASE + 140), "%s: attempting to move from %s to %s"
#define dht_msg_141 (GLFS_COMP_BASE + 141), "%s: creating symlink failed (%s)"
#define dht_msg_142 (GLFS_COMP_BASE + 142), "%s - disk layout has invalid count %d"
#define dht_msg_143 (GLFS_COMP_BASE + 143), "%s - disk layout missing"
#define dht_msg_144 (GLFS_COMP_BASE + 144), "selfhealing directory %s failed: %s"
#define dht_msg_145 (GLFS_COMP_BASE + 145), "setattr of uid/gid on %s :<gfid:%s> failed (%s)"
#define dht_msg_146 (GLFS_COMP_BASE + 146), "Setxattr failed for %s"
#define dht_msg_147 (GLFS_COMP_BASE + 147), "%s: failed to create a new synctask"
#define dht_msg_148 (GLFS_COMP_BASE + 148), "%s: failed to delete the linkfile (%s)"
#define dht_msg_149 (GLFS_COMP_BASE + 149), "%s: failed to do a stat on %s (%s)"
#define dht_msg_150 (GLFS_COMP_BASE + 150), "%s: failed to fsync on %s (%s)"
#define dht_msg_151 (GLFS_COMP_BASE + 151), "%s: failed to get "GF_XATTR_LINKINFO_KEY" key - %s"
#define dht_msg_152 (GLFS_COMP_BASE + 152), "%s: failed to get the destination node"
#define dht_msg_153 (GLFS_COMP_BASE + 153), "%s: failed to get the 'linkto' xattr from dict"
#define dht_msg_154 (GLFS_COMP_BASE + 154), "%s: failed to get the 'linkto' xattr %s"
#define dht_msg_155 (GLFS_COMP_BASE + 155), "%s: failed to get xattr from %s (%s)"
#define dht_msg_156 (GLFS_COMP_BASE + 156), "%s: failed to initialize linkfile data"
#define dht_msg_157 (GLFS_COMP_BASE + 157), "%s: failed to lookup file (%s)"
#define dht_msg_158 (GLFS_COMP_BASE + 158), "%s: failed to lookup the file on %s"
#define dht_msg_159 (GLFS_COMP_BASE + 159), "%s: failed to merge layouts"
#define dht_msg_160 (GLFS_COMP_BASE + 160), "%s: failed to migrate data"
#define dht_msg_161 (GLFS_COMP_BASE + 161), "%s: failed to migrate file with link"
#define dht_msg_162 (GLFS_COMP_BASE + 162), "%s: failed to perform removexattr on %s (%s)"
#define dht_msg_163 (GLFS_COMP_BASE + 163), "%s: failed to perform setattr on %s (%s)"
#define dht_msg_164 (GLFS_COMP_BASE + 164), "%s: failed to perform truncate on %s (%s)"
#define dht_msg_165 (GLFS_COMP_BASE + 165), "%s: failed to perform unlink on %s (%s)"
#define dht_msg_166 (GLFS_COMP_BASE + 166), "%s: failed to reset target size back to 0 (%s)"
#define dht_msg_167 (GLFS_COMP_BASE + 167), "%s: failed to set gfid"
#define dht_msg_168 (GLFS_COMP_BASE + 168), "%s: failed to set gfid in dict"
#define dht_msg_169 (GLFS_COMP_BASE + 169), "%s: failed to set gfid in dict for create"
#define dht_msg_170 (GLFS_COMP_BASE + 170), "%s: failed to set inode ctx"
#define dht_msg_171 (GLFS_COMP_BASE + 171), "%s: failed to set inode-ctx target file at %s"
#define dht_msg_172 (GLFS_COMP_BASE + 172), "%s: failed to set layout for subvol %s"
#define dht_msg_173 (GLFS_COMP_BASE + 173), "%s: failed to set 'linkto' key in dict"
#define dht_msg_174 (GLFS_COMP_BASE + 174), "%s: failed to set '%s' key"
#define dht_msg_175 (GLFS_COMP_BASE + 175), "%s: failed to set the new layout"
#define dht_msg_176 (GLFS_COMP_BASE + 176), "%s: failed to set xattr on %s (%s)"
#define dht_msg_177 (GLFS_COMP_BASE + 177), "%s: fd create failed (destination) (%s)"
#define dht_msg_178 (GLFS_COMP_BASE + 178), "%s: fd create failed (source)"
#define dht_msg_179 (GLFS_COMP_BASE + 179), "%s: file exists in destination"
#define dht_msg_180 (GLFS_COMP_BASE + 180), "%s: file has hardlinks"
#define dht_msg_181 (GLFS_COMP_BASE + 181), "%s found on cached subvol %s"
#define dht_msg_182 (GLFS_COMP_BASE + 182), "%s: gfid different on data file on %s"
#define dht_msg_183 (GLFS_COMP_BASE + 183), "%s: gfid different on %s"
#define dht_msg_184 (GLFS_COMP_BASE + 184), "%s: gfid different on the target file on %s"
#define dht_msg_185 (GLFS_COMP_BASE + 185), "%s: gfid differs on subvolume %s"
#define dht_msg_186 (GLFS_COMP_BASE + 186), "%s: gfid set failed"
#define dht_msg_187 (GLFS_COMP_BASE + 187), "%s: internal-fop set failed"
#define dht_msg_188 (GLFS_COMP_BASE + 188), "%s is not a subvolume of %s. pattern can only be scheduled only to a subvolume of %s"
#define dht_msg_189 (GLFS_COMP_BASE + 189), "%s lookup failed"
#define dht_msg_190 (GLFS_COMP_BASE + 190), "%s: lookup failed on %s (%s)"
#define dht_msg_191 (GLFS_COMP_BASE + 191), "%s: lookup failed (%s)"
#define dht_msg_192 (GLFS_COMP_BASE + 192), "%s: migrate-file called on directory"
#define dht_msg_193 (GLFS_COMP_BASE + 193), "%s: mknod failed (%s)"
#define dht_msg_194 (GLFS_COMP_BASE + 194), "%s: no pre-set layout for subvolume %s"
#define dht_msg_195 (GLFS_COMP_BASE + 195), "%s on %s found to be not a linkfile (type=0%o)"
#define dht_msg_196 (GLFS_COMP_BASE + 196), "sort failed?! how the ...."
#define dht_msg_197 (GLFS_COMP_BASE + 197), "%s: readlink on symlink failed (%s)"
#define dht_msg_198 (GLFS_COMP_BASE + 198), "%s: rename on %s failed (%s)"
#define dht_msg_199 (GLFS_COMP_BASE + 199), "%s/%s gfid not present"
#define dht_msg_200 (GLFS_COMP_BASE + 200), "%s: (subvol %s) failed to extract disk layout"
#define dht_msg_201 (GLFS_COMP_BASE + 201), "%s: (subvol %s) failed to set xattr dictionary"
#define dht_msg_202 (GLFS_COMP_BASE + 202), "subvol: %s; inode layout - %"PRIu32" - %"PRIu32"; disk layout - %"PRIu32" - %"PRIu32
#define dht_msg_203 (GLFS_COMP_BASE + 203), "Subvolume %s for %s returned -1 (%s)"
#define dht_msg_204 (GLFS_COMP_BASE + 204), "Subvolume %s returned -1 (%s)"
#define dht_msg_205 (GLFS_COMP_BASE + 205), "%s: unlink failed (%s)"
#define dht_msg_206 (GLFS_COMP_BASE + 206), "%s: unlink on %s failed (%s)"
#define dht_msg_207 (GLFS_COMP_BASE + 207), "%s - xattr dictionary is NULL"
#define dht_msg_208 (GLFS_COMP_BASE + 208), "Unable to find local subvolume, switching to dht mode"
#define dht_msg_209 (GLFS_COMP_BASE + 209), "Unknown local->xsel (%s)"
#define dht_msg_210 (GLFS_COMP_BASE + 210), "Unlinking linkfile %s on %s failed (%s)"
#define dht_msg_211 (GLFS_COMP_BASE + 211), "Unlinking %s on %s (given path %s)"
#define dht_msg_212 (GLFS_COMP_BASE + 212), "Using regex %s = %s"
#define dht_msg_213 (GLFS_COMP_BASE + 213), "Using specified subvol %s"
#define dht_msg_214 (GLFS_COMP_BASE + 214), "Using the first local subvol %s"
#define dht_msg_215 (GLFS_COMP_BASE + 215), "%u subvolume(s) are down. Skipping fix layout."
#define dht_msg_216 (GLFS_COMP_BASE + 216), "uuid_parse failed for %s",
#define dht_msg_217 (GLFS_COMP_BASE + 217), "Wrong 'directory-spread-count' value (%s)"
#define dht_msg_218 (GLFS_COMP_BASE + 218), "xattr dict set failed"
#define dht_msg_end_x GLFS_MSGID_END, "Invalid: End of messages"
/*Remove the following:*/
/**
100: Failed to set key value in dict
*/
#define DHT_SET_DICT_FAILED 100
#define DHT_SET_DICT_SUCCESS 101
gf_msg2("dht", GF_LOG_ERROR, DHT_SET_DICT_FAILED, "Failed to set xattr %s in dict ", xattr_name);
#endif
/*
Copyright (c) 2008-2012 Red Hat, Inc. <http://www.redhat.com>
This file is part of GlusterFS.
This file is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3 or
later), or the GNU General Public License, version 2 (GPLv2), in all
cases as published by the Free Software Foundation.
*/
#ifndef _CONFIG_H
#define _CONFIG_H
#include "config.h"
#endif
/* TODO: add NS locking */
#include "glusterfs.h"
#include "xlator.h"
#include "dht-common.h"
#include "defaults.h"
#include <sys/time.h>
int
dht_du_info_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
int op_ret, int op_errno, struct statvfs *statvfs,
dict_t *xdata)
{
dht_conf_t *conf = NULL;
call_frame_t *prev = NULL;
int this_call_cnt = 0;
int i = 0;
double percent = 0;
double percent_inodes = 0;
uint64_t bytes = 0;
conf = this->private;
prev = cookie;
if (op_ret == -1) {
gf_msg (this->name, GF_LOG_WARNING,
DHT_DISK_INFO_FAILED,
"Failed to get disk info from %s", prev->this->name);
goto out;
}
if (statvfs && statvfs->f_blocks) {
percent = (statvfs->f_bavail * 100) / statvfs->f_blocks;
bytes = (statvfs->f_bavail * statvfs->f_frsize);
}
if (statvfs && statvfs->f_files) {
percent_inodes = (statvfs->f_ffree * 100) / statvfs->f_files;
} else {
/* set percent inodes to 100 for dynamically allocated inode filesystems
this logic holds good so that, distribute has nothing to worry about
total inodes rather let the 'create()' to be scheduled on the hashed
subvol regardless of the total inodes. since we have no awareness on
loosing inodes this logic fits well
*/
percent_inodes = 100;
}
LOCK (&conf->subvolume_lock);
{
for (i = 0; i < conf->subvolume_cnt; i++)
if (prev->this == conf->subvolumes[i]) {
conf->du_stats[i].avail_percent = percent;
conf->du_stats[i].avail_space = bytes;
conf->du_stats[i].avail_inodes = percent_inodes;
gf_log (this->name, GF_LOG_DEBUG,
"on subvolume '%s': avail_percent is: "
"%.2f and avail_space is: %"PRIu64" "
"and avail_inodes is: %.2f",
prev->this->name,
conf->du_stats[i].avail_percent,
conf->du_stats[i].avail_space,
conf->du_stats[i].avail_inodes);
}
}
UNLOCK (&conf->subvolume_lock);
out:
this_call_cnt = dht_frame_return (frame);
if (is_last_call (this_call_cnt))
DHT_STACK_DESTROY (frame);
return 0;
}
int
dht_get_du_info_for_subvol (xlator_t *this, int subvol_idx)
{
dht_conf_t *conf = NULL;
call_frame_t *statfs_frame = NULL;
dht_local_t *statfs_local = NULL;
call_pool_t *pool = NULL;
loc_t tmp_loc = {0,};
conf = this->private;
pool = this->ctx->pool;
statfs_frame = create_frame (this, pool);
if (!statfs_frame) {
goto err;
}
/* local->fop value is not used in this case */
statfs_local = dht_local_init (statfs_frame, NULL, NULL,
GF_FOP_MAXVALUE);
if (!statfs_local) {
goto err;
}
/* make it root gfid, should be enough to get the proper info back */
tmp_loc.gfid[15] = 1;
statfs_local->call_cnt = 1;
STACK_WIND (statfs_frame, dht_du_info_cbk,
conf->subvolumes[subvol_idx],
conf->subvolumes[subvol_idx]->fops->statfs,
&tmp_loc, NULL);
return 0;
err:
if (statfs_frame)
DHT_STACK_DESTROY (statfs_frame);
return -1;
}
int
dht_get_du_info (call_frame_t *frame, xlator_t *this, loc_t *loc)
{
int i = 0;
dht_conf_t *conf = NULL;
call_frame_t *statfs_frame = NULL;
dht_local_t *statfs_local = NULL;
struct timeval tv = {0,};
loc_t tmp_loc = {0,};
conf = this->private;
gettimeofday (&tv, NULL);
/* make it root gfid, should be enough to get the proper
info back */
tmp_loc.gfid[15] = 1;
if (tv.tv_sec > (conf->refresh_interval
+ conf->last_stat_fetch.tv_sec)) {
statfs_frame = copy_frame (frame);
if (!statfs_frame) {
goto err;
}
/* In this case, 'local->fop' is not used */
statfs_local = dht_local_init (statfs_frame, loc, NULL,
GF_FOP_MAXVALUE);
if (!statfs_local) {
goto err;
}
statfs_local->call_cnt = conf->subvolume_cnt;
for (i = 0; i < conf->subvolume_cnt; i++) {
STACK_WIND (statfs_frame, dht_du_info_cbk,
conf->subvolumes[i],
conf->subvolumes[i]->fops->statfs,
&tmp_loc, NULL);
}
conf->last_stat_fetch.tv_sec = tv.tv_sec;
}
return 0;
err:
if (statfs_frame)
DHT_STACK_DESTROY (statfs_frame);
return -1;
}
gf_boolean_t
dht_is_subvol_filled (xlator_t *this, xlator_t *subvol)
{
int i = 0;
dht_conf_t *conf = NULL;
gf_boolean_t subvol_filled_inodes = _gf_false;
gf_boolean_t subvol_filled_space = _gf_false;
gf_boolean_t is_subvol_filled = _gf_false;
conf = this->private;
/* Check for values above specified percent or free disk */
LOCK (&conf->subvolume_lock);
{
for (i = 0; i < conf->subvolume_cnt; i++) {
if (subvol == conf->subvolumes[i]) {
if (conf->disk_unit == 'p') {
if (conf->du_stats[i].avail_percent <
conf->min_free_disk) {
subvol_filled_space = _gf_true;
break;
}
} else {
if (conf->du_stats[i].avail_space <
conf->min_free_disk) {
subvol_filled_space = _gf_true;
break;
}
}
if (conf->du_stats[i].avail_inodes <
conf->min_free_inodes) {
subvol_filled_inodes = _gf_true;
break;
}
}
}
}
UNLOCK (&conf->subvolume_lock);
if (subvol_filled_space && conf->subvolume_status[i]) {
if (!(conf->du_stats[i].log++ % (GF_UNIVERSAL_ANSWER * 10))) {
gf_msg (this->name, GF_LOG_WARNING, DHT_SUBVOL_OUT_OF_SPACE,
"Disk space on subvolume '%s' is getting "
"full (%.2f %%), consider adding more nodes",
subvol->name,
(100 - conf->du_stats[i].avail_percent));
}
}
if (subvol_filled_inodes && conf->subvolume_status[i]) {
if (!(conf->du_stats[i].log++ % (GF_UNIVERSAL_ANSWER * 10))) {
gf_msg (this->name, GF_LOG_CRITICAL, DHT_SUBVOL_OUT_OF_INODES,
"inodes on subvolume '%s' are at "
"(%.2f %%), consider adding more nodes",
subvol->name,
(100 - conf->du_stats[i].avail_inodes));
}
}
is_subvol_filled = (subvol_filled_space || subvol_filled_inodes);
return is_subvol_filled;
}
/*Get the best subvolume to create the file in*/
xlator_t *
dht_free_disk_available_subvol (xlator_t *this, xlator_t *subvol,
dht_local_t *local)
{
xlator_t *avail_subvol = NULL;
dht_conf_t *conf = NULL;
dht_layout_t *layout = NULL;
loc_t *loc = NULL;
conf = this->private;
if (!local)
goto out;
loc = &local->loc;
if (!local->layout) {
layout = dht_layout_get (this, loc->parent);
if (!layout) {
gf_log (this->name, GF_LOG_DEBUG,
"layout missing path=%s parent=%s",
loc->path, uuid_utoa (loc->parent->gfid));
goto out;
}
} else {
layout = dht_layout_ref (this, local->layout);
}
LOCK (&conf->subvolume_lock);
{
avail_subvol = dht_subvol_with_free_space_inodes(this, subvol,
layout);
if(!avail_subvol)
{
avail_subvol = dht_subvol_maxspace_nonzeroinode(this,
subvol,
layout);
}
}
UNLOCK (&conf->subvolume_lock);
out:
if (!avail_subvol) {
gf_log (this->name,
GF_LOG_DEBUG,
"no subvolume has enough free space and/or inodes\
to create");
avail_subvol = subvol;
}
if (layout)
dht_layout_unref (this, layout);
return avail_subvol;
}
static inline
int32_t dht_subvol_has_err (xlator_t *this, dht_layout_t *layout)
{
int ret = -1;
int i = 0;
if (!this || !layout)
goto out;
/* check if subvol has layout errors, before selecting it */
for (i = 0; i < layout->cnt; i++) {
if (!strcmp (layout->list[i].xlator->name, this->name) &&
(layout->list[i].err != 0)) {
ret = -1;
goto out;
}
}
ret = 0;
out:
return ret;
}
/*Get subvolume which has both space and inodes more than the min criteria*/
xlator_t *
dht_subvol_with_free_space_inodes(xlator_t *this, xlator_t *subvol,
dht_layout_t *layout)
{
int i = 0;
double max = 0;
double max_inodes = 0;
int ignore_subvol = 0;
xlator_t *avail_subvol = NULL;
dht_conf_t *conf = NULL;
conf = this->private;
for(i=0; i < conf->subvolume_cnt; i++) {
/* check if subvol has layout errors, before selecting it */
ignore_subvol = dht_subvol_has_err (conf->subvolumes[i],
layout);
if (ignore_subvol)
continue;
if ((conf->disk_unit == 'p') &&
(conf->du_stats[i].avail_percent > conf->min_free_disk) &&
(conf->du_stats[i].avail_inodes > conf->min_free_inodes)) {
if ((conf->du_stats[i].avail_inodes > max_inodes) ||
(conf->du_stats[i].avail_percent > max)) {
max = conf->du_stats[i].avail_percent;
max_inodes = conf->du_stats[i].avail_inodes;
avail_subvol = conf->subvolumes[i];
}
}
if ((conf->disk_unit != 'p') &&
(conf->du_stats[i].avail_space > conf->min_free_disk) &&
(conf->du_stats[i].avail_inodes > conf->min_free_inodes)) {
if ((conf->du_stats[i].avail_inodes > max_inodes) ||
(conf->du_stats[i].avail_space > max)) {
max = conf->du_stats[i].avail_space;
max_inodes = conf->du_stats[i].avail_inodes;
avail_subvol = conf->subvolumes[i];
}
}
}
return avail_subvol;
}
/* Get subvol which has atleast one inode and maximum space */
xlator_t *
dht_subvol_maxspace_nonzeroinode (xlator_t *this, xlator_t *subvol,
dht_layout_t *layout)
{
int i = 0;
double max = 0;
int ignore_subvol = 0;
xlator_t *avail_subvol = NULL;
dht_conf_t *conf = NULL;
conf = this->private;
for (i = 0; i < conf->subvolume_cnt; i++) {
/* check if subvol has layout errors, before selecting it */
ignore_subvol = dht_subvol_has_err (conf->subvolumes[i],
layout);
if (ignore_subvol)
continue;
if (conf->disk_unit == 'p') {
if ((conf->du_stats[i].avail_percent > max)
&& (conf->du_stats[i].avail_inodes > 0 )) {
max = conf->du_stats[i].avail_percent;
avail_subvol = conf->subvolumes[i];
}
} else {
if ((conf->du_stats[i].avail_space > max)
&& (conf->du_stats[i].avail_inodes > 0)) {
max = conf->du_stats[i].avail_space;
avail_subvol = conf->subvolumes[i];
}
}
}
return avail_subvol;
}
/*
Copyright (c) 2013 Red Hat, Inc. <http://www.redhat.com>
This file is part of GlusterFS.
This file is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3 or
later), or the GNU General Public License, version 2 (GPLv2), in all
cases as published by the Free Software Foundation.
*/
#ifndef _GLUSTERFSD_MESSAGES_H_
#define _GLUSTERFSD_MESSAGES_H_
#ifndef _CONFIG_H
#define _CONFIG_H
#include "config.h"
#endif
#include "glfs-message-id.h"
/* NOTE: Rules for message additions
* 1) Each instance of a message is _better_ left with a unique message ID, even
* if the message format is the same. Reasoning is that, if the message
* format needs to change in one instance, the other instances are not
* impacted or the new change does not change the ID of the instance being
* modified.
* 2) Addition of a message,
* - Should increment the GLFS_NUM_MESSAGES
* - Append to the list of messages defined, towards the end
* - Retain macro naming as glfs_msg_X (for redability across developers)
* NOTE: Rules for message format modifications
* 3) Check acorss the code if the message ID macro in question is reused
* anywhere. If reused then then the modifications should ensure correctness
* everywhere, or needs a new message ID as (1) above was not adhered to. If
* not used anywhere, proceed with the required modification.
* NOTE: Rules for message deletion
* 4) Check (3) and if used anywhere else, then cannot be deleted. If not used
* anywhere, then can be deleted, but will leave a hole by design, as
* addition rules specify modification to the end of the list and not filling
* holes.
*/
#define GLFS_COMP_BASE GLFS_MSGID_COMP_DHT
#define GLFS_NUM_MESSAGES 124
#define GLFS_MSGID_END (GLFS_COMP_BASE + GLFS_NUM_MESSAGES + 1)
/* Messages with message IDs */
#define DHT_ALLOC_FAILED (GLFS_COMP_BASE + 1)
#define DHT_ATTR_SET_FAILED (GLFS_COMP_BASE + 2)
#define DHT_CHECK_SCHED (GLFS_COMP_BASE + 3)
#define DHT_CHILD_LOC_BUILD_FAILED (GLFS_COMP_BASE + 4)
#define DHT_CHOWN_FAILED (GLFS_COMP_BASE + 5)
#define DHT_COMPILE_REGEX_FAILED (GLFS_COMP_BASE + 6)
#define DHT_CREATE (GLFS_COMP_BASE + 7)
#define DHT_CREATE_SYMLNK_FAILED (GLFS_COMP_BASE + 8)
#define DHT_DANGLING_VOLUME (GLFS_COMP_BASE + 9)
#define DHT_DATA_TO_BIN_FAILED (GLFS_COMP_BASE + 10)
#define DHT_DEL_LNKFILE_FAILED (GLFS_COMP_BASE + 11)
#define DHT_DICT_ADD_FAILED (GLFS_COMP_BASE + 12)
#define DHT_DICT_GET_FAILED (GLFS_COMP_BASE + 13)
#define DHT_DICT_GET_NODE_UUID_FAILED (GLFS_COMP_BASE + 14)
#define DHT_DICT_NEW_FAILED (GLFS_COMP_BASE + 15)
#define DHT_DICT_NULL (GLFS_COMP_BASE + 16)
#define DHT_DICT_SET_FAILED (GLFS_COMP_BASE + 17)
#define DHT_DICT_SET_STR_FAILED (GLFS_COMP_BASE + 18)
#define DHT_DICT_SET_VAL_FAILED (GLFS_COMP_BASE + 19)
#define DHT_DISK_INFO_FAILED (GLFS_COMP_BASE + 20)
#define DHT_DISK_LAYOUT_EXTRACT_FAILED (GLFS_COMP_BASE + 21)
#define DHT_DISTRIBUTE_SINGLE_SUBVOL (GLFS_COMP_BASE + 22)
#define DHT_FD_CREATE_FAILED (GLFS_COMP_BASE + 23)
#define DHT_FD_OPEN_FAILED (GLFS_COMP_BASE + 24)
#define DHT_FD_SEND_OPEN_FAILED (GLFS_COMP_BASE + 25)
#define DHT_FILE_CREATE_FAILED (GLFS_COMP_BASE + 26)
#define DHT_FILE_EXISTS (GLFS_COMP_BASE + 27)
#define DHT_FILE_GFID_MISMATCH (GLFS_COMP_BASE + 28)
#define DHT_FILE_HAS_HARDLINKS (GLFS_COMP_BASE + 29)
#define DHT_FILE_OPEN_FAILED (GLFS_COMP_BASE + 30)
#define DHT_FILETYPE_MISMATCH (GLFS_COMP_BASE + 31)
#define DHT_FSTAT_FAILED (GLFS_COMP_BASE + 32)
#define DHT_FSYNC_FAILED (GLFS_COMP_BASE + 33)
#define DHT_FTRUNCATE_FAILED (GLFS_COMP_BASE + 34)
#define DHT_GET_STATFS_FAILED (GLFS_COMP_BASE + 35)
#define DHT_GFID_DIFF (GLFS_COMP_BASE + 36)
#define DHT_GFID_MISSING (GLFS_COMP_BASE + 37)
#define DHT_GFID_NULL (GLFS_COMP_BASE + 38)
#define DHT_GFID_SET_FAILED (GLFS_COMP_BASE + 39)
#define DHT_HASH_COMPUTE_FAILED (GLFS_COMP_BASE + 40)
#define DHT_HASHED_SUBVOL_NOT_FOUND (GLFS_COMP_BASE + 41)
#define DHT_INODE_CTX_SET_FAILED (GLFS_COMP_BASE + 42)
#define DHT_INODE_LINK_LAYOUT_FAILED (GLFS_COMP_BASE + 43)
#define DHT_INT_FOP_SET_FAILED (GLFS_COMP_BASE + 44)
#define DHT_INVALID_DIR_SPREAD_COUNT (GLFS_COMP_BASE + 45)
#define DHT_INVALID_NUMBER_FORMAT (GLFS_COMP_BASE + 46)
#define DHT_LAYOUT_ANOMALIES_FAILED (GLFS_COMP_BASE + 47)
#define DHT_LAYOUT_ANOMALIES_INFO (GLFS_COMP_BASE + 48)
#define DHT_LAYOUT_CREATE_FAILED (GLFS_COMP_BASE + 49)
#define DHT_LAYOUT_FIX_FAILED (GLFS_COMP_BASE + 50)
#define DHT_LAYOUT_FIXING (GLFS_COMP_BASE + 51)
#define DHT_LAYOUT_INVALID_COUNT (GLFS_COMP_BASE + 52)
#define DHT_LAYOUT_MERGE_FAILED (GLFS_COMP_BASE + 53)
#define DHT_LAYOUT_MISMATCH (GLFS_COMP_BASE + 54)
#define DHT_LAYOUT_NO_PRESET (GLFS_COMP_BASE + 55)
#define DHT_LAYOUT_NOT_FOUND (GLFS_COMP_BASE + 56)
#define DHT_LAYOUT_NULL (GLFS_COMP_BASE + 57)
#define DHT_LAYOUT_PRESET_FAILED (GLFS_COMP_BASE + 58)
#define DHT_LAYOUT_SET_FAILED (GLFS_COMP_BASE + 59)
#define DHT_LAYOUT_SORT_FAILED (GLFS_COMP_BASE + 60)
#define DHT_LISTENER_SOCKET_UNLINK_FAILED (GLFS_COMP_BASE + 61)
#define DHT_LNK_FOUND_REVALIDATE (GLFS_COMP_BASE + 62)
#define DHT_LNK_NO (GLFS_COMP_BASE + 63)
#define DHT_LOOKUP_FAILED (GLFS_COMP_BASE + 64)
#define DHT_MEM_ACCOUNT_INIT_FAILED (GLFS_COMP_BASE + 65)
#define DHT_MEM_ALLOC_FAILED (GLFS_COMP_BASE + 66)
#define DHT_MERGE_NO_LAYOUT (GLFS_COMP_BASE + 67)
#define DHT_MIGRATE_DATA_CALLED (GLFS_COMP_BASE + 68)
#define DHT_MIGRATE_DATA_FAILED (GLFS_COMP_BASE + 69)
#define DHT_MIGRATE_FAILED (GLFS_COMP_BASE + 70)
#define DHT_MIGRATE_FILE_ON_DIR (GLFS_COMP_BASE + 71)
#define DHT_MIGRATE_FILES_STATS (GLFS_COMP_BASE + 72)
#define DHT_MIGRATE_TARGET_INSUFF_SPACE (GLFS_COMP_BASE + 73)
#define DHT_MIGRATE_TARGET_LESS_SPACE (GLFS_COMP_BASE + 74)
#define DHT_MIGRATION_COMPLETE (GLFS_COMP_BASE + 75)
#define DHT_MIGRATION_DIR_INFO (GLFS_COMP_BASE + 76)
#define DHT_MIGRATION_FILE_INFO (GLFS_COMP_BASE + 77)
#define DHT_MIGRATION_HARDLNK (GLFS_COMP_BASE + 78)
#define DHT_MKNOD_FAILED (GLFS_COMP_BASE + 79)
#define DHT_MOVE_ATTEMPT (GLFS_COMP_BASE + 80)
#define DHT_NO_HOSTNAME (GLFS_COMP_BASE + 81)
#define DHT_NO_LOCAL_BRICK (GLFS_COMP_BASE + 82)
#define DHT_NO_SUBVOL_FOR_HASH (GLFS_COMP_BASE + 83)
#define DHT_NO_SUBVOL_IN_LAYOUT (GLFS_COMP_BASE + 84)
#define DHT_NOT_LNKFILE (GLFS_COMP_BASE + 85)
#define DHT_NOT_SUBVOL (GLFS_COMP_BASE + 86)
#define DHT_NUFA_NO_LOCAL_SUBVOL (GLFS_COMP_BASE + 87)
#define DHT_OPENDIR_FAILED (GLFS_COMP_BASE + 88)
#define DHT_PARGFID_NULL (GLFS_COMP_BASE + 89)
#define DHT_PARSE_FAILED (GLFS_COMP_BASE + 90)
#define DHT_PARSE_UUID_FAILED (GLFS_COMP_BASE + 91)
#define DHT_POOL_CREATE_FAILED (GLFS_COMP_BASE + 92)
#define DHT_READDIR_FAILED (GLFS_COMP_BASE + 93)
#define DHT_READLINK_FAILED (GLFS_COMP_BASE + 94)
#define DHT_REBALANCE_STATUS (GLFS_COMP_BASE + 95)
#define DHT_REBALANCE_STOP_RECV (GLFS_COMP_BASE + 96)
#define DHT_RENAME_FAILED (GLFS_COMP_BASE + 97)
#define DHT_SELFHEAL_DIR_FAILED (GLFS_COMP_BASE + 98)
#define DHT_STALE_LINKFILE (GLFS_COMP_BASE + 99)
#define DHT_STAT_FAILED (GLFS_COMP_BASE + 100)
#define DHT_SUBVOL_DECOMMISSION (GLFS_COMP_BASE + 101)
#define DHT_SUBVOL_DOWN (GLFS_COMP_BASE + 102)
#define DHT_SUBVOL_FILE_CONFLICT (GLFS_COMP_BASE + 103)
#define DHT_SUBVOL_FILE_DIR_CONFLICT (GLFS_COMP_BASE + 104)
#define DHT_SUBVOL_GET_CACHED_FAILED (GLFS_COMP_BASE + 105)
#define DHT_SUBVOL_GET_HASHED_FAILED (GLFS_COMP_BASE + 106)
#define DHT_SUBVOL_GET_LINK_FAILED (GLFS_COMP_BASE + 107)
#define DHT_SUBVOL_LOCAL_FOUND (GLFS_COMP_BASE + 108)
#define DHT_SUBVOL_NO_CACHED (GLFS_COMP_BASE + 109)
#define DHT_SUBVOL_OUT_OF_INODES (GLFS_COMP_BASE + 110)
#define DHT_SUBVOL_OUT_OF_SPACE (GLFS_COMP_BASE + 111)
#define DHT_SUBVOL_SET_LAYOUT_FAILED (GLFS_COMP_BASE + 112)
#define DHT_SUBVOL_UNRECOVERABLE_ERROR (GLFS_COMP_BASE + 113)
#define DHT_SYNCTASK_CREATE_FAILED (GLFS_COMP_BASE + 114)
#define DHT_TASK_CREATE_FAILED (GLFS_COMP_BASE + 115)
#define DHT_TRUNCATE_FAILED (GLFS_COMP_BASE + 116)
#define DHT_UNKNOWN_LAYOUT_TYPE (GLFS_COMP_BASE + 117)
#define DHT_UNLINK_FAILED (GLFS_COMP_BASE + 118)
#define DHT_UNLINKING (GLFS_COMP_BASE + 119)
#define DHT_WRITE_FAILED (GLFS_COMP_BASE + 120)
#define DHT_XATTR_GET_FAILED (GLFS_COMP_BASE + 121)
#define DHT_XATTR_GET_NODE_UUID_FAILED (GLFS_COMP_BASE + 122)
#define DHT_XATTR_REM_FAILED (GLFS_COMP_BASE + 123)
#define DHT_XATTR_SET_FAILED (GLFS_COMP_BASE + 124)
#define glfs_msg_end_x GLFS_MSGID_END, "Invalid: End of messages"
#endif /* !_GLUSTERFSD_MESSAGES_H_ */
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
