Re: tiering: emergency demotions

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Patch for review: http://review.gluster.org/15158

Milind

On 08/12/2016 07:27 PM, Milind Changire wrote:
On 08/10/2016 12:06 PM, Milind Changire wrote:
Emergency demotions will be required whenever writes breach the
hi-watermark. Emergency demotions are required to avoid ENOSPC in case
of continuous writes that originate on the hot tier.

There are two concerns in this area:

1. enforcing max-cycle-time during emergency demotions
   max-cycle-time is the time the tiering daemon spends in promotions or
   demotions
   I tend to think that the tiering daemon skip this check for the
   emergency situation and continue demotions until the watermark drops
   below the hi-watermark

Update:
To keep matters simple and manageable, it has been decided to *enforce*
max-cycle-time to yield the worker threads to attend to impending tier
management tasks if the need arises.


2. file demotion policy
   I tend to think that evicting the largest file with the most recent
   *write* should be chosen for eviction when write-freq-threshold is
   NON-ZERO.
   Choosing a least written file is just going to delay file migration
   of an active file which might consume hot tier disk space resulting
   in a ENOSPC, in the worst case.
   In cases where write-freq-threshold are ZERO, the most recently
   *written* file can be chosen for eviction.
   In the case of choosing the largest file within the
   write-freq-threshold, a stat() on the files would be required to
   calculate the number of files that need to be demoted to take the
   watermark below the hi-watermark. Finding the number of most recently
   written files to demote could also help make demotions in parallel
   rather than in the sequential manner currently in place.

Update:
The idea of choosing the files wrt file size has been dropped.
Iteratively, the most recently written file will be chosen for eviction
from the hot tier in case of a hi-watermark breach and until the
watermark drops below hi-watermark.
The idea of parallelizing multiple promotions/demotions has been
deferred.

-----

Sustained writes creating larges files in the hot tier which
cumulatively breach the hi-watermark does NOT seem to be a good
workload for making use of tiering. The assumption is that, to make the
most of of the hot tier, the hi-watermark would be closer to 100.
In this case a sustained large file copy might easily breach the
hi-watermark and may even consume the entire hot tier space, resulting
in a ENOSPC.

eg. an example of a sustained write

# cp file1 /mnt/glustervol/dir

Workloads that would seem to make the most of tiering are:
1. Many smaller files, which are created in small bursts of write
   activity and then closed
2. Few large files where updates are in-place and the file size
   does not grow beyond the hi-watermark eg. database, with frequent
   in-line compaction/de-fragmentation policy enabled
3. Frequent reads of few large files, mostly static in size, which
   cumulatively don't breach the hi-watermark. Frequently reading
   a large number of smaller, mostly static, files would be good
   tiering workload candidates as well.



Comments are requested.

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel



[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux