On Fri, 1 Sep 2017, Felix, Evan J wrote: > Is there documentation about how to deal with a pool application > association that is not one of cephfs, rbd, or rgw? We have multiple > pools that have nothing to do with those applications, we just use the > objects in them directly using the librados API calls. I don’t really > want health warnings always showing in my status screens. Hi Evan, Just ceph osd pool application enable $pool myapp See http://docs.ceph.com/docs/master/rados/operations/pools/#associate-pool-to-application sage > > Evan Felix > > -----Original Message----- > From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Abhishek Lekshmanan > Sent: Tuesday, August 29, 2017 11:20 AM > To: ceph-devel@xxxxxxxxxxxxxxx; ceph-users@xxxxxxxx; ceph-maintainers@xxxxxxxx; ceph-announce@xxxxxxxx > Subject: v12.2.0 Luminous released > > > We're glad to announce the first release of Luminous v12.2.x long term stable release series. There have been major changes since Kraken > (v11.2.z) and Jewel (v10.2.z), and the upgrade process is non-trivial. > Please read the release notes carefully. > > For more details, links & changelog please refer to the complete release notes entry at the Ceph blog: > http://ceph.com/releases/v12-2-0-luminous-released/ > > > Major Changes from Kraken > ------------------------- > > - *General*: > * Ceph now has a simple, built-in web-based dashboard for monitoring cluster > status. > > - *RADOS*: > * *BlueStore*: > - The new *BlueStore* backend for *ceph-osd* is now stable and the > new default for newly created OSDs. BlueStore manages data > stored by each OSD by directly managing the physical HDDs or > SSDs without the use of an intervening file system like XFS. > This provides greater performance and features. > - BlueStore supports full data and metadata checksums > of all data stored by Ceph. > - BlueStore supports inline compression using zlib, snappy, or LZ4. (Ceph > also supports zstd for RGW compression but zstd is not recommended for > BlueStore for performance reasons.) > > * *Erasure coded* pools now have full support for overwrites > allowing them to be used with RBD and CephFS. > > * *ceph-mgr*: > - There is a new daemon, *ceph-mgr*, which is a required part of > any Ceph deployment. Although IO can continue when *ceph-mgr* > is down, metrics will not refresh and some metrics-related calls > (e.g., `ceph df`) may block. We recommend deploying several > instances of *ceph-mgr* for reliability. See the notes on > Upgrading below. > - The *ceph-mgr* daemon includes a REST-based management API. > The API is still experimental and somewhat limited but > will form the basis for API-based management of Ceph going forward. > - ceph-mgr also includes a Prometheus exporter plugin, which can provide Ceph > perfcounters to Prometheus. > - ceph-mgr now has a Zabbix plugin. Using zabbix_sender it sends trapper > events to a Zabbix server containing high-level information of the Ceph > cluster. This makes it easy to monitor a Ceph cluster's status and send > out notifications in case of a malfunction. > > * The overall *scalability* of the cluster has improved. We have > successfully tested clusters with up to 10,000 OSDs. > * Each OSD can now have a device class associated with > it (e.g., `hdd` or `ssd`), allowing CRUSH rules to trivially map > data to a subset of devices in the system. Manually writing CRUSH > rules or manual editing of the CRUSH is normally not required. > * There is a new upmap exception mechanism that allows individual PGs to be moved around to achieve > a *perfect distribution* (this requires luminous clients). > * Each OSD now adjusts its default configuration based on whether the > backing device is an HDD or SSD. Manual tuning generally not required. > * The prototype mClock QoS queueing algorithm is now available. > * There is now a *backoff* mechanism that prevents OSDs from being > overloaded by requests to objects or PGs that are not currently able to > process IO. > * There is a simplified OSD replacement process that is more robust. > * You can query the supported features and (apparent) releases of > all connected daemons and clients with `ceph features` > * You can configure the oldest Ceph client version you wish to allow to > connect to the cluster via `ceph osd set-require-min-compat-client` and > Ceph will prevent you from enabling features that will break compatibility > with those clients. > * Several `sleep` settings, include `osd_recovery_sleep`, > `osd_snap_trim_sleep`, and `osd_scrub_sleep` have been > reimplemented to work efficiently. (These are used in some cases > to work around issues throttling background work.) > * Pools are now expected to be associated with the application using them. > Upon completing the upgrade to Luminous, the cluster will attempt to associate > existing pools to known applications (i.e. CephFS, RBD, and RGW). In-use pools > that are not associated to an application will generate a health warning. Any > unassociated pools can be manually associated using the new > `ceph osd pool application enable` command. For more details see > `associate pool to application` in the documentation. > > - *RGW*: > > * RGW *metadata search* backed by ElasticSearch now supports end > user requests service via RGW itself, and also supports custom > metadata fields. A query language a set of RESTful APIs were > created for users to be able to search objects by their > metadata. New APIs that allow control of custom metadata fields > were also added. > * RGW now supports *dynamic bucket index sharding*. This has to be enabled via > the `rgw dyamic resharding` configurable. As the number of objects in a > bucket grows, RGW will automatically reshard the bucket index in response. > No user intervention or bucket size capacity planning is required. > * RGW introduces *server side encryption* of uploaded objects with > three options for the management of encryption keys: automatic > encryption (only recommended for test setups), customer provided > keys similar to Amazon SSE-C specification, and through the use of > an external key management service (Openstack Barbican) similar > to Amazon SSE-KMS specification. > * RGW now has preliminary AWS-like bucket policy API support. For > now, policy is a means to express a range of new authorization > concepts. In the future it will be the foundation for additional > auth capabilities such as STS and group policy. > * RGW has consolidated the several metadata index pools via the use of rados > namespaces. > * S3 Object Tagging API has been added; while APIs are > supported for GET/PUT/DELETE object tags and in PUT object > API, there is no support for tags on Policies & Lifecycle yet > * RGW multisite now supports for enabling or disabling sync at a > bucket level. > > - *RBD*: > > * RBD now has full, stable support for *erasure coded pools* via the new > `--data-pool` option to `rbd create`. > * RBD mirroring's rbd-mirror daemon is now highly available. We > recommend deploying several instances of rbd-mirror for > reliability. > * RBD mirroring's rbd-mirror daemon should utilize unique Ceph user > IDs per instance to support the new mirroring dashboard. > * The default 'rbd' pool is no longer created automatically during > cluster creation. Additionally, the name of the default pool used > by the rbd CLI when no pool is specified can be overridden via a > new `rbd default pool = <pool name>` configuration option. > * Initial support for deferred image deletion via new `rbd > trash` CLI commands. Images, even ones actively in-use by > clones, can be moved to the trash and deleted at a later time. > * New pool-level `rbd mirror pool promote` and `rbd mirror pool > demote` commands to batch promote/demote all mirrored images > within a pool. > * Mirroring now optionally supports a configurable replication delay > via the `rbd mirroring replay delay = <seconds>` configuration > option. > * Improved discard handling when the object map feature is enabled. > * rbd CLI `import` and `copy` commands now detect sparse and > preserve sparse regions. > * Images and Snapshots will now include a creation timestamp. > * Specifying user authorization capabilities for RBD clients has been > simplified. The general syntax for using RBD capability profiles is > "mon 'profile rbd' osd 'profile rbd[-read-only][ pool={pool-name}[, ...]]'". > For more details see "User Management" in the documentation. > > - *CephFS*: > > * *Multiple active MDS daemons* is now considered stable. The number > of active MDS servers may be adjusted up or down on an active CephFS file > system. > * CephFS *directory fragmentation* is now stable and enabled by > default on new filesystems. To enable it on existing filesystems > use "ceph fs set <fs_name> allow_dirfrags". Large or very busy > directories are sharded and (potentially) distributed across > multiple MDS daemons automatically. > * Directory subtrees can be explicitly pinned to specific MDS daemons in > cases where the automatic load balancing is not desired or effective. > * Client keys can now be created using the new `ceph fs authorize` command > to create keys with access to the given CephFS file system and all of its > data pools. > * When running 'df' on a CephFS filesystem comprising exactly one data pool, > the result now reflects the file storage space used and available in that > data pool (fuse client only). > > - *Miscellaneous*: > > * Release packages are now being built for *Debian Stretch*. Note > that QA is limited to CentOS and Ubuntu (xenial and trusty). The > distributions we build for now include: > > - CentOS 7 (x86_64 and aarch64) > - Debian 8 Jessie (x86_64) > - Debian 9 Stretch (x86_64) > - Ubuntu 16.04 Xenial (x86_64 and aarch64) > - Ubuntu 14.04 Trusty (x86_64) > > * A first release of Ceph for FreeBSD is available which contains a full set > of features, other than Bluestore. It will run everything needed to build a > storage cluster. For clients, all access methods are available, albeit > CephFS is only accessible through a Fuse implementation. RBD images can be > mounted on FreeBSD systems through rbd-ggate > Ceph versions are released through the regular FreeBSD ports and packages > system. The most current version is available as: net/ceph-devel. Once > Luminous goes into official release, this version will be available as > net/ceph. Future development releases will be available via net/ceph-devel > > * *CLI changes*: > > - The `ceph -s` or `ceph status` command has a fresh look. > - `ceph mgr metadata` will dump metadata associated with each mgr > daemon. > - `ceph versions` or `ceph {osd,mds,mon,mgr} versions` > summarize versions of running daemons. > - `ceph {osd,mds,mon,mgr} count-metadata <property>` similarly > tabulates any other daemon metadata visible via the `ceph > {osd,mds,mon,mgr} metadata` commands. > - `ceph features` summarizes features and releases of connected > clients and daemons. > - `ceph osd require-osd-release <release>` replaces the old > `require_RELEASE_osds` flags. > - `ceph osd pg-upmap`, `ceph osd rm-pg-upmap`, `ceph osd > pg-upmap-items`, `ceph osd rm-pg-upmap-items` can explicitly > manage `upmap` items > - `ceph osd getcrushmap` returns a crush map version number on > stderr, and `ceph osd setcrushmap [version]` will only inject > an updated crush map if the version matches. This allows crush > maps to be updated offline and then reinjected into the cluster > without fear of clobbering racing changes (e.g., by newly added > osds or changes by other administrators). > - `ceph osd create` has been replaced by `ceph osd new`. This > should be hidden from most users by user-facing tools like > `ceph-disk`. > - `ceph osd destroy` will mark an OSD destroyed and remove its > cephx and lockbox keys. However, the OSD id and CRUSH map entry > will remain in place, allowing the id to be reused by a > replacement device with minimal data rebalancing. > - `ceph osd purge` will remove all traces of an OSD from the > cluster, including its cephx encryption keys, dm-crypt lockbox > keys, OSD id, and crush map entry. > - `ceph osd ls-tree <name>` will output a list of OSD ids under > the given CRUSH name (like a host or rack name). This is useful > for applying changes to entire subtrees. For example, `ceph > osd down `ceph osd ls-tree rack1``. > - `ceph osd {add,rm}-{noout,noin,nodown,noup}` allow the > `noout`, `noin`, `nodown`, and `noup` flags to be applied to > specific OSDs. > - `ceph osd safe-to-destroy <osd(s)>` will report whether it is safe to > remove or destroy OSD(s) without reducing data durability or redundancy. > - `ceph osd ok-to-stop <osd(s)>` will report whether it is okay to stop > OSD(s) without immediately compromising availability (i.e., all PGs > should remain active but may be degraded). > - `ceph log last [n]` will output the last *n* lines of the cluster > log. > - `ceph mgr dump` will dump the MgrMap, including the currently active > ceph-mgr daemon and any standbys. > - `ceph mgr module ls` will list active ceph-mgr modules. > - `ceph mgr module {enable,disable} <name>` will enable or > disable the named mgr module. The module must be present in the > configured `mgr_module_path` on the host(s) where `ceph-mgr` is > running. > - `ceph osd crush ls <node>` will list items (OSDs or other CRUSH nodes) > directly beneath a given CRUSH node. > - `ceph osd crush swap-bucket <src> <dest>` will swap the > contents of two CRUSH buckets in the hierarchy while preserving > the buckets' ids. This allows an entire subtree of devices to > be replaced (e.g., to replace an entire host of FileStore OSDs > with newly-imaged BlueStore OSDs) without disrupting the > distribution of data across neighboring devices. > - `ceph osd set-require-min-compat-client <release>` configures > the oldest client release the cluster is required to support. > Other changes, like CRUSH tunables, will fail with an error if > they would violate this setting. Changing this setting also > fails if clients older than the specified release are currently > connected to the cluster. > - `ceph config-key dump` dumps config-key entries and their > contents. (The existing `ceph config-key list` only dumps the key > names, not the values.) > - `ceph config-key list` is deprecated in favor of `ceph config-key ls`. > - `ceph config-key put` is deprecated in favor of `ceph config-key set`. > - `ceph auth list` is deprecated in favor of `ceph auth ls`. > - `ceph osd crush rule list` is deprecated in favor of `ceph osd crush rule ls`. > - `ceph osd set-{full,nearfull,backfillfull}-ratio` sets the > cluster-wide ratio for various full thresholds (when the cluster > refuses IO, when the cluster warns about being close to full, > when an OSD will defer rebalancing a PG to itself, > respectively). > - `ceph osd reweightn` will specify the `reweight` values for > multiple OSDs in a single command. This is equivalent to a series of > `ceph osd reweight` commands. > - `ceph osd crush {set,rm}-device-class` manage the new > CRUSH *device class* feature. Note that manually creating or deleting > a device class name is generally not necessary as it will be smart > enough to be self-managed. `ceph osd crush class ls` and > `ceph osd crush class ls-osd` will output all existing device classes > and a list of OSD ids under the given device class respectively. > - `ceph osd crush rule create-replicated` replaces the old > `ceph osd crush rule create-simple` command to create a CRUSH > rule for a replicated pool. Notably it takes a `class` argument > for the *device class* the rule should target (e.g., `ssd` or > `hdd`). > - `ceph mon feature ls` will list monitor features recorded in the > MonMap. `ceph mon feature set` will set an optional feature (none of > these exist yet). > - `ceph tell <daemon> help` will now return a usage summary. > - `ceph fs authorize` creates a new client key with caps automatically > set to access the given CephFS file system. > - The `ceph health` structured output (JSON or XML) no longer contains > 'timechecks' section describing the time sync status. This > information is now available via the 'ceph time-sync-status' > command. > - Certain extra fields in the `ceph health` structured output that > used to appear if the mons were low on disk space (which duplicated > the information in the normal health warning messages) are now gone. > - The `ceph -w` output no longer contains audit log entries by default. > Add a `--watch-channel=audit` or `--watch-channel=*` to see them. > - New "ceph -w" behavior - the "ceph -w" output no longer contains > I/O rates, available space, pg info, etc. because these are no > longer logged to the central log (which is what `ceph -w` > shows). The same information can be obtained by running `ceph pg > stat`; alternatively, I/O rates per pool can be determined using > `ceph osd pool stats`. Although these commands do not > self-update like `ceph -w` did, they do have the ability to > return formatted output by providing a `--format=<format>` > option. > - Added new commands `pg force-recovery` and > `pg-force-backfill`. Use them to boost recovery or backfill > priority of specified pgs, so they're recovered/backfilled > before any other. Note that these commands don't interrupt > ongoing recovery/backfill, but merely queue specified pgs before > others so they're recovered/backfilled as soon as possible. New > commands `pg cancel-force-recovery` and `pg > cancel-force-backfill` restore default recovery/backfill > priority of previously forced pgs. > > Major Changes from Jewel > ------------------------ > > - *RADOS*: > > * We now default to the AsyncMessenger (`ms type = async`) instead > of the legacy SimpleMessenger. The most noticeable difference is > that we now use a fixed sized thread pool for network connections > (instead of two threads per socket with SimpleMessenger). > * Some OSD failures are now detected almost immediately, whereas > previously the heartbeat timeout (which defaults to 20 seconds) > had to expire. This prevents IO from blocking for an extended > period for failures where the host remains up but the ceph-osd > process is no longer running. > * The size of encoded OSDMaps has been reduced. > * The OSDs now quiesce scrubbing when recovery or rebalancing is in progress. > > - *RGW*: > > * RGW now supports the S3 multipart object copy-part API. > * It is possible now to reshard an existing bucket offline. Offline > bucket resharding currently requires that all IO (especially > writes) to the specific bucket is quiesced. (For automatic online > resharding, see the new feature in Luminous above.) > * RGW now supports data compression for objects. > * Civetweb version has been upgraded to 1.8 > * The Swift static website API is now supported (S3 support has been added > previously). > * S3 bucket lifecycle API has been added. Note that currently it only supports > object expiration. > * Support for custom search filters has been added to the LDAP auth > implementation. > * Support for NFS version 3 has been added to the RGW NFS gateway. > * A Python binding has been created for librgw. > > - *RBD*: > > * The rbd-mirror daemon now supports replicating dynamic image > feature updates and image metadata key/value pairs from the > primary image to the non-primary image. > * The number of image snapshots can be optionally restricted to a > configurable maximum. > * The rbd Python API now supports asynchronous IO operations. > > - *CephFS*: > > * libcephfs function definitions have been changed to enable proper > uid/gid control. The library version has been increased to reflect the > interface change. > * Standby replay MDS daemons now consume less memory on workloads > doing deletions. > * Scrub now repairs backtrace, and populates `damage ls` with > discovered errors. > * A new `pg_files` subcommand to `cephfs-data-scan` can identify > files affected by a damaged or lost RADOS PG. > * The false-positive "failing to respond to cache pressure" warnings have > been fixed. > > > Upgrade from Jewel or Kraken > ---------------------------- > #. Ensure that the `sortbitwise` flag is enabled:: > # ceph osd set sortbitwise > #. Make sure your cluster is stable and healthy (no down or > recoverying OSDs). (Optional, but recommended.) #. Do not create any new erasure-code pools while upgrading the monitors. > #. You can monitor the progress of your upgrade at each stage with the > `ceph versions` command, which will tell you what ceph version is > running for each type of daemon. > #. Set the `noout` flag for the duration of the upgrade. (Optional > but recommended.):: > # ceph osd set noout > #. Upgrade monitors by installing the new packages and restarting the > monitor daemons. Note that, unlike prior releases, the ceph-mon > daemons *must* be upgraded first:: > # systemctl restart ceph-mon.target > Verify the monitor upgrade is complete once all monitors are up by > looking for the `luminous` feature string in the mon map. For > example:: > # ceph mon feature ls > should include `luminous` under persistent features:: > on current monmap (epoch NNN) > persistent: [kraken,luminous] > required: [kraken,luminous] > #. Add or restart `ceph-mgr` daemons. If you are upgrading from > kraken, upgrade packages and restart ceph-mgr daemons with:: > # systemctl restart ceph-mgr.target > If you are upgrading from kraken, you may already have ceph-mgr > daemons deployed. If not, or if you are upgrading from jewel, you > can deploy new daemons with tools like ceph-deploy or ceph-ansible. > For example:: > # ceph-deploy mgr create HOST > Verify the ceph-mgr daemons are running by checking `ceph -s`:: > # ceph -s > ... > services: > mon: 3 daemons, quorum foo,bar,baz > mgr: foo(active), standbys: bar, baz > ... > #. Upgrade all OSDs by installing the new packages and restarting the > ceph-osd daemons on all hosts:: > # systemctl restart ceph-osd.target > You can monitor the progress of the OSD upgrades with the new > `ceph versions` or `ceph osd versions` command:: > # ceph osd versions > { > "ceph version 12.2.0 (...) luminous (stable)": 12, > "ceph version 10.2.6 (...)": 3, > } > #. Upgrade all CephFS daemons by upgrading packages and restarting > daemons on all hosts:: > # systemctl restart ceph-mds.target #. Upgrade all radosgw daemons by upgrading packages and restarting > daemons on all hosts:: > # systemctl restart radosgw.target > #. Complete the upgrade by disallowing pre-luminous OSDs and enabling > all new Luminous-only functionality:: > # ceph osd require-osd-release luminous > If you set `noout` at the beginning, be sure to clear it with:: > # ceph osd unset noout > #. Verify the cluster is healthy with `ceph health`. > > > Upgrading from pre-Jewel releases (like Hammer) > ----------------------------------------------- > > You *must* first upgrade to Jewel (10.2.z) before attempting an upgrade to Luminous. > > > Upgrade compatibility notes, Kraken to Luminous > ----------------------------------------------- > > * The configuration option `osd pool erasure code stripe width` has > been replaced by `osd pool erasure code stripe unit`, and given > the ability to be overridden by the erasure code profile setting > `stripe_unit`. For more details see > :ref:`erasure-code-profiles`. > > * rbd and cephfs can use erasure coding with bluestore. This may be > enabled by setting `allow_ec_overwrites` to `true` for a pool. Since > this relies on bluestore's checksumming to do deep scrubbing, > enabling this on a pool stored on filestore is not allowed. > > * The `rados df` JSON output now prints numeric values as numbers instead of > strings. > > * The `mon_osd_max_op_age` option has been renamed to > `mon_osd_warn_op_age` (default: 32 seconds), to indicate we > generate a warning at this age. There is also a new > `mon_osd_err_op_age_ratio` that is a expressed as a multitple of > `mon_osd_warn_op_age` (default: 128, for roughly 60 minutes) to > control when an error is generated. > > * The default maximum size for a single RADOS object has been reduced from > 100GB to 128MB. The 100GB limit was completely impractical in practice > while the 128MB limit is a bit high but not unreasonable. If you have an > application written directly to librados that is using objects larger than > 128MB you may need to adjust `osd_max_object_size`. > > * The semantics of the `rados ls` and librados object listing > operations have always been a bit confusing in that "whiteout" > objects (which logically don't exist and will return ENOENT if you > try to access them) are included in the results. Previously > whiteouts only occurred in cache tier pools. In luminous, logically > deleted but snapshotted objects now result in a whiteout object, and > as a result they will appear in `rados ls` results, even though > trying to read such an object will result in ENOENT. The `rados > listsnaps` operation can be used in such a case to enumerate which > snapshots are present. > This may seem a bit strange, but is less strange than having a > deleted-but-snapshotted object not appear at all and be completely > hidden from librados's ability to enumerate objects. Future > versions of Ceph will likely include an alternative object > enumeration interface that makes it more natural and efficient to > enumerate all objects along with their snapshot and clone metadata. > > * The deprecated `crush_ruleset` property has finally been removed; > please use `crush_rule` instead for the `osd pool get ...` and `osd > pool set ...` commands. > > * The `osd pool default crush replicated ruleset` option has been > removed and replaced by the `psd pool default crush rule` option. > By default it is -1, which means the mon will pick the first type > replicated rule in the CRUSH map for replicated pools. Erasure > coded pools have rules that are automatically created for them if > they are not specified at pool creation time. > > * We no longer test the FileStore ceph-osd backend in combination with > btrfs. We recommend against using btrfs. If you are using > btrfs-based OSDs and want to upgrade to luminous you will need to > add the follwing to your ceph.conf:: > > enable experimental unrecoverable data corrupting features = btrfs > > The code is mature and unlikely to change, but we are only > continuing to test the Jewel stable branch against btrfs. We > recommend moving these OSDs to FileStore with XFS or BlueStore. > * The `ruleset-*` properties for the erasure code profiles have been > renamed to `crush-*` to (1) move away from the obsolete 'ruleset' > term and to be more clear about their purpose. There is also a new > optional `crush-device-class` property to specify a CRUSH device > class to use for the erasure coded pool. Existing erasure code > profiles will be converted automatically when upgrade completes > (when the `ceph osd require-osd-release luminous` command is run) > but any provisioning tools that create erasure coded pools may need > to be updated. > * The structure of the XML output for `osd crush tree` has changed > slightly to better match the `osd tree` output. The top level > structure is now `nodes` instead of `crush_map_roots`. > * When assigning a network to the public network and not to > the cluster network the network specification of the public > network will be used for the cluster network as well. > In older versions this would lead to cluster services > being bound to 0.0.0.0:<port>, thus making the > cluster service even more publicly available than the > public services. When only specifying a cluster network it > will still result in the public services binding to 0.0.0.0. > > * In previous versions, if a client sent an op to the wrong OSD, the OSD > would reply with ENXIO. The rationale here is that the client or OSD is > clearly buggy and we want to surface the error as clearly as possible. > We now only send the ENXIO reply if the osd_enxio_on_misdirected_op option > is enabled (it's off by default). This means that a VM using librbd that > previously would have gotten an EIO and gone read-only will now see a > blocked/hung IO instead. > > * The "journaler allow split entries" config setting has been removed. > > * The 'mon_warn_osd_usage_min_max_delta' config option has been > removed and the associated health warning has been disabled because > it does not address clusters undergoing recovery or CRUSH rules that do > not target all devices in the cluster. > > * Added new configuration "public bind addr" to support dynamic > environments like Kubernetes. When set the Ceph MON daemon could > bind locally to an IP address and advertise a different IP address > `public addr` on the network. > > * The crush `choose_args` encoding has been changed to make it > architecture-independent. If you deployed Luminous dev releases or > 12.1.0 rc release and made use of the CRUSH choose_args feature, you > need to remove all choose_args mappings from your CRUSH map before > starting the upgrade. > > > - *librados*: > > * Some variants of the omap_get_keys and omap_get_vals librados > functions have been deprecated in favor of omap_get_vals2 and > omap_get_keys2. The new methods include an output argument > indicating whether there are additional keys left to fetch. > Previously this had to be inferred from the requested key count vs > the number of keys returned, but this breaks with new OSD-side > limits on the number of keys or bytes that can be returned by a > single omap request. These limits were introduced by kraken but > are effectively disabled by default (by setting a very large limit > of 1 GB) because users of the newly deprecated interface cannot > tell whether they should fetch more keys or not. In the case of > the standalone calls in the C++ interface > (IoCtx::get_omap_{keys,vals}), librados has been updated to loop on > the client side to provide a correct result via multiple calls to > the OSD. In the case of the methods used for building > multi-operation transactions, however, client-side looping is not > practical, and the methods have been deprecated. Note that use of > either the IoCtx methods on older librados versions or the > deprecated methods on any version of librados will lead to > incomplete results if/when the new OSD limits are enabled. > > * The original librados rados_objects_list_open (C) and objects_begin > (C++) object listing API, deprecated in Hammer, has finally been > removed. Users of this interface must update their software to use > either the rados_nobjects_list_open (C) and nobjects_begin (C++) API or > the new rados_object_list_begin (C) and object_list_begin (C++) API > before updating the client-side librados library to Luminous. > Object enumeration (via any API) with the latest librados version > and pre-Hammer OSDs is no longer supported. Note that no in-tree > Ceph services rely on object enumeration via the deprecated APIs, so > only external librados users might be affected. > The newest (and recommended) rados_object_list_begin (C) and > object_list_begin (C++) API is only usable on clusters with the > SORTBITWISE flag enabled (Jewel and later). (Note that this flag is > required to be set before upgrading beyond Jewel.) > > - *CephFS*: > > * When configuring ceph-fuse mounts in /etc/fstab, a new syntax is > available that uses "ceph.<arg>=<val>" in the options column, instead > of putting configuration in the device column. The old style syntax > still works. See the documentation page "Mount CephFS in your > file systems table" for details. > * CephFS clients without the 'p' flag in their authentication capability > string will no longer be able to set quotas or any layout fields. This > flag previously only restricted modification of the pool and namespace > fields in layouts. > * CephFS will generate a health warning if you have fewer standby daemons > than it thinks you wanted. By default this will be 1 if you ever had > a standby, and 0 if you did not. You can customize this using > `ceph fs set <fs> standby_count_wanted <number>`. Setting it > to zero will effectively disable the health check. > * The "ceph mds tell ..." command has been removed. It is superceded > by "ceph tell mds.<id> ..." > * The `apply` mode of cephfs-journal-tool has been removed > > Getting Ceph > ------------ > > * Git at git://github.com/ceph/ceph.git > * Tarball at http://download.ceph.com/tarballs/ceph-12.2.0.tar.gz > * For packages, see http://docs.ceph.com/docs/master/install/get-packages/ > * For ceph-deploy, see http://docs.ceph.com/docs/master/install/install-ceph-deploy > * Release git sha1: 32ce2a3ae5239ee33d6150705cdb24d43bab910c > > -- > Abhishek Lekshmanan > SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html > N?????r??y??????X??ǧv???){.n?????z?]z????ay? ʇڙ??j??f???h????? ?w??? ???j:+v???w????????????zZ+???????j"????i
