Cool, thanks! I see many new features in RGW, but where the documentation or something like that can be found? Kind regards, Vasily. 2016-04-21 21:30 GMT+03:00 Sage Weil <sage@xxxxxxxxxx>: > This major release of Ceph will be the foundation for the next > long-term stable release. There have been many major changes since > the Infernalis (9.2.x) and Hammer (0.94.x) releases, and the upgrade > process is non-trivial. Please read these release notes carefully. > > For the complete release notes, please see > > http://ceph.com/releases/v10-2-0-jewel-released/ > > > Major Changes from Infernalis > ----------------------------- > > - *CephFS*: > > * This is the first release in which CephFS is declared stable and > production ready! Several features are disabled by default, including > snapshots and multiple active MDS servers. > * The repair and disaster recovery tools are now feature-complete. > * A new cephfs-volume-manager module is included that provides a > high-level interface for creating "shares" for OpenStack Manila > and similar projects. > * There is now experimental support for multiple CephFS file systems > within a single cluster. > > - *RGW*: > > * The multisite feature has been almost completely rearchitected and > rewritten to support any number of clusters/sites, bidirectional > fail-over, and active/active configurations. > * You can now access radosgw buckets via NFS (experimental). > * The AWS4 authentication protocol is now supported. > * There is now support for S3 request payer buckets. > * The new multitenancy infrastructure improves compatibility with > Swift, which provides a separate container namespace for each > user/tenant. > * The OpenStack Keystone v3 API is now supported. There are a range > of other small Swift API features and compatibility improvements > as well, including bulk delete and SLO (static large objects). > > - *RBD*: > > * There is new support for mirroring (asynchronous replication) of > RBD images across clusters. This is implemented as a per-RBD > image journal that can be streamed across a WAN to another site, > and a new rbd-mirror daemon that performs the cross-cluster > replication. > * The exclusive-lock, object-map, fast-diff, and journaling features > can be enabled or disabled dynamically. The deep-flatten features > can be disabled dynamically but not re-enabled. > * The RBD CLI has been rewritten to provide command-specific help > and full bash completion support. > * RBD snapshots can now be renamed. > > - *RADOS*: > > * BlueStore, a new OSD backend, is included as an experimental > feature. The plan is for it to become the default backend in the > K or L release. > * The OSD now persists scrub results and provides a librados API to > query results in detail. > * We have revised our documentation to recommend *against* using > ext4 as the underlying filesystem for Ceph OSD daemons due to > problems supporting our long object name handling. > > Major Changes from Hammer > ------------------------- > > - *General*: > > * Ceph daemons are now managed via systemd (with the exception of > Ubuntu Trusty, which still uses upstart). > * Ceph daemons run as 'ceph' user instead of 'root'. > * On Red Hat distros, there is also an SELinux policy. > > - *RADOS*: > > * The RADOS cache tier can now proxy write operations to the base > tier, allowing writes to be handled without forcing migration of > an object into the cache. > * The SHEC erasure coding support is no longer flagged as > experimental. SHEC trades some additional storage space for faster > repair. > * There is now a unified queue (and thus prioritization) of client > IO, recovery, scrubbing, and snapshot trimming. > * There have been many improvements to low-level repair tooling > (ceph-objectstore-tool). > * The internal ObjectStore API has been significantly cleaned up in order > to faciliate new storage backends like BlueStore. > > - *RGW*: > > * The Swift API now supports object expiration. > * There are many Swift API compatibility improvements. > > - *RBD*: > > * The ``rbd du`` command shows actual usage (quickly, when > object-map is enabled). > * The object-map feature has seen many stability improvements. > * The object-map and exclusive-lock features can be enabled or disabled > dynamically. > * You can now store user metadata and set persistent librbd options > associated with individual images. > * The new deep-flatten features allow flattening of a clone and all > of its snapshots. (Previously snapshots could not be flattened.) > * The export-diff command is now faster (it uses aio). There is also > a new fast-diff feature. > * The --size argument can be specified with a suffix for units > (e.g., ``--size 64G``). > * There is a new ``rbd status`` command that, for now, shows who has > the image open/mapped. > > - *CephFS*: > > * You can now rename snapshots. > * There have been ongoing improvements around administration, diagnostics, > and the check and repair tools. > * The caching and revocation of client cache state due to unused > inodes has been dramatically improved. > * The ceph-fuse client behaves better on 32-bit hosts. > > Distro compatibility > -------------------- > > Starting with Infernalis, we have dropped support for many older > distributions so that we can move to a newer compiler toolchain (e.g., > C++11). Although it is still possible to build Ceph on older > distributions by installing backported development tools, we are not > building and publishing release packages for ceph.com. > > We now build packages for the following distributions and architectures: > > - x86_64: > > * CentOS 7.x. We have dropped support for CentOS 6 (and other RHEL 6 > derivatives, like Scientific Linux 6). > * Debian Jessie 8.x. Debian Wheezy 7.x's g++ has incomplete support > for C++11 (and no systemd). > * Ubuntu Xenial 16.04 and Trusty 14.04. Ubuntu Precise 12.04 is no > longer supported. > * Fedora 22 or later. > > - aarch64 / arm64: > > * Ubuntu Xenial 16.04. > > Upgrading from Infernalis or Hammer > ----------------------------------- > > * We now recommend against using ``ext4`` as the underlying file > system for Ceph OSDs, especially when RGW or other users of long > RADOS object names are used. For more information about why, please > see `Filesystem Recommendations`_. > > If you have an existing cluster that uses ext4 for the OSDs but uses only > RBD and/or CephFS, then the ext4 limitations will not affect you. Before > upgrading, be sure add the following to ``ceph.conf`` to allow the OSDs to > start:: > > osd max object name len = 256 > osd max object namespace len = 64 > > Keep in mind that if you set these lower object name limits and > later decide to use RGW on this cluster, it will have problems > storing S3/Swift objects with long names. This startup check can also be > disabled via the below option, although this is not recommended:: > > osd check max object name len on startup = false > > .. _Filesystem Recommendations: ../configuration/filesystem-recommendations > > * There are no major compatibility changes since Infernalis. Simply > upgrading the daemons on each host and restarting all daemons is > sufficient. > > * The rbd CLI no longer accepts the deprecated '--image-features' option > during create, import, and clone operations. The '--image-feature' > option should be used instead. > > * The rbd legacy image format (version 1) is deprecated with the Jewel release. > Attempting to create a new version 1 RBD image will result in a warning. > Future releases of Ceph will remove support for version 1 RBD images. > > * The 'send_pg_creates' and 'map_pg_creates' mon CLI commands are > obsolete and no longer supported. > > * A new configure option 'mon_election_timeout' is added to specifically > limit max waiting time of monitor election process, which was previously > restricted by 'mon_lease'. > > * CephFS filesystems created using versions older than Firefly (0.80) must > use the new 'cephfs-data-scan tmap_upgrade' command after upgrading to > Jewel. See 'Upgrading' in the CephFS documentation for more information. > > * The 'ceph mds setmap' command has been removed. > > * The default RBD image features for new images have been updated to > enable the following: exclusive lock, object map, fast-diff, and > deep-flatten. These features are not currently supported by the RBD > kernel driver nor older RBD clients. They can be disabled on a per-image > basis via the RBD CLI, or the default features can be updated to the > pre-Jewel setting by adding the following to the client section of the Ceph > configuration file:: > > rbd default features = 1 > > * The rbd legacy image format (version 1) is deprecated with the Jewel > release. > > * After upgrading, users should set the 'sortbitwise' flag to enable the new > internal object sort order:: > > ceph osd set sortbitwise > > This flag is important for the new object enumeration API and for > new backends like BlueStore. > > * The rbd CLI no longer permits creating images and snapshots with potentially > ambiguous names (e.g. the '/' and '@' characters are disallowed). The > validation can be temporarily disabled by adding "--rbd-validate-names=false" > to the rbd CLI when creating an image or snapshot. It can also be disabled > by adding the following to the client section of the Ceph configuration file:: > > rbd validate names = false > > Upgrading from Hammer > --------------------- > > * All cluster nodes must first upgrade to Hammer v0.94.4 or a later > v0.94.z release; only then is it possible to upgrade to Jewel > 10.2.z. > > * For all distributions that support systemd (CentOS 7, Fedora, Debian > Jessie 8.x, OpenSUSE), ceph daemons are now managed using native systemd > files instead of the legacy sysvinit scripts. For example,:: > > systemctl start ceph.target # start all daemons > systemctl status ceph-osd@12 # check status of osd.12 > > The main notable distro that is *not* yet using systemd is Ubuntu trusty > 14.04. (The next Ubuntu LTS, 16.04, will use systemd instead of upstart.) > > * Ceph daemons now run as user and group ``ceph`` by default. The > ceph user has a static UID assigned by Fedora and Debian (also used by > derivative distributions like RHEL/CentOS and Ubuntu). On SUSE the same > UID/GID as in Fedora and Debian will be used, *provided it is not already > assigned*. In the unlikely event the preferred UID or GID is assigned to a > different user/group, ceph will get a dynamically assigned UID/GID. > > If your systems already have a ceph user, upgrading the package will cause > problems. We suggest you first remove or rename the existing 'ceph' user > and 'ceph' group before upgrading. > > When upgrading, administrators have two options: > > #. Add the following line to ``ceph.conf`` on all hosts:: > > setuser match path = /var/lib/ceph/$type/$cluster-$id > > This will make the Ceph daemons run as root (i.e., not drop > privileges and switch to user ceph) if the daemon's data > directory is still owned by root. Newly deployed daemons will > be created with data owned by user ceph and will run with > reduced privileges, but upgraded daemons will continue to run as > root. > > #. Fix the data ownership during the upgrade. This is the > preferred option, but it is more work and can be very time > consuming. The process for each host is to: > > #. Upgrade the ceph package. This creates the ceph user and group. For > example:: > > ceph-deploy install --stable jewel HOST > > #. Stop the daemon(s).:: > > service ceph stop # fedora, centos, rhel, debian > stop ceph-all # ubuntu > > #. Fix the ownership:: > > chown -R ceph:ceph /var/lib/ceph > > #. Restart the daemon(s).:: > > start ceph-all # ubuntu > systemctl start ceph.target # debian, centos, fedora, rhel > > Alternatively, the same process can be done with a single daemon > type, for example by stopping only monitors and chowning only > ``/var/lib/ceph/mon``. > > * The on-disk format for the experimental KeyValueStore OSD backend has > changed. You will need to remove any OSDs using that backend before you > upgrade any test clusters that use it. > > * When a pool quota is reached, librados operations now block indefinitely, > the same way they do when the cluster fills up. (Previously they would return > -ENOSPC.) By default, a full cluster or pool will now block. If your > librados application can handle ENOSPC or EDQUOT errors gracefully, you can > get error returns instead by using the new librados OPERATION_FULL_TRY flag. > > * The return code for librbd's rbd_aio_read and Image::aio_read API methods no > longer returns the number of bytes read upon success. Instead, it returns 0 > upon success and a negative value upon failure. > > * 'ceph scrub', 'ceph compact' and 'ceph sync force' are now DEPRECATED. Users > should instead use 'ceph mon scrub', 'ceph mon compact' and > 'ceph mon sync force'. > > * 'ceph mon_metadata' should now be used as 'ceph mon metadata'. There is no > need to deprecate this command (same major release since it was first > introduced). > > * The `--dump-json` option of "osdmaptool" is replaced by `--dump json`. > > * The commands of "pg ls-by-{pool,primary,osd}" and "pg ls" now take "recovering" > instead of "recovery", to include the recovering pgs in the listed pgs. > > Upgrading from Firefly > ---------------------- > > Upgrading directly from Firefly v0.80.z is not recommended. It is > possible to do a direct upgrade, but not without downtime, as all OSDs > must be stopped, upgraded, and then restarted. We recommend that > clusters be first upgraded to Hammer v0.94.6 or a later v0.94.z > release; only then is it possible to upgrade to Jewel 10.2.z for an > online upgrade (see below). > > To do an offline upgrade directly from Firefly, all Firefly OSDs must > be stopped and marked down before any Jewel OSDs will be allowed > to start up. This fencing is enforced by the Jewel monitor, so > you should use an upgrade procedure like: > > #. Upgrade Ceph on monitor hosts > #. Restart all ceph-mon daemons > #. Set noout:: > ceph osd set noout > #. Upgrade Ceph on all OSD hosts > #. Stop all ceph-osd daemons > #. Mark all OSDs down with something like:: > ceph osd down `seq 0 1000` > #. Start all ceph-osd daemons > #. Let the cluster settle and then unset noout:: > ceph osd unset noout > #. Upgrade and restart any remaining daemons (ceph-mds, radosgw) > > > Getting Ceph > ------------ > > * Git at git://github.com/ceph/ceph.git > * Tarball at http://download.ceph.com/tarballs/ceph-10.2.0.tar.gz > * For packages, see http://ceph.com/docs/master/install/get-packages > * For ceph-deploy, see http://ceph.com/docs/master/install/install-ceph-deploy > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |