[PATCH 00/20] EXT4 encoding support

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Ted,

This patchset implements encoding support as a superblock feature, valid
for the entire disk, as an intermediate step to my goal of supporting
case-insensitiveness.

The superblock records the encoding in a reserved field, but the
encoding can also be forced through the use of a mount flag called
encoding=.

Since not every NLS tables support normalization operations, we limit
which encodings can be used by an ext4 volume.  Right now, ascii and
utf8n are supported, utf8n being a new version of the utf8 charset, but
with normalization support using the SGI patches, which are part of this
patchset.

This patchset also includes the NLS changes that I am proposing, since
encoding-awareness depends on NLS and the former patchset didn't get
much review beforehand.  As usual, I did not include the source ucd
files because they would bounce in the list, but a completely functional
implementation can be found at:

    https://gitlab.collabora.com/krisman/linux.git -b ext4-ci-directory

I am also not supporting encoding with encrypted directories, given the
cost of searching encrypted directories where the diggested name is not
normalized. This means that we need to decrypt each filename beforehand,
so I decided to simply skip this for now.  If the user tries to mount
with encoding a directory that has the encryption feature, we simply
bail out saying it is not supported.

The patchset survives without failures the smoke tests of xfstests, with
the obvious exception of generic/453.  This test, which verifies that
multiple files having similar names (which would match in utf8
normalization) are not the same file, doesn't really make sense with
this patchset, since it verifies the fs is *not* encoding aware.

I have a patch adding support for encoding detection and skipping
this test on xfstests, that I can send once we agreed on the mount point
options and the superblock layout changes for this feature.

I am also CC'ing Darick for his input on utf8n and NLS.

I am not listing this as a new version of the previous NLS patchsets,
because it includes a lot more patches than the original patchset, but I left the changelogs from the previous versions for the NLS patches.

Please let me know what you think.

Gabriel Krisman Bertazi (16):
  nls: Wrap uni2char/char2uni callers
  nls: Wrap charset field access
  nls: Wrap charset hooks in ops structure
  nls: Split default charset from NLS core
  nls: Split struct nls_charset from struct nls_table
  nls: Add support for multiple versions of an encoding
  nls: Add new interface for string comparisons
  nls: Let charsets define the behavior of tolower/toupper
  nls: Add optional normalization and casefold hooks
  nls: utf8norm: Integrate utf8norm code with NLS subsystem
  nls: utf8norm: Introduce test module for utf8norm implementation
  nls: ascii: Support casefold and normalization operations
  ext4: Include encoding information in the superblock
  ext4: Support encoding-aware file name lookups
  vfs: Handle case-exact lookup in d_add_ci
  ext4: Implement encoding-aware dcache hooks

Olaf Weber (4):
  nls: utf8norm: Add unicode character database files
  scripts: add trie generator for UTF-8
  nls: utf8norm: Introduce code for UTF-8 normalization
  nls: utf8norm: reduce the size of utf8data[]

 fs/befs/linuxvfs.c                   |    8 +-
 fs/cifs/cifs_unicode.c               |   15 +-
 fs/cifs/cifsfs.c                     |    2 +-
 fs/cifs/connect.c                    |    2 +-
 fs/cifs/dir.c                        |    7 +-
 fs/dcache.c                          |   33 +-
 fs/ext4/dir.c                        |   30 +
 fs/ext4/ext4.h                       |    8 +-
 fs/ext4/namei.c                      |   60 +-
 fs/ext4/super.c                      |  123 +
 fs/fat/dir.c                         |   13 +-
 fs/fat/inode.c                       |    6 +-
 fs/fat/namei_vfat.c                  |    6 +-
 fs/hfs/super.c                       |    6 +-
 fs/hfs/trans.c                       |    9 +-
 fs/hfsplus/options.c                 |    2 +-
 fs/hfsplus/unicode.c                 |    6 +-
 fs/isofs/inode.c                     |    5 +-
 fs/isofs/joliet.c                    |    3 +-
 fs/jfs/jfs_unicode.c                 |    9 +-
 fs/jfs/super.c                       |    3 +-
 fs/nls/Kconfig                       |   13 +
 fs/nls/Makefile                      |   19 +
 fs/nls/mac-celtic.c                  |   34 +-
 fs/nls/mac-centeuro.c                |   34 +-
 fs/nls/mac-croatian.c                |   34 +-
 fs/nls/mac-cyrillic.c                |   34 +-
 fs/nls/mac-gaelic.c                  |   34 +-
 fs/nls/mac-greek.c                   |   34 +-
 fs/nls/mac-iceland.c                 |   34 +-
 fs/nls/mac-inuit.c                   |   34 +-
 fs/nls/mac-roman.c                   |   34 +-
 fs/nls/mac-romanian.c                |   34 +-
 fs/nls/mac-turkish.c                 |   34 +-
 fs/nls/nls_ascii.c                   |   67 +-
 fs/nls/nls_core.c                    |  141 ++
 fs/nls/nls_cp1250.c                  |   34 +-
 fs/nls/nls_cp1251.c                  |   34 +-
 fs/nls/nls_cp1255.c                  |   36 +-
 fs/nls/nls_cp437.c                   |   34 +-
 fs/nls/nls_cp737.c                   |   34 +-
 fs/nls/nls_cp775.c                   |   34 +-
 fs/nls/nls_cp850.c                   |   34 +-
 fs/nls/nls_cp852.c                   |   34 +-
 fs/nls/nls_cp855.c                   |   34 +-
 fs/nls/nls_cp857.c                   |   34 +-
 fs/nls/nls_cp860.c                   |   34 +-
 fs/nls/nls_cp861.c                   |   34 +-
 fs/nls/nls_cp862.c                   |   34 +-
 fs/nls/nls_cp863.c                   |   34 +-
 fs/nls/nls_cp864.c                   |   34 +-
 fs/nls/nls_cp865.c                   |   34 +-
 fs/nls/nls_cp866.c                   |   34 +-
 fs/nls/nls_cp869.c                   |   34 +-
 fs/nls/nls_cp874.c                   |   36 +-
 fs/nls/nls_cp932.c                   |   36 +-
 fs/nls/nls_cp936.c                   |   36 +-
 fs/nls/nls_cp949.c                   |   36 +-
 fs/nls/nls_cp950.c                   |   36 +-
 fs/nls/{nls_base.c => nls_default.c} |  124 +-
 fs/nls/nls_euc-jp.c                  |   29 +-
 fs/nls/nls_iso8859-1.c               |   34 +-
 fs/nls/nls_iso8859-13.c              |   34 +-
 fs/nls/nls_iso8859-14.c              |   34 +-
 fs/nls/nls_iso8859-15.c              |   34 +-
 fs/nls/nls_iso8859-2.c               |   34 +-
 fs/nls/nls_iso8859-3.c               |   34 +-
 fs/nls/nls_iso8859-4.c               |   34 +-
 fs/nls/nls_iso8859-5.c               |   34 +-
 fs/nls/nls_iso8859-6.c               |   34 +-
 fs/nls/nls_iso8859-7.c               |   34 +-
 fs/nls/nls_iso8859-9.c               |   34 +-
 fs/nls/nls_koi8-r.c                  |   34 +-
 fs/nls/nls_koi8-ru.c                 |   30 +-
 fs/nls/nls_koi8-u.c                  |   34 +-
 fs/nls/nls_utf8.c                    |   34 +-
 fs/nls/nls_utf8n-core.c              |  276 ++
 fs/nls/nls_utf8n-norm.c              |  797 ++++++
 fs/nls/nls_utf8n-selftest.c          |  307 +++
 fs/nls/ucd/README                    |   33 +
 fs/nls/utf8n.h                       |  117 +
 fs/ntfs/inode.c                      |    2 +-
 fs/ntfs/super.c                      |    6 +-
 fs/ntfs/unistr.c                     |   13 +-
 fs/udf/super.c                       |    3 +-
 fs/udf/unicode.c                     |    4 +-
 include/linux/nls.h                  |  127 +-
 scripts/Makefile                     |    1 +
 scripts/mkutf8data.c                 | 3464 ++++++++++++++++++++++++++
 89 files changed, 7018 insertions(+), 555 deletions(-)
 create mode 100644 fs/nls/nls_core.c
 rename fs/nls/{nls_base.c => nls_default.c} (89%)
 create mode 100644 fs/nls/nls_utf8n-core.c
 create mode 100644 fs/nls/nls_utf8n-norm.c
 create mode 100644 fs/nls/nls_utf8n-selftest.c
 create mode 100644 fs/nls/ucd/README
 create mode 100644 fs/nls/utf8n.h
 create mode 100644 scripts/mkutf8data.c

-- 
2.18.0




[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux