UTF-8 & imap folder name handling

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

The Fedora Linux project and Red Hat are pushing very hard for good
internationalisation support. That means UTF-8 support in every single
app they ship even if it may cause interoperability problems with old
applications (the common feeling is if we wait for everyone to jump on
the unicode boat before using unicode, we'll still be waiting in ten
years).

Squirrelmail by default does not use UTF-8 for many of its locales,
causing character losses when you reply to a message that contains
characters not available in the encoding declared for your locale.

To workaround this all the locales in the fedora squirrelmail version
are converted to UTF-8 at build time (cf attached build script). This
fixes interoperability between locales, since all characters in a
received message now fit inside the locale encoding, so quoting people
when replying to them now work.

Unfortunately this fix has unearthed a problem with squirrelmail
handling of imap folders. squirrelmail seems to assume 7bit imap folder
names, as soon as you try do display a folder name that use >7bit
characters in an UTF-8 locale things break (it doesn't with 8bit iso
locales, probably because so many bits assume iso-8859-1 transcoding
problems cancel one another).

With an UTF-8 locale SM won't display properly non-ascii folder names
when those have been created by another mail client, if you
recreate/rename the folders in squirrelmail display is now fine inside
SM but broken for everyone else.

Many common default folders use >7bit encodings in many locales (Sent ->
"Éléments envoyés" in French) so you're almost certain to hit this bug
as soon as you use an UTF-8 locale that actually needs >7bit for naming
folders. It's not a problem that can be ignored if one wants to ship and
i18n SM.

Can someone familiar with squirrelmail fix the imap folder encoding code
when local is UTF-8-enabled ? This is a bit too much to do at the Fedora
level (especially since Fedora tries to stay close to upstream). If not
Fedora will have to accept SM won't do UTF-8 sanely any time soon, which
will mean SM removal according to Fedora internal rules.

Various bug reports related to the problem :
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=162852
https://sourceforge.net/tracker/?func=detail&aid=1339393&group_id=311&atid=100311
https://sourceforge.net/tracker/?func=detail&atid=423691&aid=1235345&group_id=311

Regards,

-- 
Nicolas Mailhot
# OPTION: Fedora (1) or RHEL (0) Splash
%define fedora_splash 1

%define contentdir /var/www

Summary: SquirrelMail webmail client
Name: squirrelmail
Version: 1.4.6
Release: 0.cvs20050812.2.fc5.1.nim
License: GPL
URL: http://www.squirrelmail.org/
Group: Applications/Internet
#Source: %{name}-%{version}.tar.bz2
Source0: %{name}-1.4.5.tar.bz2
Source1: squirrelmail.conf
Source2: squirrelmail-splash-fedora.png
Source3: squirrelmail-splash-rhel.png
Source4: squirrelmail-20050812_1242-CVS.locales.tar.bz2
Patch2: squirrelmail-1.4.3-config.patch
Patch3: squirrelmail-1.4.6-cvs20050812.patch
Patch4: squirrelmail-1.4.5-charset.patch
BuildRoot: %{_tmppath}/%{name}-%{version}-root
BuildArch: noarch
Requires: httpd, php >= 4.0.4, perl, tmpwatch >= 2.8, aspell
Requires: /usr/sbin/sendmail
Prereq: httpd, perl
Provides: squirrelmail-i18n

%description
SquirrelMail is a standards-based webmail package written in PHP4. It
includes built-in pure PHP support for the IMAP and SMTP protocols, and
all pages render in pure HTML 4.0 (with no Javascript) for maximum
compatibility across browsers.  It has very few requirements and is very
easy to configure and install. SquirrelMail has all the functionality
you would want from an email client, including strong MIME support,
address books, and folder manipulation.

%prep
#%setup -q
%setup -q -n squirrelmail-1.4.5
%patch2 -p0
%patch3 -p1
%patch4 -p1
rm -f plugins/make_archive.pl

# Rearrange the documentation
mv AUTHORS ChangeLog COPYING INSTALL README UPGRADE doc/
mv ReleaseNotes doc/ReleaseNotes.txt
mv themes/README.themes doc/
for f in `find plugins -name "README*" -or -name INSTALL \
		   -or -name CHANGES -or -name HISTORY`; do
    mkdir -p doc/`dirname $f`
    mv $f $_
done
mv doc/plugins/squirrelspell/doc/README doc/plugins/squirrelspell
rmdir doc/plugins/squirrelspell/doc
mv plugins/squirrelspell/doc/* doc/plugins/squirrelspell
rm -f doc/plugins/squirrelspell/index.php
rmdir plugins/squirrelspell/doc
perl -pi -e "s/\.\.//g" doc/index.html

# Fixup various files
echo "left_refresh=300" >> data/default_pref
for f in contrib/RPM/squirrelmail.cron contrib/RPM/config.php.redhat; do
    perl -pi -e "s|__ATTDIR__|%{_localstatedir}/spool/squirrelmail/attach/|g;"\
	 -e "s|__PREFSDIR__|%{_localstatedir}/lib/squirrelmail/prefs/|g;" $f
done

# Fix the version
%{__perl} -pi -e "s|^(\s*\\\$version\s*=\s*'[^']+)'|\1-%{release}'|g"\
    functions/strings.php
  
# replace splash screen
%if %{fedora_splash}
cp %{SOURCE2} images/sm_logo.png
%else
cp %{SOURCE3} images/sm_logo.png
%endif

%install
rm -rf $RPM_BUILD_ROOT
mkdir -p -m0755 $RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail
mkdir -p -m0755 $RPM_BUILD_ROOT%{_localstatedir}/lib/squirrelmail/prefs
mkdir -p -m0755 $RPM_BUILD_ROOT%{_localstatedir}/spool/squirrelmail/attach
mkdir -p -m0755 $RPM_BUILD_ROOT%{_datadir}/squirrelmail
mkdir -p -m0755 $RPM_BUILD_ROOT%{contentdir}/html
mkdir -p -m0755 $RPM_BUILD_ROOT%{_sysconfdir}/cron.daily

# install default_pref
install -m 0644 data/default_pref \
    $RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/
ln -s ../../../..%{_sysconfdir}/squirrelmail/default_pref \
    $RPM_BUILD_ROOT%{_localstatedir}/lib/squirrelmail/prefs/default_pref

# install the config files
mkdir -p -m0755 $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config
install -m 0644 config/*.php $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/
rm -f $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/config_local.php
install -m 0644 config/config_local.php \
    $RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/config_local.php
ln -s ../../../..%{_sysconfdir}/squirrelmail/config_local.php \
    $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/config_local.php
install -m 0644 contrib/RPM/config.php.redhat \
    $RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/config.php
ln -s ../../../..%{_sysconfdir}/squirrelmail/config.php \
    $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/config.php
install -m 0755 config/*.pl $RPM_BUILD_ROOT%{_datadir}/squirrelmail/config/

# set default_folder_prefix to be dovecot compatible
echo -e "\n\$default_folder_prefix		= '';" >> $RPM_BUILD_ROOT%{_sysconfdir}/squirrelmail/config_local.php

# install index.php
install -m 0644 index.php $RPM_BUILD_ROOT%{_datadir}/squirrelmail/

# copy over the rest
for d in class functions help images include locale plugins src themes; do
    cp -rp $d $RPM_BUILD_ROOT%{_datadir}/squirrelmail/
done

# install the cron script
install -m 0755 contrib/RPM/squirrelmail.cron \
    $RPM_BUILD_ROOT/%{_sysconfdir}/cron.daily/

# install the config file
mkdir -p $RPM_BUILD_ROOT%{_sysconfdir}/httpd/conf.d
install -m 644 $RPM_SOURCE_DIR/squirrelmail.conf \
   $RPM_BUILD_ROOT%{_sysconfdir}/httpd/conf.d/

# install locales
mkdir locale_tempdir
cd locale_tempdir
tar xfj %SOURCE4
cd squirrelmail.locales
# Convert all locales to utf-8. Not only is this probably the right thing
# to do anyway, but SquirrelMail will corrupt charsets unless the charset
# of the user's locale is a superset of the charset of any mail they reply to
# https://sf.net/tracker/?func=detail&atid=423691&aid=1235345&group_id=311
for LOCALE in `ls locale/` ; do
    case $LOCALE in
	ja_JP)
	    # ja_JP uses iso2022-jp for email but euc-jp in its interface.
	    CHARSET=euc-jp
	    ;;
	ko_KR)
	    # Not really, but I can't work out what charset the ko_KR help
	    # files are in, so we'll just leave it alone for now.
	    CHARSET=utf-8
	    ;;
	*)
	    CHARSET=`grep CHARSET locale/$LOCALE/setup.php | cut -f6 -d\'`
	    ;;
    esac
    # Check for locales where CHARSET isn't in LOCALE.
    grep LOCALE locale/$LOCALE/setup.php | grep -vi $CHARSET  || :

    if [ "$CHARSET" != "utf-8" -a "$CHARSET" != "UTF-8" ]; then
	for a in `ls help/$LOCALE/ 2>/dev/null` ; do
	   iconv -f $CHARSET -t utf-8 help/$LOCALE/$a > $a.new && mv $a.new help/$LOCALE/$a
	done
	sed -e "s/CHARSET..[ ]*= [^;]*;/CHARSET'] = 'utf-8';/" \
	    -e "s/LOCALE..[ ]*= [^;]*;/LOCALE'] = '$LOCALE.UTF-8';/" \
	    locale/$LOCALE/setup.php  > setup.php.new ; mv setup.php.new locale/$LOCALE/setup.php
    fi
done

# do the pofiles separately since they each specify their own charset 
for POFILE in `find locale -name \*.po` ; do 
    CHARSET=`grep charset= $POFILE | cut -f2 -d= | cut -f1 -d\\\\`
    if [ "$CHARSET" != "utf-8" -a "$CHARSET" != "UTF-8" ]; then
	sed s/charset=$CHARSET/charset=utf-8/ $POFILE | iconv -f $CHARSET -t utf-8 > $POFILE.new && mv $POFILE.new $POFILE
    fi
done
./compilelocales
#find -name '*.mo' |xargs rm
cp -r locale/* $RPM_BUILD_ROOT%{_datadir}/squirrelmail/locale/
cp -r images/* $RPM_BUILD_ROOT%{_datadir}/squirrelmail/images/
cp -r help/* $RPM_BUILD_ROOT%{_datadir}/squirrelmail/help/
cd ../..
rm $RPM_BUILD_ROOT%{_datadir}/squirrelmail/locale/README.locales

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root)
%config %dir %{_sysconfdir}/squirrelmail
%attr(640,root,apache) %config(noreplace) %{_sysconfdir}/squirrelmail/*.php
%attr(640,root,apache) %config(noreplace) %{_sysconfdir}/squirrelmail/default_pref
%config(noreplace) %{_sysconfdir}/httpd/conf.d/*.conf
%doc doc/*
%dir %{_datadir}/squirrelmail
%dir %{_localstatedir}/lib/squirrelmail
%dir %{_localstatedir}/spool/squirrelmail
%{_datadir}/squirrelmail/class
%{_datadir}/squirrelmail/config
%{_datadir}/squirrelmail/functions
%{_datadir}/squirrelmail/help
%{_datadir}/squirrelmail/images
%{_datadir}/squirrelmail/include
%{_datadir}/squirrelmail/locale
%{_datadir}/squirrelmail/plugins
%{_datadir}/squirrelmail/src
%{_datadir}/squirrelmail/themes
%{_datadir}/squirrelmail/index.php
%attr(0700, apache, apache) %dir %{_localstatedir}/lib/squirrelmail/prefs
%attr(0700, apache, apache) %dir %{_localstatedir}/spool/squirrelmail/attach
%{_localstatedir}/lib/squirrelmail/prefs/default_pref
%{_sysconfdir}/cron.daily/squirrelmail.cron

%changelog
* Mon Sep 12 2005 David Woodhouse <dwmw2@xxxxxxxxxx> 1.4.6-0.cvs20050812.2
- Convert all locales to UTF-8 instead of legacy character sets to
  work around bug #162852. Except for ko_KR, because iconv doesn't
  believe its help files are actually in EUC-KR as claimed.

* Sun Aug 14 2005 Warren Togami <wtogami@xxxxxxxxxx> 1.4.6-0.cvs20050812.1
- snapshot of 1.4.6 because 1.4.5 upstream was a bad release
  this hopefully will also work on PHP5 too...

* Mon Jun 20 2005 Warren Togami <wtogami@xxxxxxxxxx> 1.4.5-0.rc1
- 1.4.5-0.rc1

* Thu Jan 27 2005 Warren Togami <wtogami@xxxxxxxxxx> 1.4.4-2
- 1.4.4
- re-include translations and Provide squirrelmail-i18n
  better compatible with upstream, but we cannot split sub-package
  due to support of existing distributions
- remove unnecessary .po files

* Fri Nov 19 2004 Warren Togami <wtogami@xxxxxxxxxx> 1.4.3a-7
- CAN-2004-1036 Cross Site Scripting in encoded text
- #112769 updated splash screens

* Thu Oct 14 2004 Warren Togami <wtogami@xxxxxxxxxx> 1.4.3a-5
- default_folder_prefix dovecot compatible by default
  /etc/squirrelmail/config_local.php if you must change it

* Wed Oct 13 2004 Warren Togami <wtogami@xxxxxxxxxx> 1.4.3a-4
- HIGASHIYAMA Masato's patch to improve Japanese support
  (coordinated by Scott A. Hughes).
- real 1.4.3a tarball

* Tue Sep 21 2004 Gary Benson <gbenson@xxxxxxxxxx> 1.4.3-3
- rebuilt.

* Tue Aug 31 2004 Warren Togami <wtogami@xxxxxxxxxx> 1.4.3-2
- #125638 config_local.php and default_pref in /etc/squirrelmail/
  to match upstream RPM.  This should allow smoother drop-in
  replacements and upgrades.
- other spec cleanup.

* Mon Jun  7 2004 Gary Benson <gbenson@xxxxxxxxxx> 1.4.3-1
- upgrade to 1.4.3a.
- retain stuff after version when adding release to it.

* Wed Jun  2 2004 Gary Benson <gbenson@xxxxxxxxxx>
- upgrade to 1.4.3.

* Fri Feb 13 2004 Elliot Lee <sopwith@xxxxxxxxxx>
- rebuilt.

* Wed Jan 21 2004 Gary Benson <gbenson@xxxxxxxxxx> 1.4.2-2
- fix calendar plugin breakage (#113902).

* Thu Jan  8 2004 Gary Benson <gbenson@xxxxxxxxxx> 1.4.2-1
- upgrade to 1.4.2.
- tighten up permissions on /etc/squirrelmail/config.php (#112774).

* Mon May 12 2003 Gary Benson <gbenson@xxxxxxxxxx> 1.4.0-1
- upgrade to 1.4.0.
- fix links in /usr/share/doc/squirrelmail-X.Y.Z/index.html (#90269).

* Mon Mar 24 2003 Gary Benson <gbenson@xxxxxxxxxx> 1.2.11-1
- upgrade to 1.2.11 to fix CAN-2003-0160.

* Mon Feb 10 2003 Gary Benson <gbenson@xxxxxxxxxx> 1.2.10-4
- fix syntax error in download.php (#82600).
- resized splash screen to be the same size as the one it replaces
  (#82790)
- remove piece of squirrelmail-1.2.10-xss.patch that changed the
  version from '1.2.10' to '1.2.11 [cvs]'

* Wed Jan 22 2003 Tim Powers <timp@xxxxxxxxxx> 1.2.10-3
- rebuilt

* Wed Jan 15 2003 Tim Powers <timp@xxxxxxxxxx> 1.2.10-2
- bump and rebuild

* Mon Dec  9 2002 Gary Benson <gbenson@xxxxxxxxxx> 1.2.10-1
- patch to fix CAN-2002-1341 (#78982) and CAN-2002-1276 (#79147).

* Tue Dec 03 2002 Elliot Lee <sopwith@xxxxxxxxxx> 1.2.8-2
- fix prep macro in changelog

* Fri Sep 20 2002 Gary Benson <gbenson@xxxxxxxxxx> 1.2.8-1
- upgrade to 1.2.8 to fix CAN-2002-1131 and CAN-2002-1132 (#74313)

* Tue Aug  6 2002 Preston Brown <pbrown@xxxxxxxxxx> 1.2.7-4
- replacement splash screen.

* Mon Jul 22 2002 Gary Benson <gbenson@xxxxxxxxxx> 1.2.7-3
- get rid of long lines in the specfile.
- remove symlink in docroot and use an alias in conf.d instead.
- work with register_globals off (#68669)

* Tue Jul 09 2002 Gary Benson <gbenson@xxxxxxxxxx> 1.2.7-2
- hardwire the hostname (well, localhost) into the config file (#67635)

* Mon Jun 24 2002 Gary Benson <gbenson@xxxxxxxxxx> 1.2.7-1
- hardwire the locations into the config file and cron file.
- install squirrelmail-cleanup.cron as squirrelmail.cron.
- make symlinks relative.
- upgrade to 1.2.7.
- more dependency fixes.

* Fri Jun 21 2002 Gary Benson <gbenson@xxxxxxxxxx>
- summarize the summary, fix deps, and remove some redundant stuff.
- tidy up the prep section.
- replace directory definitions with standard RHL ones.

* Fri Jun 21 2002 Tim Powers <timp@xxxxxxxxxx> 1.2.6-3
- automated rebuild

* Wed Jun 19 2002 Preston Brown <pbrown@xxxxxxxxxx> 1.2.6-2
- adopted Konstantin Riabitsev <icon@xxxxxxxx>'s package for Red Hat
  Linux.  Nice job Konstantin!

Attachment: signature.asc
Description: Ceci est une partie de message =?ISO-8859-1?Q?num=E9riquement?= =?ISO-8859-1?Q?_sign=E9e?=

-- 
fedora-devel-list mailing list
fedora-devel-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/fedora-devel-list

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]
  Powered by Linux