Re: Characters in an uploaded text file being corrupted

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Ok, issue sorted - for now anyway - helps to get some rest, and then consider alternatives to a problem...<smile>

function morphString($sIn) {
$sOut = "";
for ($I = 0; $I < strlen($sIn); $I++) {
$sChr = $sIn[$I];
if (ord($sChr) >= 10 && ord($sChr) <= 126) {
$sOut .= chr(ord($sChr));
}
}//end of looping
return $sOut;
}//end of morphString function

I basically typed in a sort of test string including all keyboard characters I would want to allow/include, including things like \\ \" \n \r etc., as well as all capital letters, all punctuation marks, etc., and ran it through another looping function to determine max and min values for the ord() function on each character, and thus determined that anything ranging from 10 to 126 could be included, etc.

Stay well

Jacob Kruger
Blind Biker
Skype: BlindZA
'...fate had broken his body, but not his spirit...'

----- Original Message ----- From: "Jacob Kruger" <jacobk@xxxxxxxxxxxxxx>
To: <php-windows@xxxxxxxxxxxxx>
Sent: Monday, May 28, 2012 6:32 AM
Subject: Re:  Characters in an uploaded text file being corrupted


Thanks - will check that out, but only issue is that this will need end-users to try sort something like this out, and they won't necessarily be the most technical types as such, and, I also wouldn't have too much control over what they'd originally used to generate the content.

Am thinking that might be able to handle character set encoding processing a bit more simply using python, which is currently my other primary programming language, so might try out something like creating a bit of a cross platform utility app that they can try use initially to control something like this before uploading content, but, let's see...

Stay well

Jacob Kruger
Blind Biker
Skype: BlindZA
'...fate had broken his body, but not his spirit...'

----- Original Message ----- From: "Carl Roett" <carlroett@xxxxxxxxx>
To: <php-windows@xxxxxxxxxxxxx>
Sent: Sunday, May 27, 2012 6:47 PM
Subject: Re:  Characters in an uploaded text file being corrupted


Do not use Microsoft Word to save files that need to be read-in by other
software. Ever. It *always* finds a way to screw it up. Even if you cut and
paste it into the other program, I've heard cases where Word has put
unprintable control characters into the pasted output, that showed-up as
spaces in the form, then corrupted the output at runtime.

I suggest you open the saved file in NetBeans. In the NetBeans
settings dialogue, enable the options to "show control characters" and
"process file as unicode" (iirc). That will cause NetBeans to display all
characters your font set contains, display a square box for the ones it
doesn't, and display the various icons for control characters.

If column alignment in your file is important (for example a set number of spaces between tokens and operators), you should be using a monospace font.
The best one I've ever used is available here:
http://code.google.com/p/buddypress-media/downloads/detail?name=ttf-bitstream-vera-1.10.zip
It's also really easy to read.

For text streams in production site, take a look at BP-Media's sanitizer
classes. Unicode to ASCII to HTML entity conversion is a tricky business,
and these functions will ensure you get the right kind of output no matter what users throw at you. For example, people embedding unicode sequences in
an ASCII stream.
http://code.google.com/p/buddypress-media/source/browse/bp_media/trunk/core/database/class.database.sanitizers.php

^C^

===================================================



On Sun, May 27, 2012 at 8:57 AM, Jacob Kruger <jacobk@xxxxxxxxxxxxxx> wrote:

Ok, specifically told that storage field to make use of UTF8_unicode_ci,
and tried re-uploading text file that specifically saved from word using
UTF8 encoding, and, the text content is still full of garbage characters..?

Suppose might try something like copying/pasting from text file in
notepad, into something like a textarea field, but, don't think that would
always be the perfect process in this site.

Will try a couple of other options with regard to DB encoding, and see if
come right, but, let's see...

Jacob Kruger
Blind Biker
Skype: BlindZA
'...fate had broken his body, but not his spirit...'

----- Original Message ----- From: "Niel Archer" <
spam-free@xxxxxxxxxxxxxxxx>
To: <php-windows@xxxxxxxxxxxxx>
Sent: Sunday, May 27, 2012 4:22 PM
Subject: Re: Characters in an uploaded text file being corrupted


 Hi

Ok, and, FWIW, this test text file is a word document that have saved as
a
text file, and when now resaved it using specifically unicode encoding,
it
seemd to eliminate this issue,


Make sure the MySQL Db, table, field are correctly set for the encoding.
MySQL does not default to UTF-8 so you have to manually set it when
creating the Db/table. IIRC, newer versions can also have the encoding
set per field.

 but would have thought there might be a
relatively simple way to handle something like encoding conversion in PHP
itself..?


PHP cannot read minds, unfortunately. There are ways to handle encoding
conversions, but I don't think anyone would call them 'simple'. ;-)
See the Multibyte String extension for one way.

--
Niel Archer


--
PHP Windows Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php



--
PHP Windows Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php





--
PHP Windows Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php



--
PHP Windows Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php



[Index of Archives]     [PHP Home]     [PHP Users]     [PHP Database Programming]     [PHP Install]     [Kernel Newbies]     [Yosemite Forum]     [PHP Books]

  Powered by Linux