On 12/10/06, Linus Torvalds <torvalds@xxxxxxxx> wrote:
data into ASCII etc. So temp-files are almost never a better solution than
keeping things in memory (unless you use those temp-files to truly
_share_ data between processes, ie you do a shared mmap and they can
re-use the same pages actively in a way they couldn't otherwise).
Ok. Perhaps I'm doing something wrong but the following code it's
always 10% slower then the temporary file one (4.7s against 4.3s for
linux tree)
bool DataLoader::start(const QStringList& args, const QString& workDir) {
QDir::setCurrent(workDir);
_file = popen(args.join(" ").ascii(), "r");
if (!_file)
return false;
loadTime.start();
guiUpdateTimer.start(10, true); // will call on_timeout() in 10ms
return true;
}
void DataLoader::on_timeout() {
if (canceling)
deleteLater();
// int fd = fileno(_file); // read() case
ssize_t len = 0;
while (1) {
QByteArray* ba = new QByteArray(FILE_BLOCK_SIZE); // 64KB
// len = read(fd, ba->data(), ba->size()); // read() case
len = fread(ba->data(), 1, ba->size(), _file); // fread() case
if (len <= 0) {
delete ba;
break;
} else if (len < (ssize_t)ba->size()) // very rare, 4 out of 40000
on Linux tree
ba->resize(len);
loadedBytes += len;
fh->rowData.append(ba); // fh->rowData it's a pointer's list
parseSingleBuffer(ba);
// avoid reading small chunks if data producer is still running
if (len < FILE_BLOCK_SIZE)
break;
}
// if (len == 0) { // read() case
if (feof(_file)) { // fread() case
emit loaded(fh, loadedBytes, loadTime.elapsed(), true, "", ""); //
show some stats
pclose(_file);
_file = NULL;
deleteLater();
} else
guiUpdateTimer.start(100, true); // next read in 100ms
}
Uncomment 'read() case' lines and comment out the 'fread case()' ones
and you have a way slooooower code, about 10 times slower!
Marco
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html