On 07/03/2014 09:48 PM, Lists wrote:
> On 07/03/2014 12:19 PM, John R Pierce wrote:
>> you do realize, adding/removing or even changing the length of a single
>> line in a block of that pg_dump file will change every block after it as
>> the data will be offset ?
>
> Yes. And I guess this is probably where the conversation should end. I'm
> used to the capabilities of Mercurial DVCS as well as ZFS snapshots, and
> was thinking/hoping that this type of capability might exist in a file
> system. Perhaps it just doesn't belong there.
>
> On 07/03/2014 12:23 PM, Les Mikesell wrote:
>> But, since this is about postgresql, the right way is probably just to
>> set up replication and let it send the changes itself instead of doing
>> frequent dumps.
>
> Whatever we do, we need the ability to create a point-in-time history.
> We commonly use our archival dumps for audit, testing, and debugging
> purposes. I don't think PG + WAL provides this type of capability. So at
> the moment we're down to:
>
> A) run PG on a ZFS partition and snapshot ZFS.
> B) Keep making dumps (as now) and use lots of disk space.
> C) Cook something new and magical using diff, rdiff-backup, or related
> tools.
>
Check out 7z from p7zip package. I use command:
7za a -t7z $YearNum-$MonthNum.7z -i@xxxxxxxxxxx -mx$CompressionMetod
-mmt$ThreadNumber -mtc=on
for compressing a lot of similar files from sysinfo and kernel.log,
files backuped every hour that do not change much. And I find out that
it reuses already existing blocks/hashes/whatever and I guess just
reference them with a pointer instead of storing them again.
So, 742 files that uncompressed have 179 MB, compressed ocupy only 452
KB, which is only 0.2% of original size, 442 TIMES smaller :
Listing archive: 2014-03.7z
--
Path = 2014-03.7z
Type = 7z
Method = LZMA
Solid = +
Blocks = 1
Physical Size = 426210
Headers Size = 9231
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------
------------------------
2014-03-01 00:02:07 ....A 259517 416979
Silos-Srbobran-l5-kernlog-2014-03-01-00-02-07.txt
2014-03-01 01:01:52 ....A 259529
Silos-Srbobran-l5-kernlog-2014-03-01-01-01-52.txt
...............................................
...............................................
...............................................
2014-03-31 22:01:33 ....A 259502
Silos-Srbobran-l5-kernlog-2014-03-31-22-01-33.txt
2014-03-31 23:01:33 ....A 259485
Silos-Srbobran-l5-kernlog-2014-03-31-23-01-32.txt
------------------- ----- ------------ ------------
------------------------
184553028 416979 742 files, 0 folders
Maybe, if you compress them into the same file (afterwords?) you can get
the similar result.
--
Ljubomir Ljubojevic
(Love is in the Air)
PL Computers
Serbia, Europe
StarOS, Mikrotik and CentOS/RHEL/Linux consultant
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
