Hard link cutter [was: Suggestion: High speed cutting]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Udo Richter wrote:
> I've been thinking about video cutting strategies, and I think its 
> possible to speed up the cutting process noticeable, with some 
> modifications to VDR.

Hi list,

I just had to do this in self-reply. ;)
For reference, this is my 'original' post:
http://linvdr.org/mailinglists/vdr/2004/12/msg00552.html

This has been in my mind for over two years, and now I just wanted to 
see it working: Using hard file system links to speed up editing 
whenever a 00x.vdr file is copied from source to destination recording 
without modification.

A first patch is available on my web page:
http://www.udo-richter.de/vdr/patches.html#hlcutter
http://www.udo-richter.de/vdr/patches.en.html#hlcutter

Though the patch has been working for a week for me very well, it is 
HIGHLY EXPERIMENTAL and should be used with caution. The patch is 
COMPLETELY UNTESTED and NOT ADAPTED for multiple /videoxx folders. The 
safety checks should prevent data loss, but hard linking may fail more 
often than necessary.


While editing a recording, the patch searches for any 00x.vdr files that 
dont contain editing marks and would normally be copied 1:1 unmodified 
to the edited recording. In this case the current target 00x.vdr file 
will be aborted, and the cutter process attempts to duplicate the source 
file as a hard link, so that both files share the same disk space. If 
this succeeds, the editing process fast-forwards through the duplicated 
file and continues normally beginning with the next source file. If hard 
linking fails, the cutter process continues with plain old copying. (but 
does not take up the aborted last file.)

After editing, the un-edited recording can be deleted as usual, the hard 
linked copies will continue to exist as the only remaining copy.

To be effective, the default 'Max. video file size (MB)' should be 
lowered. The patch lowers the smallest possible file size to 10mb, 
though this limits the recording length to 10*255 Mb (~100 minutes). A 
value of 100mb is a good compromise between speed improvement and 
recording length (~16 hours).

The patch must be enabled in Setup-> Recordings-> Hard Link Cutter. When 
disabled, the cutter process behaves identical to VDR's default cutter.

There's a //#define HARDLINK_TEST_ONLY in the cutter.c file that enables 
a test-mode that hard-links 00x.vdr_ files only, and continues the 
classic editing. The resulting 00x.vdr and 00x.vdr_ files should be 
identical. If you delete the un-edited recording, dont forget to delete 
the *.vdr_ files too, they will now eat real disk space.

Note: 'du' displays the disk space of hard links only on first 
appearance, and usually you will see a noticeably smaller size on the 
edited recording.

Future plans
------------

To solve the file size vs. recording size conflict, dynamic file sizes 
could be implemented, so that a recording starts with small file sizes, 
and increases the file size at some point to ensure enough space for 
huge recordings before 255.vdr is reached. For example, using 32Mb up to 
192.vdr and 2000mb from 193.vdr on will give a total of 128 Gb or 84 
hours, while using small files for up to 4 hours.


To support multiple /videoxx folders, the hard link must be placed on 
the same disk as the source file. This requires a more advanced linking 
strategy.


Since original and edited copy share disk space, free space is wrong if 
one of them is moved to *.del. Free space should only count files with 
hard link count = 1. This still goes wrong if all copies get deleted.


For more safety, the hard-linked files may be made read-only, as 
modifications to one copy will affect the other copy too. (except 
deleting, of course)


SetBrokenLink may get lost on rare cases, this needs some more thoughts.


Cheers,

Udo


[Index of Archives]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [Xorg]     [Util Linux NG]     [Xfree86]     [Big List of Linux Books]     [Fedora Users]     [Fedora Women]     [ALSA Devel]     [Linux USB]

  Powered by Linux