Hi David, Using bucket notifications sounds like the right approach here. One option with the policy would be to move the suspicious objects to a quarantine bucket, so they are not lost in case of false positives, but not harmful either. I did something similar, but with inline processing using Lua scripting, see: https://www.youtube.com/watch?v=57-gHFuVb7M The script itself does the routing of the object to the quarantine bucket, and the suspicious object never reaches the original bucket. The downside with this approach is that you would need to find a virus scanner in lua (or at least wrapped with lua) and that it might slow down the upload process, as everything is done inline. Note that the bigger value of using the inline approach is if you want to scan objects that are downloaded from the system - a case where bucket notifications won't help. Yuval On Tue, Nov 8, 2022 at 5:21 PM Dave Holland <dh3@xxxxxxxxxxxx> wrote: > Hi, > > Historically our RGW S3 service has been mostly used by staff to > publish data to the world and external partners. However there's an > upcoming project to allow external partners/collaborators to upload > bulk data to us. Depending on the trust relationship with the > uploaders, I can imagine scenarios in which we might want to check > that what's been uploaded is not malicious or illegal. Is anyone doing > anything like that currently? I am imagining using bucket PUT > notifications to trigger a virus scanner or fuzzy match; but apart > from the technical side there are policy concerns too -- e.g. what > should happen when there's a hit? I'd love to hear from others who > are doing or considering this. > > Thanks, > Dave > -- > ** Dave Holland ** Systems Support -- Informatics Systems Group ** > ** dh3@xxxxxxxxxxxx ** Wellcome Sanger Institute, Hinxton, UK ** > > > -- > The Wellcome Sanger Institute is operated by Genome Research > Limited, a charity registered in England with number 1021457 and a > company registered in England with number 2742969, whose registered > office is 215 Euston Road, London, NW1 2BE. > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |