Hi Kees,
I didn't test but try this :
String file_id = "testupload_" + Id.getNext("file_id");
try {
String config_file = "/etc/ceph/ceph.conf";
// nothing fancy here
String id = "admin";
String pool = "temp";
Rados rados;
IoCTX ioctx;
// make the connection
rados = new Rados(id) {
};
rados.confReadFile(new File(config_file));
rados.connect();
ioctx = rados.ioCtxCreate(pool);
// read bytes from the inputstream
int read = 0;
long bytesRead = 0;
byte[] bytes = new byte[8192];
while ((read = inputStream.read(bytes)) !=
-1) {
//truncate to really read data
byte[] real_buff = Arrays.copyOf(bytes, read);
ioctx.write(file_id, real_buff, bytesRead);
bytesRead += read;
}
// close the connection
rados.ioCtxDestroy(ioctx);
rados.shutDown();
} catch (RadosException ex) {
Logger.getLogger(ApiFunction_dvd_upload.class.getName()).log(Level.SEVERE,
null, ex);
}
I use Rados object for my Zimbra<->Ceph connector and I don't have segfault problems with rados-java.
Best regards,
----- Le 28 Déc 15, à 13:43, KeesBoog techie2015@xxxxxxxxxxxxxx a écrit :
> Hi Florent,
> Thanks.
> I did not consider 'plain Rados objects' and I am not sure how that
> would work. If you think that would solve the problem, perhaps you could
> elaborate?
> Would I still need Rbd to create a new image and set the size?
> In the example I replaced
> RbdImage image = rbd.open(file_id);
> image.write(bytes, bytesRead, read);
> with
> ioctx.write("rbd_id."+file_id, bytes, bytesRead);
> but
> a) the image-file was empty after the upload (so I am sure I am doing
> something wrong)
> b) the SIGSEGV-error was still there (after a few minutes)
> Best Regards,
> Kees
> Op 28-12-2015 om 12:33 schreef Florent Manens:
> > Hi Kees,
> > Why don't you use plain Rados objects instead of a RBD ?
> > Best regards,
> > ----- Le 28 Déc 15, à 11:54, KeesBoog techie2015@xxxxxxxxxxxxxx a écrit :
> >> Thanks Wido,
> >> Here is the example code, taken from a Java web application.
> >> There are no other calls to Rbd/Rados in the code.
> >> // code altered for sake of the example
> >> (everything in one try/catch and random file name and initial size)
> >> // objective: read binary file(s) from an
> >> inputstream to a ceph image
> >> String file_id = "testupload_" +
> >> Id.getNext("file_id");
> >> long initialsize = 1000000000L;
> >> try {
> >> String config_file = "/etc/ceph/ceph.conf";
> >> // nothing fancy here
> >> String id = "admin";
> >> String pool = "temp";
> >> Rados rados;
> >> IoCTX ioctx;
> >> Rbd rbd;
> >> // make the connection
> >> rados = new Rados(id) {
> >> };
> >> rados.confReadFile(new File(config_file));
> >> rados.connect();
> >> ioctx = rados.ioCtxCreate(pool);
> >> rbd = new Rbd(ioctx);
> >> // create the file with initial size
> >> (larger than actual size)
> >> int features = (1 << 0);
> >> rbd.create(file_id, initialsize, features, 0);
> >> // open the image
> >> RbdImage image = rbd.open(file_id);;
> >> // read bytes from the inputstream
> >> int read = 0;
> >> long bytesRead = 0;
> >> byte[] bytes = new byte[8192];
> >> while ((read = inputStream.read(bytes)) !=
> >> -1) {
> >> image.write(bytes, bytesRead, read);
> >> bytesRead += read;
> >> }
> >> // adjust size to the actual read bytes
> >> image.resize(bytesRead);
> >> // close the image
> >> rbd.close(image);
> >> // close the connection
> >> rados.ioCtxDestroy(ioctx);
> >> rados.shutDown();
> >> } catch (RadosException ex) {
> >> Logger.getLogger(ApiFunction_dvd_upload.class.getName()).log(Level.SEVERE,
> >> null, ex);
> >> }
> >> This all works.
> >> I uploaded a very small text-file.
> >> rbd info temp/testupload_1028
> >> rbd image 'testupload_1028':
> >> size 5069 bytes in 1 objects
> >> order 22 (4096 kB objects)
> >> block_name_prefix: rbd_data.1f61b774b0dc51
> >> format: 2
> >> features: layering
> >> About 40 seconds after the successful upload (the content equals the
> >> original content) I get an unexpected SIGSEGV-error as reported (without
> >> a stacktrace).
> >> After a restart of Tomcat I did a second upload with a somewhat larger file:
> >> rbd info temp/testupload_1030
> >> rbd image 'testupload_1030':
> >> size 41151 kB in 11 objects
> >> order 22 (4096 kB objects)
> >> block_name_prefix: rbd_data.1f61c62ae8944a
> >> format: 2
> >> features: layering
> >> About 3 minutes after the successful upload I got the same SIGSEGV-error.
> >> And a third (first restart Tomcat again):
> >> rbd info temp/testupload_1032
> >> rbd image 'testupload_1032':
> >> size 127 MB in 32 objects
> >> order 22 (4096 kB objects)
> >> block_name_prefix: rbd_data.1f61fc2ae8944a
> >> format: 2
> >> features: layering
> >> About 2 minutes after the successful upload I got the same SIGSEGV-error.
> >> Thanks again.
> >> Kees
> >> Op 28-12-2015 om 10:21 schreef Wido den Hollander:
> >>> On 12/28/2015 07:18 AM, KeesBoog wrote:
> >>>> Hi,
> >>>> I do love rados-java! I do get listings of ceph pools; I even can upload
> >>>> files to a ceph pool with a Tomcat web application. That is great!
> >>> Great to hear!
> >>>> However, I get frequent SIGSEGV errors (which halt Tomcat):
> >>> Ok, that is not good. Can you reproduce it with maybe some example code?
> >>> I would like to dig down to where this is happening, but if you are able
> >>> to reproduce it with some simple example code it would be great.
> >>> Or a stacktrace on which call it actually dies.
> >>> Wido
> >>>> #
> >>>> # A fatal error has been detected by the Java Runtime Environment:
> >>>> #
> >>>> # SIGSEGV (0xb) at pc=0x00007fd4bc54166d, pid=25308, tid=140554357282560
> >>>> #
> >>>> # JRE version: OpenJDK Runtime Environment (7.0_91-b02) (build
> >>>> 1.7.0_91-b02)
> >>>> # Java VM: OpenJDK 64-Bit Server VM (24.91-b01 mixed mode linux-amd64
> >>>> compressed oops)
> >>>> # Derivative: IcedTea 2.6.3
> >>>> # Distribution: Ubuntu 14.04 LTS, package 7u91-2.6.3-0ubuntu0.14.04.1
> >>>> # Problematic frame:
> >>>> # C [librados.so.2+0x28766d] Mutex::Lock(bool)+0xd
> >>>> #
> >>>> # Failed to write core dump. Core dumps have been disabled. To enable
> >>>> core dumping, try "ulimit -c unlimited" before starting Java again
> >>>> #
> >>>> # An error report file with more information is saved as:
> >>>> # /tmp/hs_err_pid25308.log
> >>>> #
> >>>> # If you would like to submit a bug report, please include
> >>>> # instructions on how to reproduce the bug and visit:
> >>>> # http://icedtea.classpath.org/bugzilla
> >>>> #
> >>>> Please find the file hs_err_pid25308.log attached. I could send a few more.
> >>>> It would be great if someone could point me to a workaround of perhaps
> >>>> even fix the issue.
> >>>> Best Regards,
> >>>> Kees
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Florent Manens
florent@xxxxxxxxx
09 72 33 25 68
06 76 29 87 83
skype:fmanens
http://vcard.manens.org
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
