On Thu, 2006-08-03 at 15:59 +0200, hendrich@xxxxxxxxxxxxxxxxxxxxxxxxx wrote: > Hello all, > > anyone interested in torturing our Swing, awt.image and javax.image operations > a bit? Long text with a question buried at the end. Sorry, but I had > to get this > off my soul; feeling much better already :-) No problem :) Anyone's who's looked at our awt.image code is probably just as frustrated. :) The way I see it, this is the last major piece of work that needs to be done for the whole Java2D bit. (or AWT as a whole even) The rest is fairly easy stuff. [Snip: How to run the program] > 4a. Loading typical digicam images with 4+ Mpixels is somewhat slower, > but still > almost acceptable. jamvm needs about 5 seconds for a 3000x2000 JPEG for > image loading, plus about 7 seconds for calculating the histogram > (when enabled). > JDK 1.5 needs about 1.5 seconds for loading plus 0.3 secs for the > histogram. The discrepancy is rather large, since the actual loading in this case (using Toolkit.createImage) is done natively. I'm guessing most of the overhead is coming from any and all java processing we do afterwards. > 4b. Cacao calculates the histogram much faster (almost as fast as the > JDK), but > unfortunately it leaks memory and crashes after loading a few > 4+Mpixel images > (with -Xmx300m and top reporting about 320M RSS actually used.) > > It seems that cacao 0.96 never garbage-collects image data? Well, it should. But we should check up on that. See what Herr Thalinger has to say. > For the following, use smaller images (800x600 or so) to avoid frustration. > > 5. Select Tools > Sharpen > Laplace 3x3. Simple convolution filter implemented > with ConvolveOp and applied to a BufferedImage TYPE_INT_RGB. > > java.lang.ArrayIndexOutOfBoundsException: 3 > at java.awt.image.ColorModel.getComponentSize(ColorModel.java:200) > at java.awt.image.ColorModel.coerceData(ColorModel.java:641) > at java.awt.image.DirectColorModel.coerceData(DirectColorModel.java:405) > at java.awt.image.BufferedImage.coerceData(BufferedImage.java:288) > at java.awt.image.ConvolveOp.filter(ConvolveOp.java:126) Looks like a regression. Overall we're making progress on awt.image though, mainly thanks to Mr Gilbert, at the moment. > This worked a week ago, but very slowly. Try the Tools > Edges > Mexican > Hat 13x13 filter, if you don't believe me. The JDK seems to include some > optimizations for such (separable?) kernels. Yes, this is one of those things which is left to do with awt.image. There's a lot of optimization that can and should be done. > 6. Select Tools > Negative Image. Obvious implementation based on > LookupOp. Works. Performance is ok (1 sec for 800x600). The result is > a BufferedImage.TYPE_INT_RGB. > > BUT repainting suddenly takes 3 seconds for each paintComponent, > and the application is pretty much dead. For comparison, a repaint of the > BufferedImage before the filtering took about 10 msec. Right. BufferedImages created with Component.createImage(int, int) are backed by a Cairo surface. Ones created directly by BufferedImage are not. Again, this is one of these things that needs to be fixed, basically so that all BufferedImages can have some native backing buffer.) > For 3000x2000 images, each repaint takes 40 seconds on my system. > Any ideas about what I am doing wrong here are HIGHLY appreciated. Nothing really, we're to blame here (Well, Cairo is a bit slow too. But not _that_ slow.) Since we can only draw Cairo surfaces, non-CairoSurface backed BufferedImages will draw much slower, as this requires transferring (and possibly converting all) the pixels. > Load a new image. Repainting time is back to the millisecond range. Right, it's a CairoSurface-backed buffer then too. > > 7. Perhaps LookupOp and ConvolveOp are bad? Select Tools > Rotate > image left (or right). Implemented 'by hand' via getRGB and setRGB. > Much slower than LookupOp, about 4 seconds on my system at 800x600. They can probably be faster, but I don't think they're the main problem if the times you're measuring also include the repaint time. > But again, repainting suddenly takes many seconds. Same explanation as before. > Edit > Load images via ImageIO. > > Loading a 800x600 JPEG takes about 200 msecs with Toolkit, and about > 7 seconds with Toolkit. Loading a 3000x2000 JPEG takes 200+ seconds. Too slow. > The imageio GIF reader is much faster (4 seconds at 3000x2000), but now > the conversion to BufferedImage.TYPE_INT_RGB takes 90+ seconds... Really? That sounds too fast, almost. How do you mean conversion? Running it through a filter, or just drawing the image returned by IIO onto the the aforementioned BufferedImagE? > Images returned by the PNG reader render as transparent. > > 9. Select Help > Commands... A simple JTextArea in a JScrollPanel, but > with about 700 lines of text. Try scrolling. Painfully slow. The vertical > scrollbar only scrolls down on click event (but dragging works). I'll have to check this out. drawString is fairly slow nowadays. However, drawGlyphVector (and by extension TextLayout.draw) are quite snappy, comparable to the JDK in speed. We might need to tune Swing here. > I also get a lot of these: > > (.:2449): GdkPixbuf-CRITICAL **: gdk_pixbuf_new: assertion `height > > 0' failed > java.lang.InternalError: GdkPixbuf: gdk_pixbuf_new: assertion > `height > 0' failed Hmm, I "thought* we'd fixed this. It's probably a Swing regression. Or the fix didn't work well enough. At öeast the cause is obvious; you can't create a > 10. Neither JOptionPane (Help > About) nor JToolTip include support for > HTML formatting (e.g. the histogram tooltip). Audrius told me > that the HTML > parser part already works for my examples, but the parser isn't used... Yes, Swing.text needs lots of love, too. > Overall, I am quite happy about the current state of Swing. JFileChooser > obviously needs a lot of work, Agreed. I fixed a few bugs in it recently, and I think Roman fixed one or two more. Is this on the current version? > and JTextArea is unuseable when it holds more than a few lines of text. Due to speed as you said above?? > However, what should I do about the repaint performance issue? Is there > an inherent reason why images of type BufferedImage.TYPE_INT_RGB > render so slowly? (I also tried INT_ARGB, but this doesn't help, needs > more memory, and needs postprocessing for the ConvolveOp filters). Well, I think I explained this, but the bottom line here is that to paint these BufferedImages, we need to retrieve the pixel data, which is slow, and then possibly convert the pixel format, which is also slow. The solution we're going for here is for BufferedImage to create/retrive a CairoSurface-backed image whenever possible. With that in place, oimage-drawing speed is pretty much all dependent on Cairo. It also helps if Cairo can work directly with as many pixel formats as possible. This seems to be moving now; as of the last 1.2 release they'd now added support for 565 16-bit RGB (previously only ARGB32 and RGB24 have been supported.) They're also allegedly working more on speed now. So we need to fix our impl to exploit Cairo more, and there are some other hacks to make getting and converting the pixel data faster. I'm hoping for this to be done or mostly done by the release after the pending one. > Any obvious workaround? Well, one at least; which is that you should use BufferedImages created with Component.createGraphis(int, int) as much as possible if you want it to draw them quickly. Perhaps even better you could use VolatileImage (== an X pixmap). These objects are slower to draw onto, but render to the screen as fast as they possibly could, really. (Assuming X is doing a good job) So if your intention is to draw-onto-bitmap-once, display-bitmap-many, then it should make things many many times faster. /Sven
