Ok. I had a response to this partially written, but it got eaten
somehow. I am very upset about this (I lost hours of work!!)
Calvin Williamson wrote:
I think I just dont understand your notion of "image".
Let me show you what I think they are with some specific
examples. Then maybe you can say if the way you are thinking
is different
ok. Yes, what I am thinking is different, it a subtle but very
important point. An image is a bounded region, composed of pixels. The
primary problem I see with your approach is that you are trying to treat
images like ops. Very early on (at gimpcon) you and pippin both made it
very clear that edges should be images and nodes should be ops. I
didn't quite understand the power of that statement when you made it,
but I went with it anyway, trusting that you two were right.
Now though, you seem to be doing something completely different. Let
try and explain:
Ill just show how you might set up a projection stack
for gimp using images and ops and gobject properties.
My viewpoint goes like this:
combine3
/ \
L3 combine2
/ \
L2 combine1
/ \
L1 P
Each combine is an GeglOp, everything else is "data" (not
a GeglOp). Other "data" besides the above image data
(eg mask image data) is not shown but each combine op will
have other kinds of data that feeds into it as well (opacities,
modes, or whatever)
Images are edges, not nodes.
So combine ops will have two "image" input properties:
These are two regular gobject style properties called
"source1" and "source2".
(Think source2 over source1 here, so source2 is foreground
and source1 is background)
Also each combine op has one "image" output property.
Again this is also a regular gobject style property and
is called "dest".
Now output properties are read-only gobject properties,
and input properties are read-write gobject properties.
The reason for this is because output properties are only
to be set by the corresponding op as a result of the
"evaluate" method. However inputs are obviously settable
properties of the op objects (and settable by g_object_sets)
Now you set up your ops like this:
GeglNode *combine1 = g_object_new(GEGL_TYPE_COMBINE_OP,
"source1", P,
"source2", L1,
NULL);
GeglNode *combine2 = g_object_new(GEGL_TYPE_COMBINE_OP,
"source2", L2,
NULL);
GeglNode *combine3 = g_object_new(GEGL_TYPE_COMBINE_OP,
"source2", L3,
NULL);
So far all the "source2" properties are hooked up. Now
hook up the output properties for each node to the
appropriate input properties of the combines like this:
gegl_node_connect(combine2, "source1", combine1, "dest");
gegl_node_connect(combine3, "source1", combine2, "dest");
ok. The first problem I have with your example is that you are clearly
treating images and nodes differently in your api, but your diagram
makes no distinction.
Let me try to adjust your example to better fit what your api seems to
be doing.
My format will be:
opname
-prop
-prop
-------
| connection
combine3
-source1:L3
------------
\
combine2
-source1:L2
-----------
\
combine1
-source1:L1
-source2:L2
-----------
This distinction is necessary because clearly you are saying that images
and ops are different beasts.
This example doesn't do your api justice however, because you have the
problem that sometimes, your property is an image, and sometimes your
property is an op, and they are treated differently
This tells the combine2 and combine3 ops that when they do their
"evaluate" method, they should get the input property called
"source1" from the output property of the corresponding node they
are connected to along the source1 line (in this case
they are connected to the "dest" output property of another
combine op)
Now you call
gegl_node_apply(combine3, "dest");
and pass it the name of the output property you want it to compute
(You should also set a ROI on this output property as well before
doing this, and that is left out here too)
There is no reason that an op has to be associated with a particular
ROI. In fact, multiple users may want different ROI's computed from the
op simultanously! Clearly, the ROI should be assocated with the
resultant image.
Here, let me try and start my own example now. Lets start with yours:
combine3
/ \
L3 combine2
/ \
L2 combine1
/ \
L1 P
Only this time, let L3, L2, L1 and P are leaf nodes. (they could be
files, frame buffers, networks, or a v4l or GStreamer source).
L3, L2, and L1 are allocated with (as an abstract example):
g_buffered_node_new("inital_image",gegl_image1, "origin", point1);
and nodes are connected in a way similar to what you described. This
time the diagram accuartly represents the connections between nodes.
Now, every time someone wants a particular region as an image they call
something like:
GeglImage* gegl_graph_get_image(roi_rect, other_rendering_parameters
(antialising hints, etc)).
And a GeglImage object is immediately returned. This GeglImage object
doesn't actually contain tiles, it just behaves like a BufferImage, and
retrieve tiles from the underlying op as soon as you request a tiles using:
gegl_image_get_tile(image,tileX,tileY,hints)
Where hints can contain information like, you are going to need a
certain sized neighborhood, or the pattern you are planning to iterate
over the image, which lets you mimimize latency and take advantage of
parallel hardware. The roi is implicit in the size of the image that
you requested. I am going to try to go over the advantages of this api
one by one.
-Sytactic equivlence (Differences between ops and images)
First of, this method means there is no sytactic difference between
files, network sources, and in-memory tiles. It makes the node
construction api simplier.
Further, an op can (though it doesn't have to) do away with things like
bounds. And image, when connected to an op, has bounds, a specific
context in which it was rendered (say, with antilaising, a resampling
algorithm, or with a particular resolution), to which the op should know
nothing about.
-Ease of passing hints
There is a well defined way of passing hints to the rendering system.
The hints are able to be passed exactly when they are needed (and thus,
when they are known).
-multiple simultanous renderings
Using this method, a user could request several images from the graph
(essentially queueing up multiple renderings) and the rendering system
can take care of minimizing the amount of recaculations needed. This is
especially useful, because it is trivial to extend this api to support
resolution independence. (simply allow one to specify the rendering
context to the get_image function).
-asyncronous ops (latency)
When you have operators that are acting asyncronously (say, they are in
a different process, on a different thread, or over the network) it is
important to minimize the latency to getting the tiles to the node.
This is because the op can start processing input before the child nodes
are entirely finished with the image. (using your model, the image
needs to be totally calculated before the next op can being processing it).
-tile scheduling
Tile scheduling becomes easier because you can be sure that when a tile
appears in the queue, with its hints, you know that someone specifically
requested that tile and the direct prerequisites of the tile appear
right next to it in the queue. If you are calculating entire images at
once the prerequistes for a tile are not gaurenteed to be "close-to" its
prerequisits, so even if you tried to bolt on some kind of tile
scheduler, it would have to do a lot of work figuring out the prereqs of
a tile to get those tiles out as quickly as possible.
-algebra
Not really applicable to your specific example, but what happens when
you try to do algebra? Using your example, you need to make special
cases for images because images might be stored in properties to nodes.
-lower memory usage
When doing multiple renderings are needed your memory usage is much
lower, because the multiple renderings can be scheduled simultanously.
Now your result is sitting in the op's "dest" output property when
you are done.
What happens if you want another rendering to be performed by the node?
So in this view images are just another kind of data moving through the
graphs properties.
no, in this view, images are some kind of pseudo-node, and anything that
deals with nodes is going to need to make all kinds of special cases in
case its node is actully an image.
Now we certainly arrange for the set of combine ops to either operate
in-place (so each dest is actually just the projection GeglImage
P passed at the beginning) or you can have each op create dest
images as it goes along the way and have those results cached,
so that if a layer changes, then the input property that corresponds
to that data is written as dirty and this propagates up the graph
to the root, and all the properties above that one are set dirty.
In-placeness is not really something that you need to worry about.
Since the tiles are copy-on-write, just make every node do things
"in-place" and if it needs to be copied (e.g. someone else is going to
use the tile) it will be. And of course, property changes need to be
propaged up the graph, but that is easy, because nodes know about their
parents.
Anyway underneath the ops are asking images for their tiles
so and reading and writing to them, and placing pieces of
the images in caches and listening for changes to that same data so
that they can mark properties dirty if they change in some way.
only, an op should not be caching it's inputs, only it's outputs. You
are implying that the ops are caching inputs (and probably outputs too)
since there is no way for the image it has to communicate with the
underlying op. Caching inputs is inefficient. First off, the op
doesn't have any real control over its inputs, so it shouldn't be in
charge of caching them. Second, caching inputs is inefficent, since a
single op may have multiple outputs and each of those outputs would be
cahing the same data.
Is this very different from the way you are thinking?
Yeah, pretty different.
No, I think I meant what I said. GimpImage is a stack of GimpLayers,
conceptually, and a GimpLayer is a combition of a GimpDrawable and some
associated blending info. For all the reasons above, nothing should
keep around a GeglImage unless it is actually processing that region (as
this locks up some resources). Thus, GimpImage maps to a tree of ops.
GimpLayer maps to a blend op and a leaf op. A GimpDrawable maps to a
leaf node, which probably just contains a bunch of tiles (a la' a
gegl-buffered-image, but with all the op stuff connected to it).
I dont see why there is a need for leaf nodes. And how is the
drawable the output of a painthit if it is a gegl-buffered-image,
unless you additionally copy to and from it to get your paint hit
on the drawable.
because a layer is not always a collection of pixels. Sometimes it is a
text layer, or a vector layer, or an adjustment layer, or an effect
layer. Ignore that thing about gegl-buffered-image. It was confusing
and not what I meant.
There certainly is a need for there to be a certain kind of
image that is capable of participating in operator caches, yes,
but wont you want to have many different ops writing to a
drawable?
Right, and the best way to handle that is with an op that can produce
multiple, writable images, or that has procedural drawing calls.
Requsting an image from an op is a way of specifing that you are
interested in manipulating that region. The op can handle all the magic
of locking minimal regions.
| I think most of the work in bringing in a new image type is
| related to replacing TileManagers and PixelRegions code all over
| gimp to use GeglImages and GeglOps instead.
Why do you think that?
I just mean the the typical place where an op in gimp
is set up looks like this usually:
PixelRegion src1PR;
PixelRegion src2PR;
PixelRegion destPR;
pixel_region_init(&src1PR, tile_manager1, x,y,w,h);
pixel_region_init(&src2PR, tile_manager2, x,y,w,h);
pixel_region_init(&destPR, dest_tile_manager, x,y,w,h);
blah_region(&src1PR, &src2PR, &destPR ...);
Then you have the result in dest tile manager when you are done.
All these places in the code have to be changed to set up an
op and then apply the op instead of what they do now.
yeah, but all the code that uses this stuff needs to change too.
And there is more to getting the gimp to use gegl than just getting gegl
squeezed into the gimps processing model. To really take advantage of
some of the cool stuff we can do with gegl, the gimps processing model
needs to change (otherwise, what is the point of gegl?)
Beyond all this, the methods I am trying to outline here consider things
like, networked (eg. beowulf) processing, multiple processors, real-time
image sources, low latency processing, resolution independence. Now I
don't expect all these features to be implemented right away, but when
they are, I don't think the api will need to change all that much.
--
Daniel
