Goat Invasion in GIMP

Once upon a time, like 5 weeks ago, there used to be the longstanding plan to, at some point in the future, port GIMP to GEGL.

We have done a lot of refactoring in GIMP over the last ten years, but its innermost pixel manipulating core was still basically unchanged since GIMP 1.2 days. We didn’t bother to do anything about it, because the long term goal was to do all this stuff with GEGL, when GEGL was ready. Now GEGL has been ready for quite a while, and the GEGL porting got assigned a milestone. Was it 2.10, 3.0, 3.2, I don’t remember. We thought it would take us forever until it’s done, because nobody really had that kind of time.

About 5 weeks ago, I happened to pick up Øyvind Kolås, aka Pippin the Goatkeeper to stay at my place for about a week and do some hacking. After one day, without intending it, we started to do some small GEGL hacking in GIMP, just in order to verify an approach that seemed a good migration strategy for the future porting.

The Problem: All the GimpImage’s pixels are stored in legacy data structures called TileManagers, which are kept by high level objects called GimpDrawables. Each layer, channel, mask in GIMP is a GimpDrawable.

A typical way to do things is:

TileManager *tiles = gimp_drawable_get_tiles (drawable);
PixelRegion region;

pixel_region_init (&region, tiles, x, y, w, h, TRUE);

/* do legacy stuff on the pixel region in order to change pixels */

After the GEGL porting, things would look like that:

GeglBuffer *buffer = gimp_drawable_get_buffer (drawable);

/* do GEGL stuff on the buffer, like running it through a graph in order to change pixels */

Just, how would we get there? Replacing the drawable’s tile manager by a buffer, breaking all of GIMP at the same time while we move on porting things to buffers instead of tile managers? No way!

The Solution: A GeglBuffer’s tiles are stored in a GeglTileBackend, and it’s possible to write tile backends for arbitrary pixel storage, so why not write a tile backend that uses a legacy GIMP TileManager as storage.

After a few hours of hacking, Pippin had the GimpTileBackendTileManager working, and I went ahead replacing some legacy code with GEGL code, using the new backend. And it simply worked!

The next important step was to make GimpDrawable keep around a GeglBuffer on top of its TileManager all the time, and to add gimp_drawable_get_buffer(). And things just kept working, and getting easier and easier the more legacy code got replaced by GEGL code, the more GeglBuffers were being passed around instead of TileManagers and PixelRegions.

What was planned as a one week visit turned into 3 weeks of GEGL porting madness. At the time this article is written, about 90% of the GIMP application’s core are ported to GEGL, and the only thing really missing are GeglOperations for all layer modes.

As a totally unexpected extra bonus, there is now even a GEGL buffer tile backend in libgimp, for plug-ins to use, so also plug-ins can simply say gimp_drawable_get_buffer(drawable_ID), and use all of GEGL to do their stuff, instead of using the legacy pixel region API that also exists on the plug-in side.

GIMP 2.10’s core will be 100% ported to GEGL, and all of the legacy pixel fiddling API for plug-ins is going to be deprecated. Once the core is completely ported, it will be a minor effort to simply “switch on” high bit depths and whatever color models we’d like to see. Oh, and already now, instead of removing indexed mode (as originally planned), we accidentally promoted indexed images to first class citizens that can be painted on, and even color corrected, just like any other image. The code doing so doesn’t even notice because GEGL and Babl transparently handle the pixel conversion magic.

The port lives in the goat-invasion branch in GIT. That branch will become master once GIMP 2.8 is relased, so the first GIMP 2.9 developer release will already contain the port in progress.

If you want to discuss GIMP and GEGL things with us face to face, join us at this year’s Libre Graphics Meeting in Vienna, in two weeks from now, a lot of GIMP people will be there; or simply check out the goat-invasion branch and see the goats yourself.

If you have some Euros to spare, consider donating them to Libre Graphics Meeting, it’s one of the few occasions for GIMP developers, and the people hacking on other projects, to meet in person; and such meetings are always a great boost for development.

During the 3 crazy weeks, quite some work time hours were spent on the port, thanks to my employer Lanedo for sponsoring this via “Labs time”.

65 Responses to “Goat Invasion in GIMP”

  1. Zenkibou Says:

    Great work !
    Glad to see work has been done on the GEGL front !

  2. Janne Says:

    Thank you! Thank you, thank you, thank you!

  3. Francesco Riosa Says:

    simply amused, and happy, especially because this will allow 16bit per channel.
    Congratulations

  4. Tobias Says:

    As much as I like to see an GEGL enabled Gimp and the stuff you two had done is really really great, wouldn’t it be more impotent to first get Gimp 2.8 ready?

  5. Markus Says:

    Thank you very much for this work. It is highly appreciated!

    Are there any effects on GIMPs performance?

  6. Raul Diaz Says:

    Before this post, I was very frustrated that the GIMP 2.8 release seemed to slip and slip. Now I understand what you are doing. Photographers desperately need 16-bit per color, so your keep up the amazing, amazing work.
    Best, Radzfoto.

  7. Ejnaren Says:

    I cant express how seriously cool you guys are.
    I too have wondered how long it would take to get GEGL into gimp when it has taken so long to get 2.8 out the door. And then you guys go and port most of it over in weeks..!? Madness is right… :)
    It must feel very good as well making it easier to work with gimp with a better core?
    Congratulations and looking forward to LGM to hear from all of you.

  8. Øyvind Kolås Says:

    @tobias: The code of GIMP-2.8 was feature frozen when the goat-invasion started. What keeps it back is testing and packaging, by users and packagers on multiple distros - along with filed bugs and patches to fix issues arising, as well as perhaps a splash that people agree upon.

    IMHO though, getting GEGL integrated was important than releasing GIMP-2.6 The goats have been waiting to be properly let in and rule GIMP since before that release, and even back then that plan was many years overdue.

  9. David Says:

    Tobias, what a great idea! I’m sure the team are excitedly awaiting your first patch.

  10. aury88 Says:

    I just stopped to wait the 2.8 release and started to look forward for the future 2.9/2.10…thank you very much guys! … XD ;)
    great work!

  11. Reed Solomon Says:

    That’s awesome.

  12. Federico Mena Quintero Says:

    This is *fantastic* news! It seems you reached the point where the initial refactoring is really hard, but then something happens and the code starts writing itself. Can’t wait for the goatified Gimp!

  13. Mitch Says:

    Tobias: And waste the opportunity to hack together on something that’s really fun? Absolutely not :)

  14. Dom Lachowicz Says:

    Way to go, guys!

  15. Anonymous Says:

    Thank you and Øyvind Kolås, and all the GIMP/GEGL/babl devs for all your work!

  16. Thomas Eriksson Says:

    I for one, welcome our 16 bit GEGL enabled Goat overloards!
    Great hacking guys!

  17. Natterer: Goat Invasion in GIMP | Linux-Support.com Says:

    […] his blog, Michael Natterer writes about some major progress in making GIMP work with the Generic Graphics Library (GEGL), which will […]

  18. eric-yorba Says:

    Awesome news! Can’t wait for the new version of Gimp…

  19. Michael Mol Says:

    Will a full port to GEGL allow me to load in DNG files with full support?

    Say, I loaded a DNG file, performed color correction, and then exported as JPEG. Would the JPEG include all the EXIF metadata the DNG file had?

  20. Mitch Says:

    @Michael Mol: No, GEGL porting and metadata handling are completely unrelated.

  21. troubled Says:

    Woot, nice accidental work there pippin, congrats!

    So…when are you gonna merge GIMP into Blender? ;) jk, cya o/

  22. Progress by Accident | Meet the GIMP! Says:

    […] my way to bed I stumbled across a blog post by Michael Natterer (Mitch), one of the core developers of GIMP. Øyvind Kolås (Pippin) and he […]

  23. Aldi Says:

    Thanks a lot! This seems to be the best start to kick-off gimp again. I hope that you can now easily identify EasyHacks for newcomers to the project. LibreOffice did that quite successfully. I hope Gimp manages this, too! I really looking forward to testing 2.9.X

  24. antistress Says:

    Hi, i’ve donated $40.00 to Libre Graphics Meeting as suggested since i really like Gimp and want to support your work :-)
    Thank you !

  25. Helvetix Victorinox Says:

    Great, great job guys. Excellent.

  26. Mike R Says:

    Wow pal,
    Truly superb penmanship for a programmer. A fine example of when stereotypes aren’t always true. Congratulations on deciphering the surely monumental task which lied ahead of you. And gratitude for continuing the advancement of the lovable GIMP project forward.

  27. Joe Fidler Says:

    Really really really good news - well done!

  28. Matthias Urlichs Says:

    That’s great work. Too bad nobody had this brilliant idea five years ago – but then again, I probably should be thankful that you had it at all. ;-)

  29. ds Says:

    Thank you so much Mitch and pippin. :)
    Have just donated 50 bux.
    Keep up the good work!

  30. Anonymous Coward Says:

    Amazing, thanks for all your work. I’ve been using GIMP and I prefer it a lot to other solutions (Photoshop, etc).

    BTW, not to be nitpicky but you misspelled some words:

    stategy -> strategy
    lagacy -> legacy
    accidentially -> accidentally

  31. Gimp core code basically completed GEGL transplantation - Open News Says:

    […] ported to a new generation of non-destructive graphics core GEGL. Now, developer Michael Natterer announced the progress of the transplant , said to have completed 90% of the core code portability. GEGL will be able to provide many new […]

  32. Inklusiv Says:

    Found this blog post on Slashdot. As a long time Gimp user I found it interesting to read how you guys work. And it seems you know what you’re doing. Respect!

  33. Mitch Says:

    Thanks to the anonymous nitpicker fo pointing out the typos, fixed ;)

  34. guest Says:

    just wondering, if HTML5 canvas as backend is worth to be considered or not ;)
    GIMP over web….

  35. gary Says:

    @Tobias - I stopped watching for GIMP updates years ago as 8bpc is a show-stopper for photo editing (just think how much detail is lost from a camera producing 14-bit per channel images - 2×2x2×2x2×2 levels).

    Now I really look forward to a return to GIMP.

    Fantastic work - thanks.

  36. MoH Says:

    Yeah - great thing! Congratulations and thank you.

    Finally, this also gets Gimp+GEGL far far away from Duke Nukem Forever ;-)

  37. asdf Says:

    90% are done, there’s only the other 90% to do now :)

  38. GIMP 2.9 avrà un core completamente GEGL | oneOpenSource Says:

    […] già al futuro: il progetto pare aver ripreso vitalità negli ultimi mesi infatti, ed è stato formalmente annunciato che GIMP 2.9 sarà portato a GEGL in tutto il core del […]

  39. Ядро GIMP переведено на библиотеку GEGL | AllUNIX.ru — Всероссийский портал о UNIX-системах Says:

    […] GIMP объявили о завершении основных этапов перевода ядра […]

  40. Bildbearbeitung: Gimp 2.10 basiert komplett auf GEGL | www.Anime-Island.org Says:

    […] Veröffentlichung von Gimp 2.8 lässt weiter auf sich warten, dennoch gibt Hauptentwickler Michael Natterer nun in seinem Blog an, Gimp 2.10 werde vollständig auf der Grafikbibliothek Gegl basieren. Dies bringt unter anderem […]

  41. lz83 Says:

    @guest: did you see the broadway GTK+ backend demo running GIMP?
    http://blogs.gnome.org/alexl/2011/04/18/broadway-update-3/

  42. Ozcrates.CO.CC | Oliver Zdravkovic » Bildbearbeitung: Gimp 2.10 basiert komplett auf GEGL » Sport, Politik, Technik, Psychologie Says:

    […] Veröffentlichung von Gimp 2.8 lässt weiter auf sich warten, dennoch gibt Hauptentwickler Michael Natterer nun in seinem Blog an, Gimp 2.10 werde vollständig auf der Grafikbibliothek Gegl basieren. Dies bringt unter anderem […]

  43. Guest Says:

    Thank you. This will Gimp give an huge step forward.

  44. Tobias Says:

    First of all I forget to say thank you for the work. And now to the comments:

    @Øyvind
    I can see your points and the best thing is, that it helped to motivate you.

    @David
    My first patch was 2007, but C is not my language. But there are other areas where I help GIMP.

    @Mitch
    Best reason to do some thing ever.

    @gary
    The “high bit depths” support is really missing in GIMP, but GiMP is even without it a great tool.

  45. George Says:

    Awesome work, thank you so much for your effort. I have many friends that use GIMP understand what you are doing and are also very appreciative.

  46. El núcleo de GIMP 2.10 será portado a GEGL | TecnoApps.net Says:

    […] | GIMP foo Related Posts:GIMP 2.8 en la recta final, publicada la primera versión candidataGIMP 2.7.5 […]

  47. Ядро GIMP переведено на библиотеку GEGL : Записки начинающего линуксоида Says:

    […] GIMP объявили о завершении основных этапов перевода ядра […]

  48. El Núcleo De GIMP 2.10 Será Portado A GEGL | Evangelista De Fedora Says:

    […] trabajar con una profundidad de color de 16 bits en las operaciones gestionadas por la librería. Fuente | Gimp Foo Tweet (function() { var li = document.createElement(’script’); li.type = […]

  49. El núcleo de GIMP 2.10 será portado a GEGL - La Isla Buscada Says:

    […] Vía | GIMP foo […]

  50. gary Says:

    @Tobias ‘“high bit depths” support is …missing … but GiMP is even without it a great tool.’

    For some things, yes. For a photographer I’m sad to say that it’s a non-starter. As soon as high bit-depth is introduced, I’ll happily switch back - and hopefully the other essential feature - adjustment layers -won’t be too far behind.

    Again - great work, and thanks

  51. Stefan Maerz Says:

    This is really exciting. Keep up the great work. :)

  52. Rdzeń GIMP-a portowany na GEGL - GIMP.Edu.pl Says:

    […] się w końcu opracować strategię migracji. Podczas pracy nad tym zobaczyli, że zaczęli zajmować się już portowaniem. Teraz około 90% rdzenia GIMP-a została przeportowane na GEGL. Po zakończeniu tego działania, […]

  53. Links 19/4/2012: Linux 3.4 RC3, GNOME Shell 3.4.1, Fedora 17 Almost Done | Techrights Says:

    […] Goat Invasion in GIMP We have done a lot of refactoring in GIMP over the last ten years, but its innermost pixel manipulating core was still basically unchanged since GIMP 1.2 days. We didn’t bother to do anything about it, because the long term goal was to do all this stuff with GEGL, when GEGL was ready. Now GEGL has been ready for quite a while, and the GEGL porting got assigned a milestone. Was it 2.10, 3.0, 3.2, I don’t remember. We thought it would take us forever until it’s done, because nobody really had that kind of time. […]

  54. Niko Says:

    Great news! Thanks!

  55. heiko Says:

    I’m glad to see you finally bring higher bit-depth to Gimp. As Gary already wrote, 8 bit is a show-stopper for photographers (at least those that aspire more than a snapshot look).
    I’m really looking forward to give Gimp another try, once high bit-depth is available.
    Will adjustment layers also be available shortly?
    Thanks for your hard work and bringing about an important change.

  56. Episode 180: Not for Grown Ups! | Meet the GIMP! Says:

    […] then there is an invasion of goats int GIMP and 16 bits and more are in […]

  57. SteveCzajka Says:

    Hi Mitch,

    Interesting post. It would be great if you would be a contributor to GIMP Magazine on GIMP development related items. Contact info is on our http://www.twitter.com/GIMPMagazine

    Thanks
    Steve

  58. finalzone Says:

    @ heiko
    Please read this http://blog.mmiworks.net/2012/01/gimp-full-gegl-ahead.html

  59. Vladimir Says:

    Thanks for this work, we use GIMP for our work at http://www.flor360.com there’s a lot of things done in GIMP (mainly because its auto white balancing and nice unsharp filters).

    Thanks for the good work!

  60. Steve Says:

    Wonderful news, I feel very excited. Does this mean the road map will be modified and high bit depth will move into 2.10 rather than 3.0 ?

  61. BenjaminBugl Says:

    guys!
    This is so awesome i can’t describe it!
    When 16-bit/channel is ready i can simply ditch photoshop and use gimp for all my photographic needs. this is the last feature before gimp will be a real(like REAL!) photoshop alternative and i’m looking forward to this.

    thank you for all your hard work.
    much love and appreciation
    benni

  62. Libre Graphics Meeting 2012 | LukasT.mediablog.sk Says:

    […] proprietary solutions so far. Øyvind Kolås a.k.a. pippin with some Gimp developer talked about Gegl integration in GIMP. Recently a huge step was made towards integrating this very fast, important back-end (e.g. 16 bit […]

  63. El nucleo de GIMP 2.10 sera portado a GEGL Says:

    […] con una profundidad de color de 16 bits en las operaciones gestionadas por la librería. Vía | GIMP foo Fuente: […]

  64. The day Goats invaded the GIMP « Linux Photography Says:

    […] turned out to a 3 weeks hackaton were 90% of the GIMP core was ported to GEGL. It is worth reading Mitch’s blog on the subject – picked by Slashdot where the whole discussion was about the hackers […]

  65. GIMP未来版本发布的GIMP 2.10核心代码将100%移植到GEGL - Official blog for xjspace Says:

    […] GIMP开发者去年宣布了GIMP 2.8后续开发计划,其中重要的一步是将核心图形代码移植到新一代非破坏性图形处理核心GEGL。现在,开发者Michael Natterer 公布了移植进度,表示已完成90%的核心代码移植。GEGL将能提供许多新特性,包括高位深度(high-bit-depth)支持。未来发布的GIMP 2.10核心代码将100%移植到GEGL。 […]

Leave a Reply