installation sets, parallel compression, pigz etc.

i've noticed that building "archive" installation sets on my machine is
entirely CPU bound (since everything fits into RAM) and a single "gzip"
takes up most of the time, so i've toyed around a little with a parallel
implementation "pigz" (available in Fedora) and a single line change
makes quite a difference:

real    2m55.031s
user    3m7.863s
sys     0m10.980s

real    2m56.103s
user    3m9.408s
sys     0m11.484s

real    2m58.402s
user    3m10.293s
sys     0m10.523s

real    0m39.902s
user    4m40.154s
sys     0m12.257s

real    0m36.689s
user    4m39.771s
sys     0m11.989s

real    0m41.307s
user    4m40.514s
sys     0m11.741s

i wonder, is this something that would enable us to do release builds
faster; of course since i don't do those i don't know to what extent it
could help, especially as we don't release "archive" but "rpm" (and
"deb"?) installation sets, and probably those will have their
compression done by something invoked by rpm... but at least for
gzip/pigz the tools look compatible, so perhaps putting it in PATH with
the right name could be done.

also there seem to be parallel implementations of bzip2 and lzma:

http://www.zlib.net/pigz/
http://compression.ca/pbzip2/
http://jnovy.fedorapeople.org/pxz/

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On Tue, Jun 26, 2012 at 05:02:37PM +0200, Michael Stahl wrote:
>
> i've noticed that building "archive" installation sets on my machine is
> entirely CPU bound (since everything fits into RAM) and a single "gzip"
> takes up most of the time, so i've toyed around a little with a parallel
> implementation "pigz" (available in Fedora) and a single line change
> makes quite a difference:
>

Its even worse for xz compression. My release builds (with ccache take 1.5 as
much time to pack as to build):
https://plus.google.com/101094190333184858950/posts/D9x87DkdceD

Best,

Bjoern
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

Michael Stahl píše v Út 26. 06. 2012 v 17:02 +0200:
That sounds great.

Unfortunately, the patch can't be applied as is. Would you mind to add a
configure check for gzip/pigz and call it via an variable?

> i wonder, is this something that would enable us to do release builds
> faster; of course since i don't do those i don't know to what extent it
> could help, especially as we don't release "archive" but "rpm" (and
> "deb"?) installation sets, and probably those will have their
> compression done by something invoked by rpm... but at least for
> gzip/pigz the tools look compatible, so perhaps putting it in PATH with
> the right name could be done.

We could also somehow tweak the internal epm. Well, AFAIK, the tasks for
creating normal installation, help packs, lang packs, debs and rpms are
done in parallel, so we already have some kind of parallelism.

Best Regards,
Petr

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice