make check problem in libtest_smoketest building master

Hi

Building on Ubuntu 64-bit, "make check" is failing because of a missing
symbol in libtest_smoketest.so.
Doing a "make smoketest.clean" doesn't seem to help.
The library does genuinely seem to be missing the symbol (readelf log
attached).

Any ideas for tracking this down?

Thanks, Noel Grandin

Disclaimer: http://www.peralex.com/disclaimer.html



_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 04/11/2012 02:24 PM, Noel Grandin wrote:
> Building on Ubuntu 64-bit, "make check" is failing because of a missing
> symbol in libtest_smoketest.so.
> Doing a "make smoketest.clean" doesn't seem to help.
> The library does genuinely seem to be missing the symbol (readelf log
> attached).

In the case of smoketest, the lib containing cppunitTestPlugIn is not
libtest_smoketest.so but libsmoketest.so (where the former links
against the latter).  The reason for that appears to be
<http://cgit.freedesktop.org/libreoffice/core/commit/?id=8c478c911033243df90ba290b32732a1fd70130e>
"create installation set for tests."  (Petr, does that feature still
work, after gbuild'ification of smoketest?)

Looks like the dynamic loader on your system does not support dlsym to
report a symbol not exported by the lib itself, but only indirectly by a
lib the first lib links against.

Stephan
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On Wednesday 11 of April 2012, Stephan Bergmann wrote:
 Note that you can check what the dynamic loader does by doing
LD_DEBUG=symbols command (see LD_DEBUG=help /bin/true).

--
 Lubos Lunak
 [hidden email]
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 2012-04-11 16:29, Stephan Bergmann wrote:
> Looks like the dynamic loader on your system does not support dlsym to
> report a symbol not exported by the lib itself, but only indirectly by
> a lib the first lib links against.
>
>
Darn, I would have thought that by now pretty much all Linux's had the
same dynamic loader.
Is there a linker option that would cure this?
Something like
   --export-dynamic
   --export-all-symbols
And where would I try inserting these?

Thanks, Noel Grandin


Disclaimer: http://www.peralex.com/disclaimer.html


_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 04/12/2012 09:49 AM, Noel Grandin wrote:
> On 2012-04-11 16:29, Stephan Bergmann wrote:
>> Looks like the dynamic loader on your system does not support dlsym to
>> report a symbol not exported by the lib itself, but only indirectly by
>> a lib the first lib links against.
>>
>>
> Darn, I would have thought that by now pretty much all Linux's had the
> same dynamic loader.

Please find out whether the assumption is correct (see Lubos' mail on
LD_DEBUG).

Stephan
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 2012-04-12 10:22, Stephan Bergmann wrote:
>
> Please find out whether the assumption is correct (see Lubos' mail on
> LD_DEBUG).
>
I tried

LD_DEBUG=symbols make check

but it produced so much output it seems to have broken the build - it eventually froze up during a link step.

How do I run just the failing test?



Disclaimer: http://www.peralex.com/disclaimer.html


_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 04/12/2012 10:30 AM, Noel Grandin wrote:
cd smoketest && make -n
/home/noel/libo/workdir/unxlngx6.pro/CppunitTest/smoketest.test

will display the command line to run the test (following a line "[ build
CUT ] smoketest"), starting with "S=... && O=... && W=... && mkdir -p
... && (LD_LIBRARY_PATH=...".  Copy that mumbo-jumbo, stick
"LD_DEBUG=symbols " between "(" and "LD_LIBRARY_PATH=" and run that.
The output will be captured in
/home/noel/libo/workdir/unxlngx6.pro/CppunitTest/smoketest.test.log.

Stephan
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

Stephan Bergmann píše v St 11. 04. 2012 v 16:29 +0200:
I haven't tested it after gbuild'fication. I do not work much with
master this time :-(

> Looks like the dynamic loader on your system does not support dlsym to
> report a symbol not exported by the lib itself, but only indirectly by a
> lib the first lib links against.

I wonder if it might be related to --as-needed linker option. It caused
similar troubles when enabled on openSUSE linker by default.

Of course, it is more likely broken by gbuild'ification. I wonder if
Matúš could have a look at it.


Best Regards,
Petr

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 04/12/2012 10:48 AM, Noel Grandin wrote:
> Output attached.
[...]
>      10709: symbol=dlsym;  lookup in file=/lib/x86_64-linux-gnu/libdl.so.2 [0]
>      10709: symbol=cppunitTestPlugIn;  lookup in file=/home/noel/libo/workdir/unxlngx6.pro/LinkTarget/CppunitTest/libtest_smoketest.so [0]
>      10709: symbol=cppunitTestPlugIn;  lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
>      10709: symbol=cppunitTestPlugIn;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
>      10709: /home/noel/libo/workdir/unxlngx6.pro/LinkTarget/CppunitTest/libtest_smoketest.so: error: symbol lookup error: undefined symbol: cppunitTestPlugIn (fatal)

My output (Fedora 16) there is

>       4333: symbol=dlsym;  lookup in file=/lib64/libdl.so.2 [0]
>       4333: symbol=cppunitTestPlugIn;  lookup in file=/data/lo/core/workdir/unxlngx6/LinkTarget/CppunitTest/libtest_smoketest.so [0]
>       4333: symbol=cppunitTestPlugIn;  lookup in file=/data/lo/core/solver/unxlngx6/lib/libcppunit-1.12.so.1 [0]
>       4333: symbol=cppunitTestPlugIn;  lookup in file=/data/lo/core/solver/unxlngx6/lib/libsmoketest.so [0]

What is the output of

readelf -d workdir/unxlngx6.pro/LinkTarget/CppunitTest/libtest_smoketest.so

for you?

Stephan
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 04/12/2012 11:05 AM, Petr Mladek wrote:
> Stephan Bergmann píše v St 11. 04. 2012 v 16:29 +0200:
>> Looks like the dynamic loader on your system does not support dlsym to
>> report a symbol not exported by the lib itself, but only indirectly by a
>> lib the first lib links against.
>
> I wonder if it might be related to --as-needed linker option. It caused
> similar troubles when enabled on openSUSE linker by default.

Sounds plausible.  Just asked Noel for readelf -d output for
verification.  I would consider an ld that assumes --as-needed per
default as broken.

Stephan
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 2012-04-12 11:51, Stephan Bergmann wrote:
> What is the output of
>
> readelf -d
> workdir/unxlngx6.pro/LinkTarget/CppunitTest/libtest_smoketest.so
>
> for you?
>
Output attached.



Disclaimer: http://www.peralex.com/disclaimer.html



_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 12/04/12 12:32, Noel Grandin wrote:
>  0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

yeah, that's rather too little :(

this is missing here:

>  0x0000000000000001 (NEEDED)             Shared library: [libsmoketest.so]

indeed looks like your ld defaults to --as-needed.

it should work if you do this:

cd smoketest && make -r clean && make -r LDFLAGS=-Wl,--no-as-needed
subsequentcheck

wonder what we should do about ld that defaults to --as-needed...

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 2012-04-12 11:56, Stephan Bergmann wrote:
> Sounds plausible.  Just asked Noel for readelf -d output for
> verification.  I would consider an ld that assumes --as-needed per
> default as broken.
>
>

For whatever reason, Ubuntu appears to do that now:
https://wiki.ubuntu.com/NattyNarwhal/ToolchainTransition


Disclaimer: http://www.peralex.com/disclaimer.html


_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 2012-04-12 12:40, Michael Stahl wrote:
> it should work if you do this: cd smoketest && make -r clean && make
> -r LDFLAGS=-Wl,--no-as-needed subsequentcheck

Indeed that works (not entirely, but at least it gets past the symbol
loading issue).

Disclaimer: http://www.peralex.com/disclaimer.html


_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

Zitat von Michael Stahl <[hidden email]>:

As it's stated here [1]

*Caution:* Reading the documentation, you may be tempted to try the  
--no-as-needed option as a "quick fix" workaround, but it's generally  
the not the right fix. If you aren't able to get a package working  
with the --as-needed and --no-copy-dt-needed-entries options enabled,  
it's best to submit a bug report and get expert attention. (It may be  
a sign of deeper flaws in the code or linking strategy of the package.)

[1] https://wiki.ubuntu.com/NattyNarwhal/ToolchainTransition

> wonder what we should do about ld that defaults to --as-needed...
>
> _______________________________________________
> LibreOffice mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/libreoffice
>

David

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 04/12/2012 12:40 PM, Michael Stahl wrote:
> wonder what we should do about ld that defaults to --as-needed...

Insist on people compiling with an unbroken toolchain instead?

For the smoketest problem at hand, the best thing would probably be to
get rid of the broken test_smoketest vs. smoketest library design, anyway.

But there are other places where we imply --no-as-needed:  Libraries
within extensions are guaranteed to be run in an environment where all
the public URE libraries are available.  The way we ensure that is by
explicitly linking the relevant executables against all the relevant URE
libraries.  (At least that's how we used to do it; might well got broken
with some gbuild'ification, and is typically hard to detect, given that
most if not all public URE libraries get loaded into a process through
one way or another, anyway.)

Stephan
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 2012-04-12 14:40, Stephan Bergmann wrote:
> Insist on people compiling with an unbroken toolchain instead?
>
>
It looks like Fedora is also going to do this:
http://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking

I'm willing to write a configure test to add the --no-as-needed flag if
someone can point me in the right direction.
I can follow the configure.in syntax easily enough, but what would be
the easiest way to test for the fact that the linker is defaulting to
--as-needed?

Thanks, Noel Grandin



Disclaimer: http://www.peralex.com/disclaimer.html


_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 13/04/12 11:35, Noel Grandin wrote:
>
>
> On 2012-04-12 14:40, Stephan Bergmann wrote:
>> Insist on people compiling with an unbroken toolchain instead?
>>
>>
> It looks like Fedora is also going to do this:
> http://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking

this is a different change: ChangeInImplicitDSOLinking is about not
adding DT_NEEDED entries for libraries that are _not_ explicitly passed
on the ld command line, while the problematic --as-needed is about not
adding DT_NEEDED entries for libraries that _are_ explicitly passed on
the ld command line but from which no symbol is imported.

besides, the page says ChangeInImplicitDSOLinking is live since Fedora13

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 11 April 2012 14:24, Noel Grandin <[hidden email]> wrote:
> Building on Ubuntu 64-bit, "make check" is failing because of a missing
> symbol in libtest_smoketest.so.
> Doing a "make smoketest.clean" doesn't seem to help.
> The library does genuinely seem to be missing the symbol (readelf log
> attached).
>
> Any ideas for tracking this down?

Does attached diff help ?
There is
-$(eval $(call gb_CppunitTest_use_libraries,smoketest,\
+$(eval $(call gb_CppunitTest_use_library_objects,smoketest,\

I did not know what commit message to write there.
If it helps feel free to push it anybody, please.

Best,
Matus

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

On 04/13/2012 11:35 AM, Noel Grandin wrote:
> On 2012-04-12 14:40, Stephan Bergmann wrote:
>> Insist on people compiling with an unbroken toolchain instead?
>>
>>
> It looks like Fedora is also going to do this:
> http://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking

No, as already detailed by Michael.

> I'm willing to write a configure test to add the --no-as-needed flag if
> someone can point me in the right direction.
> I can follow the configure.in syntax easily enough, but what would be
> the easiest way to test for the fact that the linker is defaulting to
> --as-needed?

I would still prefer to simply insist on Ubuntu fixing their tool chain.
 
<http://wiki.services.openoffice.org/wiki/Writing_warning-free_code#When_all_else_fails>
"NOTE: The ld --as-needed default was reverted for the final natty
release, and will be re-enabled in the o-series."  This at least seems
to indicate that Ubuntu is aware their move is not without problems.
Noel, do you happen to be affected because you are on a pre-final natty
release (whatever that is)?

Björn, do you know anything about this Ubuntu-specific thing, or anyone
within Ubuntu we could discuss that with?

Stephan
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice