LD_LIBRARY_PATH vs ldconfig

Leave a reply

I’m having problems with LD_LIBRARY_PATH at link time (this question has nothing to do with run time).

The link line looks like this when I run make (this is a Linux system using g++ version 4.1.x):

<code>g++ a.o b.o c.o -o myapp \
 -L/long/path/to/libs/ \
 -L/another/long/path/ \
 -labc -ldef -lghi
</code>

The -l options reference shared libraries (e.g., libabc.so) which exist in the directories specified by the-L options. Those directories also appear in LD_LIBRARY_PATH. With that configuration, the link is successful, and I can run the application.

If I remove the directories from LD_LIBRARY_PATH, then I get a single error line such as:

<code>/usr/bin/ld: cannot find -labc
</code>

On the other hand, if I remove the directories from the list of -L options, then I get many warnings such as:

<code>/usr/bin/ld: warning: libabc.so, needed by /long/path/to/libs/libxyz.so,
    not found (try using -rpath or -rpath-link)
</code>

and then many more errors, such as:

<code>/long/path/to/libs/libdef.so: undefined reference to `Foo::Bar&lt;Baz&gt;::junk(Fred*)'
</code>

Can someone explain the difference between LD_LIBRARY_PATH and -L? I would like to understand this stuff in depth, so references are greatly appreciated!

Also, what do I have to add to the link line to avoid using LD_LIBRARY_PATH?

EDIT: When directories were missing from -L, the compiler suggested to “try using -rpath or -rpath-link”. I don’t think I’ve ever seen those options in a makefile before. Have you? Not sure if that would help the LD_LIBRARY_PATH problem though.

asked Dec 15 ’09 at 3:29
Dan
2,022718
73% accept rate

4 Answers

The settings of LD_LIBRARY_PATH has the highest precedence, so when it is set, the set of directories mentioned by LD_LIBRARY_PATH are searched first even before the standard set
of directories. So in your case setting of LD_LIBRARY_PATH is influencing the lookup of
the libraries mentioned with ‘-l’ option. Without LD_LIBRARY_PATH some of the dependencies
might have been resolved from the standard set of directories.

Though setting of LD_LIBRARY_PATH help with debugging and to try out a newer version of
a library its usage in the general development environment setup and deployment is considered bad.

Also refer this HOWTO from Linux Documentation for more details on Shared Libraries

answered Dec 15 ’09 at 4:20
sateesh
2,294411
That link is comprehensive! Thanks, just what I was looking for. – Dan Dec 15 ’09 at 17:24

LD_LIBRARY_PATH is intended for finding shared libraries when running an application. It is a side effect that it’s impacting your link, and you should not rely on that.

As an often unwanted side effect, LD_LIBRARY_PATH will also be searched at link (ld) stage after directories specified with -L (also if no -L flag is given).

Why LD_LIBRARY_PATH is bad

answered Dec 15 ’09 at 3:38
Don Neufeld
7,31421634
I agree it’s a side effect, and I would prefer not to rely on it. But I’m not sure where to put the directories instead of in LD_LIBRARY_PATH — because just having them in -L apparently isn’t good enough. Thanks for the link — good reading! – Dan Dec 15 ’09 at 16:38

There are two answers to this question, part of the answer lies in the compile-time linking (i.e gcc -lfoo -L/usr/lib … which in turn calls ld), and run-time linker lookups.

When you compile your program, the compiler checks syntax, and then the linker ensures that the symbols required for execution exist (i.e variables / methods / etc), among other things. LD_LIBRARY_PATH, as has been noted, has the side-effect of altering the way gcc/ld behave as well as the way the the run-time linker behaves by modifying the search path.

When you run your program, the run-time linker actually fetches the shared libraries (on disk or from memory if possible), and loads in the shared symbols / code / etc. Again, LD_LIBRARY_PATH affects this search path implicitly (sometimes not a good thing, as has been mentioned.)

The correct fix for this without using LD_LIBRARY_PATH on most Linux systems is to add the path that contains your shared libraries to /etc/ld.so.conf (or in some distributions, create a file in /etc/ld.so.conf.d/ with the path in it) and run ldconfig (/sbin/ldconfig as root) to update the runtime linker bindings cache.

Example on Debian:

<code>jewart@dorfl:~$ cat /etc/ld.so.conf.d/usrlocal.conf 
/usr/local/lib
</code>

Then when the program is executed, the run-time linker will look in those directories for libraries that your binary has been linked against.

If you want to know what libraries the run-time linker knows about, you can use:

<code>jewart@dorfl:~$ ldconfig -v 

/usr/lib:
libbfd-2.18.0.20080103.so -&gt; libbfd-2.18.0.20080103.so
libkdb5.so.4 -&gt; libkdb5.so.4.0
libXext.so.6 -&gt; libXext.so.6.4.0
</code>

And, if you want to know what libraries a binary is linked against, you can use ldd like such, which will tell you which library your runtime linker is going to choose:

<code>jewart@dorfl:~$ ldd /bin/ls
linux-vdso.so.1 =&gt;  (0x00007fffda1ff000)
librt.so.1 =&gt; /lib/librt.so.1 (0x00007f5d2149b000)
libselinux.so.1 =&gt; /lib/libselinux.so.1 (0x00007f5d2127f000)
libacl.so.1 =&gt; /lib/libacl.so.1 (0x00007f5d21077000)
libc.so.6 =&gt; /lib/libc.so.6 (0x00007f5d20d23000)
</code>
answered Dec 15 ’09 at 4:05
+1, good concise description of how linking & loading works. ld.so.conf seems like it could have some of the same problems as LD_LIBRARY_PATH, although it wouldn’t impact link-time, only run-time — it’s mentioned here as a <href=”xahlee.org/UnixResource%5Fdir/%5F/… attempt to improve things</a>. – Dan Dec 15 ’09 at 16:32
However, I still don’t understand why the absence of LD_LIBRARY_PATH prevents the link. Editing ld.so.conf is not an option. If it were, would it even address the problem I have at link-time when I remove directories from LD_LIBRARY_PATH? – Dan Dec 15 ’09 at 16:34
The way I read your original post is the following: Compilation with -L/path/to/lib1 -L/path/to/lib2and3 -llib1 -llib2 -llib3 works, but execution fails unless LD_LIBRARY_PATH has /path/to/lib{1,2and3} in it. If linking at compile time is failing, then something is awry with your -L paths… Can you post the actual compilation command that fails? – John Ewart Dec 15 ’09 at 17:04
No… it’s link that fails unless LD_LIBRARY_PATH has /path/to/lib{1,2,3} in it. I will check for discrepancies between the LD_LIBRARY_PATH and what is specified by -L. Unfortunately I cannot post the compilation command. Regardless of what LD_LIBRARY_PATH is set to, it does not affect what gets spit out by make as the compilation/link command. – Dan Dec 15 ’09 at 17:14
Correct, you won’t see it in the output, just whatever the Makefile has as written by a human or computed by autotools. Is it possible that the error you’re seeing is actually earlier than you compiling your specific binary? Maybe from a dependency that’s trying to compile (ala autoconf’s tests for the existence of a library), and failing because your libabc.so is not in a standard location (and not explicitly specified via -L)? That would be a scenario where compilation / autoconf works with LD_LIBRARY_PATH set… – John Ewart Dec 15 ’09 at 17:33

If I were to guess, I would say that the linker is falling back to using LD_LIBRARY_PATH to resolve libraries that your direct links (e.g., libabc.so, libdef.so, and libghi.so) are dynamically linked against. Looking at the manual page for ld, it looks like linking against an .so that was built using -rpath would affect how the lookup of dynamically bound symbols works.

 

http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>