Compilation errors on various platforms

[CamJN]

I'm working on instructions for installing the deps on various linuxes, and am running into compilation issues at the conan install . --build=missing step. My test env is docker based, running various distro images, dockerfiles included.

Debian 9:

FROM debian:9

WORKDIR /
RUN apt update
RUN apt install -y build-essential git python-pip cmake m4

RUN pip install conan

RUN git clone https://github.com/DEGoodmanWilson/luna-example
WORKDIR /luna-example
RUN conan remote add vthiery https://api.bintray.com/conan/vthiery/conan-packages
RUN conan remote add degoodmanwilson https://api.bintray.com/conan/degoodmanwilson/opensource
RUN conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
RUN conan install . --build=missing
RUN conan build .
CMD env PORT=5000 ./bin/awesomesauce

error:

/usr/bin/c++    -std=c++17 -pthread -m64 -O3 -DNDEBUG        CMakeFiles/awesomesauce.dir/logger.cpp.o CMakeFiles/awesomesauce.dir/main.cpp.o  -o bin/awesomesauce  -L/root/.conan/data/luna/5.0.4/DEGoodmanWilson/stable/package/c7add8bb08936f7bf0d20a5581bca736343277b0/lib  -L/root/.conan/data/libmicrohttpd/0.9.51/DEGoodmanWilson/stable/package/d03a64e2ddda901c98d42890167192e39eeee55c/lib  -L/root/.conan/data/libmime/0.1.1/DEGoodmanWilson/stable/package/c1840d3dde03cdb427e4fcccca3e06e39c567a19/lib  -L/root/.conan/data/base64/1.0.2/DEGoodmanWilson/stable/package/587df1d6a993e6eff4d3791612ba383ca625243c/lib  -L/root/.conan/data/jsonformoderncpp/3.1.2/vthiery/stable/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib  -L/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/package/08bb6973ba0ab01617100a7ee91241aa9cf22606/lib  -L/root/.conan/data/libgcrypt/1.7.3/DEGoodmanWilson/stable/package/b324bddf3afb93b0c235b0b436e2c7b41fcf7782/lib  -L/root/.conan/data/libiconv/1.15/bincrafters/stable/package/f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb/lib  -L/root/.conan/data/nettle/3.4/DEGoodmanWilson/stable/package/b922586c0015d9624adc2a14bf56affcad2f447a/lib  -L/root/.conan/data/zlib/1.2.11/conan/stable/package/f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb/lib  -L/root/.conan/data/libgpg-error/1.24/DEGoodmanWilson/stable/package/351053dac39760023a2758a781ec2b142ea75835/lib  -L/root/.conan/data/gmp/6.1.1/DEGoodmanWilson/stable/package/907b73170cb188d51f2fabd9a3a2b9e81ce67a74/lib -Wl,-rpath,/root/.conan/data/luna/5.0.4/DEGoodmanWilson/stable/package/c7add8bb08936f7bf0d20a5581bca736343277b0/lib:/root/.conan/data/libmicrohttpd/0.9.51/DEGoodmanWilson/stable/package/d03a64e2ddda901c98d42890167192e39eeee55c/lib:/root/.conan/data/libmime/0.1.1/DEGoodmanWilson/stable/package/c1840d3dde03cdb427e4fcccca3e06e39c567a19/lib:/root/.conan/data/base64/1.0.2/DEGoodmanWilson/stable/package/587df1d6a993e6eff4d3791612ba383ca625243c/lib:/root/.conan/data/jsonformoderncpp/3.1.2/vthiery/stable/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib:/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/package/08bb6973ba0ab01617100a7ee91241aa9cf22606/lib:/root/.conan/data/libgcrypt/1.7.3/DEGoodmanWilson/stable/package/b324bddf3afb93b0c235b0b436e2c7b41fcf7782/lib:/root/.conan/data/libiconv/1.15/bincrafters/stable/package/f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb/lib:/root/.conan/data/nettle/3.4/DEGoodmanWilson/stable/package/b922586c0015d9624adc2a14bf56affcad2f447a/lib:/root/.conan/data/zlib/1.2.11/conan/stable/package/f7e573cb501ccfc49e9e4d84de886bc1ef2e6ebb/lib:/root/.conan/data/libgpg-error/1.24/DEGoodmanWilson/stable/package/351053dac39760023a2758a781ec2b142ea75835/lib:/root/.conan/data/gmp/6.1.1/DEGoodmanWilson/stable/package/907b73170cb188d51f2fabd9a3a2b9e81ce67a74/lib -lluna -lmicrohttpd -lmime -lbase64 -lgnutls -lgcrypt -lmpi -lcompat -lrandom -lcipher -liconv -lhogweed -lnettle -lz -lgpg-error -lgmp -lgmpxx 
/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/package/08bb6973ba0ab01617100a7ee91241aa9cf22606/lib/libgnutls.a(sysrng-linux.o): In function `_rnd_get_system_entropy_getrandom':
sysrng-linux.c:(.text+0xf9): undefined reference to `getrandom'
/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/package/08bb6973ba0ab01617100a7ee91241aa9cf22606/lib/libgnutls.a(sysrng-linux.o): In function `_rnd_system_entropy_init':
sysrng-linux.c:(.text+0x1ec): undefined reference to `getrandom'
collect2: error: ld returned 1 exit status
CMakeFiles/awesomesauce.dir/build.make:123: recipe for target 'bin/awesomesauce' failed
make[2]: Leaving directory '/luna-example'
make[2]: *** [bin/awesomesauce] Error 1
CMakeFiles/Makefile2:70: recipe for target 'CMakeFiles/awesomesauce.dir/all' failed
make[1]: Leaving directory '/luna-example'
make[1]: *** [CMakeFiles/awesomesauce.dir/all] Error 2
Makefile:86: recipe for target 'all' failed
make: *** [all] Error 2
ERROR: PROJECT: Error in build() method, line 16
	cmake.build()
	ConanException: Error 512 while executing cmake --build '/luna-example' '--' '-j6'
The command '/bin/sh -c conan build .' returned a non-zero code: 1

Ubuntu Cosmic:

FROM ubuntu:cosmic

WORKDIR /
RUN apt update
RUN apt install -y build-essential git python-pip cmake m4

RUN pip install conan

RUN git clone https://github.com/DEGoodmanWilson/luna-example
WORKDIR /luna-example
RUN conan remote add vthiery https://api.bintray.com/conan/vthiery/conan-packages
RUN conan remote add degoodmanwilson https://api.bintray.com/conan/degoodmanwilson/opensource
RUN conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
RUN conan install . --build=missing
RUN conan build .
CMD env PORT=5000 ./bin/awesomesauce

error:

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I../src -I../src -I/root/.conan/data/libgpg-error/1.24/DEGoodmanWilson/stable/package/ec336a8e46e06995be9ad34dc1a7477dff42a7df/include -DNDEBUG -I/root/.conan/data/libgpg-error/1.24/DEGoodmanWilson/stable/package/ec336a8e46e06995be9ad34dc1a7477dff42a7df/include -m64 -O3 -s -fPIC -fvisibility=hidden -Wall -MT rijndael-padlock.lo -MD -MP -MF .deps/rijndael-padlock.Tpo -c rijndael-padlock.c -o rijndael-padlock.o
In file included from /usr/include/features.h:424,
                 from /usr/include/x86_64-linux-gnu/bits/libc-header-start.h:33,
                 from /usr/include/stdio.h:27,
                 from rijndael-ssse3-amd64.c:38:
rijndael-ssse3-amd64.c: In function '_gcry_aes_ssse3_do_setkey':
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:31:1: error: inlining failed in call to always_inline 'memcpy': target specific option mismatch
 __NTH (memcpy (void *__restrict __dest, const void *__restrict __src,
 ^~~~~
rijndael-ssse3-amd64.c:175:3: note: called from here
   memcpy(&ctx->keyschdec32[0][0], key, keybits / 8);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [Makefile:652: rijndael-ssse3-amd64.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
mv -f .deps/primegen.Tpo .deps/primegen.Plo
mv -f .deps/rijndael-padlock.Tpo .deps/rijndael-padlock.Plo
mv -f .deps/rijndael-aesni.Tpo .deps/rijndael-aesni.Plo
mv -f .deps/des.Tpo .deps/des.Plo
mv -f .deps/rijndael.Tpo .deps/rijndael.Plo
make[2]: Leaving directory '/root/.conan/data/libgcrypt/1.7.3/DEGoodmanWilson/stable/build/9acc871fe67e5eeb8cfb287f4ddbac259d2e74bc/sources/cipher'
make[1]: *** [Makefile:477: all-recursive] Error 1
make[1]: Leaving directory '/root/.conan/data/libgcrypt/1.7.3/DEGoodmanWilson/stable/build/9acc871fe67e5eeb8cfb287f4ddbac259d2e74bc/sources'
make: *** [Makefile:408: all] Error 2
libgcrypt/1.7.3@DEGoodmanWilson/stable: 
libgcrypt/1.7.3@DEGoodmanWilson/stable: ERROR: Package '9acc871fe67e5eeb8cfb287f4ddbac259d2e74bc' build failed
libgcrypt/1.7.3@DEGoodmanWilson/stable: WARN: Build folder /root/.conan/data/libgcrypt/1.7.3/DEGoodmanWilson/stable/build/9acc871fe67e5eeb8cfb287f4ddbac259d2e74bc
ERROR: libgcrypt/1.7.3@DEGoodmanWilson/stable: Error in build() method, line 83
	env_build.make()
	ConanException: Error 512 while executing make -j6
The command '/bin/sh -c conan install . --build=missing' returned a non-zero code: 1

Arch (latest):

FROM base/archlinux

WORKDIR /
RUN pacman -Sy --noconfirm git base-devel cmake xxd sudo
RUN git clone https://aur.archlinux.org/yay.git
WORKDIR /yay

RUN useradd builduser -m
RUN passwd -d builduser
RUN printf 'builduser ALL=(ALL) ALL\n' | tee -a /etc/sudoers
RUN chmod -R 777 /yay
USER builduser
RUN makepkg -si --noconfirm
RUN yay -S --noconfirm conan
USER root

WORKDIR /
RUN git clone https://github.com/DEGoodmanWilson/luna-example
WORKDIR /luna-example
RUN conan remote add vthiery https://api.bintray.com/conan/vthiery/conan-packages
RUN conan remote add degoodmanwilson https://api.bintray.com/conan/degoodmanwilson/opensource
RUN conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
RUN conan install . --build=missing
RUN conan build .
CMD env PORT=5000 ./bin/awesomesauce

error:

make[3]: Entering directory '/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/build/e5125037ab8f6ffcec0b20f08fbd90843f3eb95a/sources/guile'
  GEN      modules/gnutls.scm
  GUILEC   modules/gnutls/extra.go
  GUILEC   modules/gnutls.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /usr/sbin/guild
;;; compiled /root/.cache/guile/ccache/2.2-LE-8-3.A/usr/bin/guild.go
Backtrace:
In ice-9/boot-9.scm:
    705:2 19 (call-with-prompt _ _ #<procedure default-prompt-handle?>)
In ice-9/eval.scm:
    619:8 18 (_ #(#(#<directory (guile-user) 555af22ed140>)))
In /usr/sbin/guild:
    72:17 17 (main _)
In srfi/srfi-1.scm:
    640:9 16 (for-each #<procedure 555af25f9000 at scripts/compile.?> ?)
In scripts/compile.scm:
   259:26 15 (_ _)
In system/base/target.scm:
     57:6 14 (with-target _ _)
In system/base/compile.scm:
    152:6 13 (compile-file "modules/gnutls.scm" #:output-file _ # _ # ?)
     43:4 12 (call-once _)
In ice-9/boot-9.scm:
    841:4 11 (with-throw-handler _ _ _)
In system/base/compile.scm:
    59:11 10 (_)
   155:11  9 (_ #<closed: file 555af26621c0>)
   235:18  8 (read-and-compile #<input: gnutls.scm 12> #:from _ #:to ?)
   183:32  7 (compile-fold (#<procedure compile-tree-il (x e opts)>) ?)
In ice-9/boot-9.scm:
   2312:4  6 (save-module-excursion #<procedure 555af29b5e20 at lang?>)
In language/scheme/compile-tree-il.scm:
    31:15  5 (_)
In ice-9/psyntax.scm:
  1235:36  4 (expand-top-sequence ((eval-when (expand load eval) ?)) ?)
  1182:24  3 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   285:10  2 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
In unknown file:
           1 (load-extension "/root/.conan/data/gnutls/3.6.2/DEGood?" ?)
In ice-9/boot-9.scm:
   752:25  0 (dispatch-exception _ _ _)

ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
In procedure dynamic-link: file: "/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/build/e5125037ab8f6ffcec0b20f08fbd90843f3eb95a/sources/guile/src/guile-gnutls-v-2", message: "file not found"
make[3]: *** [Makefile:2241: modules/gnutls.go] Error 1
make[3]: *** Waiting for unfinished jobs....
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /usr/sbin/guild
;;; compiled /root/.cache/guile/ccache/2.2-LE-8-3.A/usr/bin/guild.go
Backtrace:
In ice-9/psyntax.scm:
   285:10 19 (parse _ (("placeholder" placeholder)) (()) _ c&e (# #) #)
In ice-9/eval.scm:
   293:34 18 (_ #<module (#{ g347}#) 55f4bc6f71e0>)
In ice-9/boot-9.scm:
   2874:4 17 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  2071:24 16 (call-with-deferred-observers #<procedure 55f4bc881730 ?>)
  2887:24 15 (_)
   222:17 14 (map1 (((gnutls))))
  2800:17 13 (resolve-interface (gnutls) #:select _ #:hide _ #:prefix ?)
In ice-9/threads.scm:
    390:8 12 (_ _)
In ice-9/boot-9.scm:
  2726:13 11 (_)
In ice-9/threads.scm:
    390:8 10 (_ _)
In ice-9/boot-9.scm:
  2994:20  9 (_)
   2312:4  8 (save-module-excursion #<procedure 55f4bcaed210 at ice-?>)
  3014:26  7 (_)
In unknown file:
           6 (primitive-load-path "gnutls" #<procedure 55f4bc378d60 ?>)
In ice-9/eval.scm:
   721:20  5 (primitive-eval (eval-when (expand load eval) (# # #) #))
In ice-9/psyntax.scm:
  1235:36  4 (expand-top-sequence ((eval-when (expand load eval) ?)) ?)
  1182:24  3 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   285:10  2 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
In unknown file:
           1 (load-extension "/root/.conan/data/gnutls/3.6.2/DEGood?" ?)
In ice-9/boot-9.scm:
   752:25  0 (dispatch-exception _ _ _)

ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
In procedure dynamic-link: file: "/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/build/e5125037ab8f6ffcec0b20f08fbd90843f3eb95a/sources/guile/src/guile-gnutls-v-2", message: "file not found"
make[3]: Leaving directory '/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/build/e5125037ab8f6ffcec0b20f08fbd90843f3eb95a/sources/guile'
make[3]: *** [Makefile:2241: modules/gnutls/extra.go] Error 1
make[2]: *** [Makefile:1732: all-recursive] Error 1
make[2]: Leaving directory '/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/build/e5125037ab8f6ffcec0b20f08fbd90843f3eb95a/sources/guile'
make[1]: Leaving directory '/root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/build/e5125037ab8f6ffcec0b20f08fbd90843f3eb95a/sources'
make[1]: *** [Makefile:1460: all-recursive] Error 1
make: *** [Makefile:1388: all] Error 2
gnutls/3.6.2@DEGoodmanWilson/stable: 
gnutls/3.6.2@DEGoodmanWilson/stable: ERROR: Package 'e5125037ab8f6ffcec0b20f08fbd90843f3eb95a' build failed
gnutls/3.6.2@DEGoodmanWilson/stable: WARN: Build folder /root/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/build/e5125037ab8f6ffcec0b20f08fbd90843f3eb95a
ERROR: gnutls/3.6.2@DEGoodmanWilson/stable: Error in build() method, line 126
	env_build.make()
	ConanException: Error 512 while executing make -j6
The command '/bin/sh -c conan install . --build=missing' returned a non-zero code: 1

[DEGoodmanWilson]

Well, the good news is that these errors are not with Luna, but with libgnutls and libgpg-error. The bad news is that that doesn't make anything easier for us ;)

Here are the relevant repos (just for reference). Let me meditate on the right way forward.
https://github.com/DEGoodmanWilson/conan-gnutls
https://github.com/DEGoodmanWilson/conan-libgpg-error

[DEGoodmanWilson]

For reference, BTW, all Linux builds are built on Ubuntu Trusty:
https://github.com/lasote/conan-docker-tools/blob/master/gcc_4.9/Dockerfile

[DEGoodmanWilson]

For now, I think the proper fix would be to build everything from source. Which will take a disgustingly long time, but it should help.

conan install . --build=all

[CamJN]

--build=all seems to be ignored:

base64/1.0.2@DEGoodmanWilson/stable: WARN: Can't find a 'base64/1.0.2@DEGoodmanWilson/stable' package for the specified settings, options and dependencies:
- Settings: arch=x86_64, build_type=Release, compiler=gcc, compiler.libcxx=libstdc++, compiler.version=8, os=Linux
- Options: shared=False
- Dependencies: 
- Package ID: 2d5685a99e2ff5a0d2b054bb1fbfa46781f8a46a

ERROR: Missing prebuilt package for 'base64/1.0.2@DEGoodmanWilson/stable'
Try to build it from sources with "--build base64"
Or read "http://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package"

The command '/bin/sh -c conan install . --build=all' returned a non-zero code: 1

[DEGoodmanWilson]

Just looking at the Debian 9 issue to start:

@CamJN The problem is that, for whatever reason, GnuTLS is not linking up right. GnuTLS calls getrandom, a syscall that is apparently not available in the kernel, but should be available with glibc. In a fit of yak shaving, I am going back to attempting to recompile GnuTLS. I have found that the org is now making the sources available on GitLab (I've had to host my own sources on Dropbox, ask me why later). These sources differ from what I had been using, and getting them to compile across all the various environments I try to support is not going so smoothly. I am hoping the difference in sources is going to have an impact on the ability to detect getrandom availability—but that's just wishful thinking. Indeed, my hope is to a) get 3.6.2 to compile, then move on to a newer version (since it turns out that 3.6.2 wasn't exactly stable anyway).

That's super long-winded and maybe not the most clear explanation ever, but the Tl;dr is that I have been yak-shaving all day, and I still haven't found a yak under all that hair. It's moments like that that lead me to despair that Conan can really support the needs of developers across all these different Linux ecosystems 😭 (Also that the GnuTLS build documentation sucks)