Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OSX: CFITSIO version mismatch warning #38

Open
pochoi opened this issue Aug 12, 2017 · 9 comments
Open

OSX: CFITSIO version mismatch warning #38

pochoi opened this issue Aug 12, 2017 · 9 comments

Comments

@pochoi
Copy link

pochoi commented Aug 12, 2017 

julia> versioninfo()
Julia Version 0.6.0
Commit 903644385b (2017-06-19 13:05 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin13.4.0)
  CPU: Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

I guess it is not your fault. Just let you know some warnings appear since your last update.

julia> using LibHealpix
INFO: Precompiling module LibHealpix.
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:146
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:146
in anonymous at <missing>
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:318
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:329
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:340
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:352
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:386
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:386
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:386
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:405
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:412
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:424
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:424
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:424
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:424
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:424
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:424
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libcfitsio.jl:424
WARNING: Compat.ASCIIString is deprecated, use String instead.
  likely near /Users/po/.julia/v0.6/FITSIO/src/libjl_uv_writecb() ERROR: operation canceled ECANCELED
jl_uv_writecb() ERROR: operation canceled ECANCELED
jl_uv_writecb() ERROR: operation canceled ECANCELED
...
... nearly 500 lines repeated
...
jl_uv_writecb() ERROR: operation canceled ECANCELED
jl_uv_writecb() ERROR: operation canceled ECANCELED

WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).

And the warning

WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).`

appears each time when using LibHealpix and then running function query_disc.
@mweastwood
Copy link
Owner

Thanks for the bug report! I believe a lot of the deprecation warnings in FITSIO are fixed on the latest master, so running Pkg.checkout("FITSIO") should help.

I also see the "version mismatch" warning in the Travis logs. Do you get this warning when using FITSIO in isolation? I suspect this is caused by FITSIO.jl and LibHealpix.jl installing two competing copies of libcfitsio. Can you run ldd on libhealpixwrapper.so? If I can figure out how to build healpix without linking to libcfitsio then this should no longer be a problem.

However I don't see the jl_uv_writecb() error in the Travis logs. Does the error appear consistently? Do you have any sense for the conditions on when it happens?

@mweastwood mweastwood mentioned this issue Aug 12, 2017
Merged
@pochoi
Copy link
Author

pochoi commented Aug 13, 2017

Went into LibHealpix/deps/usr/lib and run otool -L (the osx's ldd)

✔ ~/.julia/v0.6/LibHealpix/deps/usr/lib [v0.2.2|✔]
$ otool -L libhealpixwrapper.dylib
libhealpixwrapper.dylib:
	libhealpixwrapper.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib/libchealpix.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib/libhealpix_cxx.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

All functions in LibHealpix seem to be running fine. Just the version mismatch warning pops up when running those functions depending on that library.

I tried to reinstall LibHealpix but the error jl_uv_writecb() does not pop up again.

julia> Pkg.rm("LibHealpix")
INFO: Removing FITSIO v0.10.0
INFO: Removing LibHealpix v0.2.2
INFO: Package database updated

Restart Julia

julia> Pkg.add("LibHealpix")
INFO: Installing FITSIO v0.10.0
INFO: Installing LibHealpix v0.2.2
INFO: Building FITSIO
INFO: Building Homebrew
INFO: Building LibHealpix
WARNING: Compat.KERNEL is deprecated.
  likely near /Users/po/.julia/v0.6/LibHealpix/deps/build.jl:143
WARNING: Compat.KERNEL is deprecated.
  likely near /Users/po/.julia/v0.6/LibHealpix/deps/build.jl:143
in can_use at /Users/po/.julia/v0.6/Homebrew/src/bindeps_integration.jl
WARNING: Compat.KERNEL is deprecated.
  likely near /Users/po/.julia/v0.6/LibHealpix/deps/build.jl:143
in package_available at /Users/po/.julia/v0.6/Homebrew/src/bindeps_integration.jl

WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).


WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).


WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).


WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).

INFO: Package database updated

julia> using LibHealpix

julia> query_disc(512, LibHealpix.ring, 0, 0, deg2rad(10/60), inclusive=false)

WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).

4-element Array{Int32,1}:
 1
 2
 3
 4

The last time I actually did rm -rf .julia/v0.6 and then reinstalled anything and then got those jl_uv_writecb(). Not trying it again that this moment. 😳

@mweastwood
Copy link
Owner

Hmm. I still don't know which library is being linked against the wrong version of cfitsio. Can you check libchealpix and libhealpix_cxx as well?

@mweastwood
Copy link
Owner

Ok after doing a little digging it looks like the version mismatch warning is coming from libhealpix_cxx. This library is being built by homebrew, but otool -L shows that it's at least being linked against the libcfitsio from homebrew, as expected. I think this warning is somehow related to cfitsio being moved out of the science tap.

The deprecation warnings are mostly due to FITSIO.jl, but I think they are fixed on the master branch and that package just needs a new tag.

But I have no idea where the jl_uv_writecb() errors came from. Don't even know where to begin with that. If those show up again in a reproducible way, open a separate issue for that.

@mweastwood mweastwood changed the title Warning from CFITSIO OSX: CFITSIO version mismatch warning Aug 13, 2017
@mweastwood
Copy link
Owner

So on OSX Travis I get

$ grep -r CFITSIO_VERSION /Users/travis/.julia/v0.6/FITSIO/deps/usr
/Users/travis/.julia/v0.6/FITSIO/deps/usr/include/fitsio.h:#define CFITSIO_VERSION 3.37

$ grep -r CFITSIO_VERSION /Users/travis/.julia/v0.6/Homebrew/deps/usr
/Users/travis/.julia/v0.6/Homebrew/deps/usr/Cellar/cfitsio/3.410/include/fitsio.h:#define CFITSIO_VERSION 3.41
/Users/travis/.julia/v0.6/Homebrew/deps/usr/include/fitsio.h:#define CFITSIO_VERSION 3.41
/Users/travis/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/include/fitsio.h:#define CFITSIO_VERSION 3.41
/Users/travis/.julia/v0.6/Homebrew/deps/usr/var/homebrew/linked/cfitsio/include/fitsio.h:#define CFITSIO_VERSION 3.41

So FITSIO.jl is installing headers for version 3.37 and Homebrew.jl is installing headers for version 3.41. I have no idea where the headers for 3.39 are coming from. So Homebrew must be distributing binaries that were somehow built with the wrong headers. I don't understand how Homebrew works though. Is it even distributing binaries or build recipes?

@pochoi
Copy link
Author

pochoi commented Aug 14, 2017

In Homebrew or Homebrew.jl, Bottles are the binary packages. Usually the bottles (binaries) are downloaded and installed. For example, I have the caches of the healpix and cfitsio bottles downloaded by Homebrew.jl

✔ ~/Library/Caches/Homebrew.jl
22:48 $ ls | grep -e fits -e healpix
cfitsio-3.390.yosemite.bottle.tar.gz
cfitsio-3.410.yosemite.bottle.tar.gz
healpix-3.31.yosemite.bottle.tar.gz

@pochoi
Copy link
Author

pochoi commented Aug 14, 2017 •
edited
Loading

More otool -L

✔ ~/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib [1.2.3|✔]
$ ls
libchealpix.0.dylib    libchealpix.a          libchealpix.dylib      libhealpix_cxx.0.dylib libhealpix_cxx.a       libhealpix_cxx.dylib   pkgconfig

$ otool -L libchealpix.dylib
libchealpix.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib/libchealpix.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.39)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

$ otool -L libchealpix.0.dylib
libchealpix.0.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib/libchealpix.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.39)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

$ otool -L libhealpix_cxx.dylib
libhealpix_cxx.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib/libhealpix_cxx.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.39)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)

$ otool -L libhealpix_cxx.0.dylib
libhealpix_cxx.0.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib/libhealpix_cxx.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.39)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
✔ ~/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib [1.2.3|✔]
$ ls
libcfitsio.5.3.41.dylib libcfitsio.5.dylib      libcfitsio.a            libcfitsio.dylib        pkgconfig

$ otool -L libcfitsio.dylib
libcfitsio.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.41)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

$ otool -L libcfitsio.5.dylib
libcfitsio.5.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.41)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

$ otool -L libcfitsio.5.3.41.dylib
libcfitsio.5.3.41.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.41)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

In the Homebrew.jl, we have cfitsio, which is installed as dependency of healpix in Homebrew.jl

✔ ~/.julia/v0.6/Homebrew/deps/usr/include [1.2.3|✔]
23:50 $ grep -r CFITSIO_VERSION .
./fitsio.h:#define CFITSIO_VERSION 3.41

For FITSIO.jl, I didn't have the folder .julia/v0.6/FITSIO/deps/usr/include/. I looked into the deps.jl, it was using Homebrew's cfitsio I installed previously (not Homebrew.jl's cfitsio installed by LibHealpix.jl).

✔ /usr/local/include
$ grep -r CFITSIO_VERSION .
./fitsio.h:#define CFITSIO_VERSION 3.41

Then I just uninstalled the Homebrew's cfitsio, and then I reinstalled LibHealpix.jl and FITSIO.jl. Then FITSIO.jl downloaded and compiled cfitsio from source:

✔ ~/.julia/v0.6/FITSIO/deps/usr [v0.10.0|✔]
$ grep -r CFITSIO_VERSION .
./include/fitsio.h:#define CFITSIO_VERSION 3.37

After uninstalling Homebrew's cfitsio and reinstalling LibHealpix.jl and FITSIO.jl. The version mismatch is still here
WARNING: version mismatch between CFITSIO header (v3.39) and linked library (v3.41).

Summary (Guess):

Maybe the cfitsio of FITSIO.jl is innocent, as it got v3.37

In the Homebrew.jl, by the otool -L

/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.41)

/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.39)

Maybe it is where the version mismatch comes from?

Here the Healpix people talked about it too.

@mweastwood
Copy link
Owner

Maybe the cfitsio of FITSIO.jl is innocent, as it got v3.37

I agree.

In Homebrew or Homebrew.jl, Bottles are the binary packages. Usually the bottles (binaries) are downloaded and installed.

As far as I could tell, everything is linked correctly on your system and the Travis OSX worker. So I believe the binary package provided by Homebrew is to blame.

(note to myself: the Healpix bottles are hosted at https://bintray.com/homebrew/bottles-science/healpix/3.31#files)

Here the Healpix people talked about it too.

I wonder why they chose to strip out the version check instead of building with the right headers. They do mention that the ABI hasn't changed, so the warning is harmless (albeit annoying).

Would you mind trying brew install healpix --build-from-source to see if the error persists without using the binaries?

@pochoi
Copy link
Author

pochoi commented Aug 15, 2017

In Homebrew.jl (not my system Homebrew. Homebrew.jl is isolated from the system Homebrew for obvious reason), I install healpix build from source

import Homebrew
Homebrew.rm("homebrew/science/healpix")
Homebrew.brew(`install homebrew/science/healpix --build-from-source`)

🎉 Now the version mismatch warning in LibHealpix.jl is gone!

And also it is now 5.3.41 instead of 5.3.39

✔ ~/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib [1.2.3|✔]
19:42 $ otool -L libchealpix.dylib
libchealpix.dylib:
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/healpix/lib/libchealpix.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Users/po/.julia/v0.6/Homebrew/deps/usr/opt/cfitsio/lib/libcfitsio.5.dylib (compatibility version 5.0.0, current version 5.3.41)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

After consulting my friend Randy, I learnt that it is a dependency issue happen sometime in homebrew. I guess the warning is because the healpix bottle had been built (https://github.com/Homebrew/homebrew-science/commit/f21e92b475acc0faac890beb98bd8179a75b7c35) with old version of cfitsio in homebrew but cfitsio's bottles were just updated a few day ago (Homebrew/homebrew-core@cad302c). So it is just a coincidence, as you also updated LibHealpix.jl a few day ago.

Now the immediate solution is to manually build the healpix from source in Homebrew.jl

Homebrew.brew(`install homebrew/science/healpix --build-from-source`)

I can wait until the next update on healpix.rb, or I submit a pull request to get the bottles recompiled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mweastwood @pochoi