PythonQT Fails to Build in Debug on MSVC 2010

[awiles]

I experienced this issue while building Slicer 4.2 on Windows 7 with MSVC 2010. I then tested it outside of Slicer, building it against both the x64 version of Python built by Slicer and a x86 version of Python 2.7.2. When building inside Slicer in debug, I get this error:

2>Link:
2> Creating library D:/src/Slicer4/SlicerBuild-Debug/CTK-build/PythonQt-build/Debug/PythonQt.lib and object D:/src/Slicer4/SlicerBuild-Debug/CTK-build/PythonQt-build/Debug/PythonQt.exp
2>PythonQt.obj : error LNK2019: unresolved external symbol __imp__invalid_parameter_noinfo_noreturn referenced in function "public: int const & __cdecl std::_Vector_const_iterator<class std::_Vector_val<int,class std::allocator > >::operator*(void)const " (??D?$_Vector_const_iterator@V?$_Vector_val@HV?$allocator@H@std@@@std@@@std@@QEBAAEBHXZ)
2>D:\src\Slicer4\SlicerBuild-Debug\CTK-build\PythonQt-build\Debug\PythonQt.dll : fatal error LNK1120: 1 unresolved externals

If I change to Release mode using the PythonQt.sln inside the Slicer/CTK project, it builds fine.

I get the same error when building outside of slicer in debug, but it works fine in release both for when I build against the Slicer Python and my 32 bit Python 2.7.2/Qt 4.7.4.

I know the suggestion is to use MSVC 2008, but our software needs 2010 and I figured this should at least be reported here instead of just on the mailing lists.

[Neosettler]

Greetings everyone,

I'm having this exact same issue while compiling PythonQt path 3 that I don't know how to solve. Is there any known work around for this?

PS: Is PythonQt still maintained?

[jcfr]

Hi,

PythonQt is still maintained. The upstream is repository is [1]
We use [2] as patch "incubator" that we then contribute upstream.

We would also be happy to accept contribution to fix the reported problem.
You could do so by submitting a pull request against [2]

Thanks
Jc

[1] http://pythonqt.svn.sourceforge.net/viewvc/pythonqt/trunk/
[2] https://github.com/commontk/PythonQt#readme

On Fri, May 24, 2013 at 1:25 PM, Neosettler [email protected]:

Greetings everyone,

I'm having this exact same issue while compiling PythonQt path 3 that I
don't know how to solve. Is there any known work around for this?

PS: Is PythonQt still maintained?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-18418466
.

+1 919 869 8849

[Neosettler]

Hi jc, this issue is literately a show stopper and I'd be very glad to contribute but I've been pulling my hair for the past 2 days and I still can't figure it out, nnfortunately.

[jcfr]

Hi,

Does the problem occur for both Qt 4.7.4 and Qt 4.8.4 ?

Jc

On Fri, May 24, 2013 at 2:22 PM, Neosettler [email protected]:

Hi jc, this issue is literately a show stopper and I'd be very glad to
contribute but I've been pulling my hair for the past 2 days and I still
can't figure it out, nnfortunately.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-18421389
.

+1 919 869 8849

[Neosettler]

I'm using Python 2.7.5 and Qt 4.8.4 on Windows 8!

[jcfr]

Could you also confirme that you are using a "Debug" build of Qt ?

I was able to reproduce the error building PythonQt in Debug against a
release build of Qt. Since I don't have (yet) a debug build of Qt on my
testing machine, that is why I am asking the question.

On Fri, May 24, 2013 at 4:08 PM, Neosettler [email protected]:

I'm using Python 2.7.5 and Qt 4.8.4 on Windows 8!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-18427095
.

+1 919 869 8849

[Neosettler]

Sure,

I'm using:
python27.lib or python27_d.lib (tried with both but same result)
QtGuid4.lib
QtCored4.lib

I should also mentioned that it is a (64 bit) setup.

[Neosettler]

Side note: something is fishy with PythonQt_QtAll, somehow it is not included in the VS solution when check in CMake, thus It makes it impossible to use PythonQt_QtAll::init();

[jcfr]

Details

Looking further into the issue, I am able to compile the library after commenting the line starting with PythonQtRegisterToolClassesTemplateConverter in PythonQt.cpp [1][2]

[1] https://github.com/commontk/PythonQt/blob/patched-3/src/PythonQt.cpp#L83-89
[2] https://github.com/commontk/PythonQt/blob/patched-3/src/PythonQt.cpp#L98-130

Looking at the faulty macro PythonQtRegisterToolClassesTemplateConverter, it expands to:

PythonQtRegisterListTemplateConverter(QList, innertype); \
PythonQtRegisterListTemplateConverter(QVector, innertype); \
PythonQtRegisterListTemplateConverter(std::vector, innertype);

See https://github.com/commontk/PythonQt/blob/patched-3/src/PythonQtConversion.h#L63-66

More specifically, the line having std::vector is causing trouble:

PythonQtRegisterListTemplateConverter(std::vector, innertype);

PythonQtRegisterListTemplateConverter is also a macro. It expands to:

{
  int typeId = qRegisterMetaType<type<innertype> >(#type"<"#innertype">"); \
PythonQtConv::registerPythonToMetaTypeConverter(typeId, PythonQtConvertPythonListToListOfValueType<type<innertype>, innertype>); \
PythonQtConv::registerMetaTypeToPythonConverter(typeId, PythonQtConvertListOfValueTypeToPythonList<type<innertype>, innertype>); \
}

See https://github.com/commontk/PythonQt/blob/patched-3/src/PythonQtConversion.h#L57-61

It then turns out that the line having qRegisterMetaType it causing/illustrating the problem.

Summary

To summarize, after commenting the line referenced in [1][2], PythonQt compiled in both Debug and Release. Compilation (linking) then failed in Debug by simply re-adding the following line and compiling:

int typeId = qRegisterMetaType<std::vector<int> >("std::vector<int>");

Link error:

LNK 2001: unresolved external symbol __imp___invalid_parameter_noinfo_noreturn

Test has been done using:

• VS2010
• Python 2.7.5 32-bit - http://www.python.org/ftp/python/2.7.5/python-2.7.5.msi
• Qt 4.8.4 32-bit for VS2010 - http://download.qt-project.org/official_releases/qt/4.8/4.8.4/qt-win-opensource-4.8.4-vs2010.exe

[Neosettler]

Very well, it does fix the 64 bit version on my side.

Thank you very much for your support jc.

[jcfr]

@Neosettler Please, let me know if commit 56cdf6c also fixes the issue for 64-bit build.

[Neosettler]

Yes it does! wonderful fix and a million thx jc.

If you'd like the challenge, it would be nice if the Debug version wouldn't require python27_d.lib as it involves to recompile the whole Python suite and honestly, it makes my head spin.

[Neosettler]

***Debug version of PythonQt_QtAll.lib that is.

[Neosettler]

Hint: My guess is that there is something about the generated files (Custom Build Tool) that cant live without python27_d.lib. I tried:

editing the pyconfig.h replacing line326:
pragma comment(lib,"python27_d.lib")
by
pragma comment(lib,"python27.lib")

but there is unresolved about Py_Dealloc and others...

I also try to insert -DMS_NO_COREDLL in the Custom Build Tool with no luck still unresolved.

still digging.

[Neosettler]

in addition to line 326, commenting line 369:

//#define Py_DEBUG

...not very elegant but it works.

[jcfr]

A Debug build of PythonQt should build against a release of python without
issue. That is why there is "PYTHONQT_USE_RELEASE_PYTHON_FALLBACK".
See https://github.com/commontk/PythonQt/blob/patched-3/CMakeLists.txt#L14
and
https://github.com/commontk/PythonQt/blob/patched-3/src/PythonQtPythonInclude.h#L62-64

As a side note, building python (in release for the moment) is made very
easy :) I will publish soon some article/blog about it.
See https://github.com/davidsansome/python-cmake-buildsystem

On Mon, May 27, 2013 at 11:24 PM, Neosettler [email protected]:

in addition to line 326, commenting line 369:

//#define Py_DEBUG

...not very elegant but it works.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-18523267
.

+1 919 869 8849

[Neosettler]

Alright then, this flag was missing from the generate project. I reverted the pyconfig.h to its original state and added this PYTHONQT_USE_RELEASE_PYTHON_FALLBACK and the Debug version works like a charm now.

Your the best jc! :)

[Neosettler]

This CMake build system seems very good. Just to let you know, I've been struggling quite a lot with the PyhtonQt CMake VS for the past week. It would need more love to say that it is working out of the box. Especially for the PythonQt_QtAll not being added to the solution.

[jcfr]

Hi,

Regarding the PythonQt CMake build system, when you mentioned that
"PythonQt_Wrap_QtAll" is not added to the solution. What do you exactly
mean ?

To clarify the role of the configure time option named
"PythonQt_Wrap_QtAll", you will find more details about the build system.

At configure time, if the CMake option "PythonQt_Wrap_QtAll" is enabled
[1], the build system forces the different "PythonQt_Wrap_QtXXX" to ON [2].
It also enables the required Qt module [3].
Then, a macro "-DPYTHONQT_WRAP_Qt${qtlib}" for each wrapped library is
defined [4] and the corresponding sources are considered [5]

Finally, the macro defined previously will be used to conditionally
built-in the initialization code of each wrapped library [6] in the method
"PythonQt_init_QtBindings" that could be called from within your code by
including the "PythonQt_QtBindings.h". For example, see [7]

Let me know what you think,
Thanks
Jc

[1] https://github.com/commontk/PythonQt/blob/patched-3/CMakeLists.txt#L19
[2] https://github.com/commontk/PythonQt/blob/patched-3/CMakeLists.txt#L31
[3] https://github.com/commontk/PythonQt/blob/patched-3/CMakeLists.txt#L65
[4] https://github.com/commontk/PythonQt/blob/patched-3/CMakeLists.txt#L162
[5]
https://github.com/commontk/PythonQt/blob/patched-3/CMakeLists.txt#L168-176
[6]
https://github.com/commontk/PythonQt/blob/patched-3/generated_cpp_47/PythonQt_QtBindings.cpp
[7]
https://github.com/commontk/CTK/blob/master/Libs/Scripting/Python/Core/ctkAbstractPythonManager.cpp#L155

On Tue, May 28, 2013 at 12:01 AM, Neosettler [email protected]:

This CMake build system seems very good. Just to let you know, I've been
struggling quite a lot with the PyhtonQt CMake VS for the past week. It
would need more love to say that it is working out of the box. Especially
for the PythonQt_QtAll not being added to the solution.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-18524770
.

+1 919 869 8849

[jcfr]

If you are using the FindPythonQt.cmake module provided by CTK, you shouldn't have to defined -DPYTHONQT_USE_RELEASE_PYTHON_FALLBACK yourself.

Note that we should instead implement the mechanism allowing to configure PythonQtConfig.cmake so that FindPythonQt.cmake is not needed anymore. See #11

See also #8

[Neosettler]

Hi jc, AFAIK PythonQt and PythonQt_QtAll are two separate libraries that need to be compiled and linked against a VS dev project. Using CMake, I made about a dozen tries to embed PythonQt_QtAll::init()in the PythonQt lib without success.

I also tried the PythonQt_init_QtBindings but the #defines to enable each libraries are simply not defined anywhere in the PythonQt project. Thus, the PythonQt_init_QtBindings is not doing anything when linked in against a VS dev project.

The only way I managed to use PythonQt in my VS dev project so far is to generate the PythonQt solution with CMake and create a bach file to generate the PythonQt_QtAll VS project based on the .pro and compile it.

[ihnorton]

👍 this fixes debug build of PythonQt and CTK for MSVC 2012 Express.
Trying Slicer now with extra PYTHONQT_REVISION_TAG in cmake.

[Neosettler]

I'm trying your CMake Python and I can say that Pyhton27.lib is generated like a charm. Now, if I may, how does it work if we need the other (modules) like ctypes for instance?

[jcfr]

Thanks :) Within ccmake or cmake-gui, you should have an option named
"ENABLE_CTYPES" and "BUILTIN_CTYPES" ... the ENABLE_CTYPES should be ON by
default.

Note that ctypes is not yet available in macosx. See
python-cmake-buildsystem/python-cmake-buildsystem#18

All build option are described in the Readme file:
https://github.com/davidsansome/python-cmake-buildsystem#readme

On Tue, May 28, 2013 at 6:09 PM, Neosettler [email protected]:

I'm trying your CMake Python and I can say that Pyhton27.lib is generated
like a charm. Now, if I may, how does it work if we need the other
(modules) like ctypes for instance?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-18583680
.

+1 919 869 8849

[Neosettler]

Hi Jean-Christophe,

to make things clear, ENABLE_CTYPES and BUILTIN_CTYPES doesn't exit but ENABLE_CTYPES_TEST and BUILTIN_CTYPES_TEST does which I thought _TEST was on the confusing side. So what you are saying is that contrary to the release version of Python27 (which distributes _ctypes.lib) the _ctypes.lib will be embed inside the compiled version Python27.lib?

I compiled Python2.7.5 successfully but I'm still not sure what to do with it.

I'm sorry again if I'm stretching it but I can almost see the light at the end of the tunnel after many days of darkness. Is it possible to link my dev project to the compiled Pyhton27.lib and use the installed Python27 executable to launch my python scripts or do I need to compile the executable as well? How do I use this new generated Pyhton27.lib exactly?

[Neosettler]

Follow up...

Here is my steps:

• Compiled Python 2.7.5 from source: (BUILTIN_CTYPES_TEST and ENABLE_CTYPES_TEST checked)
• Install Python 2.7.5 (needed for the .pyd).
• Copied the Lib(from source) and DLLs(from installed .pyd) folders to my API's root/Python27 folder.
• Remove Pyhton 2.7.5 default installation.
• Add env PYTHONHOME=root\Python27
• Add env PYTHONPATH=root\Python27\Lib

When launching my app I evalFile a script that use ctypes and I'm getting :

ImportError: No module named _ctypes

Any idea what am I missing?

[Neosettler]

Edit ImportError:

File "root\Python27\Lib\ctypes__init__.py", line 10, in
from _ctypes import Union, Structure, Array
ImportError: No module named _ctypes

[jcfr]

Hi,

To install cmake'ified python in a directory of your choice, you should do the following:

cmake -DCMAKE_INSTALL_PREFIX:PATH=<root> ../python-cmake-buildsystem

or by changing the value in cmake-gui

Then, from within VisualStudio, building the target INSTALL will copy all the required files.

Within your application, setting:

• PYTHONHOME to
• PYTHONPATH to <root>/Lib;<root>/Lib/lib-dynload

should be enough. <root> should be replaced by the appropriate value.

More details about embedding python: http://docs.python.org/2/extending/embedding.html

Note also that if you depends on CTK and PythonQt, you could easily use the ctkAbstractPythonManager. See https://github.com/commontk/CTK/blob/master/Libs/Scripting/Python/Core/ctkAbstractPythonManager.h It would allow you to easily set python path, initialize context etc ...

If your application is also CMake'ified, you could very easily add a find_package(Python REQUIRED) and configuring it passing -DPython_DIR:PATH=/path/to/cmakeified-python-build-or-install-tree

Then, your application CMakeLists.txt could look like this:

cmake_minimum_required(VERSION 2.8.11)

project(myapp)

find_package(Python REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})

add_executable(myapp main.cpp)
target_link_libraries(myapp ${PYTHON_LIBRARIES})

Similar to this example: https://github.com/jcfr/python-cmake-custom-extension/blob/master/CMakeLists.txt

I think it is now time to close this issue. The discussion is out of its original scope.

If it makes sens, feel free to open an issue in the python-cmake-buildsystem tracker. See https://github.com/davidsansome/python-cmake-buildsystem/issues?state=open