[slicer-devel] Final solution to the too long build directory issue?

Steve Pieper pieper at isomics.com
Tue Aug 26 18:54:46 EDT 2014


Awesome!!!


On Tue, Aug 26, 2014 at 6:50 PM, Andras Lasso <lasso at queensu.ca> wrote:

>  Hi all,
>
>  I have good news! I had a look at this path length issue again, and it
> seems that there may be a clean and definitive solution.
>
>  => Problem
>
>  Maximum command line length on Windows for cmd.exe is 8192 (32767 for
> CreateProcess). If you pass longer than 8192 then the behavior is undefined
> (actually, 8192th character is removed and the rest of the string is
> passed). CMake passes parameters to the compiler or linker using a
> parameter file (instead of command-line arguments) when it finds that the
> command-line would be too long.
>
>  However, CMake cannot implement this mechanism for custom commands,
> that's up to the developer who adds a custom command!
>
>  => Example
>
>  The particular issue that caused build error for Jim is due to a custom
> command defined in ctkMacroWrapPythonQt.cmake:
> -----
>    add_custom_command(
>     OUTPUT
>       ${wrapper_init_cpp_filename}
>       ${wrapper_h_filename}
>     DEPENDS
>       ${SOURCES_TO_WRAP}
>       ${CTK_CMAKE_DIR}/ctkWrapPythonQt.py
>     COMMAND ${PYTHON_EXECUTABLE} ${CTK_CMAKE_DIR}/ctkWrapPythonQt.py
>       --target=${TARGET}
>       --namespace=${WRAPPING_NAMESPACE}
>       --output-dir=${CMAKE_CURRENT_BINARY_DIR}/${wrap_int_dir}
> ${extra_args}
>       ${SOURCES_TO_WRAP}
>     COMMENT "PythonQt Wrapping - Generating ${wrapper_init_cpp_filename}"
>     VERBATIM
>     )
> -----
>
>  The ${SOURCES_TO_WRAP} variable can contain hundreds of full paths,
> which of course can easily grow over 8192. For example, a command like this
> can be generated (see in the generated CTKWidgetsPythonQt.vcxproj file):
>
>  -----
>      <CustomBuild
> Include="..\..\CMakeFiles\d4e9ed124a08aa4e815590aff2c25a3c\org_commontk_CTKWidgets_init.cpp.rule">
>       <Message
> Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">PythonQt Wrapping -
> Generating
> generated_cpp/org_commontk_CTKWidgets/org_commontk_CTKWidgets_init.cpp</Message>
>       <Command
> Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">setlocal
> C:\S4D\python-install\bin\SlicerPython.exe
> C:/S4D/CTK/CMake/ctkWrapPythonQt.py --target=CTKWidgets
> --namespace=org.commontk
> --output-dir=C:/S4D/CTK-build/CTK-build/Libs/Widgets/generated_cpp/org_commontk_CTKWidgets/
> C:/S4D/CTK/Libs/Widgets/ctkActionsWidget.h
> C:/S4D/CTK/Libs/Widgets/ctkAddRemoveComboBox.h
> C:/S4D/CTK/Libs/Widgets/ctkAxesWidget.h
> C:/S4D/CTK/Libs/Widgets/ctkBasePopupWidget.h
> C:/S4D/CTK/Libs/Widgets/ctkButtonGroup.h
> C:/S4D/CTK/Libs/Widgets/ctkCheckableComboBox.h
> C:/S4D/CTK/Libs/Widgets/ctkCheckableHeaderView.h
> C:/S4D/CTK/Libs/Widgets/ctkCheckableModelHelper.h
> C:/S4D/CTK/Libs/Widgets/ctkCheckBox.h
> C:/S4D/CTK/Libs/Widgets/ctkCheckBoxPixmaps.h
> C:/S4D/CTK/Libs/Widgets/ctkCheckablePushButton.h
> C:/S4D/CTK/Libs/Widgets/ctkComboBox.h
> C:/S4D/CTK/Libs/Widgets/ctkCompleter.h
> C:/S4D/CTK/Libs/Widgets/ctkCollapsibleButton.h
> C:/S4D/CTK/Libs/Widgets/ctkCollapsibleGroupBox.h
> C:/S4D/CTK/Libs/Widgets/ctkColorDialog.h
> C:/S4D/CTK/Libs/Widgets/ctkColorPickerButton.h
> C:/S4D/CTK/Libs/Widgets/ctkConsole.h
> C:/S4D/CTK/Libs/Widgets/ctkCoordinatesWidget.h
> C:/S4D/CTK/Libs/Widgets/ctkCrosshairLabel.h
> C:/S4D/CTK/Libs/Widgets/ctkDateRangeWidget.h
> C:/S4D/CTK/Libs/Widgets/ctkDirectoryButton.h
>> many many more similar lines here
>> C:/S4D/CTK/Libs/Widgets/ctkMenuComboBoxEventTranslator.h
> C:/S4D/CTK/Libs/Widgets/ctkPathLineEditEventPlayer.h
> C:/S4D/CTK/Libs/Widgets/ctkPathLineEditEventTranslator.h
> C:/S4D/CTK/Libs/Widgets/ctkPopupWidgetEventPlayer.h
> C:/S4D/CTK/Libs/Widgets/ctkPopupWidgetEventTranslator.h
> C:/S4D/CTK/Libs/Widgets/ctkRangeSliderEventPlayer.h
> C:/S4D/CTK/Libs/Widgets/ctkRangeSliderEventTranslator.h
> C:/S4D/CTK/Libs/Widgets/ctkTreeComboBoxEventPlayer.h
> C:/S4D/CTK/Libs/Widgets/ctkTreeComboBoxEventTranslator.h
> if %errorlevel% neq 0 goto :cmEnd
> :cmEnd
> endlocal &amp; call :cmErrorLevel %errorlevel% &amp; goto :cmDone
> :cmErrorLevel
> exit /b %1
> :cmDone
> if %errorlevel% neq 0 goto :VCEnd</Command>
>  -----
>
>  => Solution
>
>  Fortunately, there is a nice, clean solution to this issue: pass the
> ${SOURCES_TO_WRAP} list to the python script through a file, not through
> command-line.
>
>  There may be other instances where long lists are attempted to pass
> through command-line parameters, so fixing this issue may not allow us to
> use arbitrarily long paths yet, but if we fix these issues one by one as
> they pop up, finally we may end up fixing all of them.
>
>  Andras
>
>
>  *From:* Jean-Christophe Fillion-Robin <jchris.fillionr at kitware.com>
> *Sent:* ‎Tuesday‎, ‎August‎ ‎26‎, ‎2014 ‎9‎:‎30‎ ‎AM
> *To:* Csaba Pinter <csaba.pinter at queensu.ca>
> *Cc:* slicer-devel at bwh.harvard.edu
>
>    Hi Folks,
>
>  @James: Yes - This is weird but this is a limitation of windows. With the
> transition to VTK6, this became a problem.
>
>
>  The good news, is that while working on Slicer issue 3809 [1], we now
> build VTK with VTK_ENABLE_KITS options which reduce the number of linked
> library and should help with problem.
>
>  It also means that CMake 3.0.0 will become a hard requirement to build
> Slicer.
>
> [1] http://na-mic.org/Mantis/view.php?id=3809
>
>
> On Tue, Aug 26, 2014 at 10:25 AM, Csaba Pinter <csaba.pinter at queensu.ca>
> wrote:
>
>>  Apparently VS handles "too long" command lines in a wrong way, which
>> means exactly that it loses one character at position 16384 or some power
>> of 2. I personally use c:\d\S4D to be on the safe side.
>>
>>
>>
>> Not sure how we could get rid of this problem, I haven't tried VS2012
>> with longer paths, but that version works nicely with Slicer for sure in
>> the same conditions VS2008 works. Could be worth a try.
>>
>>
>>
>> csaba
>>
>>
>>
>> *From:* slicer-devel-bounces at bwh.harvard.edu [mailto:
>> slicer-devel-bounces at bwh.harvard.edu] *On Behalf Of *Miller, James V (GE
>> Global Research)
>> *Sent:* August 26, 2014 10:09
>> *To:* Steve Pieper
>>
>> *Cc:* Slicer Developers Mailing List (slicer-devel at bwh.harvard.edu)
>> *Subject:* Re: [slicer-devel] VS2008 build
>>
>>
>>
>> I’m trying a shorter path. It is frustrating that this is a build that
>> used to work.
>>
>>
>>
>> But I don’t understand how the path length could produce the error being
>> reported.  It looks to me that somewhere an include file name is incorrect.
>> There is a letter missing in the middle of the include file name.
>>
>>
>>
>> I understand how the include path could be truncated because of the
>> command line length and the include file not be found.
>>
>>
>>
>> But here, the include file that cannot be found legitimately does not
>> exist.  It is missing a letter in the middle of the filename.
>>
>>
>>
>> Jim
>>
>>
>>
>>
>>
>>
>>
>> *From:* Steve Pieper [mailto:pieper at isomics.com <pieper at isomics.com>]
>> *Sent:* Monday, August 25, 2014 9:42 AM
>> *To:* Miller, James V (GE Global Research)
>> *Cc:* Slicer Developers Mailing List (slicer-devel at bwh.harvard.edu)
>> *Subject:* Re: [slicer-devel] VS2008 build
>>
>>
>>
>> That definitely looks like the 'windows path is too long' issue as Andras
>> pointed out.  I use c:/pieper/s4/s as my superbuild directory and it's
>> working for me (win 7, vs2008sp1).
>>
>>
>>
>> -Steve
>>
>>
>>
>> On Mon, Aug 25, 2014 at 9:17 AM, Miller, James V (GE Global Research) <
>> millerjv at ge.com> wrote:
>>
>> Looks to me that somewhere
>>
>>
>>
>> ctkPopupWidgetEventlayer.h
>>
>>
>>
>> is being included instead of
>>
>>
>>
>> ctkPopupWidgetEvent*P*layer.h
>>
>>
>>
>> But I haven’t been able to find it. Could there be a fault regexp
>> somewhere?
>>
>>
>>
>> Jim
>>
>>
>>
>>
>>
>> *From:* Miller, James V (GE Global Research)
>> *Sent:* Monday, August 25, 2014 8:30 AM
>>
>>
>> *To:* Slicer Developers Mailing List (slicer-devel at bwh.harvard.edu)
>>
>> *Subject:* VS2008 build
>>
>>
>>
>> I have been trying to update/redo a Windows build of Slicer over the past
>> week. I deleted my build tree and started fresh. I  had a few issues in the
>> process like having to get SP1 installed onto my corporate image of VS2008
>> (SP1 is now a requirement whereas when I configured this build tree
>> originally, SP1 was not a requirement).
>>
>>
>>
>> I usually use the default build settings with the exception of turning
>> off Python Qt With Tcl.
>>
>>
>>
>> I get a consistent build error with CTK
>>
>>
>>
>> 32>6>16>  File
>> "C:/Projects/Slicer4/Slicer-build/CTK/CMake/ctkWrapPythonQt.py", line 226,
>> in <module>
>>
>> 32>6>16>    ctk_wrap_pythonqt(options.target, options.namespace,
>> options.output_dir, args, options.extra_verbose)
>>
>> 32>6>16>  File
>> "C:/Projects/Slicer4/Slicer-build/CTK/CMake/ctkWrapPythonQt.py", line 77,
>> in ctk_wrap_pythonqt
>>
>> 32>6>16>    with open(input_file) as f:
>>
>> 32>6>16>IOError: [Errno 2] No such file or directory:
>> 'C:/Projects/Slicer4/Slicer-build/CTK/Libs/Widgets/ctkPopupWidgetEventlayer.h'
>>
>> 32>6>16>Project : error PRJ0019: A tool returned an error code from
>> "PythonQt Wrapping - Generating
>> generated_cpp/org_commontk_CTKWidgets/org_commontk_CTKWidgets_init.cpp"
>>
>>
>>
>> I have updated Slicer and wiped out the CTK trees several times.
>>
>>
>>
>> This error is not appearing on the dashboard.
>>
>>
>>
>> Jim
>>
>>
>>
>>
>>
>> Jim Miller
>>
>> Principal Scientist
>>
>> GE Research
>>
>> Interventional and Therapy
>>
>>
>>
>> GE imagination at work
>>
>>
>>
>>
>> _______________________________________________
>> slicer-devel mailing list
>> slicer-devel at bwh.harvard.edu
>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>> To unsubscribe: send email to
>> slicer-devel-request at massmail.spl.harvard.edu with unsubscribe as the
>> subject
>>
>> http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Developers/FAQ
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you
>> in error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>>
>>
>>
>> _______________________________________________
>> slicer-devel mailing list
>> slicer-devel at bwh.harvard.edu
>> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
>> To unsubscribe: send email to
>> slicer-devel-request at massmail.spl.harvard.edu with unsubscribe as the
>> subject
>>
>> http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Developers/FAQ
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you
>> in error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>>
>>
>
>
> --
> +1 919 869 8849
>
> _______________________________________________
> slicer-devel mailing list
> slicer-devel at bwh.harvard.edu
> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
> To unsubscribe: send email to
> slicer-devel-request at massmail.spl.harvard.edu with unsubscribe as the
> subject
>
> http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Developers/FAQ
>
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://massmail.spl.harvard.edu/public-archives/slicer-devel/attachments/20140826/a543320a/attachment-0001.html 


More information about the slicer-devel mailing list