[slicer-devel] Slicer batch mode

Steve Pieper pieper at isomics.com
Sun Feb 22 11:50:32 EST 2015


By the way Robin I should have mentioned that it should be possible to use
Xdummy as a non-root user so you could use that solution on your cluster.

https://www.xpra.org/trac/wiki/Xdummy

I'd be curious to know if that would solve your use case.

I know some would say that creating an X environment like this is kind of a
hack, but to be honest it's not much memory overhead by modern standards
and it solves a lot of headaches easily.

-Steve

On Sun, Feb 22, 2015 at 9:58 AM, Csaba Pinter <csaba.pinter at queensu.ca>
wrote:

>  Hi Robin,
>
>
>
> We created something along this line a few months ago
>
>
> https://subversion.assembla.com/svn/slicerrt/trunk/SlicerRt/src/BatchProcessing/
>
>
>
> Not sure if this is any close to what you’d like to have, nor if it can be
> run from the environment you have. But sure enough, this script doesn’t use
> or display any graphics, and is purely driven by command-line arguments.
>
>
>
> Hth,
>
> csaba
>
>
>
> *From:* slicer-devel-bounces at bwh.harvard.edu [mailto:
> slicer-devel-bounces at bwh.harvard.edu] *On Behalf Of *Steve Pieper
> *Sent:* February 21, 2015 09:04
> *To:* Robin Weiss
> *Cc:* Slicer Devel List
> *Subject:* Re: [slicer-devel] Slicer batch mode
>
>
>
> Hi Robin -
>
>
>
> You are totally right - making slicer run in a headless environment is
> something we should have done and just haven't gotten around to doing it.
> Encouragement and help from the community is probably what's that's needed
> to make this happen - so thanks for bringing it up!  I'll take this
> opportunity to lay out some thoughts and experiences on this topic.
>
>
>
>
>
> Internally, Slicer is pretty much organized to keep the graphics-dependent
> parts modularized from the non-graphics parts.  Much of the graphics code
> relates to vtk rendering or qt interface parts.  In slicer core and
> modules, MRML and module Logic are not allowed to depend (by policy) on
> anything related to rendering.  Only displayable managers and GUIs should
> be touching the graphics and they only act in response to events from
> MRML.  Within the Qt side of things there is the QCoreApplication which is
> the event loop and application infrastructure but free of rendering; and
> the QApplication which adds rendering.  Slicer follows this layering with
> the qSlicerCoreApplication and qSlicerApplication classes.
>
>
>
> In practice, a lot of the use cases that drive Slicer development have
> interactive graphics at their heart and so batch mode gets a little
> forgotten.  And all developers sometimes mix too much of their logic into
> the gui even though they know better.  But as a general rule we would like
> to be able to run a headless Slicer where the logic and mrml classes can be
> fully functional even without any rendering or gui.  It would be a good
> project to hunt down all the code that prevents making a headless Slicer
> now - might be a good activity for a project week, for example.
>
>
>
>
>
> You mentioned a workaround so probably you are all set, but let me mention
> a couple things I have done for working on headless compute nodes and
> explain why headless slicer hasn't been a priority, at least for me.
>
>
>
> One thing I've used when running on cluster nodes is Xvfb (virtual frame
> buffer, an aging but useful utility).  I wrote a wrapper called runvnc [1]
> so if you have your keys all set up you could run a job like
>
>
>
>  ssh compute-0 runvnc slicer2 <processing-script>
>
>
>
> or you could submit that style of command to a job queue.  This had the
> nice property that I could use vnc to connect to any of the compute nodes
> to look at the slicer process.
>
>
>
> The only real problem with that approach, as you pointed out Robin, is
> that old-school cluster administrators don't want X installed on their
> compute nodes, so you had to have root access to the nodes to install
> Xvfb.  Nowadays however cloud vms do provide root access and you can
> specify, for example, an ec2 ami that includes X without a problem.  So if
> you have that kind of a cluster you can install X libraries with no problem
> (really it's not such a big overhead to add a virtual framebuffer and a few
> extra libraries).
>
>
>
> So classic Xvfb is still an option, but getting OpenGL to run correctly
> was always a hassle.   Things have actually gotten better in virtual X
> recently though with the creation of Xpra and Xdummy.  On modern
> cloud/cluster environments where you have root access to your virtual
> machine it's pretty easy to set up.  I investigated this last year [2] and
> used it with travis-ci to make an experimental dashboard machine [3].
>
>
>
> Basically you can do something like this:
>
>
>
>  sudo apt-get install -y xpra xserver-xorg-video-dummy
>
>
>
>  xpra --xvfb=\"Xorg +extension GLX -config my.xorg.conf -logfile
> ${HOME}/.xpra/xorg.log\"  start :9
>
>
>
>  env DISPLAY=:9 ./Slicer --python-script <script>
>
>
>
> I've played a little with starcluster [4] running on amazon but I haven't
> really tried scaling this up to do something practical.  It should work
> though.  I don't know if that's an option for you or if you want to make
> use of your own cluster with the X-less nodes.
>
>
>
> Note that the Xpra/Xdummy/Xvfb option is also good for rendering, since
> you can make an arbitrarily large virtual frame buffer and render into it
> with regular slicer, for example to make publication resolution renderings.
>
>
>
>
>
> So in summary I can see the value of headless slicer and think it can and
> should be done.  But it hasn't been in my critical path because I find the
> virtual X workaround to be valuable in itself and it's becoming more
> realistic for the kind of clusters I'm likely to use.
>
>
>
> Thanks again for bringing this up!
>
> -Steve
>
>
>
>
>
>
>
> [1] https://github.com/pieper/slicer2/blob/master/Scripts/runvnc
>
>
>
> [2]
> http://www.na-mic.org/Wiki/index.php/2014_Summer_Project_Week:_Factory_and_Testing_Process_Post_NA-MIC
>
>
>
> [3] https://github.com/pieper/CTK/blob/master/.travis.yml
>
>
>
> [4] http://star.mit.edu/cluster/
>
>
>
> On Fri, Feb 20, 2015 at 6:12 PM, Robin Weiss <robinweiss at uchicago.edu>
> wrote:
>
>  Hi All,
>
>
>
> Not sure I have real a question per se (suggestions are of course welcome)
> but I wanted to toss something out there regarding Slicer in batch mode.
> I'm working on a Slicer python script that converts a big pile of dicoms to
> an organized directory structure of nrrds.  It's a pretty big pile of
> dicoms and the indexing and conversion process is taking a stupid amount of
> time.  But hey, it's a good thing I have a cluster at my disposal...
>
>
>
> After a good amount of time hunting for the --python-script option to the
> Slicer launcher (which, by the way, could probably be better documented) I
> managed to get my script running.  However, I quickly discovered that even
> "Slicer --help" won't run without an X server.  This presents a bit of a
> problem for me since I'd like to submit a bunch of copies of this script to
> my cluster's head-less X-less compute nodes.  I've managed to find a
> workaround but it's certainly less than ideal.
>
>
>
> I was wondering if there's been any conversation about creating a proper
> batch interface for Slicer.  Even if this would just look like a mode that
> delivers you directly to Slicer's python interactor with all it's Slicer-y
> goodness sans any of the graphics side of things, it would be great.  I
> assume this is easier said than done since surely there's a bunch of
> components behind the scenes that take it for granted they'll be able to
> create graphics.
>
>
>
> Anyway, I just wanted to pass this experience along to the group.  I think
> a --no-display option would definitely be a great feature to see in Slicer
> at some point so it definitely gets a +1 from me.
>
>
>
> Cheers,
>
> Robin
>
>
>
> --
>
> Robin M. Weiss
>
> Research Application Engineer
>
> Research Computing Center
>
> The University of Chicago
>
> 6030 S. Ellis Ave., Suite 289C
>
> Chicago, IL 60637
>
> 773.702.9030
>
> robinweiss at uchicago.edu
>
>
>
>
> _______________________________________________
> slicer-devel mailing list
> slicer-devel at bwh.harvard.edu
> http://massmail.spl.harvard.edu/mailman/listinfo/slicer-devel
> To unsubscribe: send email to slicer-devel-request at bwh.harvard.edu with
> unsubscribe as the subject
>
> http://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Developers/FAQ
>
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://massmail.spl.harvard.edu/public-archives/slicer-devel/attachments/20150222/b969edb3/attachment-0001.html 


More information about the slicer-devel mailing list