Date: Thu, 23 Jan 2003 09:19:37 JST
From: Joe Mildenberger <Joe.Mildenberger@triumf.ca>
Reply-To: Joe.Mildenberger@triumf.ca, Joe.Mildenberger@triumf.ca
Organization: TRIUMF
To: Shaomin Chen <chensm@sitka.triumf.ca>,
Jingliang Hu
<hujl@sitka.triumf.ca>,
Akira Konaka <konaka@sitka.triumf.ca>,
Tetsuro Sekiguchi
<sekiguti@post.kek.jp>,
David E Jaffe <djaffe@bnlku5.phy.bnl.gov>,
Dima Vavilov
<vavilov@triumf.ca>, "Lewis, Benji" <benjil@bnl.gov>
Subject: Distributed paw

Hi All,
In order to be able to run through many thousands of histogram files in a
reasonable amount of time, I have built a new facility into
$PASS2_ANAL/build_func.csh that will allow one to run separate paw jobs
simultaneously on all of the machines on which the ntuple files reside, with
each job reading only the files local to that machine. Here is what you need to
do to make use of this facility, which for the time being operates only on the
TRIUMF kdecay linux cluster.

1. setup pass2 02 ; cd $your_working_directory

2. Set up your cut functions, and choose your data set as indicated in
yesterday's message regarding "pass2 ntuple links".

3. setenv DIST_PAW dist

4. Run $PASS2_ANAL/build_func.csh on your myjob.def file. Be patient: with
approx. 23000 pass2 ntuple files it takes perhaps 15-20 minutes to build all of
the necessary kumac files for running on the different machines.

5. Edit the "master" kumac file (called run_myjob.kumac) to book your
histograms, and edit myjob.f to fill them and do whatever else you want the job
to do.

6. Copy the scripts run_dist_paw.csh and remote.setup from
nunubar:/home/joseph/pass2/phveto to your working directory;
submit the pawjobs via

run_dist_paw.csh myjob

[N.b. if you do not have your account set up for passwordless ssh logins, you
will have to enter your password for each machine the job is going to run on.]

7. Each job will produce a histogram file called histfile.myjob.$i.$host where
$host is the name of the machine that the paw job ran on, and $i is a number,
incremented every time you run that job (if you delete the old histogram files
before running the job, $i will always be 1...)

8. Merge the histogram files using the ntuple/hmerge command in paw (see the paw
help facility for details).


By doing this, you should be able to cut down on the time that it takes to loop
through the entire data set by about a factor of 10 or more, compared with
running through them sequentially on a particular machine, reading them over the
network.

[N.b. I have only tested this on my own account; it is possible that minor
adjustements may need to be made in some cases...]

Joe