************************************************************************
* The files in this directory illustrate how to use TAU to perform     *
* profiling and tracing of a C code. The TAU_COMPILER script invoked   *
* by the Makefile automatically instruments the files "*.c" and        *
* generates (temporary) files "*.inst.c" that are then compiled.       *
* See files Makefile and Makefile.inc.* for details and platform       *
* specifics.                                                           *
************************************************************************

The Makefile generates "pddrive.x"

  [modules are needed on bassi@nersc, jacquard@nersc and seaborg@nersc]

  % module load tau superlu_dist

  % make

Then, "pddrive.x" can be executed by:

  % setenv COUNTER1 GET_TIME_OF_DAY
  % setenv COUNTER2 PAPI_FP_INS

  [on bassi@nersc and seaborg@nersc]

  % poe pddrive.x -r 2 -c 4 lns_3937.rua -procs 8

  [on jacquard@nersc]

  % mpirun -np 8 pddrive.x -r 2 -c 4 lns_3937.rua

  [on datastar@sdsc]

  % poe pddrive.x -r 2 -c 4 lns_3937.rua -nodes 1 -tasks_per_node 8

or

  [on bassi@nersc and seaborg@nersc]

  % llsubmit pddrive.ll

  [on jacquard@nersc]

  % qsub pddrive.pbs

where

r = process rows
c = process columns
procs = r * c

Upon completion, files named "profile.<node>.<context>.<thread>" are
created. The results from the profiling can be seen with "pprof" (this
is the TAU version of "pprof", which is loaded with the tau module),
for example:

  % cd MULTI__PAPI_FP_INS
  % pprof > pddrive.pprof

The results can be visualized with 

  % module load java
  % paraprof &

or alternatively with

  % module load tcltk
  % racy &

If "pddrive.x" is instrumented with trace then event and trace files 
named "events.<node>.edf" and "tautrace.<node>.<context>.<thread>.trc" 
are also created. The trace files can be merged by using "tau_merge":

  % tau_merge tautrace.*.0.trc pddrive.trc

which generates a merged event description file, "tau.edf". A dump of 
the trace in text format can be obtained by doing

  % tau_convert -dump pddrive.trc tau.edf > pddrive.trc.txt

Profile information can also be obtained from the trace files, by means of

  % tau_treemerge.pl              (this merges the trace files)
  % tau2profile tau.trc tau.edf   (this generates the profiles)

Also, "tau_convert" can convert the trace files to the Vampir trace
format and then Vampir can be used to visualize the results:

  % tau_convert -pv pddrive.trc tau.edf pddrive.pv
  % module load vampir
  % vampir pddrive.pv &

Additional information:
======================

1. See Makefile and Makefile.inc.* for compilation and instrumentation 
   details. More examples can be found in $(TAUROOTDIR)/examples.
2. The files generated after execution containing information for  
   profiling and tracing are not included in this directory. 
3. If SuperLU_DIST has not been compiled with instrumentation then 
   the performance of SuperLU_DIST funtions will not be displayed.
4. For a complete list of PAPI_EVENTS please consult the TAU 
   User's Guide.
5. Vampir is a third party product.