Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'parsetiming' command #1046

Merged
merged 61 commits into from
Aug 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
39119a9
Start parsetiming command
May 19, 2023
b2cd7a1
Start storing run data in a separate class
May 19, 2023
9023ba6
Extract total time
May 19, 2023
9b07d2f
Get threads
May 19, 2023
27374ee
Store results in an array
May 19, 2023
7be107d
Sort by total time
May 19, 2023
2031999
Fix # threads detection. Add total cores calc. Better error checking
May 19, 2023
2f039fb
Add sort by time/cores option
May 19, 2023
24f96f6
Add data sets and data file. Add shorthand names for each run
May 19, 2023
7edf08d
Allow skipping over non-cpptraj-output files
May 19, 2023
b02632e
Extract detailed traj read and action process timings.
May 19, 2023
2cd264c
Start TICA
May 22, 2023
ab580d1
Set up frame from mask with coordinate info
May 22, 2023
8016e52
Add mask set up and initial loop
May 22, 2023
72713e5
Enable command
May 22, 2023
a973480
Start TICA test
May 22, 2023
5993606
Add Squared function
May 22, 2023
59eec90
Calc vector quantities
May 22, 2023
b8d4dd8
Finish initial covar calc
drroe May 23, 2023
c2233bd
Add mass
drroe May 23, 2023
53d9498
Merge branch 'master' into parse.timing
drroe Jun 9, 2023
87523bc
Merge branch 'master' into parse.timing
Jun 14, 2023
1b2cc50
Make TICA hidden until its ready for prime time
Jun 14, 2023
6eabcf2
Add directory name
Jun 14, 2023
6dcd2f3
Add reverse option
Jun 14, 2023
721bdbb
Add sort by file name
Jun 14, 2023
ac9f6cb
Better error detection
Jun 14, 2023
095313f
Option to include bad runs
Jun 14, 2023
3cb9012
Merge branch 'master' into parse.timing
drroe Jun 17, 2023
511da8a
Merge branch 'master' into parse.timing
Jul 12, 2023
535016f
Merge branch 'master' into parse.timing
Jul 27, 2023
078ec8e
Explicitly state testing FFTW from MKL
Aug 1, 2023
f6d6ef9
Test groupby output
Aug 3, 2023
4bbb1de
Add groupout keyword
Aug 4, 2023
40ad3b7
Add printout of speedup, ideal speedup, and efficiency within groups
Aug 4, 2023
84f3b99
Merge branch 'parse.timing' of github.com:drroe/cpptraj into parse.ti…
Aug 7, 2023
632461b
Add 'grouptype' option; cannot use groupby since that is already a dat
Aug 7, 2023
a4f0bab
Add type trajproc
Aug 9, 2023
591f858
Change gpu name to have G prefix
Aug 10, 2023
3cc1756
Add grouptype kind
Aug 10, 2023
78948b7
For hybrid mpi openmp runs, append number of threads per process
Aug 10, 2023
24d71ec
Write total cores instead of ideal speedup
Aug 10, 2023
741b370
Make largest time out of all run times the reference
Aug 10, 2023
ecbe869
Clean up help
Aug 17, 2023
636ce8f
Clean up output. Add showdetails keyword.
Aug 17, 2023
5a8f614
Add parsetiming test
Aug 17, 2023
ec17742
Enable test
Aug 17, 2023
19e4ecc
Add output for parsetiming test
Aug 17, 2023
dde1702
Protect against empty dir name
Aug 17, 2023
d3d879f
Add set name. Update for fixed empty dir names.
drroe Aug 17, 2023
41b391f
Add entry for parsedata
drroe Aug 17, 2023
518e858
Unhide
drroe Aug 17, 2023
0db7f37
6.20.3. Revision bump for new parsetiming command.
drroe Aug 17, 2023
57c275f
Fix test for OS that do not support file globbing
drroe Aug 17, 2023
dbf82ef
Try to use pip instead of conda to build pytraj
drroe Aug 17, 2023
667c811
Try python 3.10, remove curl
drroe Aug 17, 2023
1a7e0d4
Enclose in quotes to prevent 3.10 to 3.1
drroe Aug 17, 2023
22ac47c
Try reverting to python 3.8
drroe Aug 17, 2023
d481b5b
Try python -m pytest
drroe Aug 17, 2023
3ba19ee
Add some simpler python and pytraj tests
drroe Aug 18, 2023
c06f8dd
Try conda-forge
drroe Aug 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/merge-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,18 +141,19 @@ jobs:
- name: Set up Python 3.8
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: '3.8'
- name: Add conda to system path
run: |
# $CONDA is an environment variable pointing to the root of the miniconda directory
echo $CONDA/bin >> $GITHUB_PATH
- name: Install conda packages
run: |
conda install setuptools cython numpy pytest mock curl
conda env update --file devtools/ci/environment.yml --name base
- name: Install cpptraj
run: |
export PATH=$HOME/bin:${PATH}
export LD_LIBRARY_PATH=$HOME/lib:${LD_LIBRARY_PATH}
export MAKE_COMMAND="make -j2"
./configure --buildlibs -openmp -shared gnu
source cpptraj.sh
make -j2 libcpptraj
Expand Down
2 changes: 1 addition & 1 deletion configure
Original file line number Diff line number Diff line change
Expand Up @@ -1031,7 +1031,7 @@ EOF
fi
elif [ "$MKL_FFTW" = 'yes' ] ; then
# Using FFTW from MKL
TestProgram " Checking FFTW3" "$CXX" "$CXXFLAGS ${LIB_INCL[$LBLAS]}" testp.cpp "${LIB_FLAG[$LBLAS]}"
TestProgram " Checking MKL FFTW3" "$CXX" "$CXXFLAGS ${LIB_INCL[$LBLAS]}" testp.cpp "${LIB_FLAG[$LBLAS]}"
else
TestProgram silent " Checking FFTW3" "$CXX" "$CXXFLAGS ${LIB_INCL[$LFFTW3]}" testp.cpp "${LIB_FLAG[$LFFTW3]}"
if [ $? -ne 0 ] ; then
Expand Down
9 changes: 5 additions & 4 deletions devtools/ci/environment.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
channels:
- defaults
- conda-forge
dependencies:
- python=3.8
- pyflakes
- numpy
- mock
- pytest
- cython
- setuptools
- curl
- pytest
- mock
- curl
191 changes: 190 additions & 1 deletion doc/cpptraj.lyx
Original file line number Diff line number Diff line change
Expand Up @@ -8341,7 +8341,7 @@ The following general commands are available:
\begin_layout Standard
\align center
\begin_inset Tabular
<lyxtabular version="3" rows="34" columns="2">
<lyxtabular version="3" rows="35" columns="2">
<features tabularvalignment="middle">
<column alignment="center" valignment="top">
<column alignment="center" valignment="top">
Expand Down Expand Up @@ -8734,6 +8734,26 @@ parallelanalysis
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout
parsedata
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout
Parse timing data from CPPTRAJ output.
\end_layout

\end_inset
</cell>
</row>
<row>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout
precision
\end_layout
Expand Down Expand Up @@ -11455,6 +11475,175 @@ noprefix "false"
.
\end_layout

\begin_layout Subsection
parsedata
\end_layout

\begin_layout LyX-Code
parsetiming <filename args> ...
[out <file>] [name <setname>]
\end_layout

\begin_layout LyX-Code
[sortby {time|cores|filename}] [includebad] [showdetails]
\end_layout

\begin_layout LyX-Code
[type {trajproc|trajread|actframe}] [reverse]
\end_layout

\begin_layout LyX-Code
[groupout <file> [grouptype {prefix|name|kind}]]
\end_layout

\begin_deeper
\begin_layout Description
<filename
\begin_inset space ~
\end_inset

args> Files containing CPPTRAJ output to get timing data from.
\end_layout

\begin_layout Description
[out
\begin_inset space ~
\end_inset

<file>] Write total sorted timing sets to <file>.
\end_layout

\begin_layout Description
[name
\begin_inset space ~
\end_inset

<setname>] Set name for timing data sets.
\end_layout

\begin_layout Description
[sortby
\begin_inset space ~
\end_inset

{time|cores|filename}] Sort timing data sets by either time, number of cores
(MPI processes * OpenMP threads), or file name.
\end_layout

\begin_layout Description
[includebad] If specified, include run output for which timing data cannot
be extracted (e.g.
an incomplete/failed run).
\end_layout

\begin_layout Description
[showdetails] If specified, details about each run will be printed to STDOUT.
\end_layout

\begin_layout Description
[type
\begin_inset space ~
\end_inset

{trajproc|trajread|actframe} If specified, report time other than the total
time (which is the default):
\end_layout

\begin_deeper
\begin_layout Description
trajproc Total trajectory processing time (trajectory I/O plus action frame
time).
\end_layout

\begin_layout Description
trajread Trajectory read time (requires compiling with '-timer' configure
option/-DTIMER compiler define).
\end_layout

\begin_layout Description
actframe Action frame time (requires compiling with '-timer' configure option/-D
TIMER compiler define).
\end_layout

\end_deeper
\begin_layout Description
[reverse] Instead of longest time to shortest time, sort shortest time to
longest time.
\end_layout

\begin_layout Description
[groupout
\begin_inset space ~
\end_inset

<file>] Group run output by a property and write to a file.
Additional details will be written like speedup and efficiency (relative
to the slowest run).
\end_layout

\begin_deeper
\begin_layout Description
[grouptype
\begin_inset space ~
\end_inset

{prefix|name|kind}] Property to group runs by in group output file.
\end_layout

\begin_deeper
\begin_layout Description
prefix Group by directory prefix.
\end_layout

\begin_layout Description
name Group by run type name (see
\emph on
<setname>[name]
\emph default
below).
\end_layout

\begin_layout Description
kind Group by run type name; OpenMP runs are separated by number of OpenMP
threads.
\end_layout

\end_deeper
\end_deeper
\begin_layout Standard
DataSets Created:
\end_layout

\begin_layout Description
<setname> Set containing sorted run times.
\end_layout

\begin_layout Description
<setname>[name] Set containing shortedhand run names.
Consists of a prefix (S for serial, O for OpenMP, M for MPI, H for hybrid
MPI/OpenMP) followed by numbers indicating number of MPI processes 'x'
OpenMP threads; e.g., H16x4 means a hybrid run consisting of 16 MPI processes
and 4 OpenMP threads per process.
If CUDA is active, the name will be wrapped in 'G()', e.g.
G(H16x4).
\end_layout

\begin_layout Description
<setname>[dir] Set containing output directory prefixes.
If no directory prefix, just contains output file name.
\end_layout

\end_deeper
\begin_layout Standard
The
\series bold
\emph on
parsedata
\series default
\emph default
command can be used to extract timing data from CPPTRAJ output.
\end_layout

\begin_layout Subsection
precision
\end_layout
Expand Down
Loading
Loading