Chap_API_Struct.tex

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Chapter: Data Structures
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Data Structures and Types}
\label{chap:struct}

This chapter defines PMIx standard data structures, types, and constants.
These apply to all consumers of the PMIx interface.
Where necessary for clarification, the description of, for example, an attribute may be copied from this chapter into a section where it is used.

A PMIx implementation may define additional attributes beyond those specified in this document.

\adviceimplstart
Structures, types, and macros in the \ac{PMIx} Standard are defined in terms of the C-programming language. Implementers wishing to support other languages should provide the equivalent definitions in a language-appropriate manner.

If a PMIx implementation chooses to define additional attributes they should avoid using the \code{PMIX} prefix in their name or starting the attribute string with a \textit{pmix} prefix.
This helps the end user distinguish between what is defined by the PMIx standard and what is specific to that PMIx implementation, and avoids potential conflicts with attributes defined by the standard.
\adviceimplend


%%%%%%%%%%%
\section{Constants}

\ac{PMIx} defines a few values that are used throughout the standard to set the size of fixed arrays or as a means of identifying values with special meaning.
The community makes every attempt to minimize the number of such definitions.
The constants defined in this section may be used before calling any \ac{PMIx} library initialization routine.
Additional constants associated with specific data structures or types are defined in the section describing that data structure or type.

\begin{constantdesc}
%
\declareconstitem{PMIX_MAX_NSLEN}
Maximum namespace string length as an integer.
\end{constantdesc}

\adviceimplstart
\refconst{PMIX_MAX_NSLEN} should have a minimum value of 63 characters. Namespace arrays in \ac{PMIx} defined structures must reserve
a space of size \refconst{PMIX_MAX_NSLEN}+1 to allow room for the \code{NULL} terminator
\adviceimplend

\begin{constantdesc}
%
\declareconstitem{PMIX_MAX_KEYLEN}
Maximum key string length as an integer.
%
\end{constantdesc}

\adviceimplstart
\refconst{PMIX_MAX_KEYLEN} should have a minimum value of 63 characters. Key arrays in \ac{PMIx} defined structures must reserve
a space of size \refconst{PMIX_MAX_KEYLEN}+1 to allow room for the \code{NULL} terminator
\adviceimplend


%%%%%%%%%%%
\subsection{Error Constants}
\label{api:struct:errors}
\declarestruct{pmix_status_t}

The \refstruct{pmix_status_t} structure is an \code{int} type for return status.

The tables shown in this section define the possible values for \refstruct{pmix_status_t}.
PMIx errors are required to always be negative, with 0 reserved for \refconst{PMIX_SUCCESS}.

A PMIx implementation must define all of the constants defined in this section, even if they will never return the specific value to the caller.

\adviceuserstart
Other than \refconst{PMIX_SUCCESS} (which is required to be zero), the actual value of any \ac{PMIx} error constant is left to the \ac{PMIx} library implementer. Thus, users are advised to always refer to constant by name, and not a specific implementation's value, for portability between implementations and compatibility across library versions.
\adviceuserend

%%%%%%%%%%%
\subsubsection{PMIx v1 Error Constants}

The following list contains those constants defined in the PMIx v1 standard.
Those values in the list that were deprecated in later standards are denoted as such.
\ac{PMIx} errors are always negative, with \code{0} reserved for success.

\begin{constantdesc}
%
\declareconstitem{PMIX_SUCCESS}
Success
%
\declareconstitem{PMIX_ERROR}
General Error
%
\declareconstitem{PMIX_ERR_SILENT}
Silent error
%
\declareconstitem{PMIX_ERR_DEBUGGER_RELEASE}
Error in debugger release
%
\declareconstitem{PMIX_ERR_PROC_RESTART}
Fault tolerance: Error in process restart
%
\declareconstitem{PMIX_ERR_PROC_CHECKPOINT}
Fault tolerance: Error in process checkpoint
%
\declareconstitem{PMIX_ERR_PROC_MIGRATE}
Fault tolerance: Error in process migration
%
\declareconstitem{PMIX_ERR_PROC_ABORTED}
Process was aborted
%
\declareconstitem{PMIX_ERR_PROC_REQUESTED_ABORT}
Process is already requested to abort
%
\declareconstitem{PMIX_ERR_PROC_ABORTING}
Process is being aborted
%
\declareconstitem{PMIX_ERR_SERVER_FAILED_REQUEST}
Failed to connect to the server
%
\declareconstitem{PMIX_EXISTS}
Requested operation would overwrite an existing value
%
\declareconstitem{PMIX_ERR_INVALID_CRED}
Invalid security credentials
%
\declareconstitem{PMIX_ERR_HANDSHAKE_FAILED}
Connection handshake failed
%
\declareconstitem{PMIX_ERR_READY_FOR_HANDSHAKE}
Ready for handshake
%
\declareconstitem{PMIX_ERR_WOULD_BLOCK}
Operation would block
%
\declareconstitem{PMIX_ERR_UNKNOWN_DATA_TYPE}
Unknown data type
%
\declareconstitem{PMIX_ERR_PROC_ENTRY_NOT_FOUND}
Process not found
%
\declareconstitem{PMIX_ERR_TYPE_MISMATCH}
Invalid type
%
\declareconstitem{PMIX_ERR_UNPACK_INADEQUATE_SPACE}
Inadequate space to unpack data
%
\declareconstitem{PMIX_ERR_UNPACK_FAILURE}
Unpack failed
%
\declareconstitem{PMIX_ERR_PACK_FAILURE}
Pack failed
%
\declareconstitem{PMIX_ERR_PACK_MISMATCH}
Pack mismatch
%
\declareconstitem{PMIX_ERR_NO_PERMISSIONS}
No permissions
%
\declareconstitem{PMIX_ERR_TIMEOUT}
Timeout expired
%
\declareconstitem{PMIX_ERR_UNREACH}
Unreachable
%
\declareconstitem{PMIX_ERR_IN_ERRNO}
Error defined in \code{errno}
%
\declareconstitem{PMIX_ERR_BAD_PARAM}
Bad parameter
%
\declareconstitem{PMIX_ERR_RESOURCE_BUSY}
Resource busy
%
\declareconstitem{PMIX_ERR_OUT_OF_RESOURCE}
Resource exhausted
%
\declareconstitem{PMIX_ERR_DATA_VALUE_NOT_FOUND}
Data value not found
%
\declareconstitem{PMIX_ERR_INIT}
Error during initialization
%
\declareconstitem{PMIX_ERR_NOMEM}
Out of memory
%
\declareconstitem{PMIX_ERR_INVALID_ARG}
Invalid argument
%
\declareconstitem{PMIX_ERR_INVALID_KEY}
Invalid key
%
\declareconstitem{PMIX_ERR_INVALID_KEY_LENGTH}
Invalid key length
%
\declareconstitem{PMIX_ERR_INVALID_VAL}
Invalid value
%
\declareconstitem{PMIX_ERR_INVALID_VAL_LENGTH}
Invalid value length
%
\declareconstitem{PMIX_ERR_INVALID_LENGTH}
Invalid argument length
%
\declareconstitem{PMIX_ERR_INVALID_NUM_ARGS}
Invalid number of arguments
%
\declareconstitem{PMIX_ERR_INVALID_ARGS}
Invalid arguments
%
\declareconstitem{PMIX_ERR_INVALID_NUM_PARSED}
Invalid number parsed
%
\declareconstitem{PMIX_ERR_INVALID_KEYVALP}
Invalid key/value pair
%
\declareconstitem{PMIX_ERR_INVALID_SIZE}
Invalid size
%
\declareconstitem{PMIX_ERR_INVALID_NAMESPACE}
Invalid namespace
%
\declareconstitem{PMIX_ERR_SERVER_NOT_AVAIL}
Server is not available
%
\declareconstitem{PMIX_ERR_NOT_FOUND}
Not found
%
\declareconstitem{PMIX_ERR_NOT_SUPPORTED}
Not supported
%
\declareconstitem{PMIX_ERR_NOT_IMPLEMENTED}
Not implemented
%
\declareconstitem{PMIX_ERR_COMM_FAILURE}
Communication failure
%
\declareconstitem{PMIX_ERR_UNPACK_READ_PAST_END_OF_BUFFER}
Unpacking past the end of the buffer provided
%
\end{constantdesc}

%%%%%%%%%%%
\subsubsection{PMIx v2 Error Constants}

The following list contains constants added in the PMIx v2 standard.

\begin{constantdesc}
%
\declareconstitem{PMIX_ERR_LOST_CONNECTION_TO_SERVER}
Lost connection to server
%
\declareconstitem{PMIX_ERR_LOST_PEER_CONNECTION}
Lost connection to peer
%
\declareconstitem{PMIX_ERR_LOST_CONNECTION_TO_CLIENT}
Lost connection to client
%
\declareconstitem{PMIX_QUERY_PARTIAL_SUCCESS}
Query partial success (used by query system)
%
\declareconstitem{PMIX_NOTIFY_ALLOC_COMPLETE}
Notify that allocation is complete
%
\declareconstitem{PMIX_JCTRL_CHECKPOINT}
Job control: Monitored by PMIx client to trigger checkpoint operation
%
\declareconstitem{PMIX_JCTRL_CHECKPOINT_COMPLETE}
Job control: Sent by PMIx client and monitored by PMIx server to notify that requested checkpoint operation has completed.
%
\declareconstitem{PMIX_JCTRL_PREEMPT_ALERT}
Job control: Monitored by PMIx client to detect an \ac{RM} intending to preempt the job.
%
\declareconstitem{PMIX_MONITOR_HEARTBEAT_ALERT}
Job monitoring: Heartbeat alert
%
\declareconstitem{PMIX_MONITOR_FILE_ALERT}
Job monitoring: File alert
%
\declareconstitem{PMIX_PROC_TERMINATED}
Process terminated - can be either normal or abnormal termination
%
\declareconstitem{PMIX_ERR_INVALID_TERMINATION}
Process terminated without calling \refapi{PMIx_Finalize}, or was a member of an assemblage formed via \refapi{PMIx_Connect} and terminated or called \refapi{PMIx_Finalize} without first calling \refapi{PMIx_Disconnect} (or its non-blocking form) from that assemblage.
%
\end{constantdesc}

The following list contains operational error constants introduced in the v2 standard.

\begin{constantdesc}
%
\declareconstitem{PMIX_ERR_EVENT_REGISTRATION}
Error in event registration
%
\declareconstitem{PMIX_ERR_JOB_TERMINATED}
Error job terminated
%
\declareconstitem{PMIX_ERR_UPDATE_ENDPOINTS}
Error updating endpoints
%
\declareconstitem{PMIX_MODEL_DECLARED}
Model declared
%
\declareconstitem{PMIX_GDS_ACTION_COMPLETE}
The \ac{GDS} action has completed
%
\declareconstitem{PMIX_ERR_INVALID_OPERATION}
The requested operation is supported by the implementation and host environment, but fails to meet a requirement (e.g., requesting to \textit{disconnect} from processes without first \textit{connecting} to them).

\end{constantdesc}

The following list contains system error constants introduced in the v2 standard.

\begin{constantdesc}
%
\declareconstitem{PMIX_ERR_NODE_DOWN}
Node down
%
\declareconstitem{PMIX_ERR_NODE_OFFLINE}
Node is marked as offline
%
\end{constantdesc}

The following list contains event handler error constants introduced in the v2 standard.

\begin{constantdesc}
%
\declareconstitem{PMIX_EVENT_NO_ACTION_TAKEN}
Event handler: No action taken
%
\declareconstitem{PMIX_EVENT_PARTIAL_ACTION_TAKEN}
Event handler: Partial action taken
%
\declareconstitem{PMIX_EVENT_ACTION_DEFERRED}
Event handler: Action deferred
%
\declareconstitem{PMIX_EVENT_ACTION_COMPLETE}
Event handler: Action complete
%
\end{constantdesc}

%%%%%%%%%%%
\subsubsection{User-Defined Error Constants}

PMIx establishes an error code boundary for constants defined in the PMIx standard. Negative values larger than this (and any positive values greater than zero) are guaranteed not to conflict with PMIx values.

\begin{constantdesc}
%
\declareconstitem{PMIX_EXTERNAL_ERR_BASE}
A starting point for user-level defined error constants.
Negative values lower than this are guaranteed not to conflict with PMIx values.
Definitions should always be based on the \refconst{PMIX_EXTERNAL_ERR_BASE} constant and \emph{not} a specific value as the value of the constant may change.
%
\end{constantdesc}



%%%%%%%%%%%
\section{Data Types}

This section defines various data types used by the PMIx APIs.

%%%%%%%%%%%
\subsection{Key Structure}
\declarestruct{pmix_key_t}

The \refstruct{pmix_key_t} structure is a statically defined character array of length \refconst{PMIX_MAX_KEYLEN}+1, thus supporting keys of maximum length \refconst{PMIX_MAX_KEYLEN} while preserving space for a mandatory \code{NULL} terminator.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef char pmix_key_t[PMIX_MAX_KEYLEN+1];
\end{codepar}
\cspecificend

Characters in the key must be standard alphanumeric values supported by common utilities such as \textit{strcmp}.

\adviceuserstart
References to keys in \ac{PMIx} v1 rwere defined simply as an array of characters of size \code{PMIX_MAX_KEYLEN+1}. The \refstruct{pmix_key_t} type definition was introduced in version 2 of the standard. The two definitions are code-compatible and thus do not represent a break in backward compatibility.

Passing a \refstruct{pmix_key_t} value to the standard \textit{sizeof} utility can result in compiler warnings of incorrect returned value. Users are advised to avoid using \textit{sizeof(pmix_key_t)} and instead rely on the \refstruct{PMIX_MAX_KEYLEN} constant.
\adviceuserend

%%%%%%%%%%%
\subsection{Namespace Structure}
\declarestruct{pmix_nspace_t}

The \refstruct{pmix_nspace_t} structure is a statically defined character array of length \refconst{PMIX_MAX_NSLEN}+1, thus supporting namespaces of maximum length \refconst{PMIX_MAX_NSLEN} while preserving space for a mandatory \code{NULL} terminator.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef char pmix_nspace_t[PMIX_MAX_NSLEN+1];
\end{codepar}
\cspecificend

Characters in the namespace must be standard alphanumeric values supported by common utilities such as \textit{strcmp}.

\adviceuserstart
References to namespace values in \ac{PMIx} v1 rwere defined simply as an array of characters of size \code{PMIX_MAX_NSLEN+1}. The \refstruct{pmix_nspace_t} type definition was introduced in version 2 of the standard. The two definitions are code-compatible and thus do not represent a break in backward compatibility.

Passing a \refstruct{pmix_nspace_t} value to the standard \textit{sizeof} utility can result in compiler warnings of incorrect returned value. Users are advised to avoid using \textit{sizeof(pmix_nspace_t)} and instead rely on the \refstruct{PMIX_MAX_NSLEN} constant.
\adviceuserend


%%%%%%%%%%%
\subsection{Rank Structure}
\declarestruct{pmix_rank_t}

The \refstruct{pmix_rank_t} structure is a \code{uint32_t} type for rank values.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef uint32_t pmix_rank_t;
\end{codepar}
\cspecificend

The following constants can be used to set a variable of the type \refstruct{pmix_rank_t}. All definitions were introduced in version 1 of the standard unless otherwise marked. Valid rank values start at zero.

\begin{constantdesc}
%
\declareconstitem{PMIX_RANK_UNDEF}
A value to request job-level data where the information itself is not associated with any specific rank, or when passing a \refstruct{pmix_proc_t} identifier to an operation that only references the namespace field of that structure.
%
\declareconstitem{PMIX_RANK_WILDCARD}
A value to indicate that the user wants the data for the given key from every rank that posted that key.
%
\versionMarker{2.0}
\declareconstitem{PMIX_RANK_LOCAL_NODE}
Special rank value used to define groups of ranks for use in collectives.
This constant defines the group of all ranks on a local node.
%
\end{constantdesc}


%%%%%%%%%%%
\subsection{Process Structure}
\declarestruct{pmix_proc_t}

The \refstruct{pmix_proc_t} structure is used to identify a single process in the PMIx universe.
It contains a reference to the namespace and the \refstruct{pmix_rank_t} within that namespace.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_proc \{
    pmix_nspace_t nspace;
    pmix_rank_t rank;
\} pmix_proc_t;
\end{codepar}
\cspecificend

\subsection{Process structure support macros}
The following macros are provided to support the \refstruct{pmix_proc_t} structure.

\subsubsection{Initialize the \refstruct{pmix_proc_t} structure}
\declaremacro{PMIX_PROC_CONSTRUCT}

Initialize the \refstruct{pmix_proc_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_proc_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_proc_t} structure}
\declaremacro{PMIX_PROC_DESTRUCT}

Clear the \refstruct{pmix_proc_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_proc_t})}
\end{arglist}

This macro performs the identical operations as \refmacro{PMIX_PROC_CONSTRUCT}, but is provided for symmetry in user code.


\subsubsection{Create a \refstruct{pmix_proc_t} array}
\declaremacro{PMIX_PROC_CREATE}

Allocate and initialize an array of \refstruct{pmix_proc_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_proc_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


\subsubsection{Free a\refstruct{pmix_proc_t} array}
\declaremacro{PMIX_PROC_FREE}

Release an array of \refstruct{pmix_proc_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_proc_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

\subsubsection{Load a\refstruct{pmix_proc_t} structure}
\declaremacro{PMIX_PROC_LOAD}

Load values into a \refstruct{pmix_proc_t}

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_LOAD(m, n, r)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be loaded (pointer to \refstruct{pmix_proc_t})}
\argin{n}{Namespace to be loaded (\refstruct{pmix_nspace_t})}
\argin{r}{Rank to be assigned (\refstruct{pmix_rank_t})}
\end{arglist}


%%%%%%%%%%%
\subsection{Process State Structure}
\label{api:struct:processstate}
\declarestruct{pmix_proc_state_t}

\versionMarker{2.0}
The \refstruct{pmix_proc_state_t} structure is a \code{uint8_t} type for process state values. The following constants can be used to set a variable of the type \refstruct{pmix_proc_state_t}. All values were originally defined in version 2 of the standard unless otherwise marked.

\adviceuserstart
The fine-grained nature of the following constants may exceed the ability of an \ac{RM} to provide updated process state values during the process lifetime. This is particularly true of states in the launch process, and for short-lived processes.
\adviceuserend

\begin{constantdesc}
%
\declareconstitem{PMIX_PROC_STATE_UNDEF}
Undefined process state
%
\declareconstitem{PMIX_PROC_STATE_PREPPED}
Process is ready to be launched
%
\declareconstitem{PMIX_PROC_STATE_LAUNCH_UNDERWAY}
Process launch is underway
%
\declareconstitem{PMIX_PROC_STATE_RESTART}
Process is ready for restart
%
\declareconstitem{PMIX_PROC_STATE_TERMINATE}
Process is marked for termination
%
\declareconstitem{PMIX_PROC_STATE_RUNNING}
Process has been locally \code{fork}'ed by the \ac{RM}
%
\declareconstitem{PMIX_PROC_STATE_CONNECTED}
Process has connected to PMIx server
%
\declareconstitem{PMIX_PROC_STATE_UNTERMINATED}
Define a ``boundary'' between this constant and \refconst{PMIX_PROC_STATE_CONNECTED} so users can easily and quickly determine if a process is still running or not.
Any value less than this constant means that the process has not terminated.
%
\declareconstitem{PMIX_PROC_STATE_TERMINATED}
Process has terminated and is no longer running
%
\declareconstitem{PMIX_PROC_STATE_ERROR}
Define a boundary so users can easily and quickly determine if a process abnormally terminated.
Any value above this constant means that the process has terminated abnormally.
%
\declareconstitem{PMIX_PROC_STATE_KILLED_BY_CMD}
Process was killed by a command
%
\declareconstitem{PMIX_PROC_STATE_ABORTED}
Process was aborted by a call to \refapi{PMIx_Abort}
%
\declareconstitem{PMIX_PROC_STATE_FAILED_TO_START}
Process failed to start
%
\declareconstitem{PMIX_PROC_STATE_ABORTED_BY_SIG}
Process aborted by a signal
%
\declareconstitem{PMIX_PROC_STATE_TERM_WO_SYNC}
Process exited without calling \refapi{PMIx_Finalize}
%
\declareconstitem{PMIX_PROC_STATE_COMM_FAILED}
Process communication has failed
%
\declareconstitem{PMIX_PROC_STATE_CALLED_ABORT}
Process called \refapi{PMIx_Abort}
%
\declareconstitem{PMIX_PROC_STATE_MIGRATING}
Process failed and is waiting for resources before restarting
%
\declareconstitem{PMIX_PROC_STATE_CANNOT_RESTART}
Process failed and cannot be restarted
%
\declareconstitem{PMIX_PROC_STATE_TERM_NON_ZERO}
Process exited with a non-zero status
%
\declareconstitem{PMIX_PROC_STATE_FAILED_TO_LAUNCH}
Unable to launch process
%
\end{constantdesc}


%%%%%%%%%%%
\subsection{Process Information Structure}
\declarestruct{pmix_proc_info_t}

The \refstruct{pmix_proc_info_t} structure defines a set of information about a specific process including it's name, location, and state.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_proc_info \{
    /** Process structure */
    pmix_proc_t proc;
    /** Hostname where process resides */
    char *hostname;
    /** Name of the executable */
    char *executable_name;
    /** Process ID on the host */
    pid_t pid;
    /** Exit code of the process. Default: 0 */
    int exit_code;
    /** Current state of the process */
    pmix_proc_state_t state;
\} pmix_proc_info_t;
\end{codepar}
\cspecificend


%%%%%%%%%%%
\subsection{Process Information Structure support macros}

The following macros are provided to support the \refstruct{pmix_proc_info_t} structure.

%%%%
\subsubsection{Initialize the \refstruct{pmix_proc_info_t} structure}
\declaremacro{PMIX_PROC_INFO_CONSTRUCT}

Initialize the \refstruct{pmix_proc_info_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_INFO_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_proc_info_t})}
\end{arglist}

%%%%
\subsubsection{Destruct the \refstruct{pmix_proc_info_t} structure}
\declaremacro{PMIX_PROC_INFO_DESTRUCT}

Destruct the \refstruct{pmix_proc_info_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_INFO_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_proc_info_t})}
\end{arglist}

%%%%
\subsubsection{Create a \refstruct{pmix_proc_info_t} array}
\declaremacro{PMIX_PROC_INFO_CREATE}

Allocate and initialize a \refstruct{pmix_proc_info_t} array

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_INFO_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_proc_info_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}

%%%%
\subsubsection{Free a\refstruct{pmix_proc_info_t} array}
\declaremacro{PMIX_PROC_INFO_FREE}

Release an array of \refstruct{pmix_proc_info_t} structures

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_PROC_INFO_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_proc_info_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}


%%%%%%%
\subsection{Scope of Put Data}
\declarestruct{pmix_scope_t}

\versionMarker{1.0}
The \refstruct{pmix_scope_t} structure is a \code{uint8_t} type that defines the scope for data passed to \refapi{PMIx_Put}.
The following constants can be used to set a variable of the type \refstruct{pmix_scope_t}. All definitions were introduced in version 1 of the standard unless otherwise marked.

Specific implementations may support different scope values, but all implementations must support at least \refconst{PMIX_GLOBAL}.
If a scope value is not supported, then the \refapi{PMIx_Put} call must return \refconst{PMIX_ERR_NOT_SUPPORTED}.

\begin{constantdesc}
%
\declareconstitem{PMIX_SCOPE_UNDEF}
Undefined scope
%
\declareconstitem{PMIX_LOCAL}
The data is intended only for other application processes on the same node.
Data marked in this way will not be included in data packages sent to remote requestors --- i.e., it is only available to processes on the local node.
%
\declareconstitem{PMIX_REMOTE}
The data is intended solely for applications processes on remote nodes.
Data marked in this way will not be shared with other processes on the same node --- i.e., it is only available to  processes on remote nodes.
%
\declareconstitem{PMIX_GLOBAL}
The data is to be shared with all other requesting processes, regardless of location.
%
\versionMarker{2.0}
\declareconstitem{PMIX_INTERNAL}
The data is intended solely for this process and is not shared with other processes.
%
\end{constantdesc}



%%%%%%%%%%%
\subsection{Range of Published Data}
\declarestruct{pmix_data_range_t}

\versionMarker{1.0}
The \refstruct{pmix_data_range_t} structure is a \code{uint8_t} type that defines a range for data \textit{published} via functions other than \refapi{PMIx_Put} - e.g., the \refapi{PMIx_Publish} \ac{API}.
The following constants can be used to set a variable of the type \refstruct{pmix_data_range_t}. Several values were initially defined in version 1 of the standard but subsequently renamed and other values added in version 2. Thus, all values shown below are as they were defined in version 2 except where noted.

\begin{constantdesc}
%
\declareconstitem{PMIX_RANGE_UNDEF}
Undefined range
%
\declareconstitem{PMIX_RANGE_RM}
Data is intended for the host resource manager.
%
\declareconstitem{PMIX_RANGE_LOCAL}
Data is only available to processes on the local node.
%
\declareconstitem{PMIX_RANGE_NAMESPACE}
Data is only available to processes in the same namespace.
%
\declareconstitem{PMIX_RANGE_SESSION}
Data is only available to all processes in the session.
%
\declareconstitem{PMIX_RANGE_GLOBAL}
Data is available to all processes.
%
\declareconstitem{PMIX_RANGE_CUSTOM}
Range is specified in the \refstruct{pmix_info_t} associated with this call.
%
\declareconstitem{PMIX_RANGE_PROC_LOCAL}
Data is only available to this process.
%
\end{constantdesc}

\adviceuserstart
The names of the \refstruct{pmix_data_range_t} values changed between version 1 and version 2 of the standard, thereby breaking backward compatibility
\adviceuserend

%%%%%%%%%%%
\subsection{Data Persistence Structure}
\declarestruct{pmix_persistence_t}

\versionMarker{1.0}
The \refstruct{pmix_persistence_t} structure is a \code{uint8_t} type that defines the policy for data published by clients via the \refapi{PMIx_Publish} \ac{API}.
The following constants can be used to set a variable of the type \refstruct{pmix_persistence_t}. All definitions were introduced in version 1 of the standard unless otherwise marked.

\begin{constantdesc}
%
\declareconstitem{PMIX_PERSIST_INDEF}
Retain data until specifically deleted.
%
\declareconstitem{PMIX_PERSIST_FIRST_READ}
Retain data until the first access, then the data is deleted.
%
\declareconstitem{PMIX_PERSIST_PROC}
Retain data until the publishing process terminates.
%
\declareconstitem{PMIX_PERSIST_APP}
Retain data until the application terminates.
%
\declareconstitem{PMIX_PERSIST_SESSION}
Retain data until the session/allocation terminates.
%
\end{constantdesc}

%%%%%%%%%%%
\subsection{Value Structure}
\declarestruct{pmix_value_t}

The \refstruct{pmix_value_t} structure is used to represent the value passed to \refapi{PMIx_Put} and retrieved by \refapi{PMIx_Get}, as well as many of the other \ac{PMIx} functions.

A collection of values may be specified under a single key by passing a \refstruct{pmix_value_t} containing an array of type \refstruct{pmix_data_array_t}, with each array element containing its own object. All members shown below were introduced in version 1 of the standard unless otherwise marked.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_value \{
    pmix_data_type_t type;
    union \{
        bool flag;
        uint8_t byte;
        char *string;
        size_t size;
        pid_t pid;
        int integer;
        int8_t int8;
        int16_t int16;
        int32_t int32;
        int64_t int64;
        unsigned int uint;
        uint8_t uint8;
        uint16_t uint16;
        uint32_t uint32;
        uint64_t uint64;
        float fval;
        double dval;
        struct timeval tv;
        time_t time;                    // version 2.0
        pmix_status_t status;           // version 2.0
        pmix_rank_t rank;               // version 2.0
        pmix_proc_t *proc;              // version 2.0
        pmix_byte_object_t bo;
        pmix_persistence_t persist;     // version 2.0
        pmix_scope_t scope;             // version 2.0
        pmix_data_range_t range;        // version 2.0
        pmix_proc_state_t state;        // version 2.0
        pmix_proc_info_t *pinfo;        // version 2.0
        pmix_data_array_t *darray;      // version 2.0
        void *ptr;                      // version 2.0
        pmix_alloc_directive_t adir;    // version 2.0
        /**** DEPRECATED in PMIx 2 ****/
        pmix_info_array_t *array;
        /******************************/
    \} data;
\} pmix_value_t;
\end{codepar}
\cspecificend

%%%%%%%%%%%
\subsection{Value structure support macros}
The following macros are provided to support the \refstruct{pmix_value_t} structure.

\subsubsection{Initialize the \refstruct{pmix_value_t} structure}
\declaremacro{PMIX_VALUE_CONSTRUCT}

Initialize the \refstruct{pmix_value_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_VALUE_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_value_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_value_t} structure}
\declaremacro{PMIX_VALUE_DESTRUCT}

Destruct the \refstruct{pmix_value_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_VALUE_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_value_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Create a \refstruct{pmix_value_t} array}
\declaremacro{PMIX_VALUE_CREATE}

Allocate and initialize an array of \refstruct{pmix_value_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_VALUE_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_value_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\subsubsection{Free a\refstruct{pmix_value_t} array}
\declaremacro{PMIX_VALUE_FREE}

Release an array of \refstruct{pmix_value_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_VALUE_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_value_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

%%%%%%%%%%%
\subsection{Load a \refstruct{pmix_value_t} structure}
\declaremacro{PMIX_VALUE_LOAD}

%%%%
\summary

Load data into a \refstruct{pmix_value_t} structure.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_VALUE_LOAD(v, d, t);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{v}{The \refstruct{pmix_value_t} into which the data is to be loaded (pointer to \refstruct{pmix_value_t})}
\argin{d}{Pointer to the data value to be loaded (handle)}
\argin{t}{Type of the provided data value (\refstruct{pmix_data_type_t})}
\end{arglist}

%%%%
\descr

This macro simplifies the loading of data into a \refstruct{pmix_value_t} by correctly assigning values to the structure's fields.

\adviceuserstart
The data will be copied into the \refstruct{pmix_value_t} - thus, any data stored in the source value can be modified or free'd without affecting the copied data once the macro has completed.
\adviceuserend


%%%%%%%%%%%
\subsubsection{Transfer data between \refstruct{pmix_value_t} structures}
\declaremacro{PMIX_VALUE_XFER}

%%%%
\summary

Transfer the data value between two \refstruct{pmix_value_t} structures.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_VALUE_XFER(r, d, s);
\end{codepar}
\cspecificend

\begin{arglist}
\argout{r}{Status code indicating success or failure of the transfer (\refstruct{pmix_status_t})}
\argin{d}{Pointer to the \refstruct{pmix_value_t} destination (handle)}
\argin{s}{Pointer to the \refstruct{pmix_value_t} source (handle)}
\end{arglist}

%%%%
\descr

This macro simplifies the transfer of data between two \refstruct{pmix_value_t} structures, ensuring that all fields are properly copied.

\adviceuserstart
The data will be copied into the destination \refstruct{pmix_value_t} - thus, any data stored in the source value can be modified or free'd without affecting the copied data once the macro has completed.
\adviceuserend


%%%%%%%%%%%
\subsection{Info and Info Array Structures}
\declarestruct{pmix_info_t}
\declarestruct{pmix_info_array}

The \refstruct{pmix_info_t} structure defines a key/value pair with associated directive. All fields were defined in version 1.0 unless otherwise marked.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_info_t \{
    pmix_key_t key;
    pmix_info_directives_t flags;    // version 2.0
    pmix_value_t value;
\} pmix_info_t;
\end{codepar}
\cspecificend

The \refstruct{pmix_info_array} structure defines an array of \refstruct{pmix_info_t} structures.

\notestart
\noteheader
The \refstruct{pmix_info_array} structure has been deprecated and will be removed in future versions of the \ac{PMIx} Standard.
\noteend

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_info_array \{
    size_t size;
    pmix_info_t *array;
\} pmix_info_array_t;
\end{codepar}
\cspecificend

%%%%%%%%%%%
\subsection{Info structure support macros}
The following macros are provided to support the \refstruct{pmix_info_t} structure.

\subsubsection{Initialize the \refstruct{pmix_info_t} structure}
\declaremacro{PMIX_INFO_CONSTRUCT}

Initialize the \refstruct{pmix_info_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_info_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_info_t} structure}
\declaremacro{PMIX_INFO_DESTRUCT}

Destruct the \refstruct{pmix_info_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_info_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Create a \refstruct{pmix_info_t} array}
\declaremacro{PMIX_INFO_CREATE}

Allocate and initialize an array of \refstruct{pmix_info_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_info_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\subsubsection{Free a\refstruct{pmix_info_t} array}
\declaremacro{PMIX_INFO_FREE}

Release an array of \refstruct{pmix_info_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_info_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Load key and value data into a \refstruct{pmix_info_t}}
\declaremacro{PMIX_INFO_LOAD}

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_LOAD(v, k, d, t);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{v}{Pointer to the \refstruct{pmix_info_t} into which the key and data are to be loaded (pointer to \refstruct{pmix_info_t})}
\argin{k}{String key to be loaded - must be less than or equal to \refconst{PMIX_MAX_KEYLEN} in length (handle)}
\argin{d}{Pointer to the data value to be loaded (handle)}
\argin{t}{Type of the provided data value (\refstruct{pmix_data_type_t})}
\end{arglist}

This macro simplifies the loading of key and data into a \refstruct{pmix_info_t} by correctly assigning values to the structure's fields.

\adviceuserstart
Both key and data will be copied into the \refstruct{pmix_info_t} - thus, the key and any data stored in the source value can be modified or free'd without affecting the copied data once the macro has completed.
\adviceuserend

%%%%%%%%%%%
\subsubsection{Copy data between \refstruct{pmix_info_t} structures}
\declaremacro{PMIX_INFO_XFER}

Copy all data (including key, value, and directives) between two \refstruct{pmix_info_t} structures.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_XFER(d, s);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{d}{Pointer to the destination \refstruct{pmix_info_t} (pointer to \refstruct{pmix_info_t})}
\argin{s}{Pointer to the source \refstruct{pmix_info_t} (pointer to \refstruct{pmix_info_t})}
\end{arglist}

This macro simplifies the transfer of data between two\refstruct{pmix_info_t} structures.

\adviceuserstart
All data (including key, value, and directives) will be copied into the destination \refstruct{pmix_info_t} - thus, the source \refstruct{pmix_info_t} may be free'd without affecting the copied data once the macro has completed.
\adviceuserend


%%%%%%%%%%%
\subsubsection{Test a boolean \refstruct{pmix_info_t}}
\declaremacro{PMIX_INFO_TRUE}

A special macro for checking if a boolean \refstruct{pmix_info_t} is \code{true}

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_TRUE(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to a \refstruct{pmix_info_t} structure (handle)}
\end{arglist}

A \refstruct{pmix_info_t} structure is considered to be of type \refconst{PMIX_BOOL} and value \code{true} if:

\begin{itemize}
    \item the structure reports a type of \refconst{PMIX_UNDEF}, or
    \item the structure reports a type of \refconst{PMIX_BOOL} and the data flag is \code{true}
\end{itemize}

%%%%%%%%%%%
\subsection{Info Type Directives}
\declarestruct{pmix_info_directives_t}

\versionMarker{2.0}
The \refstruct{pmix_info_directives_t} structure is a \code{uint32_t} type that defines the behavior of command directives via \refstruct{pmix_info_t} arrays.
By default, the values in the \refstruct{pmix_info_t} array passed to a PMIx are \emph{optional}.

\adviceuserstart
A PMIx implementation or PMIx-enabled \ac{RM} may ignore any \refstruct{pmix_info_t} value passed to a \ac{PMIx} \ac{API} if it is not explicitly marked as \refconst{PMIX_INFO_REQD}.
This is because the values specified default to optional, meaning they can be ignored.
This may lead to unexpected behavior if the user is relying on the behavior specified by the \refstruct{pmix_info_t} value.
If the user relies on the behavior defined by the \refstruct{pmix_info_t} then they must set the \refconst{PMIX_INFO_REQD} flag using the \refmacro{PMIX_INFO_REQUIRED} macro.
\adviceuserend

\adviceimplstart
The top 16-bits of the \refstruct{pmix_info_directives_t} are reserved for internal use by \ac{PMIx} library implementers - the \ac{PMIx} standard will \textit{not} specify their intent, leaving them for customized use by implementers. Implementers are advised to use the provided \refmacro{PMIX_INFO_IS_REQUIRED} macro for testing this flag, and must return \refconst{PMIX_ERR_NOT_SUPPORTED} as soon as possible to the caller if the required behavior is not supported.
\adviceimplend

The following constants were introduced in version 2.0 (unless otherwise marked) and can be used to set a variable of the type \refstruct{pmix_info_directives_t}.

\begin{constantdesc}
%
\declareconstitem{PMIX_INFO_REQD}
The behavior defined in the \refstruct{pmix_info_t} array is required, and not optional. This is a bit-mask value.
%
\end{constantdesc}

\advicermstart
Host environments are advised to use the provided \refmacro{PMIX_INFO_IS_REQUIRED} macro for testing this flag and must return \refconst{PMIX_ERR_NOT_SUPPORTED} as soon as possible to the caller if the required behavior is not supported.
\advicermend


\subsection{Info Directive support macros}

The following macros are provided to support the setting and testing of \refstruct{pmix_info_t} directives.

%%%%
\subsubsection{Mark an info structure as required}
\declaremacro{PMIX_INFO_REQUIRED}

%%%%
\summary

Set the \refconst{PMIX_INFO_REQD} flag in a \refstruct{pmix_info_t} structure.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_REQUIRED(info);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{info}{Pointer to the \refstruct{pmix_info_t} (pointer to \refstruct{pmix_info_t})}
\end{arglist}

This macro simplifies the setting of the \refstruct{PMIX_INFO_REQD} flag in \refstruct{pmix_info_t} structures.

%%%%%%%%%%%
\subsubsection{Test an info structure for \textit{required} directive}
\declaremacro{PMIX_INFO_IS_REQUIRED}

%%%%
\summary

Test the \refconst{PMIX_INFO_REQD} flag in a \refstruct{pmix_info_t} structure, returning \code{true} if the flag is set.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_INFO_IS_REQUIRED(info);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{info}{Pointer to the \refstruct{pmix_info_t} (pointer to \refstruct{pmix_info_t})}
\end{arglist}

This macro simplifies the testing of the required flag in \refstruct{pmix_info_t} structures.



%%%%%%%%%%%
\subsection{Job Allocation Directives}
\declarestruct{pmix_alloc_directive_t}

\versionMarker{2.0}
The \refstruct{pmix_alloc_directive_t} structure is a \code{uint8_t} type that defines the behavior of allocation requests.
The following constants can be used to set a variable of the type \refstruct{pmix_alloc_directive_t}. All definitions were introduced in version 2 of the standard unless otherwise marked.

\begin{constantdesc}
%
\declareconstitem{PMIX_ALLOC_NEW}
A new allocation is being requested.
The resulting allocation will be disjoint (i.e., not connected in a job sense) from the requesting allocation.
%
\declareconstitem{PMIX_ALLOC_EXTEND}
Extend the existing allocation, either in time or as additional resources.
%
\declareconstitem{PMIX_ALLOC_RELEASE}
Release part of the existing allocation.
Attributes in the accompanying \refstruct{pmix_info_t} array may be used to specify permanent release of the identified resources, or ``lending'' of those resources for some period of time.
%
\declareconstitem{PMIX_ALLOC_REAQUIRE}
Reacquire resources that were previously ``lent'' back to the scheduler.
%
\declareconstitem{PMIX_ALLOC_EXTERNAL}
A value boundary above which implementers are free to define their own directive values.
%
\end{constantdesc}



%%%%%%%%%%%
\subsection{Lookup Returned Data Structure}
\declarestruct{pmix_pdata_t}

The \refstruct{pmix_pdata_t} structure is used by \refapi{PMIx_Lookup} to describe the data being accessed.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_pdata \{
    pmix_proc_t proc;
    pmix_key_t key;
    pmix_value_t value;
\} pmix_pdata_t;
\end{codepar}
\cspecificend

\subsection{Lookup data structure support macros}

The following macros are provided to support the \refstruct{pmix_pdata_t} structure.

\subsubsection{Initialize the \refstruct{pmix_pdata_t} structure}
\declaremacro{PMIX_PDATA_CONSTRUCT}

Initialize the \refstruct{pmix_pdata_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PDATA_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_pdata_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_pdata_t} structure}
\declaremacro{PMIX_PDATA_DESTRUCT}

Destruct the \refstruct{pmix_pdata_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PDATA_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_pdata_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Create a \refstruct{pmix_pdata_t} array}
\declaremacro{PMIX_PDATA_CREATE}

Allocate and initialize an array of \refstruct{pmix_pdata_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PDATA_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_pdata_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\subsubsection{Free a\refstruct{pmix_pdata_t} array}
\declaremacro{PMIX_PDATA_FREE}

Release an array of \refstruct{pmix_pdata_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PDATA_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_pdata_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Load a lookup data structure}
\declaremacro{PMIX_PDATA_LOAD}

%%%%
\summary

Load key, process identifier, and data value into a \refstruct{pmix_pdata_t} structure.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_PDATA_LOAD(m, p, k, d, t);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the \refstruct{pmix_pdata_t} structure into which the key and data are to be loaded (pointer to \refstruct{pmix_pdata_t})}
\argin{p}{Pointer to the \refstruct{pmix_proc_t} structure containing the identifier of the process being referenced (pointer to \refstruct{pmix_proc_t})}
\argin{k}{String key to be loaded - must be less than or equal to \refconst{PMIX_MAX_KEYLEN} in length (handle)}
\argin{d}{Pointer to the data value to be loaded (handle)}
\argin{t}{Type of the provided data value (\refstruct{pmix_data_type_t})}
\end{arglist}

This macro simplifies the loading of key, process identifier, and data into a \refstruct{pmix_proc_t} by correctly assigning values to the structure's fields.

\adviceuserstart
Key, process identifier, and data will all be copied into the \refstruct{pmix_pdata_t} - thus, the source information can be modified or free'd without affecting the copied data once the macro has completed.
\adviceuserend

%%%%%%%%%%%
\subsubsection{Transfer a lookup data structure}
\declaremacro{PMIX_PDATA_XFER}

%%%%
\summary

Transfer key, process identifier, and data value between two \refstruct{pmix_pdata_t} structures.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_PDATA_XFER(d, s);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{d}{Pointer to the destination \refstruct{pmix_pdata_t} (pointer to \refstruct{pmix_pdata_t})}
\argin{s}{Pointer to the source \refstruct{pmix_pdata_t} (pointer to \refstruct{pmix_pdata_t})}
\end{arglist}

This macro simplifies the transfer of key and data between two\refstruct{pmix_pdata_t} structures.

\adviceuserstart
Key, process identifier, and data will all be copied into the destination \refstruct{pmix_pdata_t} - thus, the source \refstruct{pmix_pdata_t} may free'd without affecting the copied data once the macro has completed.
\adviceuserend


%%%%%%%%%%%
\subsection{Application Structure}
\declarestruct{pmix_app_t}

The \refstruct{pmix_app_t} structure describes the application context for the \refapi{PMIx_Spawn} and \refapi{PMIx_Spawn_nb} operations.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_app \{
    /** Executable */
    char *cmd;
    /** Argument set, NULL terminated */
    char **argv;
    /** Environment set, NULL terminated */
    char **env;
    /** Current working directory */
    char *cwd;
    /** Maximum processes with this profile */
    int maxprocs;
    /** Array of info keys describing this application*/
    pmix_info_t *info;
    /** Number of info keys in 'info' array */
    size_t ninfo;
\} pmix_app_t;
\end{codepar}
\cspecificend

\subsection{App structure support macros}
The following macros are provided to support the \refstruct{pmix_app_t} structure.

\subsubsection{Initialize the \refstruct{pmix_app_t} structure}
\declaremacro{PMIX_APP_CONSTRUCT}

Initialize the \refstruct{pmix_app_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_APP_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_app_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_app_t} structure}
\declaremacro{PMIX_APP_DESTRUCT}

Destruct the \refstruct{pmix_app_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_APP_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_app_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Create a \refstruct{pmix_app_t} array}
\declaremacro{PMIX_APP_CREATE}

Allocate and initialize an array of \refstruct{pmix_app_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_APP_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_app_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\subsubsection{Free a\refstruct{pmix_app_t} array}
\declaremacro{PMIX_APP_FREE}

Release an array of \refstruct{pmix_app_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_APP_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_app_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

%%%%%%%%%%%
\subsection{Query Structure}
\declarestruct{pmix_query_t}

The \refstruct{pmix_query_t} structure is used by \refapi{PMIx_Query_info_nb} to describe a single query operation.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_query \{
    char **keys;
    pmix_info_t *qualifiers;
    size_t nqual;
\} pmix_query_t;
\end{codepar}
\cspecificend

\subsection{Query structure support macros}
The following macros are provided to support the \refstruct{pmix_query_t} structure.

\subsubsection{Initialize the \refstruct{pmix_query_t} structure}
\declaremacro{PMIX_QUERY_CONSTRUCT}

Initialize the \refstruct{pmix_query_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_QUERY_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_query_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_query_t} structure}
\declaremacro{PMIX_QUERY_DESTRUCT}

Destruct the \refstruct{pmix_query_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_QUERY_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_query_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Create a \refstruct{pmix_query_t} array}
\declaremacro{PMIX_QUERY_CREATE}

Allocate and initialize an array of \refstruct{pmix_query_t} structures

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_QUERY_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_query_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\subsubsection{Free a\refstruct{pmix_query_t} array}
\declaremacro{PMIX_QUERY_FREE}

Release an array of \refstruct{pmix_query_t} structures

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_QUERY_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_query_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

%%%%%%%%%%%
\subsection{Modex Structure}
\declarestruct{pmix_modex_data_t}

The \refstruct{pmix_modex_data_t} structure describes the \ac{BCX} information.

\notestart
\noteheader
This structure and its supporting macros have been deprecated and will be removed in future versions of the \ac{PMIx} Standard.
\noteend

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_modex_data \{
    pmix_nspace_t nspace;
    int rank;
    uint8_t *blob;
    size_t size;
\} pmix_modex_data_t;
\end{codepar}
\cspecificend

\subsection{Modex data structure support macros}
The following macros are provided to support the \refstruct{pmix_modex_t} structure.

\subsubsection{Initialize the \refstruct{pmix_modex_t} structure}
\declaremacro{PMIX_MODEX_CONSTRUCT}

Initialize the \refstruct{pmix_modex_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_MODEX_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_modex_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_modex_t} structure}
\declaremacro{PMIX_MODEX_DESTRUCT}

Destruct the \refstruct{pmix_modex_t} fields

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_MODEX_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_modex_t})}
\end{arglist}

%%%%%%%%%%%
\subsubsection{Create a \refstruct{pmix_modex_t} array}
\declaremacro{PMIX_MODEX_CREATE}

Allocate and initialize an array of \refstruct{pmix_modex_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_MODEX_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_modex_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\subsubsection{Free a\refstruct{pmix_modex_t} array}
\declaremacro{PMIX_MODEX_FREE}

Release an array of \refstruct{pmix_modex_t} structures

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
PMIX_MODEX_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_modex_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

%%%%%%%%%%%
\section{Data Packing/Unpacking Types and Structures}

This section defines types and structures used to pack and unpack data passed through the PMIx API.

%%%%%%%%%%%
\subsection{Byte Object Type}
\declarestruct{pmix_byte_object_t}

The \refstruct{pmix_byte_object_t} structure describes a raw byte sequence.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_byte_object \{
    char *bytes;
    size_t size;
\} pmix_byte_object_t;
\end{codepar}
\cspecificend

\subsection{Byte object support macros}
The following macros support the \refstruct{pmix_byte_object_t} structure.

\subsubsection{Initialize the \refstruct{pmix_byte_object_t} structure}
\declaremacro{PMIX_PROC_CONSTRUCT}

Initialize the \refstruct{pmix_byte_object_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_BYTE_OBJECT_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_byte_object_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_byte_object_t} structure}
\declaremacro{PMIX_BYTE_OBJECT_DESTRUCT}

Clear the \refstruct{pmix_byte_object_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_BYTE_OBJECT_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_byte_object_t})}
\end{arglist}

\subsubsection{Create a \refstruct{pmix_byte_object_t} structure}
\declaremacro{PMIX_BYTE_OBJECT_CREATE}

Allocate and intitialize an array of \refstruct{pmix_byte_object_t} structures

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_BYTE_OBJECT_CREATE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_byte_object_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}

\subsubsection{Free a \refstruct{pmix_byte_object_t} array}
\declaremacro{PMIX_BYTE_OBJECT_FREE}

Release an array of \refstruct{pmix_byte_object_t} structures

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_BYTE_OBJECT_FREE(m, n)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_byte_object_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

\subsubsection{Load a\refstruct{pmix_byte_object_t} structure}
\declaremacro{PMIX_BYTE_OBJECT_LOAD}

Load values into a \refstruct{pmix_byte_object_t}

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_BYTE_OBJECT_LOAD(b, d, s)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{b}{Pointer to the structure to be loaded (pointer to \refstruct{pmix_byte_object_t})}
\argin{d}{Pointer to the data to be loaded (\code{char*})}
\argin{s}{Number of bytes in the data array (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\subsection{Data Buffer Type}
\declarestruct{pmix_data_buffer_t}

The \refstruct{pmix_data_buffer_t} structure describes a data buffer used for packing and unpacking.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_data_buffer \{
    /** Start of my memory */
    char *base_ptr;
    /** Where the next data will be packed to (within the allocated
        memory starting at base_ptr) */
    char *pack_ptr;
    /** Where the next data will be unpacked from (within the
        allocated memory starting as base_ptr) */
    char *unpack_ptr;
    /** Number of bytes allocated (starting at base_ptr) */
    size_t bytes_allocated;
    /** Number of bytes used by the buffer (i.e., amount of data --
        including overhead -- packed in the buffer) */
    size_t bytes_used;
\} pmix_data_buffer_t;
\end{codepar}
\cspecificend

\subsection{Data buffer support macros}
The following macros support the \refstruct{pmix_data_buffer_t} structure.

\subsubsection{Initialize the \refstruct{pmix_data_buffer_t} structure}
\declaremacro{PMIX_PROC_CONSTRUCT}

Initialize the \refstruct{pmix_data_buffer_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_DATA_BUFFER_CONSTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_data_buffer_t})}
\end{arglist}

\subsubsection{Destruct the \refstruct{pmix_data_buffer_t} structure}
\declaremacro{PMIX_DATA_BUFFER_DESTRUCT}

Clear the \refstruct{pmix_data_buffer_t} fields

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_DATA_BUFFER_DESTRUCT(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_data_buffer_t})}
\end{arglist}

\subsubsection{Create a \refstruct{pmix_data_buffer_t} structure}
\declaremacro{PMIX_DATA_BUFFER_CREATE}

Allocate and intitialize a \refstruct{pmix_data_buffer_t} structure

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_DATA_BUFFER_CREATE(m)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{m}{Address where the pointer to the \refstruct{pmix_data_buffer_t} structure shall be stored (handle)}
\end{arglist}

\subsubsection{Free a \refstruct{pmix_data_buffer_t}}
\declaremacro{PMIX_DATA_BUFFER_RELEASE}

Release a \refstruct{pmix_data_buffer_t} structure

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
PMIX_DATA_BUFFER_RELEASE(m)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{m}{Pointer to the \refstruct{pmix_data_buffer_t} structure to be released (handle)}
\end{arglist}


%%%%%%%%%%%
\subsection{Data Array Structure}
\declarestruct{pmix_data_array_t}

The \refstruct{pmix_data_array_t} structure defines an array data structure.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef struct pmix_data_array \{
    pmix_data_type_t type;
    size_t size;
    void *array;
\} pmix_data_array_t;
\end{codepar}
\cspecificend



%%%%%%%%%%
\subsection{Generalized Data Types Used for Packing/Unpacking}
\declarestruct{pmix_data_type_t}

The \refstruct{pmix_data_type_t} structure is a \code{uint16_t} type for identifying the data type for packing/unpacking purposes.

\adviceimplstart
The following constants can be used to set a variable of the type \refstruct{pmix_data_type_t}. Data types in the \ac{PMIx} Standard are defined in terms of the C-programming language. Implementers wishing to support other languages should provide the equivalent definitions in a language-appropriate manner. Additionally, a PMIx implementation may choose to add additional types.
\adviceimplend

\subsubsection{PMIx v1 Data Types}
The following types were introduced in version 1 of the \ac{PMIx} Standard.

\begin{constantdesc}
%
\declareconstitem{PMIX_UNDEF}
Undefined
%
\declareconstitem{PMIX_BOOL}
Boolean (converted to/from native \code{true}/\code{false}) (\code{bool})
%
\declareconstitem{PMIX_BYTE}
A byte of data (\code{uint8_t})
%
\declareconstitem{PMIX_STRING}
\code{NULL} terminated string (\code{char*})
%
\declareconstitem{PMIX_SIZE}
Size \code{size_t}
%
\declareconstitem{PMIX_PID}
Operating \ac{PID} (\code{pid_t})
%
\declareconstitem{PMIX_INT}
Integer (\code{int})
%
\declareconstitem{PMIX_INT8}
8-byte integer (\code{int8_t})
%
\declareconstitem{PMIX_INT16}
16-byte integer (\code{int16_t})
%
\declareconstitem{PMIX_INT32}
32-byte integer (\code{int32_t})
%
\declareconstitem{PMIX_INT64}
64-byte integer (\code{int64_t})
%
\declareconstitem{PMIX_UINT}
Unsigned integer (\code{unsigned int})
%
\declareconstitem{PMIX_UINT8}
Unsigned 8-byte integer (\code{uint8_t})
%
\declareconstitem{PMIX_UINT16}
Unsigned 16-byte integer (\code{uint16_t})
%
\declareconstitem{PMIX_UINT32}
Unsigned 32-byte integer (\code{uint32_t})
%
\declareconstitem{PMIX_UINT64}
Unsigned 64-byte integer (\code{uint64_t})
%
\declareconstitem{PMIX_FLOAT}
Float (\code{float})
%
\declareconstitem{PMIX_DOUBLE}
Double (\code{double})
%
\declareconstitem{PMIX_TIMEVAL}
Time value (\code{struct timeval})
%
\declareconstitem{PMIX_TIME}
Time (\code{time_t})
%
\declareconstitem{PMIX_VALUE}
Value (\refstruct{pmix_value_t})
%
\declareconstitem{PMIX_PROC}
Process (\refstruct{pmix_proc_t})
%
\declareconstitem{PMIX_APP}
Application context
%
\declareconstitem{PMIX_INFO}
Info object
%
\declareconstitem{PMIX_PDATA}
Pointer to data
%
\declareconstitem{PMIX_BUFFER}
Buffer
%
\declareconstitem{PMIX_BYTE_OBJECT}
Byte object (\refstruct{pmix_byte_object_t})
%
\declareconstitem{PMIX_KVAL}
Key/value pair
%
\declareconstitemDEP{PMIX_MODEX}{2.0}
Modex
%
\declareconstitem{PMIX_PERSIST}
Persistance (\refstruct{pmix_persistence_t})
%
\declareconstitemDEP{PMIX_INFO_ARRAY}{2.0}
Info array
%
\end{constantdesc}


\subsubsection{PMIx v2 Data Types}
The following types were introduced in version 2 of the \ac{PMIx} Standard.

\begin{constantdesc}

%
\declareconstitem{PMIX_STATUS}
Status (\refstruct{pmix_status_t})
%
\declareconstitem{PMIX_POINTER}
Pointer (\code{void*})
%
\declareconstitem{PMIX_SCOPE}
Scope (\refstruct{pmix_scope_t})
%
\declareconstitem{PMIX_DATA_RANGE}
Data range (\refstruct{pmix_data_range_t})
%
\declareconstitem{PMIX_COMMAND}
Command
%
\declareconstitem{PMIX_INFO_DIRECTIVES}
Info directives
%
\declareconstitem{PMIX_DATA_TYPE}
Data type
%
\declareconstitem{PMIX_PROC_STATE}
Process state (\refstruct{pmix_proc_state_t})
%
\declareconstitem{PMIX_PROC_INFO}
Process info (\refstruct{pmix_proc_info_t})
%
\declareconstitem{PMIX_DATA_ARRAY}
Data array (\refstruct{pmix_data_array_t})
%
\declareconstitem{PMIX_PROC_RANK}
Process rank (\refstruct{pmix_rank_t})
%
\declareconstitem{PMIX_QUERY}
Query
%
\declareconstitem{PMIX_COMPRESSED_STRING}
Compressed string (with zlib)
%
\declareconstitem{PMIX_ALLOC_DIRECTIVE}
Allocation directive (\refstruct{pmix_alloc_directive_t})
%
\declareconstitem{PMIX_DATA_TYPE_MAX}
A boundary for implementers above which they can add their own data types.
%
\end{constantdesc}


%%%%%%%%%%%
\section{Reserved attributes}
\label{api:struct:attributes}

The PMIx standard defines a relatively small set of APIs and the caller may customize the behavior of the API by passing one or more attributes to that API.
Additionally, attributes may be keys passed to \refapi{PMIx_Get} calls to access the specified values from the system.

Each attribute is represented by a \var{key} string, and a type for the associated \var{value}.
This section defines a set of \textbf{reserved} keys which are prefixed with \code{pmix.} to designate them as PMIx standard reserved keys. All definitions were introduced in version 1 of the standard unless otherwise marked.

Applications or associated libraries (e.g., \ac{MPI}) may choose to define additional attributes.
The attributes defined in this section are of the system and job as opposed to the attributes that the application (or associated libraries) might choose to expose.
Due to this extensibility the \refapi{PMIx_Get} API will return \refconst{PMIX_ERR_NOT_FOUND} if the provided \var{key} cannot be found.

Attributes added in this version of the standard are shown in \textit{\textbf{\color{magenta}magenta}} to distinguish them from those defined in prior versions, which are shown in \textit{\textbf{black}}. Deprecated attributes are shown in \textit{\textbf{\color{green!80!black}green}} and will be removed in future versions of the standard.

\declareAttribute{PMIX_ATTR_UNDEF}{NULL}{NULL}{
Constant representing an undefined attribute.
}

%%%%%%%%%%%
\subsection{Initialization attributes}
\label{api:struct:attributes:init}

These attributes are defined to assist the caller with initialization.

%
\declareNewAttribute{PMIX_EVENT_BASE}{"pmix.evbase"}{struct event_base *}{
Pointer to libevent\footnote{\url{http://libevent.org/}} \code{event_base} to use in place of the internal progress thread.
}

%
\declareNewAttribute{PMIX_SERVER_TOOL_SUPPORT}{"pmix.srvr.tool"}{bool}{
The host \ac{RM} wants to declare itself as willing to accept tool connection requests.
}

%
\declareNewAttribute{PMIX_SERVER_REMOTE_CONNECTIONS}{"pmix.srvr.remote"}{bool}{
Allow connections from remote tools. Forces the PMIx server to not exclusively use loopback device.
}

%
\declareNewAttribute{PMIX_SERVER_SYSTEM_SUPPORT}{"pmix.srvr.sys"}{bool}{
The host \ac{RM} wants to declare itself as being the local system server for PMIx connection requests.
}

%
\declareAttribute{PMIX_SERVER_TMPDIR}{"pmix.srvr.tmpdir"}{char*}{
Top-level temporary directory for all \emph{client} processes connected to this server, and where the PMIx server will place its \emph{tool} rendezvous point and contact information.
}

%
\declareNewAttribute{PMIX_SYSTEM_TMPDIR}{"pmix.sys.tmpdir"}{char*}{
Temporary directory for this system, and where a PMIx server that declares itself to be a system-level server will place a \emph{tool} rendezvous point and contact information.
}

%
\declareNewAttribute{PMIX_REGISTER_NODATA}{"pmix.reg.nodata"}{bool}{
Registration is for the namespace only. Do not copy job data.
}

%
\declareNewAttribute{PMIX_SERVER_ENABLE_MONITORING}{"pmix.srv.monitor"}{bool}{
Enable PMIx internal monitoring by the PMIx server.
}

%
\declareNewAttribute{PMIX_SERVER_NSPACE}{"pmix.srv.nspace"}{char*}{
Name of the namespace to use for this PMIx server.
}

%
\declareNewAttribute{PMIX_SERVER_RANK}{"pmix.srv.rank"}{pmix_rank_t}{
Rank of this PMIx server
}


%%%%%%%%%%%
\subsection{Tool-related attributes}
\label{api:struct:attributes:tool}

These attributes are defined to assist PMIx-enabled tools to connect with the PMIx server.

%
\declareNewAttribute{PMIX_TOOL_NSPACE}{"pmix.tool.nspace"}{char*}{
Name of the namespace to use for this tool.
}

%
\declareNewAttribute{PMIX_TOOL_RANK}{"pmix.tool.rank"}{uint32_t}{
Rank of this tool.
}

%
\declareNewAttribute{PMIX_SERVER_PIDINFO}{"pmix.srvr.pidinfo"}{pid_t}{
\ac{PID} of the target PMIx server for a tool.
}

%
\declareNewAttribute{PMIX_CONNECT_TO_SYSTEM}{"pmix.cnct.sys"}{bool}{
The requestor requires that a connection be made only to a local, system-level PMIx server.
}

%
\declareNewAttribute{PMIX_CONNECT_SYSTEM_FIRST}{"pmix.cnct.sys.first"}{bool}{
Preferentially, look for a system-level PMIx server first.
}

%
\declareNewAttribute{PMIX_SERVER_URI}{"pmix.srvr.uri"}{char*}{
\ac{URI} of the PMIx server to be contacted.
}

%
\declareNewAttribute{PMIX_SERVER_HOSTNAME}{"pmix.srvr.host"}{char*}{
Host where target PMIx server is located.
}

%
\declareNewAttribute{PMIX_CONNECT_MAX_RETRIES}{"pmix.tool.mretries"}{uint32_t}{
Maximum number of times to try to connect to PMIx server.
}

%
\declareNewAttribute{PMIX_CONNECT_RETRY_DELAY}{"pmix.tool.retry"}{uint32_t}{
Time in seconds between connection attempts to a PMIx server.
}

%
\declareNewAttribute{PMIX_TOOL_DO_NOT_CONNECT}{"pmix.tool.nocon"}{bool}{
The tool wants to use internal PMIx support, but does not want to connect to a PMIx server.
}


%%%%%%%%%%%
\subsection{Identification attributes}
\label{api:struct:attributes:ident}

These attributes are defined to identify a process and it's associated PMIx-enabled library.

%
\declareAttribute{PMIX_USERID}{"pmix.euid"}{uint32_t}{
Effective user id.
}

%
\declareAttribute{PMIX_GRPID}{"pmix.egid"}{uint32_t}{
Effective group id.
}

%
\declareAttribute{PMIX_DSTPATH}{"pmix.dstpath"}{char*}{
Path to shared memory data storage (dstore) files.
}

%
\declareNewAttribute{PMIX_VERSION_INFO}{"pmix.version"}{char*}{
PMIx version of contractor.
}

%
\declareNewAttribute{PMIX_PROGRAMMING_MODEL}{"pmix.pgm.model"}{char*}{
Programming model being initialized (e.g., ``MPI'' or ``OpenMP'')
}

%
\declareNewAttribute{PMIX_MODEL_LIBRARY_NAME}{"pmix.mdl.name"}{char*}{
Programming model implementation ID (e.g., ``OpenMPI'' or ``MPICH'')
}

%
\declareNewAttribute{PMIX_MODEL_LIBRARY_VERSION}{"pmix.mld.vrs"}{char*}{
Programming model version string (e.g., ``2.1.1'')
}

%
\declareNewAttribute{PMIX_THREADING_MODEL}{"pmix.threads"}{char*}{
Threading model used (e.g., ``pthreads'')
}

%
\declareNewAttribute{PMIX_REQUESTOR_IS_TOOL}{"pmix.req.tool"}{bool}{
The requesting process is a PMIx tool.
}

%
\declareNewAttribute{PMIX_REQUESTOR_IS_CLIENT}{"pmix.req.client"}{bool}{
The requesting process is a PMIx client.
}


%%%%%%%%%%%
\subsection{UNIX socket rendezvous socket attributes}
\label{api:struct:attributes:usock}

These attributes are used to describe a UNIX socket for rendezvous with the local \ac{RM}.

%
\declareNewAttribute{PMIX_USOCK_DISABLE}{"pmix.usock.disable"}{bool}{
Disable legacy UNIX socket (usock) support
}

%
\declareNewAttribute{PMIX_SOCKET_MODE}{"pmix.sockmode"}{uint32_t}{
POSIX \var{mode_t} (9 bits valid)
}

%
\declareNewAttribute{PMIX_SINGLE_LISTENER}{"pmix.sing.listnr"}{bool}{
Use only one rendezvous socket, letting priorities and/or environment parameters select the active transport.
}


%%%%%%%%%%%
\subsection{TCP connection attributes}
\label{api:struct:attributes:tcp}

These attributes are used to describe a TCP socket for rendezvous with the local \ac{RM}.

%
\declareNewAttribute{PMIX_TCP_REPORT_URI}{"pmix.tcp.repuri"}{char*}{
If provided, directs that the TCP \ac{URI} be reported and indicates the desired method of reporting: \code{'-'} for stdout, \code{'+'} for stderr, or filename.
}

%
\declareNewAttribute{PMIX_TCP_URI}{"pmix.tcp.uri"}{char*}{
The \ac{URI} of the PMIx server to connect to, or a file name containing it in the form of \code{file:<name of file containing it>}.
}

%
\declareNewAttribute{PMIX_TCP_IF_INCLUDE}{"pmix.tcp.ifinclude"}{char*}{
Comma-delimited list of devices and/or \ac{CIDR} notation to include when establishing the TCP connection.
}

%
\declareNewAttribute{PMIX_TCP_IF_EXCLUDE}{"pmix.tcp.ifexclude"}{char*}{
Comma-delimited list of devices and/or \ac{CIDR} notation to exclude when establishing the TCP connection.
}

%
\declareNewAttribute{PMIX_TCP_IPV4_PORT}{"pmix.tcp.ipv4"}{int}{
The IPv4 port to be used.
}

%
\declareNewAttribute{PMIX_TCP_IPV6_PORT}{"pmix.tcp.ipv6"}{int}{
The IPv6 port to be used.
}

%
\declareNewAttribute{PMIX_TCP_DISABLE_IPV4}{"pmix.tcp.disipv4"}{bool}{
Set to \code{true} to disable IPv4 family of addresses.
}

%
\declareNewAttribute{PMIX_TCP_DISABLE_IPV6}{"pmix.tcp.disipv6"}{bool}{
Set to \code{true} to disable IPv6 family of addresses.
}


%%%%%%%%%%%
\subsection{Global Data Storage (GDS) attributes}
\label{api:struct:attributes:gds}

These attributes are used to define the behavior of the \ac{GDS} used to manage key/value pairs.

%
\declareNewAttribute{PMIX_GDS_MODULE}{"pmix.gds.mod"}{char*}{
Comma-delimited string of desired modules.
}


%%%%%%%%%%%
\subsection{General process-level attributes}
\label{api:struct:attributes:gproc}

These attributes are used to define process attributes.

%
\declareAttribute{PMIX_CPUSET}{"pmix.cpuset"}{char*}{
hwloc\footnote{\url{https://www.open-mpi.org/projects/hwloc/}} bitmap to be applied to the process upon launch.
}

%
\declareAttribute{PMIX_CREDENTIAL}{"pmix.cred"}{char*}{
Security credential assigned to the process.
}

%
\declareAttribute{PMIX_SPAWNED}{"pmix.spawned"}{bool}{
\code{true} if this process resulted from a call to \refapi{PMIx_Spawn}.
}

%
\declareAttribute{PMIX_ARCH}{"pmix.arch"}{uint32_t}{
Architecture flag.
}


%%%%%%%%%%%
\subsection{Scratch directory attributes}
\label{api:struct:attributes:scratchdir}

These attributes are used to define an application scratch directory.

%
\declareAttribute{PMIX_TMPDIR}{"pmix.tmpdir"}{char*}{
Full path to the top-level temporary directory assigned to the session.
}

%
\declareAttribute{PMIX_NSDIR}{"pmix.nsdir"}{char*}{
Full path to the temporary directory assigned to the namespace, under \refattr{PMIX_TMPDIR}.
}

%
\declareAttribute{PMIX_PROCDIR}{"pmix.pdir"}{char*}{
Full path to the subdirectory under \refattr{PMIX_NSDIR} assigned to the process.
}

%
\declareNewAttribute{PMIX_TDIR_RMCLEAN}{"pmix.tdir.rmclean"}{bool}{
Resource Manager will clean session directories
}


%%%%%%%%%%%
\subsection{Relative Rank Descriptive Attributes}
\label{api:struct:attributes:relrankinfo}

These attributes are used to describe information about relative ranks as assigned by the \ac{RM}.

%
\declareNewAttribute{PMIX_PROCID}{"pmix.procid"}{pmix_proc_t}{
Process identifier
}

%
\declareNewAttribute{PMIX_NSPACE}{"pmix.nspace"}{char*}{
Namespace of the job.
}

%
\declareAttribute{PMIX_JOBID}{"pmix.jobid"}{char*}{
Job identifier assigned by the scheduler.
}

%
\declareAttribute{PMIX_APPNUM}{"pmix.appnum"}{uint32_t}{
Application number within the job.
}

%
\declareAttribute{PMIX_RANK}{"pmix.rank"}{pmix_rank_t}{
Process rank within the job.
}

%
\declareAttribute{PMIX_GLOBAL_RANK}{"pmix.grank"}{pmix_rank_t}{
Process rank spanning across all jobs in this session.
}

%
\declareAttribute{PMIX_APP_RANK}{"pmix.apprank"}{pmix_rank_t}{
Process rank within this application.
}

%
\declareAttribute{PMIX_NPROC_OFFSET}{"pmix.offset"}{pmix_rank_t}{
Starting global rank of this job.
}

%
\declareAttribute{PMIX_LOCAL_RANK}{"pmix.lrank"}{uint16_t}{
Local rank on this node within this job.
}

%
\declareAttribute{PMIX_NODE_RANK}{"pmix.nrank"}{uint16_t}{
Process rank on this node spanning all jobs.
}

%
\declareAttribute{PMIX_LOCALLDR}{"pmix.lldr"}{pmix_rank_t}{
Lowest rank on this node within this job.
}

%
\declareAttribute{PMIX_APPLDR}{"pmix.aldr"}{pmix_rank_t}{
Lowest rank in this application within this job.
}

%
\declareNewAttribute{PMIX_PROC_PID}{"pmix.ppid"}{pid_t}{
\ac{PID} of specified process.
}

%
\declareNewAttribute{PMIX_SESSION_ID}{"pmix.session.id"}{uint32_t}{
Session identifier.
}

%
\declareAttribute{PMIX_NODE_LIST}{"pmix.nlist"}{char*}{
Comma-delimited list of nodes running processes for the specified namespace.
}

%
\declareNewAttribute{PMIX_ALLOCATED_NODELIST}{"pmix.alist"}{char*}{
Comma-delimited list of all nodes in this allocation regardless of whether or not they currently host processes.
}

%
\declareAttribute{PMIX_HOSTNAME}{"pmix.hname"}{char*}{
Name of the host where the specified process is running.
}

%
\declareAttribute{PMIX_NODEID}{"pmix.nodeid"}{uint32_t}{
Node identifier where the specified process is located, expressed as the node's index (beginning at zero) in an array of nodes comprising the users allocation
}

%
\declareAttribute{PMIX_LOCAL_PEERS}{"pmix.lpeers"}{char*}{
Comma-delimited list of ranks on this node within the specified namespace.
}

%
\declareAttribute{PMIX_LOCAL_PROCS}{"pmix.lprocs"}{pmix_proc_t array}{
Array of \refstruct{pmix_proc_t} of processes on the specified node.
}

%
\declareAttribute{PMIX_LOCAL_CPUSETS}{"pmix.lcpus"}{char*}{
Colon-delimited cpusets of local peers within the specified namespace.
}

%
\declareAttribute{PMIX_PROC_URI}{"pmix.puri"}{char*}{
\ac{URI} containing contact information for a given process.
}

%
\declareNewAttribute{PMIX_LOCALITY}{"pmix.loc"}{uint16_t}{
Relative locality of two processes.
}

%
\declareNewAttribute{PMIX_PARENT_ID}{"pmix.parent"}{pmix_proc_t}{
Process identifier of the parent process of the calling process.
}


%%%%%%%%%%%
\subsection{Size information attributes}
\label{api:struct:attributes:sizeinfo}

These attributes are used to describe the size of various dimensions of the PMIx universe.

%
\declareAttribute{PMIX_UNIV_SIZE}{"pmix.univ.size"}{uint32_t}{
Number of processes in this namespace.
}

%
\declareAttribute{PMIX_JOB_SIZE}{"pmix.job.size"}{uint32_t}{
Number of processes in this job.
}

%
\declareNewAttribute{PMIX_JOB_NUM_APPS}{"pmix.job.napps"}{uint32_t}{
Number of applications in this job.
}

%
\declareNewAttribute{PMIX_APP_SIZE}{"pmix.app.size"}{uint32_t}{
Number of processes in this application.
}

%
\declareAttribute{PMIX_LOCAL_SIZE}{"pmix.local.size"}{uint32_t}{
Number of processes in this job on this node.
}

%
\declareAttribute{PMIX_NODE_SIZE}{"pmix.node.size"}{uint32_t}{
Number of processes across all jobs on this node.
}

%
\declareAttribute{PMIX_MAX_PROCS}{"pmix.max.size"}{uint32_t}{
Maximum number of processes for this job.
}

%
\declareNewAttribute{PMIX_NUM_NODES}{"pmix.num.nodes"}{uint32_t}{
Number of nodes in this namespace.
}


%%%%%%%%%%%
\subsection{Memory information attributes}
\label{api:struct:attributes:meminfo}

These attributes are used to describe memory available and used in the system.

%
\declareNewAttribute{PMIX_AVAIL_PHYS_MEMORY}{"pmix.pmem"}{uint64_t}{
Total available physical memory on this node.
}

%
\declareNewAttribute{PMIX_DAEMON_MEMORY}{"pmix.dmn.mem"}{float}{
Megabytes of memory currently used by the \ac{RM} daemon.
}

%
\declareNewAttribute{PMIX_CLIENT_AVG_MEMORY}{"pmix.cl.mem.avg"}{float}{
Average Megabytes of memory used by client processes.
}


%%%%%%%%%%%
\subsection{Topology information attributes}
\label{api:struct:attributes:topoinfo}

These attributes are used to describe topology information in the PMIx universe.

%
\declareAttribute{PMIX_NET_TOPO}{"pmix.ntopo"}{char*}{
\ac{XML} representation of the network topology.
}

%
\declareAttribute{PMIX_LOCAL_TOPO}{"pmix.ltopo"}{char*}{
\ac{XML} representation of local node topology.
}

%
\declareAttribute{PMIX_NODE_LIST}{"pmix.nlist"}{char*}{
Comma-delimited list of nodes running processes for this job.
}

%
\declareAttribute{PMIX_TOPOLOGY}{"pmix.topo"}{hwloc_topology_t}{
Pointer to the PMIx client's internal hwloc topology object.
}

%
\declareNewAttribute{PMIX_TOPOLOGY_SIGNATURE}{"pmix.toposig"}{char*}{
Topology signature string.
}

%
\declareNewAttribute{PMIX_LOCALITY_STRING}{"pmix.locstr"}{char*}{
String describing a process's bound location.
The string is of the form:\\
\code{NM\%s:SK\%s:L3\%s:L2\%s:L1\%s:CR\%s:HT\%s}\\
Where the \code{\%s} is replaced with an integer index or inclusive range for hwloc.
\code{NM} identifies the numa node(s).
\code{SK} identifies the socket(s).
\code{L3} identifies the L3 cache(s).
\code{L2} identifies the L2 cache(s).
\code{L1} identifies the L1 cache(s).
\code{CR} identifies the cores(s).
\code{HT} identifies the hardware thread(s).
If your architecture does not have the specified hardware designation then it can be omitted from the signature.
\\
For example: \code{NM0:SK0:L30-4:L20-4:L10-4:CR0-4:HT0-39}.\\
This means numa node \code{0}, socket \code{0}, L3 caches \code{0,1,2,3,4}, L2 caches \code{0-4}, L1 caches \code{0-4}, cores \code{0,1,2,3,4}, and hardware threads \code{0-39}.
}

%
\declareNewAttribute{PMIX_HWLOC_SHMEM_ADDR}{"pmix.hwlocaddr"}{size_t}{
Address of the hwloc shared memory segment.
}

%
\declareNewAttribute{PMIX_HWLOC_SHMEM_SIZE}{"pmix.hwlocsize"}{size_t}{
Size of the hwloc shared memory segment.
}

%
\declareNewAttribute{PMIX_HWLOC_SHMEM_FILE}{"pmix.hwlocfile"}{char*}{
Path to the hwloc shared memory file.
}

%
\declareNewAttribute{PMIX_HWLOC_XML_V1}{"pmix.hwlocxml1"}{char*}{
\ac{XML} representation of local topology using hwloc's v1.x format.
}

%
\declareNewAttribute{PMIX_HWLOC_XML_V2}{"pmix.hwlocxml2"}{char*}{
\ac{XML} representation of local topology using hwloc's v2.x format.
}


%%%%%%%%%%%
\subsection{Request-related attributes}
\label{api:struct:attributes:request}

These attributes are used to influence the behavior of various PMIx operations.

%
\declareAttribute{PMIX_COLLECT_DATA}{"pmix.collect"}{bool}{
Collect data and return it at the end of the operation.
}

%
\declareAttribute{PMIX_TIMEOUT}{"pmix.timeout"}{int}{
Time in seconds before the specified operation should time out (\var{0} indicating infinite) in error.
The timeout parameter can help avoid ``hangs'' due to programming errors that prevent the target process from ever exposing its data.
}

%
\declareNewAttribute{PMIX_IMMEDIATE}{"pmix.immediate"}{bool}{
Specified operation should immediately return an error from the PMIx server if the requested data cannot be found - do not request it from the host \ac{RM}.
}

%
\declareAttribute{PMIX_WAIT}{"pmix.wait"}{int}{
Caller requests that the PMIx server wait until at least the specified number of values are found (\var{0} indicates all and is the default).
}

%
\declareAttribute{PMIX_COLLECTIVE_ALGO}{"pmix.calgo"}{char*}{
Comma-delimited list of algorithms to use for the collective operation.
}

%
\declareAttribute{PMIX_COLLECTIVE_ALGO_REQD}{"pmix.calreqd"}{bool}{
If \code{true}, indicates that the requested choice of algorithm is mandatory.
}

%
\declareAttribute{PMIX_NOTIFY_COMPLETION}{"pmix.notecomp"}{bool}{
Notify the parent process upon termination of child job.
}

%
\declareAttribute{PMIX_RANGE}{"pmix.range"}{pmix_data_range_t}{
Value for calls to publish/lookup/unpublish or for monitoring event notifications.
}

%
\declareAttribute{PMIX_PERSISTENCE}{"pmix.persist"}{pmix_persistence_t}{
Value for calls to \refapi{PMIx_Publish}.
}

%
\declareNewAttribute{PMIX_DATA_SCOPE}{"pmix.scope"}{pmix_scope_t}{
Scope of the data to be found in a \refapi{PMIx_Get} call.
}

%
\declareAttribute{PMIX_OPTIONAL}{"pmix.optional"}{bool}{
Look only in the client's local data store for the requested value - do not request data from the PMIx server if not found.
}

%
\declareNewAttribute{PMIX_EMBED_BARRIER}{"pmix.embed.barrier"}{bool}{
Execute a blocking fence operation before executing the specified operation.
By default, \refapi{PMIx_Finalize} does not include an internal barrier operation.
This attribute directs \refapi{PMIx_Finalize} to execute a barrier as part of the finalize operation.
}

%
\declareNewAttribute{PMIX_JOB_TERM_STATUS}{"pmix.job.term.status"}{pmix_status_t}{
Status to be returned upon job termination.
}

%
\declareNewAttribute{PMIX_PROC_STATE_STATUS}{"pmix.proc.state"}{pmix_proc_state_t}{
Process state
}


%%%%%%%%%%%
\subsection{Server-to-PMIx library attributes}
\label{api:struct:attributes:server2cl}

Attributes used by the host environment to pass data to its PMIx server library.
The data will then be parsed and provided to the local PMIx clients.

%
\declareNewAttribute{PMIX_REGISTER_NODATA}{"pmix.reg.nodata"}{bool}{
Registration is for this namespace only, do not copy job data.
}

%
\declareAttribute{PMIX_PROC_DATA}{"pmix.pdata"}{pmix_data_array_t}{
Array of process data. Starts with rank, then contains more data.
}

%
\declareAttribute{PMIX_NODE_MAP}{"pmix.nmap"}{char*}{
Regular expression of nodes containing processes for this job.
}

%
\declareAttribute{PMIX_PROC_MAP}{"pmix.pmap"}{char*}{
Regular expression describing processes on each node within this job.
}

%
\declareAttribute{PMIX_ANL_MAP}{"pmix.anlmap"}{char*}{
Process mapping in Argonne National Laboratory's PMI-1/PMI-2 notation.
}

%
\declareNewAttribute{PMIX_APP_MAP_TYPE}{"pmix.apmap.type"}{char*}{
Type of mapping used to layout the application (e.g., \code{cyclic}).
}

%
\declareNewAttribute{PMIX_APP_MAP_REGEX}{"pmix.apmap.regex"}{char*}{
Regular expression describing the result of the process mapping.
}


%%%%%%%%%%%
\subsection{Srever-to-Client attributes}
\label{api:struct:attributes:server2client}

Attributes used internally to communicate data from the PMIx server to the PMIx client.

%
\declareAttribute{PMIX_PROC_BLOB}{"pmix.pblob"}{pmix_byte_object_t}{
Packed blob of process data.
}

%
\declareAttribute{PMIX_MAP_BLOB}{"pmix.mblob"}{pmix_byte_object_t}{
Packed blob of process location.
}


%%%%%%%%%%%
\subsection{Event handler registration and notification attributes}
\label{api:struct:attributes:event}

Attributes to support event registration and notification.

\adviceuserstart
The event handler subsystem defined in the \ac{PMIx} \textit{ad hoc} version 1 Standard was completely overhauled in version 2 to resolve design flaws. Deprecated attributes shown below were therefore removed in the version 2 Standard.
\adviceuserend

%
\declareDepAttribute{PMIX_ERROR_NAME}{"pmix.errname"}{pmix_status_t}{
Specific error to be notified
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_COMM}{"pmix.errgroup.comm"}{bool}{
Set true to get comm errors notification
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_ABORT}{"pmix.errgroup.abort"}{bool}{
Set true to get abort errors notification
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_MIGRATE}{"pmix.errgroup.migrate"}{bool}{
Set true to get migrate errors notification
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_RESOURCE}{"pmix.errgroup.resource"}{bool}{
Set true to get resource errors notification
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_SPAWN}{"pmix.errgroup.spawn"}{bool}{
Set true to get spawn errors notification
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_NODE}{"pmix.errgroup.node"}{bool}{
Set true to get node status notification
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_LOCAL}{"pmix.errgroup.local"}{bool}{
Set true to get local errors notification
}
%
\declareDepAttribute{PMIX_ERROR_GROUP_GENERAL}{"pmix.errgroup.gen"}{bool}{
Set true to get notified of generic errors
}
%
\declareDepAttribute{PMIX_ERROR_HANDLER_ID}{"pmix.errhandler.id"}{int}{
Errhandler reference id of notification being reported
}
%
\declareNewAttribute{PMIX_EVENT_HDLR_NAME}{"pmix.evname"}{char*}{
String name identifying this handler.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_FIRST}{"pmix.evfirst"}{bool}{
Invoke this event handler before any other handlers.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_LAST}{"pmix.evlast"}{bool}{
Invoke this event handler after all other handlers have been called.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_FIRST_IN_CATEGORY}{"pmix.evfirstcat"}{bool}{
Invoke this event handler before any other handlers in this category.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_LAST_IN_CATEGORY}{"pmix.evlastcat"}{bool}{
Invoke this event handler after all other handlers in this category have been called.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_BEFORE}{"pmix.evbefore"}{char*}{
Put this event handler immediately before the one specified in the \code{(char*)} value.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_AFTER}{"pmix.evafter"}{char*}{
Put this event handler immediately after the one specified in the \code{(char*)} value.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_PREPEND}{"pmix.evprepend"}{bool}{
Prepend this handler to the precedence list within its category.
}

%
\declareNewAttribute{PMIX_EVENT_HDLR_APPEND}{"pmix.evappend"}{bool}{
Append this handler to the precedence list within its category.
}

%
\declareNewAttribute{PMIX_EVENT_CUSTOM_RANGE}{"pmix.evrange"}{pmix_data_array_t*}{
Array of \refstruct{pmix_proc_t} defining range of event notification.
}

%
\declareNewAttribute{PMIX_EVENT_AFFECTED_PROC}{"pmix.evproc"}{pmix_proc_t}{
The single process that was affected.
}

%
\declareNewAttribute{PMIX_EVENT_AFFECTED_PROCS}{"pmix.evaffected"}{pmix_data_array_t*}{
Array of \refstruct{pmix_proc_t} defining affected processes.
}

%
\declareNewAttribute{PMIX_EVENT_NON_DEFAULT}{"pmix.evnondef"}{bool}{
Event is not to be delivered to default event handlers.
}

%
\declareNewAttribute{PMIX_EVENT_RETURN_OBJECT}{"pmix.evobject"}{void *}{
Object to be returned whenever the registered callback function \code{cbfunc} is invoked.
The object will \emph{only} be returned to the process that registered it.
}

%
\declareNewAttribute{PMIX_EVENT_DO_NOT_CACHE}{"pmix.evnocache"}{bool}{
Instruct the PMIx server not to cache the event.
}

%
\declareNewAttribute{PMIX_EVENT_SILENT_TERMINATION}{"pmix.evsilentterm"}{bool}{
Do not generate an event when this job normally terminates.
}


%%%%%%%%%%%
\subsection{Fault tolerance attributes}
\label{api:struct:attributes:faulttolerance}

Attributes to support fault tolerance behaviors.

%
\declareNewAttribute{PMIX_EVENT_TERMINATE_SESSION}{"pmix.evterm.sess"}{bool}{
The \ac{RM} intends to terminate this session.
}

%
\declareNewAttribute{PMIX_EVENT_TERMINATE_JOB}{"pmix.evterm.job"}{bool}{
The \ac{RM} intends to terminate this job.
}

%
\declareNewAttribute{PMIX_EVENT_TERMINATE_NODE}{"pmix.evterm.node"}{bool}{
The \ac{RM} intends to terminate all processes on this node.
}

%
\declareNewAttribute{PMIX_EVENT_TERMINATE_PROC}{"pmix.evterm.proc"}{bool}{
The \ac{RM} intends to terminate just this process.
}

%
\declareNewAttribute{PMIX_EVENT_ACTION_TIMEOUT}{"pmix.evtimeout"}{int}{
The time in seconds before the \ac{RM} will execute error response.
}

%
\declareNewAttribute{PMIX_EVENT_NO_TERMINATION}{"pmix.evnoterm"}{bool}{
Indicates that the handler has satisfactorily handled the event and believes termination of the application is not required.
}

%
\declareNewAttribute{PMIX_EVENT_WANT_TERMINATION}{"pmix.evterm"}{bool}{
Indicates that the handler has determined that the application should be terminated
}


%%%%%%%%%%%
\subsection{Spawn attributes}
\label{api:struct:attributes:spawn}

Attributes used to describe \refapi{PMIx_Spawn} behavior.

%
\declareAttribute{PMIX_PERSONALITY}{"pmix.pers"}{char*}{
Name of personality to use.
}

%
\declareAttribute{PMIX_HOST}{"pmix.host"}{char*}{
Comma-delimited list of hosts to use for spawned processes.
}

%
\declareAttribute{PMIX_HOSTFILE}{"pmix.hostfile"}{char*}{
Hostfile to use for spawned processes.
}

%
\declareAttribute{PMIX_ADD_HOST}{"pmix.addhost"}{char*}{
Comma-delimited list of hosts to add to the allocation.
}

%
\declareAttribute{PMIX_ADD_HOSTFILE}{"pmix.addhostfile"}{char*}{
Hostfile listing hosts to add to existing allocation.
}

%
\declareAttribute{PMIX_PREFIX}{"pmix.prefix"}{char*}{
Prefix to use for starting spawned processes.
}

%
\declareAttribute{PMIX_WDIR}{"pmix.wdir"}{char*}{
Working directory for spawned processes.
}

%
\declareAttribute{PMIX_MAPPER}{"pmix.mapper"}{char*}{
Mapping mechanism to use for placing spawned processes.
}

%
\declareAttribute{PMIX_DISPLAY_MAP}{"pmix.dispmap"}{bool}{
Display process mapping upon spawn.
}

%
\declareAttribute{PMIX_PPR}{"pmix.ppr"}{char*}{
Number of processes to spawn on each identified resource.
}

%
\declareAttribute{PMIX_MAPBY}{"pmix.mapby"}{char*}{
Process mapping policy.
}

%
\declareAttribute{PMIX_RANKBY}{"pmix.rankby"}{char*}{
Process ranking policy.
}

%
\declareAttribute{PMIX_BINDTO}{"pmix.bindto"}{char*}{
Process binding policy.
}

%
\declareAttribute{PMIX_PRELOAD_BIN}{"pmix.preloadbin"}{bool}{
Preload binaries onto nodes.
}

%
\declareAttribute{PMIX_PRELOAD_FILES}{"pmix.preloadfiles"}{char*}{
Comma-delimited list of files to pre-position on nodes.
}

%
\declareAttribute{PMIX_NON_PMI}{"pmix.nonpmi"}{bool}{
Spawned processes will not call \refapi{PMIx_Init}.
}

%
\declareAttribute{PMIX_STDIN_TGT}{"pmix.stdin"}{uint32_t}{
Spawned process rank that is to receive \code{stdin}.
}

%
\declareNewAttribute{PMIX_FWD_STDIN}{"pmix.fwd.stdin"}{bool}{
Forward this process's \code{stdin} to the designated process.
}

%
\declareNewAttribute{PMIX_FWD_STDOUT}{"pmix.fwd.stdout"}{bool}{
Forward \code{stdout} from spawned processes to this process.
}

%
\declareNewAttribute{PMIX_FWD_STDERR}{"pmix.fwd.stderr"}{bool}{
Forward \code{stderr} from spawned processes to this process.
}

%
\declareNewAttribute{PMIX_DEBUGGER_DAEMONS}{"pmix.debugger"}{bool}{
Spawned application consists of debugger daemons.
}

%
\declareNewAttribute{PMIX_COSPAWN_APP}{"pmix.cospawn"}{bool}{
Designated application is to be spawned as a disconnected job.
Meaning that it is not part of the ``comm_world'' of the parent process.
}

%
\declareNewAttribute{PMIX_SET_SESSION_CWD}{"pmix.ssncwd"}{bool}{
Set the application's current working directory to the session working directory assigned by the \ac{RM}.
}

%
\declareNewAttribute{PMIX_TAG_OUTPUT}{"pmix.tagout"}{bool}{
Tag application output with the identity of the source process.
}

%
\declareNewAttribute{PMIX_TIMESTAMP_OUTPUT}{"pmix.tsout"}{bool}{
Timestamp output from applications.
}

%
\declareNewAttribute{PMIX_MERGE_STDERR_STDOUT}{"pmix.mergeerrout"}{bool}{
Merge \code{stdout} and \code{stderr} streams from application processes.
}

%
\declareNewAttribute{PMIX_OUTPUT_TO_FILE}{"pmix.outfile"}{char*}{
Output application output to the specified file.
}

%
\declareNewAttribute{PMIX_INDEX_ARGV}{"pmix.indxargv"}{bool}{
Mark the \code{argv} with the rank of the process.
}

%
\declareNewAttribute{PMIX_CPUS_PER_PROC}{"pmix.cpuperproc"}{uint32_t}{
Number of cpus to assign to each rank.
}

%
\declareNewAttribute{PMIX_NO_PROCS_ON_HEAD}{"pmix.nolocal"}{bool}{
Do not place processes on the head node.
}

%
\declareNewAttribute{PMIX_NO_OVERSUBSCRIBE}{"pmix.noover"}{bool}{
Do not oversubscribe the cpus.
}

%
\declareNewAttribute{PMIX_REPORT_BINDINGS}{"pmix.repbind"}{bool}{
Report bindings of the individual processes.
}

%
\declareNewAttribute{PMIX_CPU_LIST}{"pmix.cpulist"}{char*}{
List of cpus to use for this job.
}

%
\declareNewAttribute{PMIX_JOB_RECOVERABLE}{"pmix.recover"}{bool}{
Application supports recoverable operations.
}

%
\declareNewAttribute{PMIX_JOB_CONTINUOUS}{"pmix.continuous"}{bool}{
Application is continuous, all failed processes should be immediately restarted.
}

%
\declareNewAttribute{PMIX_MAX_RESTARTS}{"pmix.maxrestarts"}{uint32_t}{
Maximum number of times to restart a job.
}


%%%%%%%%%%%
\subsection{Query attributes}
\label{api:struct:attributes:query}

Attributes used to describe \refapi{PMIx_Query_info_nb} behavior.

%
\declareNewAttribute{PMIX_QUERY_NAMESPACES}{"pmix.qry.ns"}{char*}{
Request a comma-delimited list of active namespaces.
}

%
\declareNewAttribute{PMIX_QUERY_JOB_STATUS}{"pmix.qry.jst"}{pmix_status_t}{
Status of a specified, currently executing job.
}

%
\declareNewAttribute{PMIX_QUERY_QUEUE_LIST}{"pmix.qry.qlst"}{char*}{
Request a comma-delimited list of scheduler queues.
}

%
\declareNewAttribute{PMIX_QUERY_QUEUE_STATUS}{"pmix.qry.qst"}{TBD}{
Status of a specified scheduler queue.
}

%
\declareNewAttribute{PMIX_QUERY_PROC_TABLE}{"pmix.qry.ptable"}{char*}{
Input namespace of the job whose information is being requested returns (\refstruct{pmix_data_array_t}) an array of \refstruct{pmix_proc_info_t}.
}

%
\declareNewAttribute{PMIX_QUERY_LOCAL_PROC_TABLE}{"pmix.qry.lptable"}{char*}{
Input namespace of the job whose information is being requested returns (\refstruct{pmix_data_array_t}) an array of \refstruct{pmix_proc_info_t} for processes in job on same node.
}

%
\declareNewAttribute{PMIX_QUERY_AUTHORIZATIONS}{"pmix.qry.auths"}{bool}{
Return operations the PMIx tool is authorized to perform.
}

%
\declareNewAttribute{PMIX_QUERY_SPAWN_SUPPORT}{"pmix.qry.spawn"}{bool}{
Return a comma-delimited list of supported spawn attributes.
}

%
\declareNewAttribute{PMIX_QUERY_DEBUG_SUPPORT}{"pmix.qry.debug"}{bool}{
Return a comma-delimited list of supported debug attributes.
}

%
\declareNewAttribute{PMIX_QUERY_MEMORY_USAGE}{"pmix.qry.mem"}{bool}{
Return information on memory usage for the processes indicated in the qualifiers.
}

%
\declareNewAttribute{PMIX_QUERY_LOCAL_ONLY}{"pmix.qry.local"}{bool}{
Constrain the query to local information only.
}

%
\declareNewAttribute{PMIX_QUERY_REPORT_AVG}{"pmix.qry.avg"}{bool}{
Report average values.
}

%
\declareNewAttribute{PMIX_QUERY_REPORT_MINMAX}{"pmix.qry.minmax"}{bool}{
Report minimum and maximum values.
}

%
\declareNewAttribute{PMIX_QUERY_ALLOC_STATUS}{"pmix.query.alloc"}{char*}{
String identifier of the allocation whose status is being requested.
}

%
\declareNewAttribute{PMIX_TIME_REMAINING}{"pmix.time.remaining"}{char*}{
Query number of seconds (\code{uint32_t}) remaining in allocation for the specified namespace.
}


%%%%%%%%%%%
\subsection{Log attributes}
\label{api:struct:attributes:log}

Attributes used to describe \refapi{PMIx_Log_nb} behavior.

%
\declareNewAttribute{PMIX_LOG_STDERR}{"pmix.log.stderr"}{char*}{
Log string to \code{stderr}.
}

%
\declareNewAttribute{PMIX_LOG_STDOUT}{"pmix.log.stdout"}{char*}{
Log string to \code{stdout}.
}

%
\declareNewAttribute{PMIX_LOG_SYSLOG}{"pmix.log.syslog"}{char*}{
Log data to syslog.
Defaults to \code{ERROR} priority.
}

%
\declareNewAttribute{PMIX_LOG_MSG}{"pmix.log.msg"}{pmix_byte_object_t}{
Message blob to be sent somewhere.
}

%
\declareNewAttribute{PMIX_LOG_EMAIL}{"pmix.log.email"}{pmix_data_array_t}{
Log via email based on \refstruct{pmix_info_t} containing directives.
}

%
\declareNewAttribute{PMIX_LOG_EMAIL_ADDR}{"pmix.log.emaddr"}{char*}{
Comma-delimited list of email addresses that are to receive the message.
}

%
\declareNewAttribute{PMIX_LOG_EMAIL_SUBJECT}{"pmix.log.emsub"}{char*}{
Subject line for email.
}

%
\declareNewAttribute{PMIX_LOG_EMAIL_MSG}{"pmix.log.emmsg"}{char*}{
Message to be included in email.
}


%%%%%%%%%%%
\subsection{Debugger attributes}
\label{api:struct:attributes:debugger}

Attributes used to assist debuggers.

%
\declareNewAttribute{PMIX_DEBUG_STOP_ON_EXEC}{"pmix.dbg.exec"}{bool}{
Job is being spawned under debugger.
The processes are instructed to pause on start.
}

%
\declareNewAttribute{PMIX_DEBUG_STOP_IN_INIT}{"pmix.dbg.init"}{bool}{
Instruct job to stop processes during \refapi{PMIx_Init}.
}

%
\declareNewAttribute{PMIX_DEBUG_WAIT_FOR_NOTIFY}{"pmix.dbg.notify"}{bool}{
Block at desired point until receiving debugger release notification.
}

%
\declareNewAttribute{PMIX_DEBUG_JOB}{"pmix.dbg.job"}{char*}{
Namespace of the job to be debugged.
}

%
\declareNewAttribute{PMIX_DEBUG_WAITING_FOR_NOTIFY}{"pmix.dbg.waiting"}{bool}{
Job to be debugged is waiting for a release.
}


%%%%%%%%%%%
\subsection{Resource manager attributes}
\label{api:struct:attributes:rm}

Attributes used to describe the \ac{RM}.

%
\declareNewAttribute{PMIX_RM_NAME}{"pmix.rm.name"}{char*}{
String name of the \ac{RM}.
}

%
\declareNewAttribute{PMIX_RM_VERSION}{"pmix.rm.version"}{char*}{
\ac{RM} version string.
}


%%%%%%%%%%%
\subsection{Environment variable attributes}
\label{api:struct:attributes:envar}

Attributes used to adjust environment variables.

%
\declareNewAttribute{PMIX_SET_ENVAR}{"pmix.set.envar"}{char*}{
String ``\code{key=value}'' value shall be put into the environment.
}

%
\declareNewAttribute{PMIX_UNSET_ENVAR}{"pmix.unset.envar"}{char*}{
Unset the environment variable specified in the string.
}


%%%%%%%%%%%
\subsection{Job Allocation attributes}
\label{api:struct:attributes:joballoc}

Attributes used to describe the job allocation.

%
\declareNewAttribute{PMIX_ALLOC_ID}{"pmix.alloc.id"}{char*}{
Provide a string identifier for this allocation request which can later be used to query status of the request.
}

%
\declareNewAttribute{PMIX_ALLOC_NUM_NODES}{"pmix.alloc.nnodes"}{uint64_t}{
The number of nodes.
}

%
\declareNewAttribute{PMIX_ALLOC_NODE_LIST}{"pmix.alloc.nlist"}{char*}{
Regular expression of the specific nodes.
}

%
\declareNewAttribute{PMIX_ALLOC_NUM_CPUS}{"pmix.alloc.ncpus"}{uint64_t}{
Number of cpus.
}

%
\declareNewAttribute{PMIX_ALLOC_NUM_CPU_LIST}{"pmix.alloc.ncpulist"}{char*}{
Regular expression of the number of cpus for each node.
}

%
\declareNewAttribute{PMIX_ALLOC_CPU_LIST}{"pmix.alloc.cpulist"}{char*}{
Regular expression of the specific cpus indicating the cpus involved.
}

%
\declareNewAttribute{PMIX_ALLOC_MEM_SIZE}{"pmix.alloc.msize"}{float}{
Number of Megabytes.
}

%
\declareNewAttribute{PMIX_ALLOC_NETWORK}{"pmix.alloc.net"}{array}{
Array of \refstruct{pmix_info_t} describing requested network resources.
If not given as part of an \refstruct{pmix_info_t} struct that identifies the involved nodes, then the description will be applied across all nodes in the requestor's allocation.
}

%
\declareNewAttribute{PMIX_ALLOC_NETWORK_ID}{"pmix.alloc.netid"}{char*}{
Name of the network.
}

%
\declareNewAttribute{PMIX_ALLOC_BANDWIDTH}{"pmix.alloc.bw"}{float}{
Mbits/sec.
}

%
\declareNewAttribute{PMIX_ALLOC_NETWORK_QOS}{"pmix.alloc.netqos"}{char*}{
Quality of service level.
}

%
\declareNewAttribute{PMIX_ALLOC_TIME}{"pmix.alloc.time"}{uint32_t}{
Time in seconds.
}


%%%%%%%%%%%
\subsection{Job control attributes}
\label{api:struct:attributes:jobcontrol}

Attributes used to request control operations on an executing application.

%
\declareNewAttribute{PMIX_JOB_CTRL_ID}{"pmix.jctrl.id"}{char*}{
Provide a string identifier for this request.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_PAUSE}{"pmix.jctrl.pause"}{bool}{
Pause the specified processes.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_RESUME}{"pmix.jctrl.resume"}{bool}{
Resume (``un-pause'') the specified processes.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_CANCEL}{"pmix.jctrl.cancel"}{char*}{
Cancel the specified request (\code{NULL} implies cancel all requests from this requestor).
}

%
\declareNewAttribute{PMIX_JOB_CTRL_KILL}{"pmix.jctrl.kill"}{bool}{
Forcibly terminate the specified processes and cleanup.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_RESTART}{"pmix.jctrl.restart"}{char*}{
Restart the specified processes using the given checkpoint ID.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_CHECKPOINT}{"pmix.jctrl.ckpt"}{char*}{
Checkpoint the specified processes and assign the given ID to it.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_CHECKPOINT_EVENT}{"pmix.jctrl.ckptev"}{bool}{
Use event notification to trigger a process checkpoint.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_CHECKPOINT_SIGNAL}{"pmix.jctrl.ckptsig"}{int}{
Use the given signal to trigger a process checkpoint.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_CHECKPOINT_TIMEOUT}{"pmix.jctrl.ckptsig"}{int}{
Time in seconds to wait for a checkpoint to complete.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_CHECKPOINT_METHOD}{"pmix.jctrl.ckmethod"}{pmix_data_array_t}{
Array of \refstruct{pmix_info_t} declaring each method and value supported by this application.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_SIGNAL}{"pmix.jctrl.sig"}{int}{
Send given signal to specified processes.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_PROVISION}{"pmix.jctrl.pvn"}{char*}{
Regular expression identifying nodes that are to be provisioned.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_PROVISION_IMAGE}{"pmix.jctrl.pvnimg"}{char*}{
Name of the image that is to be provisioned.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_PREEMPTIBLE}{"pmix.jctrl.preempt"}{bool}{
Indicate that the job can be pre-empted.
}

%
\declareNewAttribute{PMIX_JOB_CTRL_TERMINATE}{"pmix.jctrl.term"}{bool}{
Politely terminate the specified processes.
}


%%%%%%%%%%%
\subsection{Monitoring attributes}
\label{api:struct:attributes:monitor}

Attributes used to control monitoring of an executing application.

%
\declareNewAttribute{PMIX_MONITOR_ID}{"pmix.monitor.id"}{char*}{
Provide a string identifier for this request.
}

%
\declareNewAttribute{PMIX_MONITOR_CANCEL}{"pmix.monitor.cancel"}{char*}{
Identifier to be canceled (\code{NULL} means cancel all monitoring for this process).
}

%
\declareNewAttribute{PMIX_MONITOR_APP_CONTROL}{"pmix.monitor.appctrl"}{bool}{
The application desires to control the response to a monitoring event.
}

%
\declareNewAttribute{PMIX_MONITOR_HEARTBEAT}{"pmix.monitor.mbeat"}{void}{
Register to have the PMIx server monitor the requestor for heartbeats.
}

%
\declareNewAttribute{PMIX_SEND_HEARTBEAT}{"pmix.monitor.beat"}{void}{
Send heartbeat to local PMIx server.
}

%
\declareNewAttribute{PMIX_MONITOR_HEARTBEAT_TIME}{"pmix.monitor.btime"}{uint32_t}{
Time in seconds before declaring heartbeat missed.
}

%
\declareNewAttribute{PMIX_MONITOR_HEARTBEAT_DROPS}{"pmix.monitor.bdrop"}{uint32_t}{
Number of heartbeats that can be missed before generating the event.
}

%
\declareNewAttribute{PMIX_MONITOR_FILE}{"pmix.monitor.fmon"}{char*}{
Register to monitor file for signs of life.
}

%
\declareNewAttribute{PMIX_MONITOR_FILE_SIZE}{"pmix.monitor.fsize"}{bool}{
Monitor size of given file is growing to determine if the application is running.
}

%
\declareNewAttribute{PMIX_MONITOR_FILE_ACCESS}{"pmix.monitor.faccess"}{char*}{
Monitor time since last access of given file to determine if the application is running.
}

%
\declareNewAttribute{PMIX_MONITOR_FILE_MODIFY}{"pmix.monitor.fmod"}{char*}{
Monitor time since last modified of given file to determine if the application is running.
}

%
\declareNewAttribute{PMIX_MONITOR_FILE_CHECK_TIME}{"pmix.monitor.ftime"}{uint32_t}{
Time in seconds between checking the file.
}

%
\declareNewAttribute{PMIX_MONITOR_FILE_DROPS}{"pmix.monitor.fdrop"}{uint32_t}{
Number of file checks that can be missed before generating the event.
}



%%%%%%%%%%%
\section{Callback Functions}

PMIx provides blocking and nonblocking versions of most APIs.
In the nonblocking versions, a callback is activated upon completion of the the operation.
This section describes many of those callbacks.


%%%%%%%%%%%
\subsection{Release Callback Function}
\declareapi{pmix_release_cbfunc_t}

%%%%
\summary

The \refapi{pmix_release_cbfunc_t} is used by the \refapi{pmix_modex_cbfunc_t} and \refapi{pmix_info_cbfunc_t} operations to indicate that the callback data may be reclaimed/freed by the caller.

%%%%
\format

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_release_cbfunc_t)
    (void *cbdata)
\end{codepar}
\cspecificend

\begin{arglist}
\arginout{cbdata}{Callback data passed to original API call (memory reference)}
\end{arglist}

%%%%
\descr

Since the data is ``owned'' by the host server, provide a callback function to notify the host server that we are done with the data so it can be released.


%%%%%%%%%%%
\subsection{Modex Callback Function}
\declareapi{pmix_modex_cbfunc_t}

%%%%
\summary

The \refapi{pmix_modex_cbfunc_t} is used by the \refapi{pmix_server_fencenb_fn_t} and \refapi{pmix_server_dmodex_req_fn_t} PMIx server operations to return modex \ac{BCX} data.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_modex_cbfunc_t)
    (pmix_status_t status,
     const char *data, size_t ndata,
     void *cbdata,
     pmix_release_cbfunc_t release_fn,
     void *release_cbdata)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status associated with the operation (handle)}
\argin{data}{Data to be passed (pointer)}
\argin{ndata}{size of the data (\code{size_t})}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\argin{release_fn}{Callback for releasing \argref{data} (function pointer)}
\argin{release_cbdata}{Pointer to be passed to \argref{release_fn} (memory reference)}
\end{arglist}

%%%%
\descr

A callback function that is solely used by PMIx servers, and not clients, to return modex \ac{BCX} data in response to ``fence'' and ``get'' operations.
The returned blob contains the data collected from each server participating in the operation.



%%%%%%%%%%%
\subsection{Spawn Callback Function}
\declareapi{pmix_spawn_cbfunc_t}

%%%%
\summary

The \refapi{pmix_spawn_cbfunc_t} is used on the PMIx client side by \refapi{PMIx_Spawn_nb} and on the PMIx server side by \refapi{pmix_server_spawn_fn_t}.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_spawn_cbfunc_t)
    (pmix_status_t status,
     pmix_nspace_t nspace, void *cbdata);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status associated with the operation (handle)}
\argin{nspace}{Namespace string (\refstruct{pmix_nspace_t})}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\end{arglist}


%%%%
\descr

The callback will be executed upon launch of the specified applications in \refapi{PMIx_Spawn_nb}, or upon failure to launch any of them.

The \refarg{status} of the callback will indicate whether or not the spawn succeeded.
The \refarg{nspace} of the spawned processes will be returned, along with any provided callback data.
Note that the returned \refarg{nspace} value will not be protected by the \ac{PRI} upon return from the callback function, so the receiver must copy it if it needs to be retained.


%%%%%%%%%%%
\subsection{Op Callback Function}
\declareapi{pmix_op_cbfunc_t}

%%%%
\summary

The \refapi{pmix_op_cbfunc_t} is used by operations that simply return a status.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_op_cbfunc_t)
    (pmix_status_t status, void *cbdata);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status associated with the operation (handle)}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\end{arglist}

%%%%
\descr

Used by a wide range of \ac{PMIx} API's including \refapi{PMIx_Fence_nb}, \refapi{pmix_server_client_connected_fn_t}, \refapi{PMIx_server_register_nspace}.
This callback function is used to return a status to an often nonblocking operation.


%%%%%%%%%%%
\subsection{Lookup Callback Function}
\declareapi{pmix_lookup_cbfunc_t}

%%%%
\summary

The \refapi{pmix_lookup_cbfunc_t} is used by \refapi{PMIx_Lookup_nb} to return data.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_lookup_cbfunc_t)
    (pmix_status_t status,
     pmix_pdata_t data[], size_t ndata,
     void *cbdata);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status associated with the operation (handle)}
\argin{data}{Array of data returned (\refstruct{pmix_pdata_t})}
\argin{ndata}{Number of elements in the \argref{data} array (\code{size_t})}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\end{arglist}


%%%%
\descr

A callback function for calls to \refapi{PMIx_Lookup_nb}
The function will be called upon completion of the command with the \refarg{status} indicating the success or failure of the request.
Any retrieved data will be returned in an array of \refstruct{pmix_pdata_t} structs.
The namespace and rank of the process that provided each data element is also returned.

Note that these structures will be released upon return from the callback function, so the receiver must copy/protect the data prior to returning if it needs to be retained.


%%%%%%%%%%%
\subsection{Value Callback Function}
\declareapi{pmix_value_cbfunc_t}

%%%%
\summary

The \refapi{pmix_value_cbfunc_t} is used by \refapi{PMIx_Get_nb} to return data.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_value_cbfunc_t)
    (pmix_status_t status,
     pmix_value_t *kv, void *cbdata);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status associated with the operation (handle)}
\argin{kv}{Key/value pair representing the data (\refstruct{pmix_value_t})}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\end{arglist}


%%%%
\descr

A callback function for calls to \refapi{PMIx_Get_nb}.
The \refarg{status} indicates if the requested data was found or not.
A pointer to the \refstruct{pmix_value_t} structure containing the found data is returned.
The pointer will be \code{NULL} if the requested data was not found.


%%%%%%%%%%%
\subsection{Info Callback Function}
\declareapi{pmix_info_cbfunc_t}

%%%%
\summary

The \refapi{pmix_info_cbfunc_t} is a general information callback used by various APIs.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_info_cbfunc_t)
    (pmix_status_t status,
     pmix_info_t info[], size_t ninfo,
     void *cbdata,
     pmix_release_cbfunc_t release_fn,
     void *release_cbdata);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status associated with the operation (\refattr{pmix_status_t})}
\argin{info}{Array of \refstruct{pmix_info_t} returned by the operation (pointer)}
\argin{ninfo}{Number of elements in the \argref{info} array (\code{size_t})}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\argin{release_fn}{Function to be called when done with the \argref{info} data (function pointer)}
\argin{release_cbdata}{Callback data to be passed to \argref{release_fn} (memory reference)}
\end{arglist}


%%%%
\descr

The \refarg{status} indicates if requested data was found or not.
An array of \refstruct{pmix_info_t} will contain the key/value pairs.


%%%%%%%%%%%
\subsection{Event Handler Registration Callback Function}
\declareapi{pmix_evhdlr_reg_cbfunc_t}

The \refapi{pmix_evhdlr_reg_cbfunc_t} callback function.

\adviceuserstart
The \ac{PMIx} \textit{ad hoc} v1.0 Standard defined an error handler registration callback function with a compatible signature, but with a different type definition function name (pmix_errhandler_reg_cbfunc_t). It was removed from the v2.0 Standard and is not included in this document to avoid confusion.
\adviceuserend

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_evhdlr_reg_cbfunc_t)
    (pmix_status_t status,
     size_t evhdlr_ref,
     void *cbdata)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status indicates if the request was successful or not (\refattr{pmix_status_t})}
\argin{evhdlr_ref}{Reference assigned to the event handler by \ac{PMIx} --- this reference
 * must be used to deregister the err handler (\code{size_t})}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\end{arglist}


%%%%
\descr

Define a callback function for calls to \refapi{PMIx_Register_event_handler}


%%%%%%%%%%%
\subsection{Notification Handler Completion Callback Function}
\declareapi{pmix_event_notification_cbfunc_fn_t}

%%%%
\summary

The \refapi{pmix_event_notification_cbfunc_fn_t} is called by event handlers to indicate completion of their operations.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_event_notification_cbfunc_fn_t)
    (pmix_status_t status,
     pmix_info_t *results, size_t nresults,
     pmix_op_cbfunc_t cbfunc, void *thiscbdata,
     void *notification_cbdata);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{Status returned by the event handler's operation (\refattr{pmix_status_t})}
\argin{results}{Results from this event handler's operation on the event (\refstruct{pmix_info_t})}
\argin{nresults}{Number of elements in the results array (\code{size_t})}
\argin{cbfunc}{\refapi{pmix_op_cbfunc_t} function to be executed when \ac{PMIx} completes processing the callback (function reference)}
\argin{thiscbdata}{Callback data that was passed in to the handler (memory reference)}
\argin{cbdata}{Callback data to be returned when \ac{PMIx} executes cbfunc (memory reference)}
\end{arglist}

%%%%
\descr

Define a callback by which an event handler can notify the \ac{PMIx} library that it has completed its response to the notification. The handler is \textit{required} to execute this callback so the library can determine if additional handlers need to be called. The handler shall return \refconst{PMIX_ERR_EVENT_COMPLETE} if no further action is required. The return status of each event handler and any returned \refstruct{pmix_info_t} structures will be added to the \textit{results} array of \refstruct{pmix_info_t} passed to any subsequent event handlers to help guide their operation.

If non-NULL, the provided callback function will be called to allow the event handler to release the provided info array and execute any other required cleanup operations.


%%%%%%%%%%%
\subsection{Notification Function}
\declareapi{pmix_notification_fn_t}

%%%%
\summary

The \refapi{pmix_notification_fn_t} is called by \ac{PMIx} to deliver notification of an event.

\adviceuserstart
The \ac{PMIx} \textit{ad hoc} v1.0 Standard defined an error notification function with an identical name, but different signature than the v2.0 Standard described below. The \textit{ad hoc} v1.0 version was removed from the v2.0 Standard is not included in this document to avoid confusion.
\adviceuserend


\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_notification_fn_t)
    (size_t evhdlr_registration_id,
     pmix_status_t status,
     const pmix_proc_t *source,
     pmix_info_t info[], size_t ninfo,
     pmix_info_t results[], size_t nresults,
     pmix_event_notification_cbfunc_fn_t cbfunc,
     void *cbdata);
\end{codepar}
\cspecificend

\begin{arglist}
\argin{evhdlr_registration_id}{Registration number of the handler being called (\code{size_t})}
\argin{status}{Status associated with the operation (\refattr{pmix_status_t})}
\argin{source}{Identifier of the process that generated the event (\refstruct{pmix_proc_t})}. If the source is the \ac{SMS}, then the nspace will be empty and the rank will be PMIX_RANK_UNDEF
\argin{info}{Information describing the event (\refstruct{pmix_info_t})}. This argument will be NULL if no additional information was provided by the event generator.
\argin{ninfo}{Number of elements in the info array (\code{size_t})}
\argin{results}{Aggregated results from prior event handlers servicing this event (\refstruct{pmix_info_t})}. This argument will be \code{NULL} if this is the first handler servicing the event, or if no prior handlers provided results.
\argin{nresults}{Number of elements in the results array (\code{size_t})}
\argin{cbfunc}{\refapi{pmix_event_notification_cbfunc_fn_t} callback function to be executed upon completion of the handler's operation and prior to handler return (function reference)}.
\argin{cbdata}{Callback data to be passed to cbfunc (memory reference)}
\end{arglist}

%%%%
\descr

Note that different \acp{RM} may provide differing levels of support for event notification to application processes. Thus, the \refarg{info} array may be \code{NULL} or may contain detailed information of the event. It is the responsibility of the application to parse any provided info array for defined key-values if it so desires.

\adviceuserstart
Possible uses of the \refarg{info} array include:

\begin{itemize}
\item for the host \ac{RM} to alert the process as to planned actions, such as aborting the session, in response to the reported event

\item provide a timeout for alternative action to occur, such as for the application to request an alternate response to the event
\end{itemize}

For example, the \ac{RM} might alert the application to the failure of a node that resulted in termination of several processes, and indicate that the overall session will be aborted unless the application requests an alternative behavior in the next 5 seconds. The application then has time to respond with a checkpoint request, or a request to recover from the failure by obtaining replacement nodes and restarting from some earlier checkpoint.

Support for these options is left to the discretion of the host \ac{RM}. Info keys are included in the common definitions above but may be augmented by environment vendors.
\adviceuserend

\advicermstart
On the server side, the notification function is used to inform the \ac{PMIx} server library's host of a detected event in the \ac{PMIx} server library. Events generated by \ac{PMIx} clients are communicated to the \ac{PMIx} server library, but will be relayed to the host via the \refapi{pmix_server_notify_event_fn_t} function pointer, if provided.
\advicermend

%%%%%%%%%%%
\subsection{Server Setup Application Callback Function}
\declareapi{pmix_setup_application_cbfunc_t}

The \refapi{PMIx_server_setup_application} callback function.

%%%%
\summary

Provide a function by which the resource manager can receive application-specific environmental variables and other setup data prior to launch of an application.

%%%%
\format

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_setup_application_cbfunc_t)(
                        pmix_status_t status,
                        pmix_info_t info[], size_t ninfo,
                        void *provided_cbdata,
                        pmix_op_cbfunc_t cbfunc, void *cbdata)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{returned status of the request (\refstruct{pmix_status_t})}
\argin{info}{Array of info structures (array of handles)}
\argin{ninfo}{Number of elements in the \refarg{info} array (integer)}
\argin{provided_cbdata}{Data originally passed to call to \refapi{PMIx_server_setup_application} (memory reference)}
\argin{cbfunc}{\refapi{pmix_op_cbfunc_t} function to be called when processing completed (function reference)}
\argin{cbdata}{Data to be passed to the \refarg{cbfunc} callback function (memory reference)}
\end{arglist}

\descr

Define a function to be called by the \ac{PMIx} server library for return of application-specific setup data in response to a request from the host \ac{RM}. The returned \refarg{info} array is owned by the \ac{PMIx} server library and will be free'd when the provided \refarg{cbfunc} is called.


%%%%%%%%%%%
\subsection{Server Direct Modex Response Callback Function}
\declareapi{pmix_dmodex_response_fn_t}

The \refapi{PMIx_server_dmodex_request} callback function.

%%%%
\summary

Provide a function by which the local \ac{PMIx} server library can return connection and other data posted by local application processes to the host resource manager.

%%%%
\format

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_dmodex_response_fn_t)(pmix_status_t status,
                        char *data, size_t sz,
                        void *cbdata);
\end{codepar}
\cspecificend


\begin{arglist}
\argin{status}{Returned status of the request (\refstruct{pmix_status_t})}
\argin{data}{Pointer to a data "blob" containing the requested information (handle)}
\argin{sz}{Number of bytes in the \refarg{data} blob (integer)}
\argin{cbdata}{Data passed into the initial call to \refapi{PMIx_server_dmodex_request} (memory reference)}
\end{arglist}


\descr
Define a function to be called by the PMIx server library for return of information posted by a local application process (via \refapi{PMIx_Put} with subsequent \refapi{PMIx_Commit}) in response to a request from the host RM. The returned \refarg{data} blob is owned by the PMIx server library and will be free’d upon return from the function.

%%%%%%%%%%%
\subsection{\code{pmix_connection_cbfunc_t}}
\declareapi{pmix_connection_cbfunc_t}

%%%%
\summary

Callback function for incoming connection request from a local client

%%%%
\format

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_connection_cbfunc_t)(
                             int incoming_sd, void *cbdata)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{incoming_sd}{(integer)}
\argin{cbdata}{ (memory reference)}
\end{arglist}

%%%%
\descr

Callback function for incoming connection requests from local clients - only used by host environments that wish to directly handle socket connection requests.


%%%%%%%%%%%
\subsection{\code{pmix_tool_connection_cbfunc_t}}
\declareapi{pmix_tool_connection_cbfunc_t}

%%%%
\summary

Callback function for incoming tool connections.

%%%%
\format

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
typedef void (*pmix_tool_connection_cbfunc_t)(
                             pmix_status_t status,
                             pmix_proc_t *proc, void *cbdata)
\end{codepar}
\cspecificend

\begin{arglist}
\argin{status}{\refstruct{pmix_status_t} value (handle)}
\argin{proc}{\refstruct{pmix_proc_t} structure containing the identifier assigned to the tool (handle)}
\argin{cbdata}{Data to be passed (memory reference)}
\end{arglist}

%%%%
\descr

Callback function for incoming tool connections.
The host environment shall provide a namespace/rank identifier for the connecting tool.

\advicermstart
It is assumed that \code{rank=0} will be the normal assignment, but allow for the future possibility of a parallel set of tools connecting, and thus each process requiring a unique rank.
\advicermend

%%%%%%%%%%%
\subsection{Constant String Functions}

Provide a string representation for several types of values.
Note that the provided string is statically defined and must NOT be \code{free}'d.

%%%%
\summary
\declareapi{PMIx_Error_string}

String representation of a \refstruct{pmix_status_t}.

\versionMarker{1.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Error_string(pmix_status_t status);
\end{codepar}
\cspecificend

%%%%
\summary
\declareapi{PMIx_Proc_state_string}

String representation of a \refstruct{pmix_proc_state_t}.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Proc_state_string(pmix_proc_state_t state);
\end{codepar}
\cspecificend

%%%%
\summary
\declareapi{PMIx_Scope_string}

String representation of a \refstruct{pmix_scope_t}.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Scope_string(pmix_scope_t scope);
\end{codepar}
\cspecificend

%%%%
\summary
\declareapi{PMIx_Persistence_string}

String representation of a \refstruct{pmix_persistence_t}.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Persistence_string(pmix_persistence_t persist);
\end{codepar}
\cspecificend

%%%%
\summary
\declareapi{PMIx_Data_range_string}

String representation of a \refstruct{pmix_data_range_t}.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Data_range_string(pmix_data_range_t range);
\end{codepar}
\cspecificend

%%%%
\summary
\declareapi{PMIx_Info_directives_string}

String representation of a \refstruct{pmix_info_directives_t}.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Info_directives_string(pmix_info_directives_t directives);
\end{codepar}
\cspecificend

%%%%
\summary
\declareapi{PMIx_Data_type_string}

String representation of a \refstruct{pmix_data_type_t}.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Data_type_string(pmix_data_type_t type);
\end{codepar}
\cspecificend

%%%%
\summary
\declareapi{PMIx_Alloc_directive_string}

String representation of a \refstruct{pmix_alloc_directive_t}.

\versionMarker{2.0}
\cspecificstart
\begin{codepar}
const char*
PMIx_Alloc_directive_string(pmix_alloc_directive_t directive);
\end{codepar}
\cspecificend


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%