C Overview

C++ Overview

 +------------------+      +------------------+
 |   User C++ API   |  --  |     JANA API     |
 +------------------+      +------------------+
           |
+---------------------+
|  Under the hood API |
+---------------------+

CCDB C++ library consists of separate levels (see fig. \ref{pic:cpp_api_levels}).

• C++ User API - This level is most probably what any user, who is not bound to JANA, needs. The API provides simple functionality to get constants. It automates connections number, multi-threading, data source selection, etc.

• JANA API - This level integrates CCDB and JANA framework. JANA API is now shipped with JANA framework. Architecturally it is based on Users C++ API.

• Under the hood API - This level considered to be internal and could be interesting only in terms further development of CCDB. Regular users should not use classes from this level

  Low level API is subject to change for better CCDB performance and stability. Changes are done without any remorse to any user code which that uses the API.

  That is the reason why Low level API should be kept separated and used only in CCDB higher levels.

TL;DR; User API

// In this case the generator can automatically determine 
// if it is MySQL or SQLite depending on the connection string
// Parameters:
//   connectionString
//   run
//   variation
//   time = 0   // 0 = now
// Example is for run 1500, default variation without time filtration:
CalibrationGenerator generator;
auto calib = generator.MakeCalibration("mysql://...", 1500, "default", 0))

// CCDB automatically reconnects to DB
calib->GetCalib("/constants/path"); 

// (!!!) In current implementation if CalibrationGenerator::MakeCalibration
// is used, do not not delete the connections manually (with delete)
auto calib = generator.MakeCalibration("<connection string>");
delete calib;  // <== DON'T DO THIS
calib = generator.MakeCalibration("<SAME connection string>");  // Problem. Deleted pointer. 

Connection to database

ccdb::Calibration class is responsible for holding a connection with DB.

• ccdb::MySQLCalibration and
• ccdb::SQLiteCalibration

inherit from Calibration to provide concrete implementations for MySQL and SQLite

ccdb::CalibrationGenerator can be used to automatically instantiate the right Calibration type according to connection parameter;

// Here MySQLCalibration will be created according to "mysql://..." connection string
auto calib = CalibrationGenerator::CreateCalibration("mysql://ccdb@hallddb.jlab.org",...);

Connection management

In general MySQL connection is a limited resource. That is how CCDB treats MySQL connection.

There are general ways of how one manages CCDB connection. Manually or by using CalibrationGenerator.

Managing MySQLCalibration manually

1. When the connection is really established:

// Connection is established when using CreateCalibration static method:
auto calib = CalibrationGenerator::CreateCalibration("<connection string>",...)

// Using direct constructor doesn't establish the connection .
auto calib = new MySqlConnection(...);

// 2 ways to establish the connection:
// Use Connect() method explicitly
calib->Connect("<connection string>");   // Makes database connection
calib->Reconnect();                      // Makes database connection if not exist
                                         // (Reconnect will fail if connection string is not set)

// If connection string is set but Calibration is not connected
// CCDB automatically establish the connection when it is required
auto data = calib->GetCalib(...);        // Here connection is opened/reopened

2. Suspend connection during inactivity

// Calibration class tracks the last time it retrieved data from DB. 
// GetLastActivityTime() can be used for that to check inactivity time. 
// CCDB doesn't have automatic timer. User is responsible (!) for doing like:
if (time_now - calib->GetLastActivityTime() > TIME_THRESHOLD) {   // TIME_THRESHOLD is set by user
    calib->Disconnect();   // Really closes connection from DB
}

// After you closed the connection...
auto data = calib->GetCalib(...);      // Connection is be reopened/opened

3. Disconnect from DB

calib->Disconnect();               // Close connection with DB
auto data = calib->GetCalib(...);  // Here connection will be reopened/opened
delete calib;                      // RAII closes the connection too

4. Check the connection and other handy functions

calib->IsConnected();  // To check the connection status

Managing Calibration by CalibrationGenerator

5. Using CalibrationGenerator to manage connections and their inactivity. (which automates calibration management)

// In this case the generator can automatically check inactivity of all 
// connections it spawned with MakeCalibration method. 
CalibrationGenerator generator;
auto calib = generator.MakeCalibration("<connection string>",...)

// To automatically close inactive connections user can periodically
// call UpdateInactivity() function. (CCDB doesn't utilize internal timer). 
generator.UpdateInactivity();   // Call it periodically

// If connection is closed, and data is requested after that, 
// CCDB automatically reconnects to DB
calib->GetCalib(<request>);  // <= OK, ccdb will reconnect

// (!!!) In current implementation if CalibrationGenerator::MakeCalibration
// is used, do not not delete the connections manually (with delete)
auto calib = generator.MakeCalibration("<connection string>");
delete calib;
calib = generator.MakeCalibration("<SAME connection string>");  // Problem. Deleted pointer. 

// You may control when CalibrationGenerator disconnects:
time_t GetMaxInactiveTime() const;
void SetMaxInactiveTime(time_t val);  //if 0 inactivity isn't checked by UpdateInactivity 

Connection class is made in the assumption of usage:

1. Intensive beginning, idle during event processing. In the beginning of work, analyzing software has an initialization phase. During this phase a lot of requests are made to DB. During event processing, the requests are rare or absent at all. Thus it is

    /**
     * @brief Connects to database using connection string
     *
     * Connects to database using connection string
     * the Connection String generally has form:
     * <type>://<needed information to access data>
     *
     * The examples of the Connection Strings are:
     *
     * @see MySQLCalibration
     * mysql://<username>:<password>@<mysql.address>:<port>/<database>
     *
     * @see SQLiteCalibration
     * sqlite://<path to sqlite file>
     *
     * @param connectionString the Connection String
     * @return true if connected
     */
    virtual bool Connect(std::string connectionString) = 0;


    /**
     * @brief indicates ether the connection is open or not
     *
     * @return true if  connection is open
     */
    virtual bool IsConnected() = 0;


    /**
     * @brief closes connection to data
     * Closes connection to data.
     * If underlayed @see DProvider* object is "locked"
     * (user could check this by
     *
     */
    virtual void Disconnect() = 0;
    

    /**
     * @brief Connects to database using the connection string 
     *        of the last @see Connect function call
     *
     * @remark Just returns true if already connected
     * @exception logic_error if Connect hasn't been called before
     * 
     * @return true if connected
     */
virtual bool Reconnect();

/** @brief gets connection string which is used for current provider */
virtual string GetConnectionString() const;

Performance profiling

CCDB may be build with performance profiling flag. Then, the outputs of running CCDB may be analyzed with $CCDB_HOME/python/ccdb_cpp_perf.py

It allows to evaluate performance and befaviour of C++ CCDB API on working applications

One has to compile CCDB C++ API with with-perflog=true (it defines CCDB_PERFLOG_ON in C++):

> cd $CCDB_HOME
> scons with-perflog=true

With this flag, whenever any constant is requested, CCDB spams performance info to std::cout. Example of one record:

CCDB_PERF_LOG:{"thread_id":139771280078592,"descr":"Calibration::GetAssignment=>/PHOTON_BEAM/endpoint_energy","start_stamp":1498647171590502,"elapsed":1558,"t_units":"us"}

One needs to save all the outputs to a file and provide a path to the file to ccdb_cpp_perf.py

So after CCDB is compiled using with-perflog=true, the commands are:

> <analysing_soft> ... > ccdb_perf.log
> python $CCDB_HOME/python/ccdb_cpp_perf.py ccdb_cpp_perf

Troubleshouting

If you don't see CCDB_PERF_LOG:... messages and you are sure that ccdb is called at all, check with 'ldd' that the right copy of ccdb.so is loaded.