Skip to content
This repository has been archived by the owner on Jun 8, 2022. It is now read-only.

Commit

Permalink
Refactor development (#14)
Browse files Browse the repository at this point in the history
* Refactor out more code, add prepared statements to reduce repeated code, some code cleanup, develop unit tests more
* Updated documentation, feedback and cleaned up some layout
* More unit testing and a testing infrastructure.
* Updated to Catch2 for testing
* Splitting out the attribute cache into its own class.
* Updated logging system and cleaned up some code
* Use Uuid generator correctly
* Remove unused spectrum dim variables
* Version information updated
* New config parameter "store_diag_time"
  • Loading branch information
chedburgh authored Jul 16, 2018
1 parent 0af5f67 commit 59c68dd
Show file tree
Hide file tree
Showing 35 changed files with 13,473 additions and 10,856 deletions.
1 change: 0 additions & 1 deletion .libhdbpp
Submodule .libhdbpp deleted from a0734d
13 changes: 11 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,24 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

## [Unreleased]

## [0.12.0] - 2018-07-11

### Added

* Prepared statements implemented along with a string cache for the queries. This will speed up many repeated database calls.
* Prepared statements implemented along with a string cache for the queries.
* Added as a new class, PreparedStatementsCache + Unit Tests.
* Tango Events are now bound in a separate templated class to improve maintainability and reduce repeated code.
* More unit tests and a mean to test database based calls via a simple ad hoc connection to a cassandra database. This infrastructure can open a connection to a cassandra cluster and prime the database ready for use with the create_hdb_cassandra.cql file.
* More unit tests and a means to test database based calls via a simple ad hoc connection.
* Includes a DbCommands class and a CassandraConnection class.
* AttributeCache class to separate out attribute caching functionality.
* Also removes code from main class.
* New parameters, store_diag_time, added to interface.

### Changed

* Lots more cleaning and documenting the code, such as todos now showing up in the doxygen documents.
* Unit test binary now located under the build directory.
* Library binary now located under the build directory

## [0.11.0] - 2018-01-05

Expand Down
14 changes: 11 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,12 @@ option(HDBPP_CASS_BUILD_STATIC "Build static library" OFF)
option(HDBPP_CASS_BUILD_TESTS "Build unit tests" OFF)
option(HDBPP_CASS_INSTALL_SCRIPTS "Install scripts and utility files on install" OFF)
option(HDBPP_CASS_DEV_INSTALL "Install development files" OFF)
option(HDBPP_CASS_ENABLE_TRACING "Enable tracing on all function calls" OFF)

if(HDBPP_CASS_ENABLE_TRACING)
message(STATUS "Enabled function tracing for libhdbpp-cassandra")
endif(HDBPP_CASS_ENABLE_TRACING)

# Debug the flags
if(HDBPP_CASS_DEV_INSTALL)
message(STATUS "Development install requested")
endif(HDBPP_CASS_DEV_INSTALL)
Expand Down Expand Up @@ -142,8 +146,12 @@ add_library(object_library OBJECT ${SRC_FILES})
target_compile_options(object_library PRIVATE -fPIC)
set_property(TARGET ${OBJECT_LIBRARY} PROPERTY POSITION_INDEPENDENT_CODE 1)

target_compile_options(object_library
PUBLIC -std=c++11 -Wall -Wextra)
target_compile_options(object_library PUBLIC -std=c++11 -Wall -Wextra)
target_compile_definitions(object_library PUBLIC -DDEBUG_ENABLED)

if(HDBPP_CASS_ENABLE_TRACING)
target_compile_definitions(object_library PUBLIC -DTRACE_ENABLED)
endif(HDBPP_CASS_ENABLE_TRACING)

target_include_directories(object_library
PUBLIC ${HDBPP_CASS_INCLUDE_PATHS}
Expand Down
10 changes: 5 additions & 5 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
- [Building and Installation](#building-and-installation)
- [Dependencies](#dependencies)
- [Datastax CPP Driver](#datastax-cpp-driver)
- [Debian Stetch Packages](#debian-stetch-packages)
- [Debian Stretch Packages](#debian-stretch-packages)
- [Libhdbpp Debian Package](#libhdbpp-debian-package)
- [Datastax CPP Driver Release 2.2.1 Debian Package](#datastax-cpp-driver-release-221-debian-package)
- [Build flags](#build-flags)
Expand Down Expand Up @@ -45,7 +45,7 @@ Libhdbpp-Cassandra was developed on a debian system, so both libuv and the Datas
cmake -DLIBUV_INCLUDE_DIR=/my/custom/include -DLIBUV_LIBRARY=/my/custom/lib ..
```

### Debian Stetch Packages
### Debian Stretch Packages

#### Libhdbpp Debian Package

Expand All @@ -65,8 +65,8 @@ The build system is CMake therefore standard CMake flags can be used to influenc
| HDBPP_CASS_BUILD_STATIC | OFF | Build the static library. This will also be installed if make install is run. |
| HDBPP_CASS_BUILD_TESTS | OFF | Build unit tests |
| HDBPP_CASS_DEV_INSTALL | OFF | Install development files and libraries |
| HDBPP_CASS_INSTALL_SCRIPTS | OFF | Install cql script to /usr/share/libdhb++cassandra |
| HDBPP_CASS_ADDITIONAL_LIBS | | Cludge to allow additional libraries to be linked against the shared library |
| HDBPP_CASS_INSTALL_SCRIPTS | OFF | Install cql scripts |
| HDBPP_CASS_ENABLE_TRACING | OFF | Enabled function tracing debug |

The following is a list of common useful CMake flags and their use:

Expand Down Expand Up @@ -179,4 +179,4 @@ After the build has completed, simply run:
make install
```

On installation the cql scripts under etc/ will be placed in /usr/shared/libhdb++cassandra/
On installation the cql scripts under etc/ will be placed in <CMAKE_INSTALL_PREFIX>/shared/libhdb++cassandra/
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

## Version

The current release version is 0.11.0.
The current release version is 0.12.0.

### **Important Changes** 0.9.1 -> 0.10.0

Expand Down Expand Up @@ -77,8 +77,7 @@ The compatibility matrix with dependencies is as follows:
| Libhdbpp-Cassandra Version | Libray Soname | Cassandra Version | Datastax C++ Driver | Libuv |
|----------------------------|---------------|-------------------|---------------------|-------|
| 0.11.0 | 7.1.0 | 2.2.11 | 2.2.1 | 1.4.2 |
| 0.10.0 | 7.0.0 | 2.2.11 | 2.2.1 | 1.4.2 |
| 0.10.0 | 7.0.0 | 2.2.9 | 2.2.1 | 1.4.2 |
| 0.10.0 | 7.0.0 | 2.2.9, 2.2.11 | 2.2.1 | 1.4.2 |

Soname version mapping to libhdbpp.so:

Expand Down
4 changes: 2 additions & 2 deletions cmake/ReleaseVersion.cmake
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Project version
set(LIBHDBPP_CASSANDRA_VERSION_MAJOR "0")
set(LIBHDBPP_CASSANDRA_VERSION_MINOR "11")
set(LIBHDBPP_CASSANDRA_VERSION_MINOR "12")
set(LIBHDBPP_CASSANDRA_VERSION_REVISION "0")
string(TIMESTAMP LIBHDBPP_CASSANDRA_TIMESTAMP "%Y-%m-%d %H:%M:%S")

# Version the shared library
set(LIBRARY_VERSION_MAJOR 7)
set(LIBRARY_VERSION_MINOR 1)
set(LIBRARY_VERSION_MINOR 2)
set(LIBRARY_VERSION_PATCH 0)

# Create the soname string
Expand Down
2 changes: 1 addition & 1 deletion debian/changelog
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
libhdb++cassandra (0.11.0-1) unstable; urgency=low
libhdb++cassandra (0.12.0-1) unstable; urgency=low

* Initial release

Expand Down
4 changes: 2 additions & 2 deletions debian/copyright
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ Upstream-Name: libhdbpp-cassandra
Source: https://github.com/tango-controls-hdbpp/libhdbpp-cassandra

Files: *
Copyright: 2014-2017 European Synchrotron Radiation Facility,
Copyright: 2014-2018 European Synchrotron Radiation Facility,
BP 220, Grenoble 38043, FRANCE
License: LGPL-3.0+

Files: debian/*
Copyright: 2014-2017 European Synchrotron Radiation Facility,
Copyright: 2014-2018 European Synchrotron Radiation Facility,
BP 220, Grenoble 38043, FRANCE
License: LGPL-3.0+

Expand Down
7 changes: 3 additions & 4 deletions etc/create_hdb_cassandra.cql
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
-- Create hdb keyspace
-- Please adapt the replication factor (3 by default here) to your use case
CREATE KEYSPACE IF NOT EXISTS hdb WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'DC1' : 3 };
-- Please adapt the replication factor (1 by default here) to your use case
CREATE KEYSPACE IF NOT EXISTS hdb WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'DC1' : 1 };

USE hdb;

Expand Down Expand Up @@ -33,7 +33,6 @@ PRIMARY KEY(att_conf_id, time, time_us)
)
WITH comment='Attribute Configuration Events History Table';


CREATE TABLE IF NOT EXISTS att_scalar_devboolean_ro (
att_conf_id timeuuid,
period text,
Expand Down Expand Up @@ -968,4 +967,4 @@ insert_time timestamp,
insert_time_us int,
PRIMARY KEY ((att_conf_id ,period),data_time,data_time_us)
)
WITH comment='Time Statistics Table';
WITH comment='Time Statistics Table';
206 changes: 206 additions & 0 deletions src/AttributeCache.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
/* Copyright (C) : 2014-2017
European Synchrotron Radiation Radiation Facility
BP 220, Grenoble 38043, FRANCE
This file is part of libhdb++cassandra.
libhdb++cassandra is free software: you can redistribute it and/or modify
it under the terms of the Lesser GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
libhdb++cassandra is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the Lesser
GNU General Public License for more details.
You should have received a copy of the Lesser GNU General Public License
along with libhdb++cassandra. If not, see <http://www.gnu.org/licenses/>. */

#include "AttributeCache.h"
#include "LibHdb++Defines.h"
#include "Log.h"
#include <tango.h>

using namespace std;
using namespace Utils;

namespace HDBPP
{
//=============================================================================
//=============================================================================
CassUuid AttributeCache::find_attr_uuid(const AttributeName &attr_name)
{
TRACE_LOGGER;
LOG(Debug) << "Requesting uuid for attr: " << attr_name.fully_qualified_attribute_name() << endl;

// First look into the cached attribute
if(_last_lookup_params != nullptr && attr_name.fully_qualified_attribute_name() == _last_lookup_name)
return _last_lookup_params->_uuid;

auto result = _attribute_cache.find(attr_name.fully_qualified_attribute_name());

if (result == _attribute_cache.end())
{
stringstream error_desc;
error_desc << "Error: no cached uuid for attribute: " << attr_name.fully_qualified_attribute_name() << ends;
LOG(Error) << error_desc.str() << endl;
Tango::Except::throw_exception(EXCEPTION_ATTR_CACHE, error_desc.str().c_str(), __func__);
}

// cache this lookup
_last_lookup_params = &((*result).second);
_last_lookup_name = attr_name.fully_qualified_attribute_name();

//return the uuid
return (*result).second._uuid;
}

//=============================================================================
//=============================================================================
unsigned int AttributeCache::find_attr_ttl(const AttributeName &attr_name)
{
TRACE_LOGGER;
LOG(Debug) << "Requesting ttl for attr: " << attr_name.fully_qualified_attribute_name() << endl;

// First look into the cached attribute
if(_last_lookup_params != nullptr && attr_name.fully_qualified_attribute_name() == _last_lookup_name)
return _last_lookup_params->_ttl;

auto result = _attribute_cache.find(attr_name.fully_qualified_attribute_name());

if (result == _attribute_cache.end())
{
stringstream error_desc;
error_desc << "Error: no cached ttl for attribute: " << attr_name.fully_qualified_attribute_name() << ends;
LOG(Error) << error_desc.str() << endl;
Tango::Except::throw_exception(EXCEPTION_ATTR_CACHE, error_desc.str().c_str(), __func__);
}

// cache this lookup
_last_lookup_params = &((*result).second);
_last_lookup_name = attr_name.fully_qualified_attribute_name();

// return the ttl
return (*result).second._ttl;
}

//=============================================================================
//=============================================================================
pair<CassUuid, unsigned int> AttributeCache::find_attr_id_and_ttl(const AttributeName &attr_name)
{
TRACE_LOGGER;
LOG(Debug) << "Requesting id and ttl for attr: " << attr_name.fully_qualified_attribute_name() << endl;

// First look into the cached attribute
if(_last_lookup_params != nullptr && attr_name.fully_qualified_attribute_name() == _last_lookup_name)
return pair<CassUuid, unsigned int>(_last_lookup_params->_uuid, _last_lookup_params->_ttl);

auto result = _attribute_cache.find(attr_name.fully_qualified_attribute_name());

if (result == _attribute_cache.end())
{
stringstream error_desc;
error_desc << "Error: no cached ttl for attribute: " << attr_name.fully_qualified_attribute_name() << ends;
LOG(Error) << error_desc.str() << endl;
Tango::Except::throw_exception(EXCEPTION_ATTR_CACHE, error_desc.str().c_str(), __func__);
}

// cache this lookup
_last_lookup_params = &((*result).second);
_last_lookup_name = attr_name.fully_qualified_attribute_name();

// return the ttl
return pair<CassUuid, unsigned int>((*result).second._uuid, (*result).second._ttl);
}

//=============================================================================
//=============================================================================
void AttributeCache::update_attr_ttl(const AttributeName &attr_name, unsigned int new_ttl)
{
TRACE_LOGGER;
LOG(Debug) << "Updating ttl for attr: " << attr_name.fully_qualified_attribute_name() << endl;

// First look into the cached attribute
if(_last_lookup_params != nullptr && attr_name.fully_qualified_attribute_name() == _last_lookup_name)
_last_lookup_params->_ttl = new_ttl;

auto result = _attribute_cache.find(attr_name.fully_qualified_attribute_name());

if (result == _attribute_cache.end())
{
stringstream error_desc;
error_desc << "Error: attribute is not cached: " << attr_name.fully_qualified_attribute_name() << ends;
LOG(Error) << error_desc.str() << endl;
Tango::Except::throw_exception(EXCEPTION_ATTR_CACHE, error_desc.str().c_str(), __func__);
}

// cache this lookup
_last_lookup_params = &((*result).second);
_last_lookup_name = attr_name.fully_qualified_attribute_name();

LOG(Debug) << "Updated attribute: " << attr_name << " ttl from: " << (*result).second._ttl
<< " to: " << new_ttl << endl;

// update the ttl
(*result).second._ttl = new_ttl;
}

//=============================================================================
//=============================================================================
void AttributeCache::cache_attribute(const AttributeName &attr_name, const CassUuid &uuid, unsigned int ttl)
{
TRACE_LOGGER;

if (_attribute_cache.find(attr_name.fully_qualified_attribute_name()) != _attribute_cache.end())
{
stringstream error_desc;
error_desc << "Error: attribute is already cached: " << attr_name.fully_qualified_attribute_name() << ends;
LOG(Error) << error_desc.str() << endl;
Tango::Except::throw_exception(EXCEPTION_ATTR_CACHE, error_desc.str().c_str(), __func__);
}

_attribute_cache.insert(
make_pair(attr_name.fully_qualified_attribute_name(), AttributeParams(uuid, ttl)));

auto result = _attribute_cache.find(attr_name.fully_qualified_attribute_name());

// cache this attribute for quick lookup
_last_lookup_params = &((*result).second);
_last_lookup_name = attr_name.fully_qualified_attribute_name();

char uuid_str[CASS_UUID_STRING_LENGTH];
cass_uuid_string(uuid, uuid_str);

LOG(Debug) << "Cached attribute: " << attr_name << " to cache with uuid: " << uuid_str
<< " and ttl: " << ttl << endl;
}

//=============================================================================
//=============================================================================
bool AttributeCache::cached(const AttributeName &attr_name)
{
TRACE_LOGGER;
auto result = _attribute_cache.find(attr_name.fully_qualified_attribute_name());

if (result == _attribute_cache.end())
return false;

// cache this attribute for quick lookup
_last_lookup_params = &((*result).second);
_last_lookup_name = attr_name.fully_qualified_attribute_name();

return true;
}

//=============================================================================
//=============================================================================
ostream &operator<<(ostream &os, const AttributeCache &attr_cache)
{
os << "cache size: " << attr_cache.cache_size() << " "
<< "_last_lookup_params set: "
<< (attr_cache._last_lookup_params == nullptr ? "false" : attr_cache._last_lookup_name);

return os;
}
}
Loading

0 comments on commit 59c68dd

Please sign in to comment.