Merge of wolk_tracker_features. (#72)

* Wolk with external data provider * Expanded data handler api; Fixed building tests * Split handling of external device data and statuses * Don't run tests when building * Don't build and install gateway app * Fixed routing of device messages * Don't build and install app if used as subproject * Newest WolkSDK-Cpp - fixing sensors & alarm messages with multiple references (#49) * Change ExternalDataService to `make_shared` * Fix setupGatewayDataService to not pass ref to invalid pointer * Link the data service to subscribe to topics for config/actuation for gateway * Added the `FileListener` class to `FileDownloadService`. (#52) * Introduced FileListener * Pass listener to service, and fix passed path * Enabled Firmware Update * FileListener control of files * Add possiblity to add file to repo * Removed fileCreationLambda * Custom protocol and persistence (#53) * Custom protocols; Publish external message from custom protocol as is * FileSystem persistence * sdk; Fixes includes and formatting * Improved use of filesystem persistence; Circular persistence * Fixed publishing * Persistence mechanism uses ordering * Check messages retreived from buffer * code review fixes * Use recursive mutex for persistence (#56) * Report update success & Store version in folder (#57) * Report successful update on return true * Save version file in separate folder * Bugfix/blocking manager start (#59) * New WolkSDK-Cpp with non-blocking log manager start * Point to master * Fix/json protocol sensor topics (#60) * New SDK with fixed topics * Fixed sensor topics * Point to master commit of WolkSDK * Fixed readings path when checking file size; (#61) Fixed building tests * Disable gw update for custom protocol (#62) * Updated WolkSDK-Cpp submodule (#63) * Bugfix/wolk destructor hanging (#64) * New WolkSDK with Buffer::stop() method * Call the stop method on buffer, and check if thread is joinable first * New WolkSDK with AWS args based on Debug/Release (#65) * Improvement/WolkSDK-Cpp with new setters and getters. (#66) * Updated WolkSDK-Cpp submodule * New WolkSDK with more getters * Updated WolkSDK (reformat) * Point to master WolkSDK-Cpp Co-authored-by: djesic <danilo.jesic@wolkabout.com> * Improvement - Use FileSystem as source of truth about files (#67) * Started implementing the FSFileRepository * Completely remove the .hash-file * Implemented the full unit tests for FSFileRepository * Use FSFileRepository and not SQLiteFileRepository * Applied the request code review changes * Change to newest WolkSDK-Cpp (optional description field in payloads) * Wolk with external data provider * Expanded data handler api; Fixed building tests * Split handling of external device data and statuses * Don't run tests when building * Don't build and install gateway app * Fixed routing of device messages * Don't build and install app if used as subproject * Change ExternalDataService to `make_shared` * Fix setupGatewayDataService to not pass ref to invalid pointer * Link the data service to subscribe to topics for config/actuation for gateway * Added the `FileListener` class to `FileDownloadService`. (#52) * Introduced FileListener * Pass listener to service, and fix passed path * Enabled Firmware Update * FileListener control of files * Add possiblity to add file to repo * Removed fileCreationLambda * Custom protocol and persistence (#53) * Custom protocols; Publish external message from custom protocol as is * FileSystem persistence * sdk; Fixes includes and formatting * Improved use of filesystem persistence; Circular persistence * Fixed publishing * Persistence mechanism uses ordering * Check messages retreived from buffer * code review fixes * Use recursive mutex for persistence (#56) * Report update success & Store version in folder (#57) * Report successful update on return true * Save version file in separate folder * Fix/json protocol sensor topics (#60) * New SDK with fixed topics * Fixed sensor topics * Point to master commit of WolkSDK * Fixed readings path when checking file size; (#61) Fixed building tests * Disable gw update for custom protocol (#62) * Improvement - Use FileSystem as source of truth about files (#67) * Started implementing the FSFileRepository * Completely remove the .hash-file * Implemented the full unit tests for FSFileRepository * Use FSFileRepository and not SQLiteFileRepository * Applied the request code review changes * Fixed enough after rebase to get it working * Changed call of `addSensorReading` in tests to reflect new protocol changes * Initialize the FSFileRepository later * Fixed the initialization sequence for FSFileRepository * Cleaned-up duplicates and made `build()` and `setupWith(External/Internal)Data()` method cleaner * Create the device repositories only in internal * Bump the version to v4.3.0 * Added the platform connection status getter and callback * Update to newest WolkSDK-Cpp * Added `POCO_INSTALL_DIR` value and reformated CMakeLists.txt * Added the include dir too * Fix files test * Added licence headers; Formatting Co-authored-by: mzlicic <milan.zlicic@wolkabout.com> Co-authored-by: nvuletic <56640944+nvuletic@users.noreply.github.com> Co-authored-by: Danilo Ješić <34022788+djesic@users.noreply.github.com> Co-authored-by: djesic <danilo.jesic@wolkabout.com>
Wolkabout · Jun 29, 2021 · 91474ca · 91474ca
1 parent d6f0fe6
commit 91474ca
Show file tree

Hide file tree

Showing 98 changed files with 3,050 additions and 720 deletions.
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -33,13 +33,24 @@ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}    -Wall -Wextra -pedantic
                                             -Werror=shadow -Werror=stack-protector                                     \
                                             -Wstrict-aliasing=2 -Wno-unused                                            \
                                             -Werror=unreachable-code                                                   \
-                                            -Wvariadic-macros                                                          \
+                                            -Wvariadic-macros -Werror=overloaded-virtual                               \
                                             -Wwrite-strings -Werror=non-virtual-dtor -Werror=return-type")
 
+
+get_directory_property(HAS_PARENT PARENT_DIRECTORY)
+if (HAS_PARENT)
+    option(BUILD_APP "" OFF)
+else ()
+    option(BUILD_APP "" ON)
+    message(STATUS "Skipping build of ${PROJECT_NAME} app")
+endif ()
+
 # library output
 set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
 set(CMAKE_PREFIX_PATH ${CMAKE_BINARY_DIR}/build)
 set(CMAKE_LIBRARY_INCLUDE_DIRECTORY ${CMAKE_PREFIX_PATH}/include)
+set(POCO_INSTALL_DIR ${CMAKE_BINARY_DIR})
+include_directories(SYSTEM ${CMAKE_BINARY_DIR}/include)
 link_directories(${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
 link_directories(${CMAKE_PREFIX_PATH}/lib)
 
@@ -52,11 +63,10 @@ set(BUILD_GTEST ON CACHE BOOL "Build the library with GTest.")
 set(BUILD_AWS_LOG_UPLOADER OFF CACHE BOOL "Build the library with AwsLogUploader.")
 option(POCO_BUILD_DATA "" ON)
 option(POCO_BUILD_NET "" ON)
+option(BUILD_AWS_LOG_UPLOADER "" OFF)
 add_subdirectory(WolkSDK-Cpp)
 
 # WolkGateway library
-include_directories("WolkSDK-Cpp/core")
-
 file(GLOB_RECURSE LIB_HEADER_FILES "src/*.h" "src/*.hpp")
 file(GLOB_RECURSE LIB_SOURCE_FILES "src/*.cpp")
 
@@ -79,35 +89,37 @@ target_include_directories(${PROJECT_NAME}Tests SYSTEM PRIVATE ${CMAKE_LIBRARY_I
 set_target_properties(${PROJECT_NAME}Tests PROPERTIES INSTALL_RPATH "$ORIGIN/lib")
 add_dependencies(${PROJECT_NAME}Tests libgtest)
 
-add_custom_target(tests ALL ${PROJECT_NAME}Tests)
+add_custom_target(tests ${PROJECT_NAME}Tests)
 add_test(gtest ${PROJECT_NAME}Tests)
 
 
 # WolkGateway executable
 file(GLOB_RECURSE BIN_HEADER_FILES "application/*.h")
 file(GLOB_RECURSE BIN_SOURCE_FILES "application/*.cpp")
 
-add_executable(${PROJECT_NAME}App ${BIN_SOURCE_FILES})
-target_link_libraries(${PROJECT_NAME}App WolkGateway)
-set_target_properties(${PROJECT_NAME}App PROPERTIES INSTALL_RPATH "$ORIGIN/lib")
+if (${BUILD_APP})
+    add_executable(${PROJECT_NAME}App ${BIN_SOURCE_FILES})
+    target_link_libraries(${PROJECT_NAME}App WolkGateway)
+    set_target_properties(${PROJECT_NAME}App PROPERTIES INSTALL_RPATH "$ORIGIN/lib")
 
-# Install target
-install(DIRECTORY ${CMAKE_LIBRARY_OUTPUT_DIRECTORY} DESTINATION /usr
-        FILES_MATCHING
-        PATTERN "*so*")
+    # Install target
+    install(DIRECTORY ${CMAKE_LIBRARY_OUTPUT_DIRECTORY} DESTINATION /usr
+            FILES_MATCHING
+            PATTERN "*so*")
 
-# Place the executable file in /usr/bin
-install(TARGETS ${PROJECT_NAME}App DESTINATION /usr/bin)
+    # Place the executable file in /usr/bin
+    install(TARGETS ${PROJECT_NAME}App DESTINATION /usr/bin)
 
-# Place the ca.crt file in /usr/bin
-install(FILES out/ca.crt DESTINATION /etc/wolkGateway
-        PERMISSIONS OWNER_WRITE OWNER_READ GROUP_WRITE GROUP_READ WORLD_READ)
+    # Place the ca.crt file in /usr/bin
+    install(FILES out/ca.crt DESTINATION /etc/wolkGateway
+            PERMISSIONS OWNER_WRITE OWNER_READ GROUP_WRITE GROUP_READ WORLD_READ)
 
-# Place the config files and service unit file.
-install(FILES out/gatewayConfiguration.json DESTINATION /etc/wolkGateway
-        PERMISSIONS OWNER_WRITE OWNER_READ GROUP_WRITE GROUP_READ WORLD_READ)
+    # Place the config files and service unit file.
+    install(FILES out/gatewayConfiguration.json DESTINATION /etc/wolkGateway
+            PERMISSIONS OWNER_WRITE OWNER_READ GROUP_WRITE GROUP_READ WORLD_READ)
 
-install(CODE "MESSAGE(\"Installed WolkGateway. Configuration files for the module are placed in /etc/wolkGateway.\")")
+    install(CODE "MESSAGE(\"Installed WolkGateway. Configuration files for the module are placed in /etc/wolkGateway.\")")
+endif ()
 
 # CMake utilities
 add_subdirectory(cmake)
diff --git a/RELEASE_NOTES.txt b/RELEASE_NOTES.txt
@@ -5,6 +5,16 @@
 
 WolkGateway bridges communication between WolkAbout IoT platform and multiple devices connected to it.
 
+**Version 4.3.0**
+    - [IMPROVEMENT] - Created WolkDefault and WolkExternal that provide different interfaces for custom subdevice data providing.
+    - [IMPROVEMENT] - Introduced the FSFileRepository to provide info about FileManagement files from the File System.
+    - [IMPROVEMENT] - Disabled the Gateway Update message sending and mechanisms as they have been found unnecessary.
+    - [IMPROVEMENT] - Imported the newest WolkSDK-Cpp that introduces various fixes and improvements.
+    - [IMPROVEMENT] - Moved the version file when firmware installing to a different location. Also, finishing the install method with `true` now reports success.
+    - [IMPROVEMENT] - Introduced the custom data and device status protocols, that allow us to define custom MQTT topics/payloads for those messages.
+    - [IMPROVEMENT] - Added the FileListener interface for FileManagementService to allow input into the File Management process and allow for hooks.
+    - [IMPROVEMENT] - Made optimizations for the application to be built as a submodule.
+
 **Version 4.2.9**
     - [BUGFIX] - Fixed the gateway update to allow us to pass if it has already been updated.
     - [BUGFIX] - Disabled the gateway update fow now, as it doesn't really have a purpose right now.

diff --git a/WolkSDK-Cpp b/WolkSDK-Cpp
diff --git a/application/Application.cpp b/application/Application.cpp
@@ -91,9 +91,9 @@ int main(int argc, char** argv)
 
     wolkabout::GatewayDevice device(gatewayConfiguration.getKey(), gatewayConfiguration.getPassword(),
                                     gatewayConfiguration.getSubdeviceManagement());
-    auto builder = wolkabout::Wolk::newBuilder(device)
-                     .gatewayHost(gatewayConfiguration.getLocalMqttUri())
-                     .platformHost(gatewayConfiguration.getPlatformMqttUri());
+    auto builder = std::move(wolkabout::Wolk::newBuilder(device)
+                               .gatewayHost(gatewayConfiguration.getLocalMqttUri())
+                               .platformHost(gatewayConfiguration.getPlatformMqttUri()));
 
     if (gatewayConfiguration.getPlatformTrustStore())
     {

diff --git a/application/Configuration.cpp b/application/Configuration.cpp
@@ -16,8 +16,8 @@
 
 #include "Configuration.h"
 
-#include "utilities/FileSystemUtils.h"
-#include "utilities/json.hpp"
+#include "core/utilities/FileSystemUtils.h"
+#include "core/utilities/json.hpp"
 
 #include <algorithm>
 #include <stdexcept>

diff --git a/application/Configuration.h b/application/Configuration.h
@@ -14,8 +14,8 @@
  * limitations under the License.
  */
 
+#include "core/model/WolkOptional.h"
 #include "model/SubdeviceManagement.h"
-#include "model/WolkOptional.h"
 
 #include <string>
 

diff --git a/debian/changelog b/debian/changelog
@@ -1,7 +1,12 @@
-wolkgateway (4.2.9) stable; urgency=medium
+wolkgateway (4.3.0) stable; urgency=medium
 
-  * Fixed the gateway update to allow us to pass if it has already been updated.
-  * Disabled the gateway update fow now, as it doesn't really have a purpose right now.
-  * Imported newest WolkSDK-Cpp that allows for optional `description` properties in JsonDto messages.
+  * Created WolkDefault and WolkExternal that provide different interfaces for custom subdevice data providing.
+  * Introduced the FSFileRepository to provide info about FileManagement files from the File System.
+  * Disabled the Gateway Update message sending and mechanisms as they have been found unnecessary.
+  * Imported the newest WolkSDK-Cpp that introduces various fixes and improvements.
+  * Moved the version file when firmware installing to a different location. Also, finishing the install method with `true` now reports success.
+  * Introduced the custom data and device status protocols, that allow us to define custom MQTT topics/payloads for those messages.
+  * Added the FileListener interface for FileManagementService to allow input into the File Management process and allow for hooks.
+  * Made optimizations for the application to be built as a submodule.
 
- -- Wolkabout ELab <elab@wolkabout.com>  Tue, 11 May 2021 00:00:00 +0100
+ -- Wolkabout ELab <elab@wolkabout.com> Fri, 14 May 2021 00:00:00 +0100
diff --git a/src/ActuatorStatusProvider.h b/src/ActuatorStatusProvider.h
@@ -17,7 +17,7 @@
 #ifndef ACTUATORSTATUSPROVIDER_H
 #define ACTUATORSTATUSPROVIDER_H
 
-#include "model/ActuatorStatus.h"
+#include "core/model/ActuatorStatus.h"
 
 #include <string>
 

diff --git a/src/ConfigurationHandler.h b/src/ConfigurationHandler.h
@@ -17,7 +17,7 @@
 #ifndef CONFIGURATIONHANDLER_H
 #define CONFIGURATIONHANDLER_H
 
-#include "model/ConfigurationItem.h"
+#include "core/model/ConfigurationItem.h"
 
 #include <vector>
 

diff --git a/src/ConfigurationProvider.h b/src/ConfigurationProvider.h
@@ -17,7 +17,7 @@
 #ifndef CONFIGURATIONPROVIDER_H
 #define CONFIGURATIONPROVIDER_H
 
-#include "model/ConfigurationItem.h"
+#include "core/model/ConfigurationItem.h"
 
 #include <vector>
 

diff --git a/src/FileHandler.cpp b/src/FileHandler.cpp
@@ -16,8 +16,8 @@
 
 #include "FileHandler.h"
 
-#include "model/BinaryData.h"
-#include "utilities/FileSystemUtils.h"
+#include "core/model/BinaryData.h"
+#include "core/utilities/FileSystemUtils.h"
 
 namespace wolkabout
 {

diff --git a/src/FileHandler.h b/src/FileHandler.h
@@ -17,7 +17,7 @@
 #ifndef FILEHANDLER_H
 #define FILEHANDLER_H
 
-#include "utilities/ByteUtils.h"
+#include "core/utilities/ByteUtils.h"
 
 #include <string>
 

diff --git a/src/FileListener.h b/src/FileListener.h
@@ -0,0 +1,72 @@
+/*
+ * Copyright 2021 WolkAbout Technology s.r.o.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef FILELISTENER_H
+#define FILELISTENER_H
+
+#include <functional>
+#include <string>
+
+namespace wolkabout
+{
+/**
+ * This class presents an interface for an object that is interested in ongoing of the `FileDownloadService`, and it
+ * will be notified of new files that are downloaded/deleted.
+ */
+class FileListener
+{
+public:
+    /**
+     * The default overridable destructor.
+     */
+    virtual ~FileListener() = default;
+
+    /**
+     * This is the method that is invoked by the service to notify of the directory that is being used to store files,
+     * and in which they are all accessible.
+     *
+     * @param absolutePath The absolute path of the directory where files are stored.
+     */
+    virtual void receiveDirectory(const std::string& absolutePath) = 0;
+
+    /**
+     * This is the method that is invoked by the service to notify that a new file has been made available to download,
+     * and we can say whether or not we want to download the file.
+     *
+     * @param fileName The name of the newly initiated file.
+     * @return Whether or not we want to download the file.
+     */
+    virtual bool chooseToDownload(const std::string& fileName) = 0;
+
+    /**
+     * This is the method that is invoked by the service to notify that a new file has been downloaded and is placed in
+     * the file system. From this point on, the service can manipulate the file further, and cause action based on that.
+     *
+     * @param fileName The name of the newly downloaded file.
+     */
+    virtual void onNewFile(const std::string& fileName) = 0;
+
+    /**
+     * This is the method that is invoked by the service to notify that a file has been deleted or purged with all the
+     * other ones off the file system. From this point on, the service can no longer count on the file existing.
+     *
+     * @param fileName The name of the removed file.
+     */
+    virtual void onRemovedFile(const std::string& fileName) = 0;
+};
+}    // namespace wolkabout
+
+#endif    // FILELISTENER_H
diff --git a/src/GatewayInboundDeviceMessageHandler.cpp b/src/GatewayInboundDeviceMessageHandler.cpp
@@ -16,10 +16,10 @@
 
 #include "GatewayInboundDeviceMessageHandler.h"
 
-#include "model/Message.h"
+#include "core/model/Message.h"
+#include "core/utilities/Logger.h"
+#include "core/utilities/StringUtils.h"
 #include "protocol/GatewayProtocol.h"
-#include "utilities/Logger.h"
-#include "utilities/StringUtils.h"
 
 #include <algorithm>
 

diff --git a/src/GatewayInboundDeviceMessageHandler.h b/src/GatewayInboundDeviceMessageHandler.h
@@ -18,7 +18,7 @@
 #define GATEWAYINBOUNDDEVICEMESSAGEHANDLER_H
 
 #include "InboundDeviceMessageHandler.h"
-#include "utilities/CommandBuffer.h"
+#include "core/utilities/CommandBuffer.h"
 
 #include <map>
 #include <memory>

diff --git a/src/GatewayInboundPlatformMessageHandler.cpp b/src/GatewayInboundPlatformMessageHandler.cpp
@@ -16,10 +16,10 @@
 
 #include "GatewayInboundPlatformMessageHandler.h"
 
-#include "model/Message.h"
-#include "protocol/Protocol.h"
-#include "utilities/Logger.h"
-#include "utilities/StringUtils.h"
+#include "core/model/Message.h"
+#include "core/protocol/Protocol.h"
+#include "core/utilities/Logger.h"
+#include "core/utilities/StringUtils.h"
 
 #include <algorithm>
 

diff --git a/src/GatewayInboundPlatformMessageHandler.h b/src/GatewayInboundPlatformMessageHandler.h
@@ -18,7 +18,7 @@
 #define GATEWAYINBOUNDPLATFORMMESSAGEHANDLER_H
 
 #include "InboundPlatformMessageHandler.h"
-#include "utilities/CommandBuffer.h"
+#include "core/utilities/CommandBuffer.h"
 
 #include <map>
 #include <memory>

diff --git a/src/InboundDeviceMessageHandler.h b/src/InboundDeviceMessageHandler.h
@@ -13,6 +13,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
+
 #ifndef INBOUNDDEVICEMESSAGEHANDLER_H
 #define INBOUNDDEVICEMESSAGEHANDLER_H
 

diff --git a/src/InboundPlatformMessageHandler.h b/src/InboundPlatformMessageHandler.h
@@ -13,6 +13,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
+
 #ifndef INBOUNDPLATFORMMESSAGEHANDLER_H
 #define INBOUNDPLATFORMMESSAGEHANDLER_H
 

diff --git a/src/OutboundRetryMessageHandler.cpp b/src/OutboundRetryMessageHandler.cpp
@@ -17,9 +17,9 @@
 #include "OutboundRetryMessageHandler.h"
 
 #include "OutboundMessageHandler.h"
-#include "model/Message.h"
-#include "utilities/Logger.h"
-#include "utilities/StringUtils.h"
+#include "core/model/Message.h"
+#include "core/utilities/Logger.h"
+#include "core/utilities/StringUtils.h"
 
 namespace
 {

diff --git a/src/OutboundRetryMessageHandler.h b/src/OutboundRetryMessageHandler.h
@@ -17,7 +17,7 @@
 #ifndef OUTBOUNDRETRYMESSAGEHANDLER_H
 #define OUTBOUNDRETRYMESSAGEHANDLER_H
 
-#include "utilities/Timer.h"
+#include "core/utilities/Timer.h"
 
 #include <atomic>
 #include <condition_variable>

diff --git a/src/RegistrationMessageRouter.cpp b/src/RegistrationMessageRouter.cpp
@@ -16,10 +16,10 @@
 
 #include "RegistrationMessageRouter.h"
 
-#include "model/Message.h"
+#include "core/model/Message.h"
+#include "core/protocol/RegistrationProtocol.h"
+#include "core/utilities/Logger.h"
 #include "protocol/GatewaySubdeviceRegistrationProtocol.h"
-#include "protocol/RegistrationProtocol.h"
-#include "utilities/Logger.h"
 
 namespace wolkabout
 {

diff --git a/src/StatusMessageRouter.cpp b/src/StatusMessageRouter.cpp
@@ -16,10 +16,10 @@
 
 #include "StatusMessageRouter.h"
 
-#include "model/Message.h"
+#include "core/model/Message.h"
+#include "core/protocol/StatusProtocol.h"
+#include "core/utilities/Logger.h"
 #include "protocol/GatewayStatusProtocol.h"
-#include "protocol/StatusProtocol.h"
-#include "utilities/Logger.h"
 
 namespace wolkabout
 {