Mqtt broker adapter

CMakeLists.txt

@@ -1,9 +1,6 @@
 # The version number.
-set(AGENT_VERSION_MAJOR 2)
-set(AGENT_VERSION_MINOR 8)
-set(AGENT_VERSION_PATCH 0)
-set(AGENT_VERSION_BUILD 4)
-set(AGENT_VERSION_RC "")
+include(cmake/get_version_number.cmake)
+
 
 # This minimum version is to support Visual Studio 2019 and C++ feature checking and FetchContent
 cmake_minimum_required(VERSION 3.23 FATAL_ERROR)
@@ -28,7 +25,9 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS OFF)
 set(CXX_COMPILE_FEATURES cxx_std_20)
 
-project(cppagent LANGUAGES C CXX)
+project(cppagent
+  VERSION ${AGENT_VERSION_MAJOR}.${AGENT_VERSION_MINOR}.${AGENT_VERSION_PATCH}.${AGENT_VERSION_BUILD}
+  LANGUAGES C CXX)
 
 set(WITH_RUBY ON CACHE STRING "With Ruby Support")
 

README.md

@@ -270,6 +270,44 @@ MqttService {
 
 Additional message mapping rules may be needed depending on your topic structure and payload format.
 
+### Embedded MQTT Broker Adapter
+
+The agent can also act as the MQTT broker itself (`Protocol = mqtt_broker`). Devices connect and publish directly to the agent; every publication is forwarded to any other subscribers, and publications matching `Topics` are ingested with the same topic and payload mapping as the MQTT client adapter (`Protocol = mqtt`).
+
+```
+Adapters {
+  Cell {
+    Protocol = mqtt_broker
+    Device = Mill
+    Topics = Mill/#
+    MqttHost = 0.0.0.0          # bind address, default 0.0.0.0
+    MqttPort = 8883             # default 1883, or 8883 with TLS
+    MqttTls = true
+    TlsCertificateChain = server.crt
+    TlsPrivateKey = server.key
+    MqttUserName = device
+    MqttPassword = secret
+  }
+}
+```
+
+* Listener, transport, and credential settings (`MqttHost`, `MqttPort`, `MqttTls`, `MqttWs`, `MqttUserName`, `MqttPassword`, `MqttBrokerSecurity`) are only read from the adapter block, so they do not conflict with the MQTT sink settings. TLS certificate settings fall back to the top level values.
+* `TlsVerifyClientCertificate = true` with `TlsClientCAs` rejects TLS clients without a valid client certificate. `TlsCertificatePassword` and `TlsDHKey` are optional.
+* `MqttBrokerSecurity` is the path to an mqtt_cpp broker security JSON file for multiple users, salted `sha256` passwords, groups, and per-topic publish/subscribe ACLs. It replaces `MqttUserName`/`MqttPassword`. For example:
+
+  ```
+  {
+    "authentication": [
+      { "name": "device", "method": "sha256", "salt": "abc", "digest": "<sha256(salt + password) in hex>" },
+      { "name": "dashboard", "method": "plain_password", "password": "viewer" }
+    ],
+    "authorization": [
+      { "topic": "Mill/#", "allow": { "pub": ["device"], "sub": ["device", "dashboard"] } }
+    ]
+  }
+  ```
+* With neither credentials nor a security file, clients connect anonymously and a warning is logged.
+
 ---
 
 ## Ruby Extensions
@@ -405,6 +443,12 @@ This section provides an overview of all the necessary configurable parameters.
 
   _Default_: 1000
 
+* `DisconnectRemovesAllAssets` - When an adapter disconnects, remove all assets for the
+  device it is the primary adapter for (the adapter's `Device`). Assets of devices listed in
+  `AdditionalDevices` are not removed. This can be overridden on a per adapter basis.
+
+  _Default_: false
+
 * `IgnoreTimestamps` - Overwrite timestamps with the agent time. This will correct
   clock drift but will not give as accurate relative time since it will not take into
   consideration network latencies. This can be overridden on a per adapter basis.

SECURITY.md

@@ -0,0 +1,43 @@
+# Security Policy
+
+## Supported Versions
+
+Security fixes are made for every 2.x release line of the MTConnect C++ Agent, from 2.0 on, when the line contains the affected code. A fix ships as a new patch release of each affected line. If you run a 1.x version, upgrade to a 2.x release to get them.
+
+| Version | Supported |
+| ------- | --------- |
+| 2.x     | Yes       |
+| < 2.0   | No        |
+
+## Reporting a Vulnerability
+
+Please do not report security vulnerabilities in public issues, pull requests or discussions.
+
+Report them privately through GitHub instead:
+
+1. Go to the [Security tab](https://github.com/mtconnect/cppagent/security) of this repository.
+2. Select **Report a vulnerability**.
+3. Fill in the form.
+
+Please include:
+
+- The agent version, and whether you built it yourself or used a published binary or Docker image.
+- The relevant parts of your `agent.cfg`, such as `BrowserView`, `AllowPut`, the sinks and the sources.
+- Steps to reproduce the problem, with a request, adapter data or file that shows it if you have one.
+- What an attacker can do with it, and what they need first (for example network access to the agent, `AllowPut`, or control of an adapter).
+- Any mitigation you have found.
+
+## What to Expect
+
+We will acknowledge your report, look into it, and keep you updated through the advisory on GitHub. When a fix is ready we will publish a release and a security advisory. We are glad to credit you in the advisory; tell us if you would rather not be named.
+
+## Scope
+
+The agent is usually deployed on a plant network and often has no authentication in front of it. Reports are in scope when they affect the agent's own code, for example:
+
+- The REST sink (HTTP and WebSocket), including the browser viewer in `styles/`.
+- The MQTT sinks and sources.
+- Parsing of adapter (SHDR and MQTT) data, assets, `Devices.xml` and `agent.cfg`.
+- The mruby scripting integration.
+
+Problems in a dependency (Boost, libxml2, mruby, and others) should be reported to that project. Let us know as well if the agent needs to update the dependency or work around it.

agent_lib/CMakeLists.txt

@@ -238,6 +238,7 @@ set(AGENT_SOURCES
         "${SOURCE_DIR}/source/adapter/agent_adapter/session.hpp"
         "${SOURCE_DIR}/source/adapter/agent_adapter/session_impl.hpp"
         "${SOURCE_DIR}/source/adapter/mqtt/mqtt_adapter.hpp"
+        "${SOURCE_DIR}/source/adapter/mqtt/mqtt_broker_adapter.hpp"
         "${SOURCE_DIR}/source/adapter/shdr/connector.hpp"
         "${SOURCE_DIR}/source/adapter/shdr/shdr_adapter.hpp"
         "${SOURCE_DIR}/source/adapter/shdr/shdr_pipeline.hpp"
@@ -249,6 +250,7 @@ set(AGENT_SOURCES
 
         "${SOURCE_DIR}/source/adapter/adapter_pipeline.cpp"
         "${SOURCE_DIR}/source/adapter/mqtt/mqtt_adapter.cpp"
+        "${SOURCE_DIR}/source/adapter/mqtt/mqtt_broker_adapter.cpp"
         "${SOURCE_DIR}/source/adapter/shdr/connector.cpp"
         "${SOURCE_DIR}/source/adapter/shdr/shdr_adapter.cpp"
         "${SOURCE_DIR}/source/adapter/shdr/shdr_pipeline.cpp"
@@ -368,6 +370,7 @@ if(MSVC)
     "${SOURCE_DIR}/sink/mqtt_entity_sink/mqtt_entity_sink.cpp"
     "${SOURCE_DIR}/sink/rest_sink/session_impl.cpp"
     "${SOURCE_DIR}/source/adapter/mqtt/mqtt_adapter.cpp"
+    "${SOURCE_DIR}/source/adapter/mqtt/mqtt_broker_adapter.cpp"
     "${SOURCE_DIR}/source/adapter/agent_adapter/agent_adapter.cpp"
     "${SOURCE_DIR}/source/adapter/shdr/shdr_pipeline.cpp"
     "${SOURCE_DIR}/source/adapter/adapter_pipeline.cpp"

agent_version.txt

@@ -0,0 +1 @@
+2.8.0.5

cmake/get_version_number.cmake

@@ -0,0 +1,22 @@
+set(AGENT_VERSION_FILE "${CMAKE_CURRENT_SOURCE_DIR}/agent_version.txt")
+
+# Read the first line and trim whitespace / stray \r from CRLF files
+file(STRINGS "${AGENT_VERSION_FILE}" _agent_version_line LIMIT_COUNT 1)
+string(STRIP "${_agent_version_line}" _agent_version_line)
+
+# MAJOR.MINOR.PATCH.BUILD with an optional RC suffix: 2.8.0.4, 2.8.0.4-RC1, 2.8.0.4_RC1, 2.8.0.4RC1
+if(NOT _agent_version_line MATCHES
+   "^([0-9]+)\\.([0-9]+)\\.([0-9]+)\\.([0-9]+)([-_]?(RC[0-9]+))?$")
+  message(FATAL_ERROR
+    "Invalid version '${_agent_version_line}' in ${AGENT_VERSION_FILE}; "
+    "expected MAJOR.MINOR.PATCH.BUILD[-RCn]")
+endif()
+
+set(AGENT_VERSION_MAJOR ${CMAKE_MATCH_1})
+set(AGENT_VERSION_MINOR ${CMAKE_MATCH_2})
+set(AGENT_VERSION_PATCH ${CMAKE_MATCH_3})
+set(AGENT_VERSION_BUILD ${CMAKE_MATCH_4})
+set(AGENT_VERSION_RC    "${CMAKE_MATCH_6}")   # "" when there's no RC
+
+# Re-run configure automatically when the version file changes
+set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS "${AGENT_VERSION_FILE}")

conanfile.py

@@ -1,15 +1,14 @@
 from conan import ConanFile
 from conan.tools.microsoft import is_msvc, is_msvc_static_runtime
 from conan.tools.cmake import CMake, CMakeToolchain, CMakeDeps, cmake_layout
-from conan.tools.files import copy
+from conan.tools.files import copy, load
 
 import os
 import io
 import re
 
 class MTConnectAgentConan(ConanFile):
     name = "mtconnect_agent"
-    version = "2.7"
     url = "https://github.com/mtconnect/cppagent.git"
     license = "Apache License 2.0"
     settings = "os", "compiler", "arch", "build_type"
@@ -27,7 +26,6 @@ class MTConnectAgentConan(ConanFile):
                  "cpack_name": [None, "ANY"],
                  "cpack_generator": [None, "ANY"]
                  }
-
     description = "MTConnect reference C++ agent copyright Association for Manufacturing Technology"
 
     build_policy = "missing"
@@ -74,7 +72,7 @@ class MTConnectAgentConan(ConanFile):
         }
 
     exports_sources = "*", "!build", "!test_package/build", "!*~"
-    exports = "conan/mqtt_cpp/*", "conan/mruby/*", "conan/oniguruma/*"
+    exports = "conan/mqtt_cpp/*", "conan/mruby/*", "conan/oniguruma/*", "agent_version.txt"
 
     def validate(self):
         if is_msvc(self) and self.options.shared and self.settings.compiler.runtime != 'dynamic':
@@ -90,6 +88,11 @@ def config_options(self):
         if is_msvc(self):
             self.options.rm_safe("fPIC")
 
+    def set_version(self):
+        if not self.version:
+            self.version = load(self, os.path.join(self.recipe_folder, "agent_version.txt")).splitlines()[0].strip()
+            print(f"Building MTConnect Version {self.version}")
+
     def tool_requires_version(self, package, version):
         self.output.info(f"Checking version of {package} > {version}")
         buf = io.StringIO()

demo/styles/viewer.html

@@ -18,6 +18,7 @@
 <html lang="en">
   <head>
     <meta charset="utf-8">
+    <meta http-equiv="Content-Security-Policy" content="default-src 'self'">
     <meta name="viewport" content="width=device-width, initial-scale=1">
     <title>MTConnect Agent</title>
     <link href="/styles/bootstrap.min.css" rel="stylesheet">

src/mtconnect/agent.cpp

@@ -697,6 +697,23 @@ namespace mtconnect {
           LOG(warning) << "Free data item found in device data items";
       }
     }
+
+    // Remove the assets of the device this adapter is the primary adapter for
+    auto source = std::dynamic_pointer_cast<source::adapter::Adapter>(findSource(adapter));
+    if (source && IsOptionSet(source->getOptions(), config::DisconnectRemovesAllAssets))
+    {
+      auto name = GetOption<string>(source->getOptions(), config::Device);
+      DevicePtr device;
+      if (name)
+        device = findDeviceByUUIDorName(*name);
+      if (device && device->getUuid())
+      {
+        asset::AssetList list;
+        removeAllAssets(*device->getUuid(), nullopt, nullopt, list);
+        LOG(info) << "Adapter " << adapter << " disconnected, removed " << list.size()
+                  << " assets for device " << *device->getComponentName();
+      }
+    }
   }
 
   void Agent::connected(const std::string& adapter, const StringList& devices, bool autoAvailable)

src/mtconnect/configuration/agent_config.cpp

@@ -65,6 +65,7 @@
 #include "mtconnect/sink/rest_sink/rest_service.hpp"
 #include "mtconnect/source/adapter/agent_adapter/agent_adapter.hpp"
 #include "mtconnect/source/adapter/mqtt/mqtt_adapter.hpp"
+#include "mtconnect/source/adapter/mqtt/mqtt_broker_adapter.hpp"
 #include "mtconnect/source/adapter/shdr/shdr_adapter.hpp"
 #include "mtconnect/version.h"
 
@@ -115,6 +116,7 @@ namespace mtconnect::configuration {
     sink::mqtt_entity_sink::MqttEntitySink::registerFactory(m_sinkFactory);
     adapter::shdr::ShdrAdapter::registerFactory(m_sourceFactory);
     adapter::mqtt_adapter::MqttAdapter::registerFactory(m_sourceFactory);
+    adapter::mqtt_adapter::MqttBrokerAdapter::registerFactory(m_sourceFactory);
     adapter::agent_adapter::AgentAdapter::registerFactory(m_sourceFactory);
 
 #if _WINDOWS
@@ -791,6 +793,7 @@ namespace mtconnect::configuration {
     ConfigOptions options;
     GetOptions(config, options,
                {{configuration::PreserveUUID, true},
+                {configuration::DisconnectRemovesAllAssets, false},
                 {configuration::DisableAgentDevice, false},
                 {configuration::MonitorResources, false},
                 {configuration::WorkingDirectory, m_working.string()},

src/mtconnect/configuration/config_options.hpp

@@ -110,6 +110,7 @@ namespace mtconnect {
     DECLARE_CONFIGURATION(MqttMaxTopicDepth);
     DECLARE_CONFIGURATION(MqttLastWillTopic);
     DECLARE_CONFIGURATION(MqttXPath);
+    DECLARE_CONFIGURATION(MqttBrokerSecurity);
     DECLARE_CONFIGURATION(ObservationTopicPrefix);
     DECLARE_CONFIGURATION(DeviceTopicPrefix);
     DECLARE_CONFIGURATION(AssetTopicPrefix);
@@ -132,6 +133,7 @@ namespace mtconnect {
     DECLARE_CONFIGURATION(Path);
     DECLARE_CONFIGURATION(PollingInterval);
     DECLARE_CONFIGURATION(PreserveUUID);
+    DECLARE_CONFIGURATION(DisconnectRemovesAllAssets);
     DECLARE_CONFIGURATION(Protocol);
     DECLARE_CONFIGURATION(RealTime);
     DECLARE_CONFIGURATION(ReconnectInterval);

src/mtconnect/mqtt/mqtt_client_impl.hpp

@@ -188,8 +188,8 @@ namespace mtconnect {
                                            mqtt::buffer contents) {
           if (packet_id)
             LOG(debug) << "packet_id: " << *packet_id;
-          LOG(debug) << "topic_name: " << topic_name;
-          LOG(debug) << "contents: " << contents;
+          LOG(trace) << "topic_name: " << topic_name;
+          LOG(trace) << "contents: " << contents;
 
           if (m_running)
           {

src/mtconnect/mqtt/mqtt_server_impl.hpp

@@ -212,8 +212,8 @@ namespace mtconnect {
             if (packet_id)
               LOG(debug) << "Server packet_id: " << *packet_id;
 
-            LOG(debug) << "Server topic_name: " << topic_name;
-            LOG(debug) << "Server contents: " << contents;
+            LOG(trace) << "Server topic_name: " << topic_name;
+            LOG(trace) << "Server contents: " << contents;
 
             for (const auto& sub : m_subs)
             {

src/mtconnect/printer/html_printer.cpp

@@ -17,6 +17,8 @@
 
 #include "mtconnect/printer/html_printer.hpp"
 
+#include <boost/beast/core/detail/base64.hpp>
+
 #include <fstream>
 #include <iterator>
 
@@ -65,10 +67,20 @@ namespace mtconnect::printer {
     if (!m_loaded)
       return content;
 
-    // Single allocation and copy: prefix + content + suffix
+    // The document is base64 encoded so nothing in it can end the script element it is placed in.
+    // Extended assets keep their raw XML, including CDATA sections, so the document can contain
+    // any text, such as </script>. The base64 alphabet has no markup characters.
+    namespace base64 = boost::beast::detail::base64;
+    auto encodedSize = base64::encoded_size(content.size());
+
+    // Single allocation: prefix + encoded content + suffix
     string result;
-    result.reserve(m_prefix.size() + content.size() + m_suffix.size());
-    result.append(m_prefix).append(content).append(m_suffix);
+    result.reserve(m_prefix.size() + encodedSize + m_suffix.size());
+    result.append(m_prefix);
+    auto pos = result.size();
+    result.resize(pos + encodedSize);
+    result.resize(pos + base64::encode(result.data() + pos, content.data(), content.size()));
+    result.append(m_suffix);
     return result;
   }
 

src/mtconnect/printer/html_printer.hpp

@@ -67,7 +67,7 @@ namespace mtconnect::printer {
     /// @param in function to be called to resolve the file name.
     void resolveBrowserView(FileResolver&& resolver) { loadFile(std::move(resolver)); }
 
-    /// @brief Inject the content into the browser view at the placeholder.
+    /// @brief Inject the base64 encoded content into the browser view at the placeholder.
     /// @param content the document generated by the delegate printer
     /// @return the html document, or the content if the browser view is not available
     std::string formatResult(const std::string& content) const;

src/mtconnect/source/adapter/mqtt/mqtt_adapter.hpp

@@ -80,11 +80,10 @@ namespace mtconnect::source::adapter::mqtt_adapter {
     /// @brief subcribe to topics
     void subscribeToTopics();
 
-  protected:
     /// @brief load all topics
     /// @param ptree the property tree coming from configuration parser
     /// @param options configation options
-    void loadTopics(const boost::property_tree::ptree& tree, ConfigOptions& options);
+    static void loadTopics(const boost::property_tree::ptree& tree, ConfigOptions& options);
 
   protected:
     boost::asio::io_context& m_ioContext;