feat(tools-api): add lightweight package mode support

- Introduced support for lightweight package mode in the Tools API.
- The API now respects the `DDS_LIGHTWEIGHT_PACKAGE` environment variable,
  allowing for smaller worker packages (~50KB) without binaries.
- Updated `CSession` and `SSubmitRequestData` to automatically enable
  lightweight mode based on the environment variable.
- Fixed logic in `DDSWorker.sh` to handle both full and lightweight
  package deployments correctly.
- Added tests to verify the behavior of the lightweight mode.

Refs: https://its.cern.ch/jira/browse/EPN-566

Author: AnarManafov

CHANGELOG.md

@@ -5,6 +5,16 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- **dds-tools-lib**: Tools API now respects `DDS_LIGHTWEIGHT_PACKAGE` environment variable. When set, both `CSession::create()` and `SSubmitRequestData` constructor automatically enable lightweight mode.
+
+### Fixed
+
+- **DDSWorker.sh**: Fixed inverted logic bug that caused worker package deployment to fail when pre-compiled binaries were present. The script now correctly handles both full packages (with binaries) and lightweight packages (without binaries).
+
 ## [3.15.0] - 2025-10-08
 
 ### Added

ReleaseNotes.md

@@ -1,5 +1,65 @@
 # Release Notes
 
+## [Unreleased]
+
+### 🎉 New Features
+
+#### Tools API Environment Variable Support
+
+- **Automatic Lightweight Mode**: Tools API now automatically detects and respects the `DDS_LIGHTWEIGHT_PACKAGE` environment variable
+- **No More `make wn_bin`**: When using lightweight mode, you no longer need to build the worker binary package with `make wn_bin` - a huge time saver!
+- **Simplified API Usage**: Users no longer need to manually set the `enable_lightweight` flag when the environment variable is set
+- **Consistent Behavior**: Tools API now behaves consistently with command-line tools (`dds-session` and `dds-submit`)
+- **Smaller Packages**: Worker packages reduced from ~15MB to ~50KB in lightweight mode
+
+### 🐛 Bug Fixes
+
+#### Critical Worker Package Deployment Fix
+
+- **DDSWorker.sh Logic Error**: Fixed inverted logic bug that caused worker package deployment to fail when pre-compiled binaries were present
+- **Impact**: This bug prevented users from deploying full worker packages (with binaries) even though the binaries were correctly packaged
+- **Resolution**: The script now correctly:
+  - Extracts and uses binaries when they exist (full package mode)
+  - Validates lightweight mode requirements when binaries are absent (lightweight package mode)
+
+### 🚀 For Users
+
+#### If You Use Tools API
+
+Before this fix, you had to explicitly set the lightweight flag:
+
+```cpp
+submitInfo.setFlag(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight, true);
+```
+
+Now, simply set the environment variable before running your application:
+
+```bash
+export DDS_LIGHTWEIGHT_PACKAGE=1
+./my_dds_app
+```
+
+The Tools API will automatically:
+
+- Start sessions with `--lightweight` flag
+- Configure submit requests for lightweight mode
+
+#### If You Experienced Worker Package Failures
+
+If you previously encountered errors like:
+
+```text
+Error: Can't find WN pre-compiled bin.: /path/to/dds-wrk-bin-3.14-Linux-x86_64.tar.gz
+```
+
+This was caused by the DDSWorker.sh bug and is now fixed. Your worker packages will deploy correctly regardless of whether they contain pre-compiled binaries or are in lightweight mode.
+
+### 📝 Complete Changelog
+
+For a complete list of all changes, see [CHANGELOG.md](CHANGELOG.md).
+
+---
+
 ## [3.15.0] - 2025-10-08
 
 ### 🎉 New Features
@@ -79,4 +139,4 @@ For a complete list of all changes, see [CHANGELOG.md](CHANGELOG.md).
 
 ### 📋 Known Issues
 
-See [GitHub Issues](https://github.com/FairRootGroup/DDS/issues) for current known issues.
+See [GitHub Issues](https://github.com/FairRootGroup/DDS/issues) for current known issues.

dds-tools-lib/README.md

@@ -332,6 +332,41 @@ int main()
 }
 ```
 
+## Lightweight Package Mode
+
+The Tools API respects the `DDS_LIGHTWEIGHT_PACKAGE` environment variable. When set to `1`, `true`, `yes`, or `on` (case-insensitive), DDS creates minimal worker packages (~50KB) without binaries.
+
+**Benefits:**
+- **No need to build `make wn_bin` target** - Eliminates the worker binary compilation step
+- Significantly smaller worker packages (~50KB vs ~15MB)
+- Faster deployment to worker nodes
+
+**Using environment variable:**
+```cpp
+// Set environment variable before creating session
+setenv("DDS_LIGHTWEIGHT_PACKAGE", "1", 1);
+
+dds::tools_api::CSession session;
+session.create();  // Automatically uses lightweight mode
+```
+
+**Using explicit flag:**
+```cpp
+dds::tools_api::CSession session;
+
+dds::tools_api::SSubmitRequestData submitInfo;
+submitInfo.m_rms = "localhost";
+submitInfo.m_config = "dds-submit.cfg";
+submitInfo.m_enable_lightweight = true;  // Explicitly enable lightweight mode
+
+session.syncSendRequest<dds::tools_api::SSubmitRequest>(submitInfo, ...);
+```
+
+**Prerequisites for lightweight mode:**
+- DDS must be pre-installed on worker nodes
+- `DDS_COMMANDER_BIN_LOCATION` environment variable must point to DDS binaries directory (e.g., `/opt/dds/bin`)
+- `DDS_COMMANDER_LIBS_LOCATION` environment variable must point to DDS libraries directory (e.g., `/opt/dds/lib`)
+
 ## Example: Subscribing to Task Done Events
 
 ```cpp
@@ -500,4 +535,4 @@ catch (const std::exception& e)
 2. Use synchronous requests for simple, one-off operations
 3. Use asynchronous requests with callbacks for more complex scenarios or when receiving multiple responses
 4. Handle exceptions properly to avoid program crashes
-5. Subscribe to message callbacks to receive important notifications from the server
+5. Subscribe to message callbacks to receive important notifications from the server

dds-tools-lib/src/Tools.cpp

@@ -8,6 +8,7 @@
 #include <chrono>
 #include <sstream>
 // BOOST
+#include <boost/algorithm/string.hpp>
 #include <boost/any.hpp>
 #include <boost/filesystem.hpp>
 #if __has_include(<boost/process/v1.hpp>)
@@ -102,6 +103,13 @@ boost::uuids::uuid CSession::create()
 
     stringstream ssCmd;
     ssCmd << bp::search_path("dds-session").string() << " start";
+
+    // Check environment variable for lightweight mode
+    if (isLightweightModeEnabledByEnv())
+    {
+        ssCmd << " --lightweight";
+    }
+
     execute(ssCmd.str(), chrono::seconds(g_WAIT_PROCESS_SEC), &sOut, &sErr, &nExitCode);
 
     if (nExitCode != 0 || !sErr.empty())

dds-tools-lib/src/ToolsProtocol.cpp

@@ -166,6 +166,11 @@ constexpr const char* SSubmitRequestData::_protocolTag;
 
 SSubmitRequestData::SSubmitRequestData()
 {
+    // Check environment variable for lightweight mode and set flag automatically
+    if (isLightweightModeEnabledByEnv())
+    {
+        setFlag(ESubmitRequestFlags::enable_lightweight, true);
+    }
 }
 
 SSubmitRequestData::SSubmitRequestData(const boost::property_tree::ptree& _pt)

dds-tools-lib/src/ToolsProtocol.h

@@ -9,10 +9,12 @@
 // STD
 #include <bitset>
 #include <chrono>
+#include <cstdlib>
 #include <ostream>
 #include <string>
 #include <vector>
 // BOOST
+#include <boost/algorithm/string.hpp>
 #include <boost/property_tree/ptree.hpp>
 // DDS
 #include "Intercom.h"
@@ -22,6 +24,19 @@ namespace dds
 {
     namespace tools_api
     {
+        /// \brief Helper function to check DDS_LIGHTWEIGHT_PACKAGE environment variable
+        inline bool isLightweightModeEnabledByEnv()
+        {
+            const char* envLightweight = std::getenv("DDS_LIGHTWEIGHT_PACKAGE");
+            if (envLightweight != nullptr)
+            {
+                std::string envValue(envLightweight);
+                boost::algorithm::to_lower(envValue);
+                return (envValue == "1" || envValue == "true" || envValue == "yes" || envValue == "on");
+            }
+            return false;
+        }
+
         /// \brief Structure holds information of a done response.
         DDS_TOOLS_DECLARE_DATA_CLASS(SBaseResponseData, SDoneResponseData, "done")
 

dds-tools-lib/tests/TestProtocol.cpp

@@ -290,4 +290,140 @@ BOOST_AUTO_TEST_CASE(test_dds_tools_protocol_submit_response)
     BOOST_CHECK_EQUAL(data.m_jobIDs[2], "125.job");
 }
 
+BOOST_AUTO_TEST_CASE(test_dds_tools_protocol_lightweight_env_helper)
+{
+    // Test helper function with different environment variable values
+
+    // Test with value "1"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "1", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == true);
+
+    // Test with value "true"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "true", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == true);
+
+    // Test with value "TRUE" (case insensitive)
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "TRUE", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == true);
+
+    // Test with value "yes"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "yes", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == true);
+
+    // Test with value "YES" (case insensitive)
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "YES", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == true);
+
+    // Test with value "on"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "on", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == true);
+
+    // Test with value "ON" (case insensitive)
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "ON", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == true);
+
+    // Test with value "0" (should be false)
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "0", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == false);
+
+    // Test with value "false"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "false", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == false);
+
+    // Test with value "no"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "no", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == false);
+
+    // Test with invalid value
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "invalid", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == false);
+
+    // Test with empty value
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "", 1);
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == false);
+
+    // Test with unset variable
+    unsetenv("DDS_LIGHTWEIGHT_PACKAGE");
+    BOOST_CHECK(isLightweightModeEnabledByEnv() == false);
+}
+
+BOOST_AUTO_TEST_CASE(test_dds_tools_protocol_submit_request_env_lightweight)
+{
+    // Test that SSubmitRequestData constructor respects DDS_LIGHTWEIGHT_PACKAGE
+
+    // Test with environment variable set to "1"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "1", 1);
+    {
+        SSubmitRequestData data;
+        BOOST_CHECK(data.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == true);
+    }
+
+    // Test with environment variable set to "true"
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "true", 1);
+    {
+        SSubmitRequestData data;
+        BOOST_CHECK(data.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == true);
+    }
+
+    // Test with environment variable unset
+    unsetenv("DDS_LIGHTWEIGHT_PACKAGE");
+    {
+        SSubmitRequestData data;
+        BOOST_CHECK(data.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == false);
+    }
+
+    // Test that explicit flag setting overrides environment variable
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "1", 1);
+    {
+        SSubmitRequestData data;
+        // First check it's enabled by env
+        BOOST_CHECK(data.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == true);
+
+        // Now explicitly disable it
+        data.setFlag(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight, false);
+        BOOST_CHECK(data.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == false);
+
+        // And enable it again
+        data.setFlag(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight, true);
+        BOOST_CHECK(data.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == true);
+    }
+
+    // Clean up
+    unsetenv("DDS_LIGHTWEIGHT_PACKAGE");
+}
+
+BOOST_AUTO_TEST_CASE(test_dds_tools_protocol_submit_request_serialization_with_lightweight)
+{
+    // Test that lightweight flag is properly serialized/deserialized
+
+    setenv("DDS_LIGHTWEIGHT_PACKAGE", "1", 1);
+    {
+        SSubmitRequestData dataTest;
+        dataTest.m_rms = "slurm";
+        dataTest.m_instances = 10;
+        dataTest.m_slots = 32;
+        dataTest.m_config = "/path/to/config";
+        dataTest.m_pluginPath = "/path/to/plugin";
+        dataTest.m_requestID = 456;
+        // Flag should already be set by constructor due to environment variable
+
+        BOOST_CHECK(dataTest.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == true);
+
+        // Serialize to JSON and back
+        stringstream strBuf(dataTest.toJSON());
+
+        ptree pt;
+        read_json(strBuf, pt);
+        const ptree& childPT = pt.get_child("dds.tools-api.submit");
+
+        SSubmitRequestData data(childPT);
+
+        BOOST_CHECK(data == dataTest);
+        BOOST_CHECK(data.isFlagEnabled(SSubmitRequestData::ESubmitRequestFlags::enable_lightweight) == true);
+    }
+
+    // Clean up
+    unsetenv("DDS_LIGHTWEIGHT_PACKAGE");
+}
+
 BOOST_AUTO_TEST_SUITE_END()

etc/DDSWorker.sh.in

@@ -253,8 +253,14 @@ LIGHTWEIGHT_PKG=""
 # use pre-compiled binaries from the worker package
 # ***** prepare pre-compiled wn binaries *****
 WN_BIN_ARC="$WD/$BASE_NAME-$PKG_VERSION-$OS-$host_arch.tar.gz"
-if [ ! -f "$WN_BIN_ARC" ]; then
-	# Checking for a lightweight package
+if [ -f "$WN_BIN_ARC" ]; then
+	# Binary archive exists - use full package mode
+	logMsg "Found pre-compiled binary archive: $WN_BIN_ARC"
+	logMsg "Extracting binaries..."
+	# un-tar without creating a sub-directory
+	tar --strip-components=1 -xzf $WN_BIN_ARC || clean_up 1
+else
+	# Binary archive doesn't exist - check for lightweight package mode
 	logMsg "Worker package is in lightweight mode. Validating environment..."
 
 	# Validate DDS_COMMANDER_BIN_LOCATION
@@ -294,14 +300,6 @@ if [ ! -f "$WN_BIN_ARC" ]; then
 	DDS_BINARIES_LOCATION=$DDS_COMMANDER_BIN_LOCATION
 	DDS_LIBS_LOCATION=$DDS_COMMANDER_LIBS_LOCATION
 	LIGHTWEIGHT_PKG="YES"
-else
-   logMsg "Error: Can't find WN pre-compiled bin.: $WN_BIN_ARC"
-   clean_up 1
-fi
-
-# un-tar without creating a sub-directory
-if [ -z "$LIGHTWEIGHT_PKG" ]; then
-	tar --strip-components=1 -xzf $WN_BIN_ARC || clean_up 1
 fi
 
 export PATH=$DDS_BINARIES_LOCATION:$PATH