Improve usage of config files, and encourage it for Keycloak tokens

ChangeLog.md

@@ -5,6 +5,14 @@
 
 # CaPyCli - Clearing Automation Python Command Line Tool for SW360
 
+## NEXT
+
+* Warn if Keycloak client_id and client_secret are passed on the command line.
+  See Readme.md for recommendations how to store them.
+* Config file can be located in home directory or current working directory.
+* Document config file usage and add missing aliases so all command line
+  options can be used in the config file.
+
 ## 2.12.0
 
 * Because of security reasons `-client_id` and `-client_secret` should only

Readme.md

@@ -230,16 +230,78 @@ More examples and usage notes can be found in [examples.md](examples.md).
 ## API Access
 
 Access to the SW360 REST API requires an access token.
-The token can be requested on SW360/Preferences/REST API Token.
+Tokens can be requested on SW360/Preferences/REST API Token.
+Starting with SW360 v20, access tokens can also be generated
+using a Keycloak client_id and _secret you can request from your
+SW360 admin team. For more information, see the SW360 documentation.
+
+You can provide a valid token in the config file (see next section),
+the environment variable ``SW360ProductionToken`` or using the
+command line option `-t`/`--token`. For the Keycloak workflow,
+use the variables ``SW360Client_id`` and ``SW360Client_secret`` or
+the config file keys or command line options `client_id` and
+`client_secret`.
+
+WARNING: Most operating systems allow easy access to the command line
+arguments of running processes. Therefore, don't pass the long-lived
+Keycloak `client_secret` on the command line. Use the environment
+variables or store it in a config file **in your home directory**
+with restricted file permissions. Don't put credentials into a project-local
+`./.capycli.cfg`, as it may accidentally end up in version control.
+
+NOTE: If your server uses the Keycloak workflow, it allows to request read-only
+or write tokens. As of now, CaPyCli automatically requests the necessary
+permission level depending on the command (e.g. read token for "bom map" and
+write token for "project create").
 
-The scripts in this repository expect, that a valid token
-is stored in the environment variable ``SW360ProductionToken``.
-Alternatively you can specify a token using the `-t` option.
-
-For proper access to an SW360 instance the correct url must be own.
 The SW360 url can be specified on the commandline with the `-url`
 parameter, via the environment variable ``SW360ServerUrl`` or in the
-config file (`.capycli.cfg`).
+config file using `sw360_url`.
+
+## Configuration File
+
+In addition to using environment variables and command line parameters,
+common settings can be preset via an optional configuration file named
+`.capycli.cfg`.
+
+CaPyCli looks for this file in the following order and uses the first
+one found:
+
+1. `./.capycli.cfg` in the current working directory
+2. `~/.capycli.cfg` in the user's home directory (`%USERPROFILE%`
+   on Windows).
+
+Note that only the **first** file found is used; merging settings from multiple
+files is not implemented yet. `./.capycli.cfg` is useful for project-specific
+settings (e.g. project name and report paths) while `~/.capycli.cfg` is the
+recommended place if you want to store server credentials, as putting sensitive
+data in a project-local config file risks accidental commit to version control.
+
+The file must be in [TOML](https://toml.io/) format and all settings
+must be placed inside a `[capycli]` table, for example:
+
+```toml
+[capycli]
+sw360_url = "https://sw360.example.com"
+oauth2 = true
+name = "MyProjectName"
+version = "1.0.0"
+```
+
+The keys correspond to the internal argument names used by CaPyCli, which
+usually match the long command line option name (e.g.  `oauth2` for
+`-oa`/`--oauth2`). For short options, the following table shows the
+corresponding internal names which you can also use for better readability
+in the configuration file.
+
+| config key alias | internal name (also accepted) |
+|------------------|-------------------------------|
+| `url`            | `sw360_url` |
+| `token`          | `sw360_token` |
+| `rr`             | `result_required` |
+| `if`             | `inputformat` |
+| `of`             | `outputformat` |
+| `X`              | `debug` |
 
 ## SBOM Format
 

capycli/main/options.py

@@ -9,6 +9,7 @@
 """Contains the logic for all of the default options for CaPyCli."""
 
 import os
+import pathlib
 import tomllib
 from typing import Any, Dict
 
@@ -22,6 +23,29 @@
 
 class CommandlineSupport():
     CONFIG_FILE_NAME = ".capycli.cfg"
+    # Maps configuration file keys (derived from the long command line option
+    # name) to the internal name for options where the two differ.
+    CONFIG_KEY_ALIASES: Dict[str, str] = {
+        "url": "sw360_url",
+        "token": "sw360_token",
+        "raw-input": "raw_input",
+        "search-meta-data": "search_meta_data",
+        "old-version": "old_version",
+        "package-source": "package_source",
+        "forceexit": "force_exit",
+        "overview": "create_overview",
+        "mapresult": "write_mapresult",
+        "rr": "result_required",
+        "if": "inputformat",
+        "of": "outputformat",
+        "remote-granularity": "remote_granularity_list",
+        "local-granularity": "local_granularity_list",
+        "remote-checklist": "remote_check_list",
+        "local-checklist": "local_checklist_list",
+        "X": "debug",
+        "forceerror": "force_error",
+        "project-mainline-state": "project_mainline_state",
+    }
 
     def __init__(self) -> None:
         custom_prog = capycli.get_app_signature()
@@ -459,7 +483,18 @@ def register_options(self) -> None:
 
     def read_config(self, filename: str = "", config_string: str = "") -> Dict[str, Any]:
         """
-        Read configuration from string or config file.
+        Read configuration from a TOML string or config file.
+
+        Lookup order (first source that is found wins, no merging):
+        1. config_string, if non-empty
+        2. filename, if non-empty
+        3. ./.capycli.cfg in the current working directory, if it exists
+        4. ~/.capycli.cfg, if it exists (%USERPROFILE%\\.capycli.cfg on Windows)
+
+        The TOML document must contain a [capycli] table; the keys of that
+        table are returned. Returns an empty dict if no source is found, if
+        the [capycli] table is missing, or on any parse or read error
+        (an error is logged in that case).
         """
 
         toml_dict = None
@@ -469,9 +504,13 @@ def read_config(self, filename: str = "", config_string: str = "") -> Dict[str,
             elif filename:
                 with open(filename, "rb") as f:
                     toml_dict = tomllib.load(f)
+            elif os.path.isfile(self.CONFIG_FILE_NAME):
+                with open(self.CONFIG_FILE_NAME, "rb") as f:
+                    toml_dict = tomllib.load(f)
             else:
-                if os.path.isfile(self.CONFIG_FILE_NAME):
-                    with open(self.CONFIG_FILE_NAME, "rb") as f:
+                home_config = pathlib.Path.home() / self.CONFIG_FILE_NAME
+                if os.path.isfile(home_config):
+                    with open(home_config, "rb") as f:
                         toml_dict = tomllib.load(f)
 
             if not toml_dict:
@@ -491,31 +530,16 @@ def read_config(self, filename: str = "", config_string: str = "") -> Dict[str,
     def process_commandline(self, argv: Any) -> Any:
         """Reads the command line arguments"""
         args = self.parser.parse_args(argv)
+        if args.client_id or args.client_secret:
+            LOG.warning("Providing client_id and client_secret on the command line is not recommended for security"
+                        " reasons. Please use the environment variables SW360Client_id/SW360Client_secret or a config"
+                        " file in your home directory instead (see Readme.md).")
         cfg = self.read_config()
 
         if cfg:
             for key in cfg:
-                args_key = key
-
-                # handle some common naming mistakes
-                if args_key == "url":
-                    args_key = "sw360_url"
-                if args_key == "url":
-                    args_key = "sw360_url"
-                if args_key == "raw-input":
-                    args_key = "raw_input"
-                if args_key == "token":
-                    args_key = "sw360_token"
-                if args_key == "oa":
-                    args_key = "oauth2"
-                if args_key == "search-meta-data":
-                    args_key = "search_meta_data"
-                if args_key == "old-version":
-                    args_key = "old_version"
-                if args_key == "package-source":
-                    args_key = "package_source"
-                if args_key == "forceexit":
-                    args_key = "force_exit"
+                # replace command line options by internal arguments in case they differ
+                args_key = self.CONFIG_KEY_ALIASES.get(key, key)
 
                 if hasattr(args, args_key) and not args.__getattribute__(args_key):
                     args.__setattr__(args_key, cfg[key])