请您Star Please Star

如果你觉得此工具不错，请轻轻点击此页面右上角Star按钮增加项目曝光度，谢谢！

If you think this tool is good, please gently click the Star button in the upper right corner at this page to increase the project exposure, thank you!

中文文档

点此查看中文文档

Easy-to-use Commandline Configuration Tool

A library for users to write (experiment in research) configurations in Python Dict or JSON format, read and write parameter value via dot . in code, while can read parameters from the command line to modify values.

标签 Labels： Python, Command Line, commandline, config, configuration, parameters, 命令行，配置，传参，参数值修改。

Github URL: https://github.com/NaiboWang/CommandlineConfig

Reserved Fields

The following fields are reserved and cannot be used as parameter names: config_name.

New Features

v2.2.*

• Support infinite level nesting of parameters in dictionary
• Automatic version checking
• Support parameter value constrained to specified value (enumeration)
• Support for tuple type
• Support reading configuration from local JSON file
• Support for setting parameter help and printing parameter descriptions via command line -h
• Documentation updates, provide simple example

Simple Example

# Install via pip
pip3 install commandline_config

# import package
from commandline_config import Config

# Define configuration dictionary
config = {
  "index":1,
  "lr": 0.1,
  "dbinfo":{
    "username": "NUS"
  }
}

# Generate configuration class based on configuration dict
c = Config(config)

# Print the configuration of the parameters
print(c)

# Read and write parameters directly via dot . and support multiple layers.
c.index = 2
c.dbinfo.username = "ZJU"
print(c.index, c.dbinfo.username, c["lr"])

# On the command line, modify the parameter values with --
python example.py --index 3 --dbinfo.username XDU

# Get the parameter descriptions via the help method in the code, or on the command line via -h or -help (customization required, see detailed documentation below for details)
c.help()

python example.py -h

Catalogue

• 请您Star Please Star
• 中文文档
• Easy-to-use Commandline Configuration Tool
  • Reserved Fields
  • New Features
    • v2.2.*
  • Simple Example
  • Catalogue
  • Usage
    • Please submit issue
    • Installation
    • Configuration Way
    • Configuration parameters read and write method
      • Write method
      • Reading method
      • Pass configuration to functions
      • Copy configuration
      • Store configuration parameters to local file or database
  • Advanced options
    • Restrict parameter input values to fixed enum types
    • Print parameter help descriptions
      • Set parameter descriptions
      • Print parameter help
  • Things need attention
    • Conflict with Argparse
    • Input value forced conversion
    • The list parameter needs to be assigned with a backslash before the string element quotes when passing by commandline
    • Quotes are required for command-line assignment of tuple parameters, and string elements must be preceded by a backslash
    • Parameter naming convention
    • Unlimited layer of nested objects
    • Parameter integrity check, all parameters to be modified must be predefined
    • Special configurations in zsh environment
  • Full conversion example
  • Example Running Script
  • Shattered thoughts
  • TODO

Usage

Please submit issue

If you encounter any problems during using with this tool, please raise an issue in the github page of this project, I will solve the bugs and problems encountered at the first time.

Meanwhile, welcome to submit issues to propose what functions you want to add to this tool and I will implement them when possible.

Installation

There are two ways to install this library:

• 1. Install via pip:

  pip3 install commandline_config

  If already installed, you can upgrade it by the following command:

  pip3 install commandline_config --upgrade

• 2. Import the commandline_config.py file directly from the /commandline_config folder of the github project into your own project directory, you need to install the dependency package prettytable:

  pip3 install prettytable

  Or install via requirements.txt:

  pip3 install -r requirements.txt

Configuration Way

• 0. Import library:

  from commandline_config import Config

• 1. Set the parameter name and initial value in JSON/Python Dict format, and add the parameter description by # comment. Currently supports nesting a dict inside another dict, and can nest unlimited layers.

  preset_config = {
        "index": 1,  # Index of party
        "dataset": "mnist",
        'lr': 0.01,  # learning rate
        'normalization': True,
        "pair": (1,2),
        "multi_information": [1, 0.5, 'test', "TEST"],  # list
        "dbinfo": {
            "username": "NUS",
            "password": 123456,
            "retry_interval_time": 5.5,
            "save_password": False,
            "pair": ("test",3),
            "multi":{
                "test":0.01,
            },
            "certificate_info": ["1", 2, [3.5]],
        }
    }

  That is, the initial configuration of the program is generated. Each key defined in preset_config dict is the parameter name and each value is the initial value of the parameter, and at the same time, the initial value type of the parameter is automatically detected according to the type of the set value.

  The above configuration contains seven parameters: index, dataset, batch, normalization, pair, multi_information and dbinfo, where the type of the parameter index is automatically detected as int, the default value is 1 and the description is "Index of party".

  Similarly, The type and default value of the second to fifth parameter are string: "mnist"; float:0.01; bool:True; tuple:(1,2); list:[1,0.5,'test', "TEST"].

  The seventh parameter is a nested dictionary of type dict, which also contains 7 parameters, with the same type and default values as the first 7 parameters, and will not be repeated here.

• 2. Create a configuration class object by passing preset_config dict to Config in any function you want.

  if __name__ == '__main__':
      config = Config(preset_config)
      # Or give the configuration a name:
      config_with_name = Config(preset_config, name="Federated Learning Experiments")
  
      # Or you can store the preset_config in local file configuration.json and pass the filename to the Config class.
      config_from_file = Config("configuration.json")

  This means that the configuration object is successfully generated.

• 3. Configuration of parameters can be printed directly via print function:

  print(config_with_name)

  The output results are:

  Configurations of Federated Learning Experiments:
  +-------------------+-------+--------------------------+
  |        Key        |  Type | Value                    |
  +-------------------+-------+--------------------------+
  |       index       |  int  | 1                        |
  |      dataset      |  str  | mnist                    |
  |         lr        | float | 0.01                     |
  |   normalization   |  bool | True                     |
  |        pair       | tuple | (1, 2)                   |
  | multi_information |  list | [1, 0.5, 'test', 'TEST'] |
  |       dbinfo      |  dict | See sub table below      |
  +-------------------+-------+--------------------------+
  
  Configurations of dict dbinfo:
  +---------------------+-------+---------------------+
  |         Key         |  Type | Value               |
  +---------------------+-------+---------------------+
  |       username      |  str  | NUS                 |
  |       password      |  int  | 123456              |
  | retry_interval_time | float | 5.5                 |
  |    save_password    |  bool | False               |
  |         pair        | tuple | ('test', 3)         |
  |        multi        |  dict | See sub table below |
  |   certificate_info  |  list | ['1', 2, [3.5]]     |
  +---------------------+-------+---------------------+
  
  Configurations of dict multi:
  +------+-------+-------+
  | Key  |  Type | Value |
  +------+-------+-------+
  | test | float | 0.01  |
  +------+-------+-------+
  

  Here the information of all parameters will be printed in table format. If you want to change the printing style, you can modify it by config_with_name.set_print_style(style=''). The values that can be taken for style are: both, table, json which means print both table and json at the same time, print only table, and json dictionary only.

  E.g.:

  # Only print json 
  config_with_name.set_print_style('json')
  print(config_with_name)
  print("----------")
  # Print table and json at the same time
  config_with_name.set_print_style('table')
  print(config_with_name)

  The output results are:

  Configurations of Federated Learning Experiments:
  {'index': 1, 'dataset': 'mnist', 'lr': 0.01, 'normalization': True, 'pair': (1, 2), 'multi_information': [1, 0.5, 'test', 'TEST'], 'dbinfo': 'See below'}
  
  Configurations of dict dbinfo:
  {'username': 'NUS', 'password': 123456, 'retry_interval_time': 5.5, 'save_password': False, 'pair': ('test', 3), 'multi': 'See below', 'certificate_info': ['1', 2, [3.5]]}
  
  Configurations of dict multi:
  {'test': 0.01}
  
  ----------
    
  Configurations of Federated Learning Experiments:
  +-------------------+-------+--------------------------+
  |        Key        |  Type | Value                    |
  +-------------------+-------+--------------------------+
  |       index       |  int  | 1                        |
  |      dataset      |  str  | mnist                    |
  |         lr        | float | 0.01                     |
  |   normalization   |  bool | True                     |
  |        pair       | tuple | (1, 2)                   |
  | multi_information |  list | [1, 0.5, 'test', 'TEST'] |
  |       dbinfo      |  dict | See sub table below      |
  +-------------------+-------+--------------------------+
  {'index': 1, 'dataset': 'mnist', 'lr': 0.01, 'normalization': True, 'pair': (1, 2), 'multi_information': [1, 0.5, 'test', 'TEST'], 'dbinfo': 'See below'}
  
  Configurations of dict dbinfo:
  +---------------------+-------+---------------------+
  |         Key         |  Type | Value               |
  +---------------------+-------+---------------------+
  |       username      |  str  | NUS                 |
  |       password      |  int  | 123456              |
  | retry_interval_time | float | 5.5                 |
  |    save_password    |  bool | False               |
  |         pair        | tuple | ('test', 3)         |
  |        multi        |  dict | See sub table below |
  |   certificate_info  |  list | ['1', 2, [3.5]]     |
  +---------------------+-------+---------------------+
  {'username': 'NUS', 'password': 123456, 'retry_interval_time': 5.5, 'save_password': False, 'pair': ('test', 3), 'multi': 'See below', 'certificate_info': ['1', 2, [3.5]]}
  
  Configurations of dict multi:
  +------+-------+-------+
  | Key  |  Type | Value |
  +------+-------+-------+
  | test | float | 0.01  |
  +------+-------+-------+
  {'test': 0.01}
  

Configuration parameters read and write method

Write method

Configuration parameter values can be written in three ways.

• 1. To receive command line arguments, simply pass --index 1 on the command line to modify the value of index to 1. Also, the considerations for passing values to different types of arguments are：

  • When passing bool type, you can use 0 or False for False, 1 or True or no value after the parameter for True: --normalization 1 or --normalization True or --normalization all can set the value of parameter normalization in the configuration to True.
  • When passing list type, empty array and multi-dimensional arrays can be passed.
  • To modify the value in the nested dict, please use --nested-parameter-name.sub-parameter-name.sub-parameter-name.….sub-parameter-name value to modify the value in the nested object, such as --dbinfo.password 987654 to change the value of the password parameter in the dbinfo subobject to 987654; --dbinfo.multi.test 1 to change the value of the test parameter in the multi dict which is in dbinfo subobject to ```. Currently this tool can supports unlimited layers/levels of nesting.
  • Note that the argument index must be in the preset_config object defined above:

  python test.py --dbinfo.password 987654 --dbinfo.multi.test 1 --index 0 --dataset emnist --normalization 0 --multi_information [\'sdf\',1,\"3.3\",,True,[1,[]]] 

• 2. Use config.index = 2 directly in the code to change the value of the parameter index to 2. Again, list type parameters can be assigned as empty or multidimensional arrays. For nested objects, you can use config.dbinfo.save_password=True to modify the value of the save_password parameter in sub dict dbinfo to True.
• 3. Way 1 and 2 will trigger type checking, that is, if the type of the assigned value and the type of the default value in the predefined dict preset_config does not match, the program will report an error, therefore, if you do not want to force type checking, you can use config["index"] = "sdf" to force the value of the parameter index to the string sdf (not recommended, it will cause unexpected impact).

Reading method

Read the value of the parameter dataset directly by means of config.dataset or config["dataset"].

print(config.dataset, config["index"])

The value of an argument a will be read by this order: the last value modified by config.a = * > the value of --a 2 specified by the command line > the initial value specified by "a":1 defined by preset_config.

For the list type, if a multidimensional array is passed, the information can be read via standard slice of python:

config.dbinfo.certificate_info = [1,[],[[2]]]
print(config.dbinfo.certificate_info[2][0][0])

For parameters in a single nested object, there are four ways to read the values of the parameters, all of which can be read successfully:

print(config.dbinfo.username)
print(config["dbinfo"].password)
print(config.dbinfo["retry_interval_time"])
print(config["dbinfo"]["save_password"])

Pass configuration to functions

Simply pass the above config object as a parameter to the function and call it:

def print_dataset_name(c):
  print(c.dataset, c["dataset"], c.dbinfo.certificate_info)

print_dataset_name(c=config)

Copy configuration

A deep copy of the