This project is no longer recommended or maintained 🛑

This project is no longer maintained, it will not get any fixes or support. It will be soon fully archived. Modern Conan 2.0 extensions can be found in https://github.com/conan-io/conan-extensions

Conan 2.0 support :warning:

The project Conan Package Tools does not support Conan 2.x and there is no current planned support.

In case you need such support, please, open an issue explaining your current case with more details.

Conan Package Tools

• Conan Package Tools
  • Introduction
  • Installation
  • How it works
    • Basic, but not very practical, example
  • Generating the build configurations automatically
  • Filtering or modifying the configurations
  • Package Version based on Commit Checksum
  • Save created packages summary
  • Using all values for custom options
  • Using Docker
    • Running scripts and executing commands before to build on Docker
    • Using with your own Docker images
    • Installing extra python packages before to build
    • Passing additional Docker parameters during build
    • Installing custom Conan config
  • Specifying a different base profile
  • Specifying build context for cross building
• The CI integration
  • Travis integration
  • Appveyor integration
  • Bamboo CI integration
  • Jenkins CI integration
  • GitLab CI integration
  • Upload packages
  • Upload dependencies (#237)
  • Pagination
    • Sequencial distribution
    • Named pages
    • Generating multiple references for the same recipe
  • Working with Bintray: Configuring repositories
  • Visual Studio auto-configuration
  • MinGW builds
  • Clang builds
• FULL REFERENCE
  • ConanMultiPackager parameters reference
  • Commit messages reference
  • Complete ConanMultiPackager methods reference:
  • Environment configuration
• Full example

Introduction

This package allows to automate the creation of conan packages for different configurations.

It eases the integration with CI servers like TravisCI and Appveyor, so you can use the cloud to generate different binary packages for your conan recipe.

Also supports Docker to create packages for different GCC and Clang versions.

Installation

$ pip install conan_package_tools

Or you can clone this repository and store its location in PYTHONPATH.

How it works

Using only conan C/C++ package manager (without conan package tools), you can use the conan create command to generate, for the same recipe, different binary packages for different configurations. The easier way to do it is using profiles:

$ conan create myuser/channel --profile win32
$ conan create myuser/channel --profile raspi
$ ...

The profiles can contain, settings, options, environment variables and build_requires. Take a look to the conan docs to know more.

Conan package tools allows to declare (or autogenerate) a set of different configurations (different profiles). It will call conan create for each one, uploading the generated packages to a remote (if needed), and using optionally docker images to ease the creation of different binaries for different compiler versions (gcc and clang supported).

Basic, but not very practical, example

Create a build.py file in your recipe repository, and add the following lines:

from cpt.packager import ConanMultiPackager

if __name__ == "__main__":
    builder = ConanMultiPackager(username="myusername")
    builder.add(settings={"arch": "x86", "build_type": "Debug"},
                options={}, env_vars={}, build_requires={})
    builder.add(settings={"arch": "x86_64", "build_type": "Debug"},
                options={}, env_vars={}, build_requires={})
    builder.run()

Now we can run the python script, the ConanMutiPackager will run the conan create command two times, one to generate x86 Debug package and another one for x86_64 Debug.

> python build.py

############## CONAN PACKAGE TOOLS ######################

INFO: ******** RUNNING BUILD **********
conan create myuser/testing --profile /var/folders/y1/9qybgph50sjg_3sm2_ztlm6dr56zsd/T/tmpz83xXmconan_package_tools_profiles/profile

[build_requires]
[settings]
arch=x86
build_type=Debug
[options]
[scopes]
[env]

...


############## CONAN PACKAGE TOOLS ######################

INFO: ******** RUNNING BUILD **********
conan create myuser/testing --profile /var/folders/y1/9qybgph50sjg_3sm2_ztlm6dr56zsd/T/tmpMiqSZUconan_package_tools_profiles/profile

[build_requires]
[settings]
arch=x86_64
build_type=Debug
[options]
[scopes]
[env]


#########################################################

...

If we inspect the local cache we can see that there are two binaries generated for our recipe, in this case the zlib recipe:

$ conan search zlib/1.2.11@myuser/testing

Existing packages for recipe zlib/1.2.11@myuser/testing:

Package_ID: a792eaa8ec188d30441564f5ba593ed5b0136807
    [options]
        shared: False
    [settings]
        arch: x86
        build_type: Debug
        compiler: apple-clang
        compiler.version: 9.0
        os: Macos
    outdated from recipe: False

Package_ID: e68b263f26a4d7513e28c9cae1673aa0466af777
    [options]
        shared: False
    [settings]
        arch: x86_64
        build_type: Debug
        compiler: apple-clang
        compiler.version: 9.0
        os: Macos
    outdated from recipe: False

Now, we could add new build configurations, but in this case we only want to add Visual Studio configurations and the runtime, but, of course, only if we are on Windows:

import platform
from cpt.packager import ConanMultiPackager

if __name__ == "__main__":
    builder = ConanMultiPackager(username="myusername")
    if platform.system() == "Windows":
        builder.add(settings={"arch": "x86", "build_type": "Debug", "compiler": "Visual Studio", "compiler.version": 14, "compiler.runtime": "MTd"},
                    options={}, env_vars={}, build_requires={})
        builder.add(settings={"arch": "x86_64", "build_type": "Release", "compiler": "Visual Studio", "compiler.version": 14, "compiler.runtime": "MT"},
                    options={}, env_vars={}, build_requires={})
    else:
        builder.add(settings={"arch": "x86", "build_type": "Debug"},
                    options={}, env_vars={}, build_requires={})
        builder.add(settings={"arch": "x86_64", "build_type": "Debug"},
                    options={}, env_vars={}, build_requires={})
    builder.run()

In the previous example, when we are on Windows, we are adding two build configurations:

- "Visual Studio 14, Debug, MTd runtime"
- "Visual Studio 14, Release, MT runtime"

We can also adjust the options, environment variables and build_requires:

from cpt.packager import ConanMultiPackager

if __name__ == "__main__":
    builder = ConanMultiPackager(username="myuser")
    builder.add({"arch": "x86", "build_type": "Release"},
                {"mypackage:option1": "ON"},
                {"PATH": "/path/to/custom"},
                {"*": ["MyBuildPackage/1.0@lasote/testing"]})
    builder.add({"arch": "x86_64", "build_type": "Release"}, {"mypackage:option1": "ON"})
    builder.add({"arch": "x86", "build_type": "Debug"}, {"mypackage:option2": "OFF", "mypackage:shared": True})
    builder.run()

We could continue adding configurations, but probably you realized that it would be such a tedious task if you want to generate many different configurations in different operating systems, using different compilers, different compiler versions etc.

Generating the build configurations automatically

Conan package tools can generate automatically a matrix of build configurations combining architecture, compiler, compiler.version, compiler.runtime, compiler.libcxx, build_type and and shared/static options.

from cpt.packager import ConanMultiPackager

if __name__ == "__main__":
    builder = ConanMultiPackager()
    builder.add_common_builds()
    builder.run()

If you run the python build.py command, for instance, in Mac OSX, it will add the following configurations automatically:

{'compiler.version': '7.3', 'arch': 'x86', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '7.3', 'arch': 'x86', 'build_type': 'Debug', 'compiler': 'apple-clang'})
{'compiler.version': '7.3', 'arch': 'x86_64', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '7.3', 'arch': 'x86_64', 'build_type': 'Debug', 'compiler': 'apple-clang'})
{'compiler.version': '8.0', 'arch': 'x86', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '8.0', 'arch': 'x86', 'build_type': 'Debug', 'compiler': 'apple-clang'})
{'compiler.version': '8.0', 'arch': 'x86_64', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '8.0', 'arch': 'x86_64', 'build_type': 'Debug', 'compiler': 'apple-clang'})
{'compiler.version': '8.1', 'arch': 'x86', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '8.1', 'arch': 'x86', 'build_type': 'Debug', 'compiler': 'apple-clang'})
{'compiler.version': '8.1', 'arch': 'x86_64', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '8.1', 'arch': 'x86_64', 'build_type': 'Debug', 'compiler': 'apple-clang'})

These are all the combinations of arch=x86/x86_64, build_type=Release/Debug for different compiler versions.

But having different apple-clang compiler versions installed in the same machine is not common at all. We can adjust the compiler versions using a parameter or an environment variable, specially useful for a CI environment:

from cpt.packager import ConanMultiPackager

if __name__ == "__main__":
    builder = ConanMultiPackager(apple_clang_versions=["9.0"]) # or declare env var CONAN_APPLE_CLANG_VERSIONS=9.0
    builder.add_common_builds()
    builder.run()

In this case, it will call conan create with only this configurations:

{'compiler.version': '9.0', 'arch': 'x86', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '9.0', 'arch': 'x86', 'build_type': 'Debug', 'compiler': 'apple-clang'})
{'compiler.version': '9.0', 'arch': 'x86_64', 'build_type': 'Release', 'compiler': 'apple-clang'})
{'compiler.version': '9.0', 'arch': 'x86_64', 'build_type': 'Debug', 'compiler': 'apple-clang'})

You can adjust other constructor parameters to control the build configurations that will be generated:

• gcc_versions: Generate only build configurations for the specified gcc versions (Ignored if the current machine is not Linux)
• visual_versions: Generate only build configurations for the specified Visual Studio versions (Ignore if the current machine is not Windows)
• visual_runtimes: Generate only build configurations for the specified runtimes. (only for Visual Studio)
• visual_toolsets: Specify the toolsets per each specified Visual Studio version. (only for Visual Studio)
• msvc_versions: Generate only build configurations for the specified msvc versions (Ignore if the current machine is not Windows)
• msvc_runtimes: Generate only build configurations for the specified runtimes. (only for msvc)
• msvc_runtime_types: Specify the runtime types per each specified msvc version. (only for msvc)
• apple_clang_versions: Generate only build configurations for the specified apple clang versions (Ignored if the current machine is not OSX)
• archs: Generate build configurations for the specified architectures, by default, ["x86", "x86_64"].
• build_types: Generate build configurations for the specified build_types, by default ["Debug", "Release"].

Or you can adjust environment variables:

• CONAN_GCC_VERSIONS
• CONAN_VISUAL_VERSIONS
• CONAN_VISUAL_RUNTIMES
• CONAN_VISUAL_TOOLSETS
• CONAN_MSVC_VERSIONS
• CONAN_MSVC_RUNTIMES
• CONAN_MSVC_RUNTIME_TYPES
• CONAN_APPLE_CLANG_VERSIONS
• CONAN_CLANG_VERSIONS
• CONAN_ARCHS
• CONAN_BUILD_TYPES

Check the REFERENCE section to see all the parameters and ENVIRONMENT VARIABLES available.

---

IMPORTANT! Both the constructor parameters and the corresponding environment variables of the previous list ONLY have effect when using builder.add_common_builds().

---

So, if we want to generate packages for x86_64 and armv8 but only for Debug and apple-clang 9.0:

$ export CONAN_ARCHS=x86_64,armv8
$ export CONAN_APPLE_CLANG_VERSIONS=9.0
$ export CONAN_BUILD_TYPES=Debug

$ python build.py

There are also two additional parameters of the add_common_builds:

• pure_c: (Default True) If your project is C++, pass the pure_c=False to add both combinations using libstdc and libstdc++11 for the setting compiler.libcxx. When True, the default profile value of libcxx will be applied. If you don't want libcxx value to apply to your binary packages you have to use the configure method to remove it:

    def configure(self):
        del self.settings.compiler.libcxx

• shared_option_name: If your conanfile.py have an option shared, the generated builds will contain automatically the "True/False" combination for that option. Pass "False" to deactivate it or "lib_name:shared_option_name" to specify a custom option name, e.j: boost:my_shared``
• dll_with_static_runtime: Will add also the combination of runtime MT with shared libraries.
• header_only: If your conanfile.py have an option header_only, the generated builds will contain automatically the "True/False" combination for that option #454.
• build_all_options_values: It includes all possible values for the listed options #457.

from cpt.packager import ConanMultiPackager

if __name__ == "__main__":
    builder = ConanMultiPackager()
    builder.add_common_builds(shared_option_name="mypackagename:shared", pure_c=False)
    builder.run()

Filtering or modifying the configurations

Use the remove_build_if helper with a lambda function to filter configurations:

from cpt.packager import ConanMultiPackager