Set Up the Project
Last updated
Last updated
Concrete-ML is a Python library, so Python should be installed to develop Concrete-ML. v3.8 and v3.9 are the only supported versions. Concrete-ML also uses Poetry and Make.
First of all, you need to git clone the project:
Some tests require files tracked by git-lfs to be downloaded. To do so please follow the instructions on then run git lfs pull.
A simple way to have everything installed is to use the development Docker (see the guide). On Linux and macOS you have to run the script in ./script/make_utils/setup_os_deps.sh. Specify the --linux-install-python flag if you want to install python3.8 as well on apt-enabled Linux distributions. The script should install everything you need for Docker and bare OS development (you can first check the content of the file to check what it will do).
For Windows users, the setup_os_deps.sh script does not install dependencies because of how many different installation methods there are/lack of a single package manager.
The first step is to (as some of our dev tools depend on it), then . In addition to installing Python, you are still going to need the following software available on path on Windows, as some of our basic dev tools depend on them:
git
jq
make
Development on Windows only works with the Docker environment. Follow .
To manually install Python, you can follow guide (alternatively, you can google how to install Python 3.8 (or 3.9)).
As there is no concrete-compiler package for Windows, only the dev dependencies can be installed. This requires Poetry >= 1.2.
At the time of writing (June 2022), there is only an alpha version of Poetry 1.2 that you can install. Use the official installer to install preview versions.
The dev tools use make to launch the various commands.
On Linux, you can install make from your distribution's preferred package manager.
On macOS, you can install a more recent version of make via brew:
In the following sections, be sure to use the proper make tool for your system: make, gmake, or other.
To get the source code of Concrete-ML, clone the code repository using the link for your favourite communication protocol (ssh or https).
We are going to make use of virtual environments. This helps to keep the project isolated from other Python projects in the system. The following commands will create a new virtual environment under the project directory and install dependencies to it.
The following command will not work on Windows if you don't have Poetry >= 1.2.
Finally, activate the newly created environment using the following command:
Docker automatically creates and sources a venv in ~/dev_venv/
The venv persists thanks to volumes. We also create a volume for ~/.cache to speed up later reinstallations. You can check which Docker volumes exist with:
You can still run all make commands inside Docker (to update the venv, for example). Be mindful of the current venv being used (the name in parentheses at the beginning of your command prompt).
After your work is done, you can simply run the following command to leave the environment:
From time to time, new dependencies will be added to the project or the old ones will be removed. The command below will make sure the project has the proper environment. So run it regularly!
If you are having issues, consider using the dev Docker exclusively (unless you are working on OS specific bug fixes or features).
Here are the steps you can take on your OS to try and fix issues:
At this point, you should consider using Docker as nobody will have the exact same setup as you. If, however, you need to develop on your OS directly, you can ask us for help but may not get a solution right away.
Here are the steps you can take in your Docker to try and fix issues:
If the problem persists at this point, you should ask for help. We're here and ready to assist!
Poetry is ised as the package manager. It drastically simplifies dependency and environment management. You can follow official guide to install it.
It is possible to install gmake as make. Check this for more info.
On Windows, check .