Skip to content

Add README.rst to etc/scripts/ #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JonoYang merged 1 commit into from
Sep 8, 2021
Merged

Add README.rst to etc/scripts/ #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 1 addition & 155 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,161 +9,7 @@ our existing ones as well.

Usage
=====
A brand new project
-------------------
.. code-block:: bash

git init my-new-repo
cd my-new-repo
git pull git@github.com:nexB/skeleton

# Create the new repo on GitHub, then update your remote
git remote set-url origin git@github.com:nexB/your-new-repo.git

From here, you can make the appropriate changes to the files for your specific project.

Update an existing project
---------------------------
.. code-block:: bash

cd my-existing-project
git remote add skeleton git@github.com:nexB/skeleton
git fetch skeleton
git merge skeleton/main --allow-unrelated-histories

This is also the workflow to use when updating the skeleton files in any given repository.

Customizing
-----------

You typically want to perform these customizations:

- remove or update the src/README.rst and tests/README.rst files
- set project info and dependencies in setup.cfg
- check the configure and configure.bat defaults

Initializing a project
----------------------

All projects using the skeleton will be expected to pull all of it dependencies
from thirdparty.aboutcode.org/pypi or the local thirdparty directory, using
requirements.txt and/or requirements-dev.txt to determine what version of a
package to collect. By default, PyPI will not be used to find and collect
packages from.

In the case where we are starting a new project where we do not have
requirements.txt and requirements-dev.txt and whose dependencies are not yet on
thirdparty.aboutcode.org/pypi, we run the following command after adding and
customizing the skeleton files to your project:

.. code-block:: bash

./configure --init

This will initialize the virtual environment for the project, pull in the
dependencies from PyPI and add them to the virtual environment.

Generating requirements.txt and requirements-dev.txt
----------------------------------------------------

After the project has been initialized, we can generate the requirements.txt and
requirements-dev.txt files.

Ensure the virtual environment is enabled.

.. code-block:: bash

source venv/bin/activate

To generate requirements.txt:

.. code-block:: bash

python etc/scripts/gen_requirements.py -s venv/lib/python<version>/site-packages/

Replace \<version\> with the version number of the Python being used, for example: ``venv/lib/python3.6/site-packages/``

To generate requirements-dev.txt after requirements.txt has been generated:

.. code-block:: bash
./configure --init --dev
python etc/scripts/gen_requirements_dev.py -s venv/lib/python<version>/site-packages/

Note: on Windows, the ``site-packages`` directory is located at ``venv\Lib\site-packages\``

.. code-block:: bash

python .\\etc\\scripts\\gen_requirements.py -s .\\venv\\Lib\\site-packages\\
.\configure --init --dev
python .\\etc\\scripts\\gen_requirements_dev.py -s .\\venv\\Lib\\site-packages\\

Collecting and generating ABOUT files for dependencies
------------------------------------------------------

Ensure that the dependencies used by ``etc/scripts/bootstrap.py`` are installed:

.. code-block:: bash

pip install -r etc/scripts/requirements.txt

Once we have requirements.txt and requirements-dev.txt, we can fetch the project
dependencies as wheels and generate ABOUT files for them:

.. code-block:: bash

python etc/scripts/bootstrap.py -r requirements.txt -r requirements-dev.txt --with-deps

There may be issues with the generated ABOUT files, which will have to be
corrected. You can check to see if your corrections are valid by running:

.. code-block:: bash

python etc/scripts/check_thirdparty.py -d thirdparty

Once the wheels are collected and the ABOUT files are generated and correct,
upload them to thirdparty.aboutcode.org/pypi by placing the wheels and ABOUT
files from the thirdparty directory to the pypi directory at
https://github.com/nexB/thirdparty-packages


Usage after project initialization
----------------------------------

Once the ``requirements.txt`` and ``requirements-dev.txt`` have been generated
and the project dependencies and their ABOUT files have been uploaded to
thirdparty.aboutcode.org/pypi, you can configure the project without using the
``--init`` option.

If the virtual env for the project becomes polluted, or you would like to remove
it, use the ``--clean`` option:

.. code-block:: bash

./configure --clean

Then you can run ``./configure`` again to set up the project virtual environment.

To set up the project for development use:

.. code-block:: bash

./configure --dev

To update the project dependencies (adding, removing, updating packages, etc.),
update the dependencies in ``setup.cfg``, then run:

.. code-block:: bash

./configure --clean # Remove existing virtual environment
./configure --init # Create project virtual environment, pull in new dependencies
source venv/bin/activate # Ensure virtual environment is activated
python etc/scripts/gen_requirements.py -s venv/lib/python<version>/site-packages/ # Regenerate requirements.txt
python etc/scripts/gen_requirements_dev.py -s venv/lib/python<version>/site-packages/ # Regenerate requirements-dev.txt
pip install -r etc/scripts/requirements.txt # Install dependencies needed by etc/scripts/bootstrap.py
python etc/scripts/bootstrap.py -r requirements.txt -r requirements-dev.txt --with-deps # Collect dependency wheels and their ABOUT files

Ensure that the generated ABOUT files are valid, then take the dependency wheels
and ABOUT files and upload them to thirdparty.aboutcode.org/pypi.
Usage instructions can be found in ``docs/skeleton-usage.rst``.

Release Notes
=============
Expand Down
157 changes: 157 additions & 0 deletions docs/skeleton-usage.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
Usage
=====
A brand new project
-------------------
.. code-block:: bash

git init my-new-repo
cd my-new-repo
git pull git@github.com:nexB/skeleton

# Create the new repo on GitHub, then update your remote
git remote set-url origin git@github.com:nexB/your-new-repo.git

From here, you can make the appropriate changes to the files for your specific project.

Update an existing project
---------------------------
.. code-block:: bash

cd my-existing-project
git remote add skeleton git@github.com:nexB/skeleton
git fetch skeleton
git merge skeleton/main --allow-unrelated-histories

This is also the workflow to use when updating the skeleton files in any given repository.

Customizing
-----------

You typically want to perform these customizations:

- remove or update the src/README.rst and tests/README.rst files
- set project info and dependencies in setup.cfg
- check the configure and configure.bat defaults

Initializing a project
----------------------

All projects using the skeleton will be expected to pull all of it dependencies
from thirdparty.aboutcode.org/pypi or the local thirdparty directory, using
requirements.txt and/or requirements-dev.txt to determine what version of a
package to collect. By default, PyPI will not be used to find and collect
packages from.

In the case where we are starting a new project where we do not have
requirements.txt and requirements-dev.txt and whose dependencies are not yet on
thirdparty.aboutcode.org/pypi, we run the following command after adding and
customizing the skeleton files to your project:

.. code-block:: bash

./configure --init

This will initialize the virtual environment for the project, pull in the
dependencies from PyPI and add them to the virtual environment.

Generating requirements.txt and requirements-dev.txt
----------------------------------------------------

After the project has been initialized, we can generate the requirements.txt and
requirements-dev.txt files.

Ensure the virtual environment is enabled.

.. code-block:: bash

source venv/bin/activate

To generate requirements.txt:

.. code-block:: bash

python etc/scripts/gen_requirements.py -s venv/lib/python<version>/site-packages/

Replace \<version\> with the version number of the Python being used, for example: ``venv/lib/python3.6/site-packages/``

To generate requirements-dev.txt after requirements.txt has been generated:

.. code-block:: bash
./configure --init --dev
python etc/scripts/gen_requirements_dev.py -s venv/lib/python<version>/site-packages/

Note: on Windows, the ``site-packages`` directory is located at ``venv\Lib\site-packages\``

.. code-block:: bash

python .\\etc\\scripts\\gen_requirements.py -s .\\venv\\Lib\\site-packages\\
.\configure --init --dev
python .\\etc\\scripts\\gen_requirements_dev.py -s .\\venv\\Lib\\site-packages\\

Collecting and generating ABOUT files for dependencies
------------------------------------------------------

Ensure that the dependencies used by ``etc/scripts/bootstrap.py`` are installed:

.. code-block:: bash

pip install -r etc/scripts/requirements.txt

Once we have requirements.txt and requirements-dev.txt, we can fetch the project
dependencies as wheels and generate ABOUT files for them:

.. code-block:: bash

python etc/scripts/bootstrap.py -r requirements.txt -r requirements-dev.txt --with-deps

There may be issues with the generated ABOUT files, which will have to be
corrected. You can check to see if your corrections are valid by running:

.. code-block:: bash

python etc/scripts/check_thirdparty.py -d thirdparty

Once the wheels are collected and the ABOUT files are generated and correct,
upload them to thirdparty.aboutcode.org/pypi by placing the wheels and ABOUT
files from the thirdparty directory to the pypi directory at
https://github.com/nexB/thirdparty-packages


Usage after project initialization
----------------------------------

Once the ``requirements.txt`` and ``requirements-dev.txt`` have been generated
and the project dependencies and their ABOUT files have been uploaded to
thirdparty.aboutcode.org/pypi, you can configure the project without using the
``--init`` option.

If the virtual env for the project becomes polluted, or you would like to remove
it, use the ``--clean`` option:

.. code-block:: bash

./configure --clean

Then you can run ``./configure`` again to set up the project virtual environment.

To set up the project for development use:

.. code-block:: bash

./configure --dev

To update the project dependencies (adding, removing, updating packages, etc.),
update the dependencies in ``setup.cfg``, then run:

.. code-block:: bash

./configure --clean # Remove existing virtual environment
./configure --init # Create project virtual environment, pull in new dependencies
source venv/bin/activate # Ensure virtual environment is activated
python etc/scripts/gen_requirements.py -s venv/lib/python<version>/site-packages/ # Regenerate requirements.txt
python etc/scripts/gen_requirements_dev.py -s venv/lib/python<version>/site-packages/ # Regenerate requirements-dev.txt
pip install -r etc/scripts/requirements.txt # Install dependencies needed by etc/scripts/bootstrap.py
python etc/scripts/bootstrap.py -r requirements.txt -r requirements-dev.txt --with-deps # Collect dependency wheels and their ABOUT files

Ensure that the generated ABOUT files are valid, then take the dependency wheels
and ABOUT files and upload them to thirdparty.aboutcode.org/pypi.
Loading