Skip to content
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AUTHORS.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The following organizations or individuals have contributed to this repo:

- Shivam Sandbhor
- nexB Inc.
- Philippe Ombredanne
- Shivam Sandbhor @sbs2001
- Philippe Ombredanne @pombredanne
- Hritik Vijay @Hritik14

69 changes: 43 additions & 26 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ univers: mostly universal version and version ranges comparison and conversion

.. |Build Status| image:: https://api.travis-ci.com/sbs2001/univers.svg?branch=main&status=passed
.. |License| image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg
:target: https://opensource.org/licenses/Apache-2.0
:target: https://scancode-licensedb.aboutcode.org/apache-2.0.html
.. |Python 3.6+| image:: https://img.shields.io/badge/python-3.6+-blue.svg
:target: https://www.python.org/downloads/release/python-380/



univers was born out of the need for a mostly univeral way to perform software
package version comparisons in VulnerableCode.
**univers** was born out of the need for a mostly univeral way to store version
ranges and to compare software package versions in VulnerableCode.

Package version ranges and version constraints are useful and essential:

Expand All @@ -27,38 +27,55 @@ Package version ranges and version constraints are useful and essential:
range of affected bar versions.

Existing tools support typically a single algorithm to parse and compare
versions and this is not accurate across different ecosystems, since each
follow different versioning rules. For example there's no concept of 'epoch' in
semver versioning as used in package types and ecosystem such as npm or
rubygems, but epochs do exist in debian versions. A tool designed for semver or
dpkg versions processing would not be able to handle correctly the other version
scheme.
versions and a single range syntax and this quite different across ecosystems,
Comment thread
pombredanne marked this conversation as resolved.
Outdated
each following different versioning rules. For example there's no concept of
'epoch' in semver versioning such used in npm which uses the "node-semver" range
Comment thread
pombredanne marked this conversation as resolved.
Outdated
syntax, which is simialr but subtly different from the Rubygems conventions; but
Comment thread
pombredanne marked this conversation as resolved.
Outdated
epochs do exist in debian and RPM versions. A tool designed for semver or
Comment thread
pombredanne marked this conversation as resolved.
Outdated
dpkg versions processing would not be able to understand the version other
schemes and ranges correctly.

univers is different and considers the ecosystem-specific version scheme used.
**univers** is different:

- It tracks each ecosystem version scheme and how two versions are compared

How does univers work ?
=========================
- It can parse native version ranges notation into the common "vers" notation
and can return back native version ranges from a "vers".

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear.

"parse native version ranges notation into the common "vers" notation" means converting ">1.2.3" to "vers:pypi/>1.2.3"

"can return back native version ranges from a 'vers' " . What is this "native version ranges" ? Is it a string representation of the range or is it an object ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it a string representation of the range or is it an object ?

This is a string, for instance the one seen in an npm package.json for dependencies or in a setup.py install_requires


univers wraps, embeds or implements multiple version comparision libraries, each
focused on specific ecosystem version scheme.
- It is designed for use with Package URLs (purl)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe link to purl. But it doesn't make sense since this supposed to be in purl repo :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link added



How does **univers** work ?
============================

**univers** wraps, embeds or implements multiple version comparison libraries, each

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't "wraps, embeds or implements" be "wraps, embeds and implements" since the lib does all of this for every version expression

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in latest commit

focused on a specific ecosystem versionning scheme.

It also implements an experimental unified syntax for version ranges specifier
and can parse and convert existing version range strings to this unified syntax.


The supported package ecosystems versioning schemes and underlying libraries are:
The supported package ecosystems versioning schemes and underlying libraries is
a wrok in progress and there are some elements of support for:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: "wrok"-> "work"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in latest commit


- semver (for versions since there is no range notation)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I completely missed the fact that semver has no official range notation. https://github.com/nexB/univers/blob/12ebed2e97bc98865bb9b689e12e885dd64000bb/src/univers/version_specifier.py#L21 :(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, in fact semver alone is only for versions and seldom seems to be used strictly (beyond npm and Go may be). The range notation is node-semver for npm and works with the "semver" version syntax.... and I decided to use the package type for the version scheme as a default (see #11 ) for simplicity.
So we have:

  • version syntaxes (e.g., semver)
  • range notations (e.g., node-semver tightly bound to semver)

Quite often both are combined in one such as with PEP440

This is supported in part by the `semantic_version <https://github.com/rbarrois/python-semanticversion>`_ library.
- npm that use node-semver ranges and semver versions
- golang (using semver)
- PHP composer
- Rubygems which use a semver-like but not-quite-semver scheme and a slightly different range notation from node-semver

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a short hint at the difference ? Like the pessimistic operator, caret operator which are ecosystem specific.


- semver: npm, golang, PHP composer, rubygems and others that follow the semver
spec, using `semantic_version <https://github.com/rbarrois/python-semanticversion>`_ library.
- debian: handled by the
`debian-inspector <https://github.com/sbs2001/univers/blob/main/src/univers/debian.py.ABOUT>`_

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This points to my fork, please update it to nexB fork

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

library.

- pypi: handled by Python's packaging library and the standard ``packaging.version`` module.
- maven: handled by the embedded `rpm_vercmp <https://github.com/sbs2001/univers/blob/main/src/univers/rpm.py.ABOUT>`_ library.
- ebuild/gentoo: handled by the embedded `gentoo_vercmp <https://github.com/sbs2001/univers/blob/main/src/univers/gentoo.py.ABOUT>`_ module.
- maven: handled by the embedded `pymaven <https://github.com/nexB/univers/blob/main/src/univers/pymaven.py.ABOUT>`_ library.
- rpm: handled by the embedded `rpm_vercmp <https://github.com/nexB/univers/blob/main/src/univers/rpm.py.ABOUT>`_ library.
- ebuild/gentoo: handled by the embedded `gentoo_vercmp <https://github.com/nexB/univers/blob/main/src/univers/gentoo.py.ABOUT>`_ module.
- arch linux : handled by the embedded `arch utility borrowed from msys2 <https://github.com/nexB/univers/blob/main/src/univers/arch.py.ABOUT>`_ module.

As we grow, new schemes will be implemented accordingly.
As we grow, new schemes and support for more package types will be implemented accordingly.


Alternative
Expand All @@ -69,6 +86,7 @@ is to use a single procedure for all the versions as implemented in `libversion
<https://github.com/repology/libversion>`_. This works in the most common case
but may not work correctly for specific tasks that demand accurate version
comparison such as for dependency resolution and vulnerabilities checks.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"vulnerabilities checks" sounds wrong gramatically.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit

It does not handle version range notations.


Installation
Expand All @@ -95,14 +113,12 @@ Test if a version is within or outside of a version range:
.. code:: python

from univers.version import PYPIVersion
from univers.version_specifier import VersionSpecifier
from univers.version_range import VersionRange

vs = VersionSpecifier.from_scheme_version_spec_string("pypi", ">=1.2.4")
v1 = PYPIVersion("1.2.4")
v2 = PYPIVersion("1.2.3")
range = VersionRange.from_vers("vers:pypi/>=1.2.4")

assert (v1 in vs ) == True
assert (v2 in vs ) == False
assert PypiVersion("1.2.4") in range
assert PypiVersion("1.2.4") not in range


Development
Expand All @@ -121,5 +137,6 @@ Visit https://github.com/nexB/univers and
https://gitter.im/aboutcode-org/vulnerablecode and
https://gitter.im/aboutcode-org/aboutcode for support and chat.


Primary license: Apache-2.0
SPDX-License-Identifier: Apache-2.0 AND BSD-3-Clause AND MIT
14 changes: 1 addition & 13 deletions conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,6 @@
# Copyright (c) nexB Inc. and others.
# SPDX-License-Identifier: Apache-2.0
#
# Visit https://aboutcode.org and https://github.com/nexB/ for support and download.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.

collect_ignore = ["setup.py"]
7 changes: 5 additions & 2 deletions src/univers/arch.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
#
# Copyright 2016-2020 Christoph Reiter
# Copyright (c) Christoph Reiter
# SPDX-License-Identifier: MIT
# Version comparision utility extracted from msys2 and further stripped down.
# Version utility extracted from msys2 https://github.com/msys2/msys2-web/
# and further stripped down.
#
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.

import re
from itertools import zip_longest
Expand Down
2 changes: 1 addition & 1 deletion src/univers/arch.py.ABOUT
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@ license_expression: MIT
homepage_url: https://github.com/msys2/msys2-web/

notes: |
The version comparision utility is extracted from msys2 and further stripped down.
The version comparison utility is extracted from msys2 and further stripped down.

notice_file: arch.py.NOTICE
4 changes: 3 additions & 1 deletion src/univers/debian.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#
# Copyright (c) nexB Inc. and others.
# Exatrcted from http://nexb.com and https://github.com/nexB/debian_inspector/
# Extracted from http://nexb.com and https://github.com/nexB/debian_inspector/
# Copyright (c) Peter Odding
# Author: Peter Odding <peter@peterodding.com>
# URL: https://github.com/xolox/python-deb-pkg-tools
# SPDX-License-Identifier: MIT
#
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.

import logging
import operator as operator_module
Expand Down
13 changes: 12 additions & 1 deletion src/univers/gentoo.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
#
# Copyright (c) 2006-2019, pkgcore contributors
# SPDX-License-Identifier: BSD-3-Clause
# Version comparision utility extracted from pkgcore and further stripped down.
# Version comparison utility extracted from pkgcore and further stripped down.
#
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.

import re

from univers.utils import cmp
from univers.utils import remove_spaces

_is_gentoo_version = re.compile(
r"^(?:\d+)(?:\.\d+)*[a-zA-Z]?(?:_(p(?:re)?|beta|alpha|rc)\d*)*$"
).match

suffix_regexp = re.compile("^(alpha|beta|rc|pre|p)(\\d*)$")

Expand All @@ -19,6 +25,11 @@
"""


def is_valid(string):
version, _ = parse_version_and_revision(remove_spaces(string))
return _is_gentoo_version(version)


def parse_version_and_revision(version_string):
"""
Return a tuple of (version string, revision int) given a ``version_string``.
Expand Down
2 changes: 1 addition & 1 deletion src/univers/gentoo.py.ABOUT
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@ copyright: |
license_expression: BSD-3-Clause
homepage_url: https://github.com/pkgcore/pkgcore/blob/master/src/pkgcore/ebuild/cpv.py

notes: The version comparision utility is extracted from pkgcore and further stripped down.
notes: The version comparison utility is extracted from pkgcore and further stripped down.

notice_file: gentoo.py.NOTICE
4 changes: 3 additions & 1 deletion src/univers/maven.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
#
# Copyright (c) SAS Institute Inc.
# SPDX-License-Identifier: Apache-2.0
# Version comparision utility extracted from pymaven and further stripped down
# Version comparison utility extracted from pymaven and further stripped down
# and significantly modified from the original at pymaven
#
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.

import functools
from itertools import zip_longest
Expand Down
6 changes: 4 additions & 2 deletions src/univers/rpm.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
#
# Copyright (c) SAS Institute Inc.
# SPDX-License-Identifier: Apache-2.0
# Version comparision utility extracted from python-rpm-vercmp and further
# Version comparison utility extracted from python-rpm-vercmp and further
# stripped down and significantly modified from the original at python-rpm-vercmp
#
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.

import re


class Vercmp(object):
class Vercmp:
R_NONALNUMTILDE = re.compile(br"^([^a-zA-Z0-9~]*)(.*)$")
R_NUM = re.compile(br"^([\d]+)(.*)$")
R_ALPHA = re.compile(br"^([a-zA-Z]+)(.*)$")
Expand Down
2 changes: 1 addition & 1 deletion src/univers/rpm.py.README
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Pure Python implementation of rpmvercmp.

The RPM Package Manager (http://rpm.org) has a version comparision algorithm,
The RPM Package Manager (http://rpm.org) has a version comparison algorithm,
implemented in its C library, which performs the comparison in a certain way.

In certain circumstances, where the C library is not installable (for example,
Expand Down
86 changes: 86 additions & 0 deletions src/univers/semver.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
#
# Copyright (c) nexB Inc. and others.
# SPDX-License-Identifier: Apache-2.0
#
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.

import semantic_version

from univers.utils import remove_spaces
from univers.version_constraint import VersionConstraint

"""
node-semver and Rubygems semver-like related utilities.
"""


def get_caret_constraints(string):
"""
Return a tuple of two VersionConstraint representing the lower and upper
bound of version constraint ``string`` that contains a caret node-semver-
like range. Raise a ValueError if this is not a caret range.

For example:
>>> lower_bound, upper_bound = get_caret_constraints("^1.0.2")
>>> vlow = semantic_version.Version("1.0.2")
>>> vup = semantic_version.Version("2.0.0")
>>> assert lower_bound == VersionConstraint(comparator=">=", version=vlow)
>>> assert upper_bound == VersionConstraint(comparator="<", version=vup)
"""
string = remove_spaces(string)
if not string or not string.startswith("^"):
raise ValueError(f"Invalid caret version range: {string!r}")

version = string.lstrip("^")
lower_bound = semantic_version.Version(version)
upper_bound = lower_bound.next_major()

return (
VersionConstraint(comparator=">=", version=lower_bound),
VersionConstraint(comparator="<", version=upper_bound),
)


def get_tilde_constraints(string, operator="~"):
"""
Return a tuple of two VersionConstraint representing the lower and upper
bound of a version range ``string`` that contains a tilde node-semver-like
range. Raise a ValueError if this is not a tilde range.

For example:
>>> lower_bound, upper_bound = get_tilde_constraints("~1.0.2")
>>> vlow = semantic_version.Version("1.0.2")
>>> vup = semantic_version.Version("1.1.0")
>>> assert lower_bound == VersionConstraint(comparator=">=", version=vlow)
>>> assert upper_bound == VersionConstraint(comparator="<", version=vup)
"""
string = remove_spaces(string)
if not string or not string.startswith(operator):
raise ValueError(f"Invalid version range: {string!r} " f"does not start with {operator!r}")

version = string.lstrip(operator)
lower_bound = semantic_version.Version(version)
upper_bound = lower_bound.next_minor()

return (
VersionConstraint(comparator=">=", version=lower_bound),
VersionConstraint(comparator="<", version=upper_bound),
)


# FIXME: this is unlikely correct https://github.com/npm/node-semver/issues/112
def get_pessimistic_constraints(string):
"""
Return a tuple of two VersionConstraint representing the lower and upper
bound of version range ``string`` that contains a pessimistic Ruby range.
Raise a ValueError if this is not a pessimistic Rubygems range.


For example:
>>> lower_bound, upper_bound = get_pessimistic_constraints("~>2.0.8")
>>> vlow = semantic_version.Version("2.0.8")
>>> vup = semantic_version.Version("2.1.0")
>>> assert lower_bound == VersionConstraint(comparator=">=", version=vlow)
>>> assert upper_bound == VersionConstraint(comparator="<", version=vup)
"""
return get_tilde_constraints(string, operator="~>")
4 changes: 2 additions & 2 deletions src/univers/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
# Copyright (c) nexB Inc. and others.
# SPDX-License-Identifier: Apache-2.0
#
# Visit https://aboutcode.org and https://github.com/nexB/ for support and download.
# Visit https://aboutcode.org and https://github.com/nexB/univers for support and download.


def remove_spaces(string):
return string.replace(" ", "")
return "".join(string.split())


def cmp(x, y):
Expand Down
Loading