Skip to content

Commit d720912

Browse files
steven-essersteven-esser
committed
Merge branch 'develop' into 62-pass-only-score-during-delta-creation
Signed-off-by: Steven Esser <sesser@nexb.com>
2 parents 3132830 + 9794515 commit d720912

31 files changed

Lines changed: 6265 additions & 1126 deletions

‎.travis.yml‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,26 @@
11
language: python
2+
23
python:
3-
- '2.7'
4+
- '2.7'
5+
6+
env:
7+
global:
8+
- PYTEST="bin/py.test -n 2 -vvs"
9+
10+
matrix:
11+
- TEST_SUITE="$PYTEST"
12+
- TEST_SUITE="./etc/release/release.sh"
13+
414
install:
515
- ./configure
6-
script: bin/py.test -n 2
16+
17+
script:
18+
- $TEST_SUITE
19+
720
notifications:
821
slack:
922
on_success: always
1023
on_failure: always
1124
rooms:
12-
- secure: VODG+P5REBhL7psD3XydwJBbEpB+3HBKsmEIHEgxrdPDvY/KDP35JTpJdkzhb+nc7EvIPdgVbVcpbiacDtXOskk4Si7X9sAXnQt+P0LTol5Pk0/qbC5BlS+HebPB5LUmUv4gYBhyLHbWvxOrBOJ9K7fNuH2I1+NWRUzHUWmlNwqwVvWZwuFNISVEqVvVP/+a9NUYJLYKR9h6TOXjCCVUdOfho4RW/0MPG5+R337QgaAFzoi90Ug3vR/IU29e1PMWbAxgxMVcDt+cSdqcu7mkn6esPIglxFW4rW2qWZFLRfOdJqvq0QTvx/zK2BYl1U5fHxyslb5eO1UhzfQjpSQcgy/Ex7K62YB7B6hVaOWg7j7/3+hbVARe0MKOFwDYEeQkLA5PQcquk7rAvnr2QfAesENXDIzxL/ghXzB94JAqm4pcElMfZIV/H9+5FSsW1OmOEF2g+HnytyCYGyRFZIgKahVHt+QUFXny9D53UujqqZe4Za223D4tRCToHkKpHvRkbPP66XRSxHDuoLRzO3hJLHAxHfoV+1rIkFq+ymryaobUX1kojHx8ihRDZn/ptNGIIm+GGDBQQwXcjf0oQW4keeKnA9bvi5MtHIjpGEZimW0UK1EvgtP0cN/xBEr2EHJDinFYpWoF1zX61P/S34z6OOAlt8NARLyZy/R+9ck8lRQ=
25+
- secure: VODG+P5REBhL7psD3XydwJBbEpB+3HBKsmEIHEgxrdPDvY/KDP35JTpJdkzhb+nc7EvIPdgVbVcpbiacDtXOskk4Si7X9sAXnQt+P0LTol5Pk0/qbC5BlS+HebPB5LUmUv4gYBhyLHbWvxOrBOJ9K7fNuH2I1+NWRUzHUWmlNwqwVvWZwuFNISVEqVvVP/+a9NUYJLYKR9h6TOXjCCVUdOfho4RW/0MPG5+R337QgaAFzoi90Ug3vR/IU29e1PMWbAxgxMVcDt+cSdqcu7mkn6esPIglxFW4rW2qWZFLRfOdJqvq0QTvx/zK2BYl1U5fHxyslb5eO1UhzfQjpSQcgy/Ex7K62YB7B6hVaOWg7j7/3+hbVARe0MKOFwDYEeQkLA5PQcquk7rAvnr2QfAesENXDIzxL/ghXzB94JAqm4pcElMfZIV/H9+5FSsW1OmOEF2g+HnytyCYGyRFZIgKahVHt+QUFXny9D53UujqqZe4Za223D4tRCToHkKpHvRkbPP66XRSxHDuoLRzO3hJLHAxHfoV+1rIkFq+ymryaobUX1kojHx8ihRDZn/ptNGIIm+GGDBQQwXcjf0oQW4keeKnA9bvi5MtHIjpGEZimW0UK1EvgtP0cN/xBEr2EHJDinFYpWoF1zX61P/S34z6OOAlt8NARLyZy/R+9ck8lRQ=
1326
email: false

‎CHANGELOG.rst‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
Changelog
22
=========
33

4-
0.0.1.beta (2018-02-05)
4+
1.0.0 (2018-02-21)
55
------------------
6-
This is the initial beta release.
6+
* Initial release of DeltaCode
7+
8+
* support for finding codebase differences from two scancode output files
9+
* simple command line with JSON and csv output formats

‎MANIFEST.in‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
graft src
22
graft etc
33
graft thirdparty
4+
graft samples
45

56
recursive-exclude thirdparty/dev *
67
prune tests
@@ -18,6 +19,8 @@ include setup.cfg
1819
include .gitignore
1920
include configure
2021
include configure.bat
22+
include deltacode
23+
include deltacode.bat
2124
include .travis.yml
2225

2326
global-exclude *.py[co] __pycache__ *.*~

‎NOTICE‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Software license
22
================
33

4-
Copyright (c) 2017 nexB Inc. and others. All rights reserved.
4+
Copyright (c) 2017-2018 nexB Inc. and others. All rights reserved.
55
http://nexb.com and https://github.com/nexB/deltacode/
66
The DeltaCode software is licensed under the Apache License version 2.0.
77
Data generated with DeltaCode require an acknowledgment.

‎README.rst‎

Lines changed: 72 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -9,68 +9,94 @@ Deltacode
99
| | :target: https://travis-ci.com/nexB/deltacode |
1010
+--------------+----------------------------------------------------------------------------------------------+
1111

12-
About:
13-
======
1412

15-
``deltacode`` is a simple command line utility that leverages the power
16-
of `scancode-toolkit <https://github.com/nexB/scancode-toolkit>`_ to determine file-level differences between two
17-
codebases.
13+
DeltaCode is a simple command line utility that leverages the power
14+
of `scancode-toolkit <https://github.com/nexB/scancode-toolkit>`_
15+
to determine file-level differences between two codebases.
1816

19-
How to use:
17+
During a typical software release cycle, development teams and software
18+
compliance experts want insight into how a codebase has changed during each
19+
release iteration. Specifically, these users need a utility that can point out
20+
places in a codebase where material license and other provenance changes have
21+
occurred. This is where DeltaCode comes in.
22+
23+
DeltaCode provides an accurate means of comparing two ScanCode result files,
24+
and returning any possible changes that have occurred between the two given
25+
scanned codebases. DeltaCode currently has the ability to detect file size and
26+
license changes, as well as means to detect when files have been moved to new
27+
locations.
28+
29+
We are continuously working on new features, such as detecting copyright changes
30+
and detecting package version changes.
31+
32+
33+
Quick Start
2034
===========
35+
Make sure you have Python 2.7 installed:
36+
* Download and install Python 2.7 32 bits for Windows
37+
https://www.python.org/ftp/python/2.7.14/python-2.7.14.msi
38+
* Download and install Python 2.7 for Mac
39+
https://www.python.org/ftp/python/2.7.14/python-2.7.14-macosx10.6.pkg
40+
* Download and install Python 2.7 for Linux via distro package manager
2141

22-
In order to calculate these differences (i.e. 'deltas'), the user must have two individual
23-
scancode scan files, with the ``--info`` scan option at minimum.
42+
Next, download and extract the latest DeltaCode release from::
2443

25-
For example::
44+
https://github.com/nexB/deltacode/releases/
2645

27-
$ pip install scancode-toolkit
28-
$ scancode --info ~/postgresql-9.6/ ~/psql-9.6-fileinfo.json
29-
$ scancode --info ~/postgresql-10.0/ ~/psql-10.0-fileinfo.json
46+
Open a terminal, extract the downloaded release archive, then `cd` to
47+
the extracted directory and run this command to display the command
48+
help. DeltaCode will self-configure if needed::
3049

50+
./deltacode --help
3151

52+
Run a sample delta
3253

33-
Once you have the two codebase scans, simply install/configure
34-
deltacode, and run it on the two scans::
54+
./deltacode -n samples/samples.json -o samples/samples.json
3555

36-
(deltacode) $ deltacode --help
37-
Usage: deltacode [OPTIONS]
56+
Run a simple delta saved to the `output.csv` file::
3857

39-
Identify the changes that need to be made to the 'old' scan file (-o or
40-
--old) in order to generate the 'new' scan file (-n or --new). Write the
41-
results to a .csv file (-c or --csv-file) or a .json file (-j or --json-
42-
file) at a user-designated location. If no file option is selected, print
43-
the JSON results to the console.
58+
./deltacode -n samples/samples.json -o samples/samples.json -c output.csv
4459

45-
Options:
46-
-h, --help Show this message and exit.
47-
-n, --new PATH Identify the path to the "new" scan file
48-
[required]
49-
-o, --old PATH Identify the path to the "old" scan file
50-
[required]
51-
-c, --csv-file PATH Identify the path to the .csv output file
52-
-j, --json-file FILENAME Identify the path to the .json output file
53-
-a, --all-delta-types Include unmodified files as well as all changed
54-
files in the .json or .csv output. If not
55-
selected, only changed files are included.
56-
57-
(deltacode) $ deltacode -n ~/psql-10.0-fileinfo.json -o ~/psql-9.6-fileinfo.json -j ~/psql-10.0-psql-9.6-delta.json
60+
Then open `output.csv` to view the delta results.
61+
62+
To get DeltaCode results for your codebase, install
63+
`scancode-toolkit <https://github.com/nexB/scancode-toolkit>`_ and generate a
64+
scan for each of the codebases you wish to 'Delta'
65+
66+
67+
Support
68+
=======
69+
70+
If you have a problem, a suggestion or found a bug, please enter a ticket at:
71+
https://github.com/nexB/deltacode/issues
72+
73+
For other questions, discussions, and chats, we have:
74+
75+
- an official Gitter channel at https://gitter.im/aboutcode-org/discuss
76+
Gitter also has an IRC bridge at https://irc.gitter.im/
77+
78+
- an official #aboutcode IRC channel on freenode (server chat.freenode.net)
79+
for DeltaCode and other related tools. Note that this receives
80+
notifications from repos so it can be a tad noisy. You can use your
81+
favorite IRC client or use the web chat at
82+
https://webchat.freenode.net/
83+
84+
85+
Source code
86+
===========
87+
88+
* https://github.com/nexB/deltacode.git
5889

59-
=========
60-
Problems?
61-
=========
6290

63-
Open an `issue <https://www.github.com/nexb/deltacode/issues>`_.
91+
License
92+
=======
6493

65-
Notes:
66-
======
94+
* Apache-2.0 with an acknowledgement required to accompany the delta output.
6795

68-
- ``deltacode`` also collects statistical information, like # of files and other percentages. Currently, this output is not present in the ``csv`` output
96+
See the NOTICE file for more details.
6997

70-
Future TODOs/additions:
71-
=======================
7298

73-
- Use ``copyright`` information from scans to identify changes in copyright holders or dates for attribution.
74-
- Detect package or file changes that are only a version change for the same package or file.
75-
- Determine whether some added/removed files or packages are actually cases where the files or packages were moved.
99+
Documentation & FAQ
100+
===================
76101

102+
https://github.com/nexB/deltacode/wiki

‎configure‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/bin/bash
22
#
3-
# Copyright (c) 2017 nexB Inc. http://www.nexb.com/ - All rights reserved.
3+
# Copyright (c) 2017-2018 nexB Inc. http://www.nexb.com/ - All rights reserved.
44
#
55
################################
66
# change these variables to customize this script locally

‎configure.bat‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
@echo OFF
22

3-
@rem Copyright (c) 2015 nexB Inc. http://www.nexb.com/ - All rights reserved.
3+
@rem Copyright (c) 2015-2018 nexB Inc. http://www.nexb.com/ - All rights reserved.
44

55
@rem ################################
66
@rem # change these variables to customize this script locally

‎deltacode‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
#!/bin/bash
2+
#
3+
# Copyright (c) 2018 nexB Inc. http://www.nexb.com/ - All rights reserved.
4+
#
5+
6+
# A minimal shell wrapper to the CLI entry point fo DeltaCode
7+
8+
9+
###################################################################################
10+
# from https://raw.githubusercontent.com/mkropat/sh-realpath/58c03982cfd8accbcf0c4426a4adf0f120a8b2bb/realpath.sh
11+
# realpath emulation for portability on *nix
12+
# this allow running scancode from aribtrary locations and from symlinks
13+
#
14+
# Copyright (c) 2014 Michael Kropat
15+
#
16+
# Permission is hereby granted, free of charge, to any person obtaining a copy
17+
# of this software and associated documentation files (the "Software"), to deal
18+
# in the Software without restriction, including without limitation the rights
19+
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
20+
# copies of the Software, and to permit persons to whom the Software is
21+
# furnished to do so, subject to the following conditions:
22+
#
23+
# The above copyright notice and this permission notice shall be included in
24+
# all copies or substantial portions of the Software.
25+
#
26+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
27+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
28+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
29+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
30+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
31+
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
32+
# THE SOFTWARE.
33+
34+
realpath() {
35+
canonicalize_path "$(resolve_symlinks "$1")"
36+
}
37+
38+
resolve_symlinks() {
39+
_resolve_symlinks "$1"
40+
}
41+
42+
_resolve_symlinks() {
43+
_assert_no_path_cycles "$@" || return
44+
45+
local dir_context path
46+
path=$(readlink -- "$1")
47+
if [ $? -eq 0 ]; then
48+
dir_context=$(dirname -- "$1")
49+
_resolve_symlinks "$(_prepend_dir_context_if_necessary "$dir_context" "$path")" "$@"
50+
else
51+
printf '%s\n' "$1"
52+
fi
53+
}
54+
55+
_prepend_dir_context_if_necessary() {
56+
if [ "$1" = . ]; then
57+
printf '%s\n' "$2"
58+
else
59+
_prepend_path_if_relative "$1" "$2"
60+
fi
61+
}
62+
63+
_prepend_path_if_relative() {
64+
case "$2" in
65+
/* ) printf '%s\n' "$2" ;;
66+
* ) printf '%s\n' "$1/$2" ;;
67+
esac
68+
}
69+
70+
_assert_no_path_cycles() {
71+
local target path
72+
73+
target=$1
74+
shift
75+
76+
for path in "$@"; do
77+
if [ "$path" = "$target" ]; then
78+
return 1
79+
fi
80+
done
81+
}
82+
83+
canonicalize_path() {
84+
if [ -d "$1" ]; then
85+
_canonicalize_dir_path "$1"
86+
else
87+
_canonicalize_file_path "$1"
88+
fi
89+
}
90+
91+
_canonicalize_dir_path() {
92+
(cd "$1" 2>/dev/null && pwd -P)
93+
}
94+
95+
_canonicalize_file_path() {
96+
local dir file
97+
dir=$(dirname -- "$1")
98+
file=$(basename -- "$1")
99+
(cd "$dir" 2>/dev/null && printf '%s/%s\n' "$(pwd -P)" "$file")
100+
}
101+
102+
###################################################################################
103+
# Now run scancode proper
104+
105+
DELTACODE_BIN="$( realpath "${BASH_SOURCE[0]}" )"
106+
DELTACODE_ROOT_DIR="$( cd "$( dirname "${DELTACODE_BIN}" )" && pwd )"
107+
108+
DELTACODE_CONFIGURED_PYTHON="$DELTACODE_ROOT_DIR/bin/python"
109+
if [ ! -f "$DELTACODE_CONFIGURED_PYTHON" ]; then
110+
echo "* Configuring DeltaCode for first use..."
111+
CONFIGURE_QUIET=1 "$DELTACODE_ROOT_DIR/configure" etc/conf
112+
fi
113+
114+
"$DELTACODE_ROOT_DIR/bin/deltacode" "$@"

‎deltacode.ABOUT‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
about_resource: .
22

33
name: Deltacode
4-
version: 0.0.1.beta
4+
version: 1.0.0
55

66
owner: nexB Inc.
77

@@ -16,4 +16,4 @@ license: apache-2.0
1616
license_text_file: apache-2.0.LICENSE
1717
notice_file: NOTICE
1818

19-
copyright: Copyright (c) 2017 nexB Inc. and others
19+
copyright: Copyright (c) 2017-2018 nexB Inc. and others

‎deltacode.bat‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
@echo OFF
2+
@rem Copyright (c) 2018 nexB Inc. http://www.nexb.com/ - All rights reserved.
3+
4+
@rem A minimal shell wrapper to the CLI entry point
5+
6+
set DELTACODE_ROOT_DIR=%~dp0
7+
@rem Use a trailing space in the next line to set the variable to an empty string
8+
set DELTACODE_CMD_LINE_ARGS=
9+
set DELTACODE_CONFIGURED_PYTHON=%DELTACODE_ROOT_DIR%\bin\python.exe
10+
11+
@rem Collect all command line arguments in a variable
12+
:collectarg
13+
if ""%1""=="""" goto continue
14+
call set DELTACODE_CMD_LINE_ARGS=%DELTACODE_CMD_LINE_ARGS% %1
15+
shift
16+
goto collectarg
17+
18+
:continue
19+
20+
if not exist "%DELTACODE_CONFIGURED_PYTHON%" goto configure
21+
goto deltacode
22+
23+
:configure
24+
echo * Configuring DeltaCode for first use...
25+
set CONFIGURE_QUIET=1
26+
call "%DELTACODE_ROOT_DIR%\configure" etc/conf
27+
if %errorlevel% neq 0 (
28+
exit /b %errorlevel%
29+
)
30+
31+
:deltacode
32+
"%DELTACODE_ROOT_DIR%\bin\deltacode" %DELTACODE_CMD_LINE_ARGS%
33+
34+
:EOS

0 commit comments

Comments
 (0)