SCANOSS Scanner

The SCANOSS Scanner is a simple Python script performs a scan of a folder using SCANOSS API.

Usage

Run scanner.py as a python script, passing as argument the path to the folder to be scanned.

Example:

python3 scanner.py /path/to/dir/to/scan

scanner.py generates a WFP file that is saved as scan_wfp in the current folder. This file is uploaded to the SCANOSS API, to perform a scan and return the output as in json format.

Winnowing

SCANOSS implements an adaptation of the original winnowing algorithm by S. Schleimer, D. S. Wilkerson and A. Aikenas described in their seminal article which can be found here: https://theory.stanford.edu/~aiken/publications/papers/sigmod03.pdf

The winnowing algorithm is configured using two parameters, the gram size and the window size. For SCANOSS the values need to be:

GRAM: 30
WINDOW: 64

The result of performing the Winnowing algorithm is a string called WFP (Winnowing FingerPrint). A WFP contains optionallythe name of the source component and the results of the Winnowing algorithm for each file.

EXAMPLE output: test-component.wfp

component=f9fc398cec3f9dd52aa76ce5b13e5f75,test-component.zipfile=cae3ae667a54d731ca934e2867b32aaa,948,test/test-file1.c4=579be9fb5=9d9eefda,58533be6,6bb116976=80188a22,f9bb922010=750988e0,b6785a0d12=600c7ec913=595544cc18=e3cb3b0f19=e8f7133dfile=cae3ae667a54d731ca934e2867b32aaa,1843,test/test-file2.c2=58fb3eed3=f5f7f4584=aba6add18=53762a72,0d274008,6be2454a10=239c7dfa12=0b2188c915=bd9c4b10,d5c8f9fb16=eb7309dd,63aebec519=316e10eb[...]

Here, component is the MD5 hash and path of the component (It could be a path to a compressed file or a URL).file is the MD5 hash, file length and file path being fingerprinted, followed bya list of WFP fingerprints with their corresponding line numbers.