Reproduce the numbers
Scripts, session-level data and statistical outputs are public at a fixed commit. Raw packet captures, trained detector files and credentials are not.
Get the code and data
The archived state is commit 639e9b4 of zilantix/praxis.
git clone https://github.com/zilantix/praxis.git
cd praxis
git checkout 639e9b4
pip install -r requirements.txt
Rerun the confirmation statistics
The analysis scripts were written for the lab machine and read inputs from /opt/praxis. Point the Phase 17 script at the public session file, then run it. The output matches the archived Phase 17 statistics.
# in artifacts/scripts/phase17_statistical_validation.py, set the input path to:
# artifacts/data_csv/phase17_confirm_session_scores.csv
python artifacts/scripts/phase17_statistical_validation.py
Main data files
All in artifacts/data_csv/. One row is one session unless noted.
| File | Contents |
|---|---|
phase17_confirm_session_scores.csv | 150 confirmation sessions: features, detector probability, benign distance, overhead, score |
phase17_confirm_config_scores.csv | Phase 17 means per configuration |
phase20_holdout_*_session_scores.csv | Holdout detector probabilities for the same sessions |
phase20b_threshold_validation.csv | Calibrated thresholds and detection rates (one row per threshold and configuration) |
phase21_rq3_curl_catalog.csv | Resolution-path sessions |
phase29_feature_delta_effect_sizes.csv | Feature changes between configurations |
phase29b_artifact_control_scores.csv | Artifact-control detectors by feature set |
phase30_multidetector_session_scores.csv | Five detector families |
phase31_mawi_cicids_rescored_sessions.csv | Sessions rescored against five benign references |
mawi_public_flows_20240302_1m.csv | MAWI day-2 benign features (one row per flow) |
Public and restricted
Public
- Analysis scripts
- Session-level CSVs for each phase
- Statistics tables and figures
- MAWI day-2 benign features
Restricted
- Raw packet captures, because of size
- Trained detector files
- Lab logs, credentials and the archive bucket
- Full benign datasets, under their license terms
Benign references come from their original sources: MAWI Working Group traffic archive and CIC-IDS 2017.
How the workbench matches the study
The workbench reimplements the study’s feature extraction and score in JavaScript. Its benign reference is the median and interquartile range of the public MAWI day-2 file, embedded in the page. With the study’s 150 confirmation sessions it gives these configuration scores:
| Configuration | Workbench | Phase 31, MAWI day 2 | Phase 17, MAWI day 1 |
|---|---|---|---|
| D, default profile, multiplexing c2 | 0.4331 | 0.4331 | 0.4329 |
| C, camouflaged profile, multiplexing c2 | 0.4628 | 0.4628 | 0.4626 |
| B, default profile, multiplexing off | 0.6536 | 0.6536 | 0.6541 |
Phase 17 originally used MAWI day 1, which is not in the public repository, so the workbench uses day 2. The two references differ by less than 0.001 in every score.
The session CSV the workbench downloads uses the study’s column names (packet_count through len_20, pcap_size_bytes, public_robust_log_distance), so it can feed the repository’s scripts.