-
Notifications
You must be signed in to change notification settings - Fork 125
feature: Add support for sysman python bindings #420
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
82d04e5
3ef4c34
9086388
244e5f5
9be93d5
37fca5a
51bc498
3faa738
aede7a8
5a15047
f430f1a
bb0935a
6208ada
594c78a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| ## | ||
| # Copyright (C) 2026 Intel Corporation | ||
| # | ||
| # SPDX-License-Identifier: MIT | ||
| # | ||
| ## | ||
|
|
||
| name: Python Bindings - Security Scan | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| - '**' | ||
| paths: | ||
| - 'bindings/sysman/python/**' | ||
| push: | ||
| branches: | ||
| - main | ||
| - master | ||
| - python_bindings | ||
| paths: | ||
| - 'bindings/sysman/python/**' | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| bandit: | ||
| name: Bandit Security Analysis | ||
| runs-on: ubuntu-latest | ||
| defaults: | ||
| run: | ||
| working-directory: bindings/sysman/python | ||
| permissions: | ||
| contents: read | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.x' | ||
|
|
||
| - name: Install Bandit | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install bandit | ||
|
|
||
| - name: Run Bandit security scan | ||
| run: | | ||
| bandit -r source/ test/ -f json -o bandit-report.json | ||
| bandit -r source/ test/ -ll -f screen | ||
|
|
||
| - name: Upload Bandit results as artifact | ||
| uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: bandit-security-report | ||
| path: bandit-report.json | ||
| retention-days: 30 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,307 @@ | ||
| ## | ||
| # Copyright (C) 2026 Intel Corporation | ||
| # | ||
| # SPDX-License-Identifier: MIT | ||
| # | ||
| ## | ||
|
|
||
| name: Python Bindings - Unit Tests & Coverage | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we have the ability to Copyright/License this file, please do if .yml supports comments
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added |
||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| - '**' | ||
| paths: | ||
| - 'bindings/sysman/python/**' | ||
| push: | ||
| branches: | ||
| - main | ||
| - master | ||
| - python_bindings | ||
| paths: | ||
| - 'bindings/sysman/python/**' | ||
| workflow_dispatch: | ||
|
|
||
| env: | ||
| PYTHON_VERSION: '3.10' | ||
|
|
||
| jobs: | ||
| unit-tests-linux: | ||
| name: Linux - Unit Tests & Coverage Analysis | ||
| runs-on: ubuntu-latest | ||
| defaults: | ||
| run: | ||
| working-directory: bindings/sysman/python | ||
| permissions: | ||
| contents: read | ||
| checks: write # For check results | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 # Full history for better coverage comparison | ||
|
|
||
| - name: Set up Python ${{ env.PYTHON_VERSION }} | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ env.PYTHON_VERSION }} | ||
| cache: 'pip' | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install pytest pytest-cov pytest-html pytest-xdist | ||
| pip install coverage[toml] | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Coverage uses Apache license right ? |
||
| # Linting, formatting, and type checking tools | ||
| pip install flake8 black isort mypy | ||
|
|
||
| # Install any project-specific dependencies if requirements.txt exists | ||
| if [ -f requirements.txt ]; then | ||
| pip install -r requirements.txt | ||
| fi | ||
|
|
||
| - name: Code Quality Checks | ||
| run: | | ||
| echo "Running code quality checks..." | ||
|
|
||
| # Check code formatting with black | ||
| echo "Checking code formatting..." | ||
| black --check --diff source/ test/ || { | ||
| echo "❌ Code formatting issues found. Run 'black source/ test/' to fix." | ||
| exit 1 | ||
| } | ||
|
|
||
| # Check import sorting | ||
| echo "Checking import sorting..." | ||
| isort --check-only --diff source/ test/ || { | ||
| echo "❌ Import sorting issues found. Run 'isort source/ test/' to fix." | ||
| exit 1 | ||
| } | ||
|
|
||
| # Run linting | ||
| echo "Running flake8 linting..." | ||
| flake8 source/ test/ || { | ||
| echo "❌ Linting issues found. Check flake8 output above." | ||
| exit 1 | ||
| } | ||
|
|
||
| # Run type checking | ||
| echo "Running mypy type checking..." | ||
| mypy source/ --ignore-missing-imports --no-strict-optional || { | ||
| echo "❌ Type checking issues found. Check mypy output above." | ||
| exit 1 | ||
| } | ||
|
|
||
| echo "✅ All code quality checks passed!" | ||
|
|
||
| - name: Run Unit Tests with Coverage | ||
| run: | | ||
| # Run tests with coverage and generate multiple report formats | ||
| python -m pytest test/unit_tests/ \ | ||
| --cov=source \ | ||
| --cov-config=pyproject.toml \ | ||
| --cov-report=term-missing \ | ||
| --cov-report=html:htmlcov \ | ||
| --cov-report=xml:coverage.xml \ | ||
| --cov-report=json:coverage.json \ | ||
| --junit-xml=test-results.xml \ | ||
| --html=test-report.html \ | ||
| --self-contained-html \ | ||
| -v \ | ||
| --tb=short \ | ||
| --durations=10 | ||
|
|
||
| - name: Extract Coverage Percentage | ||
| id: coverage | ||
| run: | | ||
| # Extract coverage percentage from JSON report | ||
| COVERAGE_PCT=$(python -c 'import json; data = json.load(open("coverage.json")); print("{:.1f}".format(data["totals"]["percent_covered"]))') | ||
| echo "coverage_pct=$COVERAGE_PCT" >> $GITHUB_OUTPUT | ||
| echo "Current Coverage: $COVERAGE_PCT%" | ||
|
|
||
| - name: Get Baseline Coverage from Target Branch | ||
| id: baseline | ||
| continue-on-error: true | ||
| run: | | ||
| # ============================================================================ | ||
| # TEMPORARY: First Baseline Handling | ||
| # TODO: Remove the special handling below after first PR merge to master | ||
| # Once master has test coverage baseline, this graceful fallback is no longer needed | ||
| # ============================================================================ | ||
| SKIP_BASELINE_ON_MISSING=true # Set to false after first merge | ||
|
|
||
| # Get the target branch (base of the PR or parent commit for push events) | ||
| if [ "${{ github.event_name }}" == "pull_request" ]; then | ||
| TARGET_BRANCH="${{ github.event.pull_request.base.ref }}" | ||
| echo "Pull request detected - comparing against base branch: $TARGET_BRANCH" | ||
| else | ||
| # For push events, compare against the parent commit | ||
| TARGET_BRANCH="${{ github.ref_name }}" | ||
| echo "Push event detected - comparing against parent commit on branch: $TARGET_BRANCH" | ||
| # Use HEAD~1 to get the previous commit | ||
| git checkout HEAD~1 2>/dev/null || { | ||
| if [ "$SKIP_BASELINE_ON_MISSING" = true ]; then | ||
| echo "⚠️ No parent commit found (possibly first commit). Skipping baseline comparison." | ||
| echo "baseline_coverage=0" >> $GITHUB_OUTPUT | ||
| exit 0 | ||
| else | ||
| echo "❌ No parent commit found" | ||
| exit 1 | ||
| fi | ||
| } | ||
| fi | ||
|
|
||
| echo "Target branch/commit: $TARGET_BRANCH" | ||
|
|
||
| # Checkout target branch/commit to get baseline coverage | ||
| if [ "${{ github.event_name }}" == "pull_request" ]; then | ||
| git fetch origin $TARGET_BRANCH | ||
| git checkout origin/$TARGET_BRANCH 2>/dev/null || { | ||
| if [ "$SKIP_BASELINE_ON_MISSING" = true ]; then | ||
| echo "⚠️ Could not checkout target branch. Skipping baseline comparison." | ||
| echo "baseline_coverage=0" >> $GITHUB_OUTPUT | ||
| exit 0 | ||
| else | ||
| echo "❌ Could not checkout target branch: $TARGET_BRANCH" | ||
| exit 1 | ||
| fi | ||
| } | ||
| fi | ||
|
|
||
| # Check if tests exist in baseline | ||
| if [ ! -d "bindings/sysman/python/test/unit_tests" ]; then | ||
| if [ "$SKIP_BASELINE_ON_MISSING" = true ]; then | ||
| echo "⚠️ No unit tests found in baseline. This is likely the first commit with tests." | ||
| echo "Skipping baseline comparison." | ||
| echo "baseline_coverage=0" >> $GITHUB_OUTPUT | ||
| exit 0 | ||
| else | ||
| echo "❌ No unit tests found in baseline branch" | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| # Install dependencies and run tests to get baseline coverage | ||
| python -m pip install --upgrade pip | ||
| pip install pytest pytest-cov coverage[toml] | ||
|
|
||
| # Run tests with coverage for baseline | ||
| python -m pytest test/unit_tests/ \ | ||
| --cov=source \ | ||
| --cov-report=json:baseline-coverage.json \ | ||
| -q || { | ||
| if [ "$SKIP_BASELINE_ON_MISSING" = true ]; then | ||
| echo "⚠️ Baseline tests failed or not found. Skipping baseline comparison." | ||
| echo "baseline_coverage=0" >> $GITHUB_OUTPUT | ||
| exit 0 | ||
| else | ||
| echo "❌ Baseline tests failed on $TARGET_BRANCH" | ||
| echo "The target branch has broken tests. Fix the baseline before merging." | ||
| exit 1 | ||
| fi | ||
| } | ||
|
|
||
| # Extract baseline coverage | ||
| BASELINE_COVERAGE=$(python -c 'import json; data = json.load(open("baseline-coverage.json")); print("{:.1f}".format(data["totals"]["percent_covered"]))') | ||
| echo "baseline_coverage=$BASELINE_COVERAGE" >> $GITHUB_OUTPUT | ||
| echo "Baseline Coverage: $BASELINE_COVERAGE%" | ||
|
|
||
| # Switch back to PR branch | ||
| git checkout ${{ github.sha }} | ||
|
|
||
| - name: Check Coverage Threshold | ||
| run: | | ||
| CURRENT_COVERAGE="${{ steps.coverage.outputs.coverage_pct }}" | ||
| BASELINE_COVERAGE="${{ steps.baseline.outputs.baseline_coverage }}" | ||
|
|
||
| echo "Current Coverage: $CURRENT_COVERAGE%" | ||
| echo "Baseline Coverage: $BASELINE_COVERAGE%" | ||
|
|
||
| # ============================================================================ | ||
| # TEMPORARY: First Baseline Handling | ||
| # TODO: Remove this section after first PR merge to master | ||
| # After first merge, baseline should always exist and this check is unnecessary | ||
| # ============================================================================ | ||
| # If baseline is 0, this is the first commit with tests - always pass | ||
| if [ "$BASELINE_COVERAGE" == "0" ] || [ -z "$BASELINE_COVERAGE" ]; then | ||
| echo "✅ No baseline coverage found (first commit with tests)." | ||
| echo "Current coverage: $CURRENT_COVERAGE%" | ||
| echo "Establishing baseline for future comparisons." | ||
| exit 0 | ||
| fi | ||
| # ============================================================================ | ||
| # END TEMPORARY SECTION | ||
| # ============================================================================ | ||
|
|
||
| # Use awk for floating point comparison (more portable than bc) | ||
| if [ $(echo "$CURRENT_COVERAGE >= $BASELINE_COVERAGE" | awk '{print ($1 >= $3)}') -eq 1 ]; then | ||
| DELTA=$(echo "$CURRENT_COVERAGE - $BASELINE_COVERAGE" | awk '{printf "%.1f", $1 - $3}') | ||
| echo "✅ Coverage check passed: $CURRENT_COVERAGE% >= $BASELINE_COVERAGE% (Δ ${DELTA}%)" | ||
| else | ||
| REGRESSION=$(echo "$BASELINE_COVERAGE - $CURRENT_COVERAGE" | awk '{printf "%.1f", $1 - $3}') | ||
| echo "❌ Coverage regression detected!" | ||
| echo "Current coverage ($CURRENT_COVERAGE%) is below baseline coverage ($BASELINE_COVERAGE%)" | ||
| echo "Regression: -${REGRESSION}%" | ||
| echo "This would cause coverage to regress from the baseline." | ||
| echo "Please add tests to maintain or improve coverage." | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Coverage Summary | ||
| if: always() | ||
| run: | | ||
| echo "## 📊 Test Coverage Report" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "| Metric | Value |" >> $GITHUB_STEP_SUMMARY | ||
| echo "|--------|-------|" >> $GITHUB_STEP_SUMMARY | ||
| echo "| Current Coverage | ${{ steps.coverage.outputs.coverage_pct }}% |" >> $GITHUB_STEP_SUMMARY | ||
| echo "| Target Branch Coverage | ${{ steps.baseline.outputs.baseline_coverage }}% |" >> $GITHUB_STEP_SUMMARY | ||
|
|
||
| CURRENT_COV="${{ steps.coverage.outputs.coverage_pct }}" | ||
| BASELINE_COV="${{ steps.baseline.outputs.baseline_coverage }}" | ||
|
|
||
| # Use awk for comparison (handles empty/zero baseline gracefully) | ||
| if [ -z "$BASELINE_COV" ] || [ "$BASELINE_COV" == "0" ]; then | ||
| echo "| Status | ✅ PASSED (Baseline Established) |" >> $GITHUB_STEP_SUMMARY | ||
| elif awk "BEGIN {exit !($CURRENT_COV >= $BASELINE_COV)}"; then | ||
| echo "| Status | ✅ PASSED |" >> $GITHUB_STEP_SUMMARY | ||
| else | ||
| echo "| Status | ❌ FAILED |" >> $GITHUB_STEP_SUMMARY | ||
| fi | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
|
|
||
| # Add detailed coverage report (only if .coverage file exists) | ||
| if [ -f .coverage ]; then | ||
| echo "### Detailed Coverage Report" >> $GITHUB_STEP_SUMMARY | ||
| echo '```' >> $GITHUB_STEP_SUMMARY | ||
| python -m coverage report >> $GITHUB_STEP_SUMMARY || echo "Coverage report generation failed" >> $GITHUB_STEP_SUMMARY | ||
| echo '```' >> $GITHUB_STEP_SUMMARY | ||
| else | ||
| echo "### Detailed Coverage Report" >> $GITHUB_STEP_SUMMARY | ||
| echo "Coverage data file not found. Report not available." >> $GITHUB_STEP_SUMMARY | ||
| fi | ||
|
|
||
| - name: Upload Test Results | ||
| uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: test-results-${{ github.run_number }} | ||
| path: | | ||
| test-results.xml | ||
| test-report.html | ||
| htmlcov/ | ||
| coverage.xml | ||
| coverage.json | ||
| retention-days: 30 | ||
|
|
||
| - name: Upload Coverage Reports | ||
| uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: coverage-report-${{ github.run_number }} | ||
| path: | | ||
| htmlcov/ | ||
| coverage.xml | ||
| coverage.json | ||
| retention-days: 30 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| [flake8] | ||
| max-line-length = 140 | ||
| extend-ignore = | ||
| # F403: 'from module import *' used; unable to detect undefined names | ||
| # F405: name may be undefined, or defined from star imports | ||
| # Acceptable for ctypes bindings where wildcard imports are idiomatic | ||
| F403, | ||
| F405, | ||
| # E266: too many leading '#' for block comment | ||
| # Used for visual section separators | ||
| E266 | ||
| per-file-ignores = | ||
| # Allow module-level imports not at top of file in examples | ||
| source/examples/pyzes_example.py:E402 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we have the ability to Copyright/License this file, please do if .yml supports comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added