diff --git a/.github/prompts/ml/generate-pr-summary.prompt.md b/.github/prompts/ml/generate-pr-summary.prompt.md new file mode 100644 index 000000000000..92a2bd536d5b --- /dev/null +++ b/.github/prompts/ml/generate-pr-summary.prompt.md @@ -0,0 +1,60 @@ +--- +agent: agent +description: Create a PR summary for azure-ai-ml changes in the current branch +name: generate-pr-summary +--- + +Analyze changes in the `sdk/ml/azure-ai-ml` package ONLY. Check both committed changes (if any) and uncommitted changes. Generate a clear PR summary highlighting key features, bug fixes, type safety improvements, and other significant changes for reviewers. Focus exclusively on azure-ai-ml - ignore all other packages. + +## Git Commands for Gathering Information + +Use these commands to analyze azure-ai-ml changes: + +```powershell +# Check uncommitted changes in azure-ai-ml (staged and unstaged) +git status -- sdk/ml/azure-ai-ml/ +git diff --name-status -- sdk/ml/azure-ai-ml/ +git diff --cached --name-status -- sdk/ml/azure-ai-ml/ + +# Get committed changes in azure-ai-ml compared to main +git diff main..HEAD --name-status -- sdk/ml/azure-ai-ml/ + +# Get commit messages for azure-ai-ml changes +git log main..HEAD --oneline -- sdk/ml/azure-ai-ml/ + +# Get change statistics for azure-ai-ml only +git diff --shortstat -- sdk/ml/azure-ai-ml/ +git diff main..HEAD --shortstat -- sdk/ml/azure-ai-ml/ + +# View sample diff for analysis +git diff -- sdk/ml/azure-ai-ml/ | Select-Object -First 100 +``` + +**Priority**: If uncommitted changes exist in azure-ai-ml, use those for the summary. Otherwise use committed changes. + +## PR Summary Template + +Keep the summary concise and focused: + +```markdown +## PR Summary + +**Title:** [Brief description of the changes] + +[Brief 1-2 sentence description of what changed and why] + +**Changes:** +- [Key change 1] +- [Key change 2] +- [Key change 3] + +**Files:** [N] files in [affected areas: operations/entities/tests/docs] + +**Testing:** [Brief status - e.g., "tests passing, mypy/pylint compliant"] +``` + +### Guidelines + +- **Title**: Brief description in lowercase, imperative mood, no period +- **Keep it brief**: 3-5 bullet points max, focus on what matters +- **No tables**: Simple list format only \ No newline at end of file diff --git a/.github/skills/ml/do-code-review/SKILL.md b/.github/skills/ml/do-code-review/SKILL.md new file mode 100644 index 000000000000..93787f310a8e --- /dev/null +++ b/.github/skills/ml/do-code-review/SKILL.md @@ -0,0 +1,311 @@ +--- +name: do-code-review +description: Reviews code changes in azure-ai-ml package for quality, Azure SDK compliance, and best practices. Use when reviewing code, checking pull requests, or when user asks to review changes or check code quality in azure-ai-ml. +--- + +# Azure AI ML Code Review + +Reviews uncommitted changes (staged and unstaged files) in the azure-ai-ml package, focusing on Azure SDK Python design guidelines, type safety, testing patterns, and API consistency. + +## Default Review Scope + +Unless otherwise specified, review all uncommitted changes in the current branch (staged and unstaged files) within `sdk/ml/azure-ai-ml/`. This includes new files, modified files, and any pending changes that haven't been committed yet. + +## Review Focus Areas + +### 1. Azure SDK Design Guidelines Compliance + +- **Check**: Adherence to [Azure SDK Python Design Guidelines](https://azure.github.io/azure-sdk/python_design.html) +- **Look for**: Proper client naming, method patterns, parameter order +- **Flag**: Non-compliant naming (use `create_or_update` not `create_or_replace`) +- **Example Issue**: `def get_job(name, subscription_id)` should be `def get_job(name, **kwargs)` + +**Key Patterns:** +- Client methods: `begin_*` for LROs, `list_*` for paginators +- Naming: snake_case for methods, PascalCase for classes +- Parameters: Required positional, optional keyword-only +- Return types: Explicit type hints for all public APIs + +### 2. Type Annotations & MyPy Compliance + +- **Check**: Complete type annotations on all public APIs +- **Look for**: Proper use of `Optional`, `Union`, `TYPE_CHECKING` +- **Flag**: Missing return types, `Any` without justification, bare `dict`/`list` +- **Example Issue**: `def process_data(data)` should be `def process_data(data: Dict[str, Any]) -> ProcessedData` + +**Common Fixes:** +```python +# Bad +def get_config(name): + return config + +# Good +def get_config(name: str) -> Optional[Dict[str, Any]]: + return config +``` + +### 3. Pylint Compliance + +- **Check**: Code passes pylint with azure-sdk-for-python rules +- **Look for**: Proper docstrings, no unused imports, correct argument names +- **Flag**: Violations of naming conventions, too many arguments (>5), long lines (>120) +- **Reference**: [Azure Pylint Guidelines](https://github.com/Azure/azure-sdk-tools/blob/main/tools/pylint-extensions/azure-pylint-guidelines-checker/README.md) + +**Watch for:** +- `client-method-should-not-use-static-method` +- `missing-client-constructor-parameter-credential` +- `client-method-has-more-than-5-positional-arguments` + +### 4. Async/Await Patterns + +- **Check**: Proper async implementation in `_async` modules +- **Look for**: Using `async with` for clients, awaiting coroutines correctly +- **Flag**: Blocking calls in async code, missing `await`, sync code in async modules +- **Example Issue**: `self._client.get()` in async should be `await self._client.get()` + +**Pattern:** +```python +# In azure/ai/ml/aio/operations/ +async def create_or_update( + self, + entity: Job, + **kwargs: Any +) -> Job: + async with self._lock: + result = await self._service_client.create_or_update(...) + return result +``` + +### 5. Error Handling & Validation + +- **Check**: Proper exception handling with Azure SDK exceptions +- **Look for**: Use of `HttpResponseError`, `ResourceNotFoundError`, proper validation +- **Flag**: Bare `except:`, catching `Exception` without re-raising, missing validation +- **Example Issue**: Missing parameter validation before API calls + +**Pattern:** +```python +from azure.core.exceptions import ResourceNotFoundError, HttpResponseError + +try: + result = self._operation.get(name) +except ResourceNotFoundError: + raise ResourceNotFoundError(f"Job '{name}' not found") +except HttpResponseError as e: + raise HttpResponseError(f"Failed to retrieve job: {e.message}") +``` + +### 6. API Design & Operations + +- **Check**: Consistent CRUD patterns across operations classes +- **Look for**: Proper separation of sync/async, operations returning correct types +- **Flag**: Business logic in client, missing operations class, inconsistent method names + +**Structure:** +``` +azure/ai/ml/ +├── operations/ # Sync operations +│ ├── job_operations.py +│ └── model_operations.py +└── aio/operations/ # Async operations (mirror structure) + ├── job_operations.py + └── model_operations.py +``` + +### 7. Entity & Schema Patterns + +- **Check**: Proper use of marshmallow schemas, correct entity inheritance +- **Look for**: Schema validation, proper serialization/deserialization +- **Flag**: Direct dict manipulation instead of entities, missing schema validation + +**Entity Pattern:** +```python +@dataclass +class Job(Resource): + """Job entity.""" + + name: str + experiment_name: Optional[str] = None + + def _to_rest_object(self) -> RestJob: + """Convert to REST representation.""" + ... + + @classmethod + def _from_rest_object(cls, obj: RestJob) -> "Job": + """Create from REST representation.""" + ... +``` + +### 8. Testing Patterns + +- **Check**: Proper unit tests, recorded tests for operations +- **Look for**: Use of `pytest`, proper test isolation, fixture usage +- **Flag**: Missing tests for new features, tests with external dependencies, hardcoded values + +**Test Structure:** +```python +class TestJobOperations: + """Test job operations.""" + + def test_create_job(self, client: MLClient, mock_workspace: Mock) -> None: + """Test job creation.""" + job = Job(name="test-job") + result = client.jobs.create_or_update(job) + assert result.name == "test-job" + + @pytest.mark.recorded + def test_get_job_recorded(self, client: MLClient) -> None: + """Test getting job with recording.""" + ... +``` + +### 9. Documentation & Docstrings + +- **Check**: Complete docstrings following Google/NumPy style +- **Look for**: Parameter descriptions, return types, examples, raises +- **Flag**: Missing docstrings on public APIs, incomplete parameter docs + +**Docstring Pattern:** +```python +def create_or_update( + self, + job: Job, + **kwargs: Any +) -> Job: + """Create or update a job. + + :param job: The job entity to create or update. + :type job: ~azure.ai.ml.entities.Job + :keyword bool skip_validation: Skip validation of the job. + :return: The created or updated job. + :rtype: ~azure.ai.ml.entities.Job + :raises ~azure.core.exceptions.HttpResponseError: If the request fails. + + .. admonition:: Example: + + .. code-block:: python + + from azure.ai.ml.entities import Job + job = Job(name="my-job") + result = ml_client.jobs.create_or_update(job) + """ +``` + +### 10. Backwards Compatibility + +- **Check**: No breaking changes without major version bump +- **Look for**: Deprecated parameters, migration paths, version notes +- **Flag**: Removing public methods, changing signatures, removing parameters + +**Deprecation Pattern:** +```python +import warnings + +def old_method(self, param: str) -> None: + """Deprecated method. + + .. deprecated:: 1.2.0 + Use :meth:`new_method` instead. + """ + warnings.warn( + "old_method is deprecated, use new_method instead", + DeprecationWarning, + stacklevel=2 + ) + self.new_method(param) +``` + +### 11. Security & Credentials + +- **Check**: Proper credential handling, no secrets in logs +- **Look for**: Use of `TokenCredential`, proper token refresh, sanitized logging +- **Flag**: Credentials in error messages, API keys in code, secrets in tests + +**Pattern:** +```python +from azure.core.credentials import TokenCredential + +class MLClient: + def __init__( + self, + credential: TokenCredential, + subscription_id: str, + **kwargs: Any + ): + self._credential = credential # Store, don't log + # Never log credential or tokens +``` + +### 12. Performance & Efficiency + +- **Check**: Efficient API calls, proper pagination, lazy loading +- **Look for**: Batching operations, caching where appropriate, avoiding N+1 queries +- **Flag**: Loading all items in memory, multiple API calls in loops, no pagination + +**Pagination Pattern:** +```python +def list(self, **kwargs: Any) -> Iterable[Job]: + """List jobs with pagination. + + :return: An iterable of jobs. + :rtype: ~azure.core.paging.ItemPaged[~azure.ai.ml.entities.Job] + """ + return self._operation.list(...) # Returns ItemPaged +``` + +## Analysis Instructions + +1. **Get uncommitted changes**: Use git to identify modified files in `sdk/ml/azure-ai-ml/` +2. **Read changed sections**: Focus on modified lines and surrounding context +3. **Check each focus area**: Go through all 12 areas systematically +4. **Priority levels**: Critical (breaking/security) > High (bugs/types) > Medium (style/docs) +5. **Provide specific examples**: Show actual code with file paths and line numbers +6. **Cross-reference**: Check consistency across sync/async, operations/entities + +## Output Format + +Organize findings by priority and category: + +### ✅ Positive Observations +Good patterns worth highlighting + +### 🔴 Critical Issues +- Breaking changes without migration path +- Missing credential validation +- Type safety violations causing runtime errors +- Security vulnerabilities + +### ⚠️ High Priority Issues +- Missing type annotations on public APIs +- Pylint/MyPy errors +- Missing tests for new functionality +- Improper async patterns + +### 📋 Medium Priority Issues +- Missing or incomplete docstrings +- Code style inconsistencies +- Performance optimizations +- Better error messages + +### 💡 Suggestions +- Refactoring opportunities +- Additional test coverage +- Documentation improvements + +For each issue: +1. **Location**: File path and line numbers +2. **Current code**: Show the problematic code +3. **Issue**: Explain what's wrong and why +4. **Recommended fix**: Show corrected code +5. **References**: Link to relevant guidelines + +### Summary + +- Total files changed: X +- Critical issues: X +- High priority: X +- Medium priority: X +- Overall assessment: Ready/Needs work/Blocked + +Focus on issues that impact SDK quality, user experience, backwards compatibility, and Azure SDK guideline compliance. \ No newline at end of file diff --git a/.github/skills/ml/fix-mypy/SKILL.md b/.github/skills/ml/fix-mypy/SKILL.md new file mode 100644 index 000000000000..a248c7bc2d70 --- /dev/null +++ b/.github/skills/ml/fix-mypy/SKILL.md @@ -0,0 +1,389 @@ +--- +name: fix-mypy +description: Automatically fix mypy type checking issues in azure-ai-ml package following Azure SDK Python patterns. Expects GitHub issue URL and optional virtual env path in the request. Format "fix mypy issue [using venv ]" +--- + +# Fix MyPy Issues Skill + +This skill automatically fixes mypy type checking errors in the azure-ai-ml package by analyzing existing code patterns and applying fixes with 100% confidence based on GitHub issues. + +## Overview + +Intelligently fixes mypy issues by: +1. Getting the GitHub issue URL from the user +2. Reading and analyzing the issue details +3. Setting up or using existing virtual environment +4. Installing required dependencies +5. Running mypy on the specific files/areas mentioned in the issue +6. Analyzing the mypy output to identify type errors +7. Searching codebase for existing type annotation patterns +8. Applying fixes only with 100% confidence +9. Re-running mypy to verify fixes +10. Creating a pull request that references the GitHub issue +11. Providing a summary of what was fixed + +## Running MyPy + +**Command for entire package:** +```powershell +cd sdk/ml/azure-ai-ml +tox -e mypy --c ../../../eng/tox/tox.ini --root . +``` + +**Command for specific file/module:** +```powershell +cd sdk/ml/azure-ai-ml +tox -e mypy --c ../../../eng/tox/tox.ini --root . -- path/to/file.py +``` + +## Reference Documentation + +- [Azure SDK Python MyPy Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/static_type_checking_cheat_sheet.md) +- [Tox Guidance](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/tests.md#tox) +- [Official MyPy Documentation](https://mypy.readthedocs.io/en/stable/) +- [MyPy Common Issues](https://mypy.readthedocs.io/en/stable/common_issues.html) + +## Fixing Strategy + +### Step 0: Get GitHub Issue Details + +**Check if user provided in their request:** +- GitHub issue URL (look for `https://github.com/Azure/azure-sdk-for-python/issues/...` in user's message) +- Virtual environment path (look for phrases like "using venv", "use env", "virtual environment at", or just the venv name) + +**If GitHub issue URL is missing:** +Ask: "Please provide the GitHub issue URL for the mypy type checking problems you want to fix." + +**If virtual environment is missing:** +Ask: "Do you have an existing virtual environment path, or should I create 'env'?" + +**Once you have the issue URL:** +Read the issue to understand which files/modules and specific error codes to fix. + +### Step 1: CRITICAL - Activate Virtual Environment FIRST + +**IMMEDIATELY activate the virtual environment before ANY other command:** + +```powershell +# Activate the provided virtual environment (e.g., envml, env, venv) +.\\Scripts\Activate.ps1 + +# If creating new virtual environment (Python 3.9+): +python -m venv env +.\env\Scripts\Activate.ps1 +``` + +**⚠️ IMPORTANT: ALL subsequent commands MUST run within the activated virtual environment. Never run commands outside the venv.** + +### Step 2: Install Dependencies (within activated venv) + +```powershell +# Navigate to azure-ai-ml directory (within activated venv) +cd sdk/ml/azure-ai-ml + +# Install dev dependencies from dev_requirements.txt (within activated venv) +pip install -r dev_requirements.txt + +# Install the package in editable mode (within activated venv) +pip install -e . +``` + +**Important:** Use Python 3.9 compatible environment for mypy checks. + +### Step 3: Identify Target Files (within activated venv) + +Based on the GitHub issue details, determine which files to check: + +**Option A - Issue specifies files:** +```powershell +# Ensure you're in azure-ai-ml directory (within activated venv) +cd sdk/ml/azure-ai-ml + +# Run mypy on specific files mentioned in the issue (within activated venv) +tox -e mypy --c ../../../eng/tox/tox.ini --root . -- path/to/specific_file.py +``` + +**Option B - Issue mentions module/directory:** +```powershell +# Run mypy on specific module (within activated venv) +cd sdk/ml/azure-ai-ml +tox -e mypy --c ../../../eng/tox/tox.ini --root . -- azure/ai/ml/specific_module/ +``` + +**Option C - Check modified files (if no specific target):** +```powershell +git diff --name-only HEAD | Select-String "sdk/ml/azure-ai-ml" +git diff --cached --name-only | Select-String "sdk/ml/azure-ai-ml" +``` + +### Step 4: Run MyPy (within activated venv) + +**⚠️ Ensure virtual environment is still activated before running:** + +```powershell +# Navigate to azure-ai-ml directory +cd sdk/ml/azure-ai-ml + +# Run mypy targeting the specific area from the issue (within activated venv) +tox -e mypy --c ../../../eng/tox/tox.ini --root . -- +``` + +### Step 5: Analyze Type Errors + +Parse the mypy output to identify: +- Error type and code (e.g., [arg-type], [return-value], [assignment]) +- File path and line number +- Specific error description +- Expected vs actual types +- **Cross-reference with the GitHub issue** to ensure you're fixing the right problems + +### Step 6: Search for Existing Type Annotation Patterns + +Before fixing, search the codebase for how similar types are annotated: +```powershell +# Example: Search for similar function signatures +grep -r "def similar_function" sdk/ml/azure-ai-ml/ -A 5 + +# Search for type imports +grep -r "from typing import" sdk/ml/azure-ai-ml/ +``` + +Use the existing type annotation patterns to ensure consistency. + +### Step 7: Apply Fixes (ONLY if 100% confident) + +**ALLOWED ACTIONS:** + Fix type errors with 100% confidence + Use existing type annotation patterns as reference + Follow Azure SDK Python type checking guidelines + Add missing type hints + Fix incorrect type annotations + Make minimal, targeted changes + +**FORBIDDEN ACTIONS:** + Fix errors without complete confidence + Create new files for solutions + Import non-existent types or modules + Add new dependencies or imports outside typing module + Use `# type: ignore` without clear justification + Change code logic to avoid type errors + Delete code without clear justification + +### Step 7: Verify Fixes + +Re-run mypy to ensure: +- The type error is resolved +- No new errors were introduced +- The code still functions correctly + +### Step 8: Summary + +Provide a summary: +- GitHub issue being addressed +- Number of type errors fixed +- Number of errors remaining +- Types of fixes applied (e.g., added type hints, fixed return types) +- Any errors that need manual review + +### Step 9: Create Pull Request + +After successfully fixing mypy issues, create a pull request: + +**Stage and commit the changes:** +```powershell +# Stage all modified files +git add . + +# Create a descriptive commit message referencing the issue +git commit -m "fix(azure-ai-ml): resolve mypy type checking errors (#) + +- Fixed +- Added type hints to +- All mypy checks now pass + +Closes #" +``` + +**Create pull request using GitHub CLI or MCP server:** + +Option 1 - Using GitHub CLI (if available): +```powershell +# Create a new branch +$branchName = "fix/azure-ai-ml-mypy-" +git checkout -b $branchName + +# Push the branch +git push origin $branchName + +# Create PR using gh CLI +gh pr create ` + --title "fix(azure-ai-ml): Resolve mypy type checking errors (#)" ` + --body "## Description +This PR fixes mypy type checking errors in the azure-ai-ml package as reported in #. + +## Changes +- Fixed mypy type checking errors following Azure SDK Python guidelines +- Ensured consistency with existing type annotation patterns +- Targeted fixes for: +- All mypy checks now pass for the affected areas + +## Testing +- [x] Ran mypy on affected files and verified all errors are resolved +- [x] No new errors introduced +- [x] Verified fixes follow existing patterns +- [x] Used Python 3.9 compatible type hints + +## Related Issues +Fixes #" ` + --base main ` + --repo Azure/azure-sdk-for-python +``` + +Option 2 - Manual PR creation (if GitHub CLI not available): +1. Push branch: `git push origin ` +2. Navigate to: https://github.com/Azure/azure-sdk-for-python/compare/main... +3. Create the pull request manually with the description above + +Option 3 - Using GitHub MCP server (if available): +Use the GitHub MCP tools to create a pull request programmatically against the Azure/azure-sdk-for-python repository, main branch. + +## Common MyPy Issues and Fixes + +### Missing Type Hints + +**Error:** `Function is missing a type annotation` + +**Fix:** Add proper type hints: +```python +from typing import Optional + +def function_name(param: str, optional_param: Optional[int] = None) -> None: + """Brief description.""" + pass +``` + +### Argument Type Mismatch + +**Error:** `Argument 1 to "function" has incompatible type "X"; expected "Y"` + +**Fix:** Ensure the argument matches the expected type or add proper type conversion: +```python +# If expecting str but passing int +value: str = str(int_value) + +# Or fix the function signature if the type is incorrect +def function(param: Union[str, int]) -> None: + pass +``` + +### Return Type Mismatch + +**Error:** `Incompatible return value type (got "X", expected "Y")` + +**Fix:** Ensure the return type matches the annotation: +```python +from typing import Optional + +def get_value() -> Optional[str]: + if condition: + return "value" + return None # Not empty string if Optional +``` + +### Type Annotation with Optional + +**Error:** `Item "None" of "Optional[X]" has no attribute "Y"` + +**Fix:** Add None check before accessing attributes: +```python +from typing import Optional + +def process(value: Optional[str]) -> str: + if value is None: + return "" + return value.upper() +``` + +### Union Types + +**Error:** `Argument has incompatible type` + +**Fix:** Use Union for multiple acceptable types: +```python +from typing import Union + +def function(param: Union[str, int, None]) -> str: + if param is None: + return "" + return str(param) +``` + +### List/Dict Type Annotations + +**Error:** `Need type annotation for variable` + +**Fix:** Add specific type annotations for collections: +```python +from typing import List, Dict, Any + +items: List[str] = [] +config: Dict[str, Any] = {} +``` + +## Example Workflow + +```powershell +# 0. Get issue details +# User provides: https://github.com/Azure/azure-sdk-for-python/issues/67890 +# Issue mentions: mypy errors in azure/ai/ml/entities/job.py + +# 1. CRITICAL - Activate virtual environment FIRST +.\\Scripts\Activate.ps1 # Use the venv name provided by user +cd sdk/ml/azure-ai-ml +pip install -r dev_requirements.txt +pip install -e . + +# 2. Identify target from issue +$targetFile = "azure/ai/ml/operations/job_operations.py" + +# 3. Run mypy on specific file +tox -e mypy --c ../../../eng/tox/tox.ini --root . -- $targetFile + +# 4. Analyze output and identify fixable issues +# Cross-reference with GitHub issue #12345 + +# 5. Search for existing type annotation patterns +grep -r "from typing import" azure/ai/ml/ | findstr "operations" + +# 6. Apply fixes to identified files + +# 7. Re-run mypy to verify +tox -e mypy --c ../../../eng/tox/tox.ini --root . -- $targetFile + +# 8. Report results + +# 9. Create PR referencing the issue +$branchName = "fix/azure-ai-ml-mypy-12345" +git checkout -b $branchName +git add . +git commit -m "fix(azure-ai-ml): resolve mypy type checking errors (#12345) + +Closes #12345" +git push origin $branchName +gh pr create ` + --title "fix(azure-ai-ml): Resolve mypy type checking errors (#12345)" ` + --body "Fixes #12345" ` + --base main ` + --repo Azure/azure-sdk-for-python +``` + +## Notes + +- Always read the existing code to understand type annotation patterns before making changes +- Prefer following existing patterns over adding new complex types +- Use Python 3.9+ compatible type hints (use `Optional[X]` instead of `X | None`) +- If unsure about a fix, mark it for manual review +- Some errors may require architectural changes - don't force fixes +- Test the code after fixing to ensure functionality is preserved +- Always reference the GitHub issue in commits and PRs +- Avoid using `# type: ignore` unless absolutely necessary and document why diff --git a/.github/skills/ml/fix-pylint/SKILL.md b/.github/skills/ml/fix-pylint/SKILL.md new file mode 100644 index 000000000000..0e0b9f2ebc21 --- /dev/null +++ b/.github/skills/ml/fix-pylint/SKILL.md @@ -0,0 +1,336 @@ +--- +name: fix-pylint +description: Automatically fix pylint issues in azure-ai-ml package following Azure SDK Python guidelines and existing code patterns. Expects GitHub issue URL and optional virtual env path in the request. Format "fix pylint issue [using venv ]" +--- + +# Fix Pylint Issues Skill + +This skill automatically fixes pylint warnings in the azure-ai-ml package by analyzing existing code patterns and applying fixes with 100% confidence based on GitHub issues. + +## Overview + +Intelligently fixes pylint issues by: +1. Getting the GitHub issue URL from the user +2. Reading and analyzing the issue details +3. Setting up or using existing virtual environment +4. Installing required dependencies +5. Running pylint on the specific files/areas mentioned in the issue +6. Analyzing the pylint output to identify warnings +7. Searching codebase for existing patterns to follow +8. Applying fixes only with 100% confidence +9. Re-running pylint to verify fixes +10. Creating a pull request that references the GitHub issue +11. Providing a summary of what was fixed + +## Running Pylint + +**Command for entire package:** +```powershell +cd sdk/ml/azure-ai-ml +tox -e pylint --c ../../../eng/tox/tox.ini --root . +``` + +**Command for specific file/module:** +```powershell +cd sdk/ml/azure-ai-ml +tox -e pylint --c ../../../eng/tox/tox.ini --root . -- path/to/file.py +``` + +## Reference Documentation + +- [Azure SDK Python Pylint Guidelines](https://github.com/Azure/azure-sdk-tools/blob/main/tools/pylint-extensions/azure-pylint-guidelines-checker/README.md) +- [Official Pylint Documentation](https://pylint.readthedocs.io/en/stable/user_guide/checkers/features.html) +- [Azure SDK Python Pylint Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/pylint_checking.md) +- [Tox Formatting Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/tests.md#tox) + +## Fixing Strategy + +### Step 0: Get GitHub Issue Details + +**Check if user provided in their request:** +- GitHub issue URL (look for `https://github.com/Azure/azure-sdk-for-python/issues/...` in user's message) +- Virtual environment path (look for phrases like "using venv", "use env", "virtual environment at", or just the venv name) + +**If GitHub issue URL is missing:** +Ask: "Please provide the GitHub issue URL for the pylint problems you want to fix." + +**If virtual environment is missing:** +Ask: "Do you have an existing virtual environment path, or should I create 'env'?" + +**Once you have the issue URL:** +Read the issue to understand which files/modules and specific warnings to fix. + +### Step 1: CRITICAL - Activate Virtual Environment FIRST + +**IMMEDIATELY activate the virtual environment before ANY other command:** + +```powershell +# Activate the provided virtual environment (e.g., envml, env, venv) +.\\Scripts\Activate.ps1 + +# If creating new virtual environment: +python -m venv env +.\env\Scripts\Activate.ps1 +``` + +**⚠️ IMPORTANT: ALL subsequent commands MUST run within the activated virtual environment. Never run commands outside the venv.** + +### Step 2: Install Dependencies (within activated venv) + +```powershell +# Navigate to azure-ai-ml directory (within activated venv) +cd sdk/ml/azure-ai-ml + +# Install dev dependencies from dev_requirements.txt (within activated venv) +pip install -r dev_requirements.txt + +# Install the package in editable mode (within activated venv) +pip install -e . +``` + +### Step 3: Identify Target Files (within activated venv) + +Based on the GitHub issue details, determine which files to check: + +**Option A - Issue specifies files:** +```powershell +# Ensure you're in azure-ai-ml directory (within activated venv) +cd sdk/ml/azure-ai-ml + +# Run pylint on specific files mentioned in the issue (within activated venv) +tox -e pylint --c ../../../eng/tox/tox.ini --root . -- path/to/specific_file.py +``` + +**Option B - Issue mentions module/directory:** +```powershell +# Run pylint on specific module (within activated venv) +cd sdk/ml/azure-ai-ml +tox -e pylint --c ../../../eng/tox/tox.ini --root . -- azure/ai/ml/specific_module/ +``` + +**Option C - Check modified files (if no specific target):** +```powershell +git diff --name-only HEAD | Select-String "sdk/ml/azure-ai-ml" +git diff --cached --name-only | Select-String "sdk/ml/azure-ai-ml" +``` + +### Step 4: Run Pylint (within activated venv) + +**⚠️ Ensure virtual environment is still activated before running:** + +```powershell +# Navigate to azure-ai-ml directory +cd sdk/ml/azure-ai-ml + +# Run pylint targeting the specific area from the issue (within activated venv) +tox -e pylint --c ../../../eng/tox/tox.ini --root . -- +``` + +### Step 5: Analyze Warnings + +Parse the pylint output to identify: +- Warning type and code (e.g., C0103, W0212, R0913) +- File path and line number +- Specific issue description +- **Cross-reference with the GitHub issue** to ensure you're fixing the right problems + +### Step 6: Search for Existing Patterns + +Before fixing, search the codebase for how similar issues are handled: +```powershell +# Example: Search for similar function patterns +grep -r "pattern" sdk/ml/azure-ai-ml/ +``` + +Use the existing code patterns to ensure consistency. + +### Step 7: Apply Fixes (ONLY if 100% confident) + +**ALLOWED ACTIONS:** + Fix warnings with 100% confidence + Use existing file patterns as reference + Follow Azure SDK Python guidelines + Make minimal, targeted changes + +**FORBIDDEN ACTIONS:** + Fix warnings without complete confidence + Create new files for solutions + Import non-existent modules + Add new dependencies or imports + Make unnecessary large changes + Change code style without clear reason + Delete code without clear justification + +### Step 7: Verify Fixes + +Re-run pylint to ensure: +- The warning is resolved +- No new warnings were introduced +- The code still functions correctly + +### Step 8: Summary + +Provide a summary: +- GitHub issue being addressed +- Number of warnings fixed +- Number of warnings remaining +- Types of fixes applied +- Any warnings that need manual review + +### Step 9: Create Pull Request + +After successfully fixing pylint issues, create a pull request: + +**Stage and commit the changes:** +```powershell +# Stage all modified files +git add . + +# Create a descriptive commit message referencing the issue +git commit -m "fix(azure-ai-ml): resolve pylint warnings (#) + +- Fixed +- Updated +- All pylint checks now pass + +Closes #" +``` + +**Create pull request using GitHub CLI or MCP server:** + +Option 1 - Using GitHub CLI (if available): +```powershell +# Create a new branch +$branchName = "fix/azure-ai-ml-pylint-" +git checkout -b $branchName + +# Push the branch +git push origin $branchName + +# Create PR using gh CLI +gh pr create ` + --title "fix(azure-ai-ml): Resolve pylint warnings (#)" ` + --body "## Description +This PR fixes pylint warnings in the azure-ai-ml package as reported in #. + +## Changes +- Fixed pylint warnings following Azure SDK Python guidelines +- Ensured consistency with existing code patterns +- Targeted fixes for: +- All pylint checks now pass for the affected areas + +## Testing +- [x] Ran pylint on affected files and verified all warnings are resolved +- [x] No new warnings introduced +- [x] Verified fixes follow existing code patterns + +## Related Issues +Fixes #" ` + --base main ` + --repo Azure/azure-sdk-for-python +``` + +Option 2 - Manual PR creation (if GitHub CLI not available): +1. Push branch: `git push origin ` +2. Navigate to: https://github.com/Azure/azure-sdk-for-python/compare/main... +3. Create the pull request manually with the description above + +Option 3 - Using GitHub MCP server (if available): +Use the GitHub MCP tools to create a pull request programmatically against the Azure/azure-sdk-for-python repository, main branch. + +## Common Pylint Issues and Fixes + +### Missing Docstrings + +**Warning:** `C0111: Missing module/function/class docstring` + +**Fix:** Add proper docstring following Azure SDK style: +```python +def function_name(param: str) -> None: + """Brief description of function. + + :param param: Description of parameter + :type param: str + :return: None + """ +``` + +### Too Many Arguments + +**Warning:** `R0913: Too many arguments` + +**Fix:** Consider grouping related parameters into a configuration object or using keyword-only arguments. + +### Line Too Long + +**Warning:** `C0301: Line too long` + +**Fix:** Break line at logical points following PEP 8 style. + +### Unused Import + +**Warning:** `W0611: Unused import` + +**Fix:** Remove the unused import statement. + +### Invalid Name + +**Warning:** `C0103: Invalid name` + +**Fix:** Rename to follow snake_case for functions/variables, PascalCase for classes. + +## Example Workflow + +```powershell +# 0. Get issue details +# User provides: https://github.com/Azure/azure-sdk-for-python/issues/12345 +# Issue mentions: pylint warnings in azure/ai/ml/operations/job_operations.py + +# 1. CRITICAL - Activate virtual environment FIRST +.\\Scripts\Activate.ps1 # Use the venv name provided by user +cd sdk/ml/azure-ai-ml +pip install -r dev_requirements.txt +pip install -e . + +# 2. Identify target from issue +$targetFile = "azure/ai/ml/operations/job_operations.py" + +# 3. Run pylint on specific file +tox -e pylint --c ../../../eng/tox/tox.ini --root . -- $targetFile + +# 4. Analyze output and identify fixable issues +# Cross-reference with GitHub issue #12345 + +# 5. Search for existing patterns in codebase +grep -r "similar_pattern" azure/ai/ml/ + +# 6. Apply fixes to identified files + +# 7. Re-run pylint to verify +tox -e pylint --c ../../../eng/tox/tox.ini --root . -- $targetFile + +# 8. Report results + +# 9. Create PR referencing the issue +$branchName = "fix/azure-ai-ml-pylint-12345" +git checkout -b $branchName +git add . +git commit -m "fix(azure-ai-ml): resolve pylint warnings (#12345) + +Closes #12345" +git push origin $branchName +gh pr create ` + --title "fix(azure-ai-ml): Resolve pylint warnings (#12345)" ` + --body "Fixes #12345" ` + --base main ` + --repo Azure/azure-sdk-for-python +``` + +## Notes + +- Always read the existing code to understand patterns before making changes +- Prefer following existing patterns over strict rule compliance +- If unsure about a fix, mark it for manual review +- Some warnings may require architectural changes - don't force fixes +- Test the code after fixing to ensure functionality is preserved +- Always reference the GitHub issue in commits and PRs diff --git a/.vscode/cspell.json b/.vscode/cspell.json index a42733f5cf0f..0a3313eff763 100644 --- a/.vscode/cspell.json +++ b/.vscode/cspell.json @@ -2190,6 +2190,10 @@ { "filename": "sdk/ai/azure-ai-voicelive/**", "words": ["viseme","VISEME","ulaw","ULAW","logprobs","pyaudio","PyAudio","libasound","webrtc","WEBRTC"] + }, + { + "filename": ".github/**/ml/**", + "words": ["envml", "paginators"] } ], "allowCompoundWords": true diff --git a/eng/ignore-links.txt b/eng/ignore-links.txt index ac740a7f9314..0dd024dbec83 100644 --- a/eng/ignore-links.txt +++ b/eng/ignore-links.txt @@ -5,3 +5,4 @@ https://github.com/Azure/azure-rest-api-specs-pr http://localhost:8000/samples/pyodide_integration https://pypi.org/project/azure-mynewpackage/ https://aka.ms/azsdk/python/migrate/my-new-package +https://github.com/Azure/azure-sdk-for-python/compare/main..