-
Notifications
You must be signed in to change notification settings - Fork 277
Speed up -Expr, -SumExpr, -ProdExpr and -Constant
#1179
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Refactors the __neg__ method in the Expr class to use Cython's PyDict_Next and PyDict_SetItem for more efficient negation of terms, replacing the previous Python dict comprehension.
Introduces a copy method to GenExpr for duplicating expression objects, with support for deep or shallow copying. Also implements the __neg__ method for ProdExpr to allow negation of product expressions by negating their constant term.
Added explicit return type annotations to the __neg__ methods in Expr and ProdExpr classes, and updated the corresponding type hints in scip.pyi. This improves type checking and code clarity.
Refactors SumExpr to use cpython.array for storing coefficients instead of Python lists, improving performance and memory efficiency. Adds a __neg__ method for SumExpr to efficiently negate coefficients and the constant term. Updates the copy method to properly clone arrays when copying SumExpr instances.
Introduces the test_neg function to verify correct behavior when negating ProdExpr and SumExpr objects in the expression API. Ensures that negated expressions have the expected types, string representations, and coefficients.
Added an entry noting the speedup of `Expr.__neg__`, `ProdExpr.__neg__`, and `Constant.__neg__` using the C-level API.
The @disjoint_base decorator was removed from the UnaryExpr class in the type stub, possibly to correct or update the class hierarchy or decorator usage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR aims to speed up expression negation by moving __neg__ implementations to faster C-level loops for core expression types.
Changes:
- Optimized
Expr.__neg__by iterating the underlying term dictionary using CPython C-API helpers. - Added specialized
__neg__implementations forSumExprandProdExpr(including newGenExpr.copyhelper). - Added a new unit test covering negation behavior for
ProdExprandSumExpr, and refined type stubs for__neg__.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
tests/test_expr.py |
Adds a test_neg test validating negation results/types for product and sum expressions. |
src/pyscipopt/scip.pyi |
Tightens typing for __neg__ on Expr and GenExpr. |
src/pyscipopt/expr.pxi |
Implements faster negation paths and introduces GenExpr.copy; changes SumExpr.coefs storage to array('d'). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Changed the type of 'coefs' from list to memoryview (double[:]) in SumExpr._evaluate for more efficient access during evaluation.
Refactors the negation method in SumExpr to correctly create a new instance, negate the constant, copy children, and set the operator. This ensures proper behavior when negating sum expressions.
Implemented the __neg__ method for the Constant class, allowing unary negation of constant expressions.
Imported Constant from pyscipopt.scip and added an assertion to test the string representation of the negated Constant expression in test_neg.
-Expr and -ProdExpr and -Constant-Expr, SumExpr, -ProdExpr and -Constant
-Expr, SumExpr, -ProdExpr and -Constant-Expr, -SumExpr, -ProdExpr and -Constant
Added assertions to test_neg to verify correct negation and string representation of expressions involving powers. This enhances test coverage for expression negation logic.
Clarified that `SumExpr.__neg__` is also sped up via the C-level API, in addition to other negation methods.
src/pyscipopt/expr.pxi
Outdated
| def __init__(self): | ||
| self.constant = 0.0 | ||
| self.coefs = [] | ||
| self.coefs = array("d") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Joao-Dionisio Could you have a loop about this problem? SumExpr.coefs has a bug, so this CI failed.
If SumExpr.coefs is not [1, 1], the result from expr_to_array to addCons won't be right.
All GenExpr will use the expr_to_array function to add to SCIP. But expr_to_array doesn't have any code to handle SumExpr.coefs. SumExpr.coefs doesn't work at all.
PySCIPOpt/src/pyscipopt/expr.pxi
Lines 829 to 849 in bc24757
| def expr_to_array(expr, nodes): | |
| """adds expression to array""" | |
| op = expr._op | |
| if op == Operator.const: # FIXME: constant expr should also have children! | |
| nodes.append(tuple([op, [expr.number]])) | |
| elif op != Operator.varidx: | |
| indices = [] | |
| nchildren = len(expr.children) | |
| for child in expr.children: | |
| pos = expr_to_array(child, nodes) # position of child in the final array of nodes, 'nodes' | |
| indices.append(pos) | |
| if op == Operator.power: | |
| pos = value_to_array(expr.expo, nodes) | |
| indices.append(pos) | |
| elif (op == Operator.add and expr.constant != 0.0) or (op == Operator.prod and expr.constant != 1.0): | |
| pos = value_to_array(expr.constant, nodes) | |
| indices.append(pos) | |
| nodes.append( tuple( [op, indices] ) ) | |
| else: # var | |
| nodes.append( tuple( [op, expr.children] ) ) | |
| return len(nodes) - 1 |
Two solutions to fix this.
- Remove
SumExpr.coefs. Any coefficient to aSumExprwill be aProdExpr. And each element coefficient should be 1. We follow current behavior. So,SumExpr.coefsrelated codes could be removed. - Let
expr_to_arraysupportSumExpr.coefs.
Replaces usage of cpython.array for storing coefficients in SumExpr with standard Python lists. Simplifies code by removing array-specific imports and clone operations, improving maintainability and compatibility.
Removed the unused GenExpr.copy() method and refactored the __neg__ methods for SumExpr and ProdExpr to avoid using the copy method. This simplifies the code and clarifies object construction during negation.
Applied the @disjoint_base decorator to the UnaryExpr class in scip.pyi to clarify its role in the type hierarchy.
Use a C-level to speed up
Expr.__neg__,ProdExpr.__neg, andConstant.__neg__.Expris 1.3x faster than before,SumExpris 2.1x faster than before, andProdExpris 4.9x faster than before.-Exprtime: 0.51s-SumExprtime: 1.38s-ProdExprtime: 1.39s-Exprtime: 0.39s-SumExprtime: 0.64s-ProdExprtime: 0.28s