fix: Java E2E pipeline — direct JVM benchmarking, JUnit detection, and instrumentation fixes#1580
Open
mashraf-222 wants to merge 35 commits intoomni-javafrom
Open
fix: Java E2E pipeline — direct JVM benchmarking, JUnit detection, and instrumentation fixes#1580mashraf-222 wants to merge 35 commits intoomni-javafrom
mashraf-222 wants to merge 35 commits intoomni-javafrom
Conversation
- Check dependencyManagement section in pom.xml for test dependencies - Recursively check submodule pom.xml files (test, tests, etc.) - Change default fallback from JUnit 5 to JUnit 4 (more common in legacy) - Add debug logging for framework detection decisions - Fixes Bug #7: 64% of optimizations blocked by incorrect JUnit 5 detection
- Add cache dict to avoid repeated rglob calls for same test files - Cache both positive and negative results - Significantly reduces file system traversals during benchmark parsing - Partially addresses Bug #2 (still need to filter irrelevant test cases)
- Add detection for cast expressions, ternary, array access, etc. - Skip instrumentation when method call is inside complex expression - Prevents syntax errors when instrumenting tests with casts like (Long)list.get(2) - Addresses Bug #6: instrumentation breaking complex Java expressions
- Detect JUnit 4 vs JUnit 5 and use appropriate runner (JUnitCore vs ConsoleLauncher) - Include all module target/classes in classpath for multi-module projects - Add stderr logging for debugging when direct execution fails - Fixes Bug #3: Direct JVM now works, avoiding slow Maven fallback (~0.3s vs ~5-10s)
…culation Bug #10: Timing marker sum was 0 because perf_stdout was never set for Java tests. The timing markers were being parsed correctly but the raw stdout containing them was not stored in TestResults.perf_stdout, causing calculate_function_throughput_from_test_results to return 0 and skip all optimizations. This fix ensures the subprocess stdout is preserved in perf_stdout field for Java performance tests, allowing throughput calculation to work correctly.
The instrumented Java test code was storing "{class_name}Test" as the
test_function_name in SQLite instead of the actual test method name
(e.g., "testAdd"). This fixes parity with Python instrumentation.
- Add _extract_test_method_name() with compiled regex patterns
- Inject _cf_test variable with actual method name in behavior code
- Fix setString(3, ...) to use _cf_test instead of hardcoded class name
- Optimize _byte_to_line_index() with bisect.bisect_right()
- Update all behavior mode test expectations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Direct JVM execution with ConsoleLauncher was always failing because junit-platform-console-standalone is not included in the standard junit-jupiter dependency tree. The _get_test_classpath() function now finds and adds the console standalone JAR from ~/.m2, downloading it via Maven if needed. This enables direct JVM test execution for JUnit 5 projects, avoiding the Maven overhead (~500ms vs ~5-10s per invocation) and Surefire configuration issues (e.g., custom <includes> that ignore -Dtest). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TestConfig.test_framework was an uncached @Property that called _detect_java_test_framework() -> detect_java_project() -> _detect_test_deps_from_pom() (parses pom.xml) on every access. During test result parsing, this was accessed once per testcase, causing 300K+ redundant pom.xml parses and massive debug log spam. Cache the result after first detection using _test_framework field. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…s probing The previous detection ran `java -cp ... JUnitCore -version` to check for JUnit 4, but JUnit 5 projects include JUnit 4 classes via junit-vintage-engine, causing false positive detection. This made direct JVM execution always fail and fall back to Maven. Now checks for JUnit 5 JAR names (junit-jupiter, junit-platform, console-standalone) in the classpath string instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Check dependencyManagement section in pom.xml for test dependencies - Recursively check submodule pom.xml files (test, tests, etc.) - Change default fallback from JUnit 5 to JUnit 4 (more common in legacy) - Add debug logging for framework detection decisions - Fixes Bug #7: 64% of optimizations blocked by incorrect JUnit 5 detection
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![codeflash-ai[bot]](https://avatars.githubusercontent.com/in/413177?s=60&v=4){kind=small}
| _FALLBACK_METHOD_PATTERN = re.compile(r"\b(\w+)\s*\(") | ||
|
|
||
|
|
||
| def _extract_test_method_name(method_lines: list[str]) -> str: |
Contributor
There was a problem hiding this comment.
⚡️Codeflash found 58% (0.58x) speedup for _extract_test_method_name in codeflash/languages/java/instrumentation.py
⏱️ Runtime : 11.1 milliseconds → 6.98 milliseconds (best of 165 runs)
📝 Explanation and details
This optimization achieves a 58% runtime improvement (from 11.1ms to 6.98ms) by exploiting a common-case optimization: searching for method signatures line-by-line before falling back to the expensive string join operation.
Key Changes:
-
Early line-by-line search: The optimized code first iterates through individual lines, attempting to match
_METHOD_SIG_PATTERNon each line separately. This allows the function to return immediately when a method signature is found on a single line. -
Deferred string joining: The expensive
" ".join(method_lines).strip()operation is only performed if the line-by-line search fails to find a match. This saves both string allocation and regex execution time in the common case.
Why This Is Faster:
-
String joining is expensive: Creating a single concatenated string from multiple lines involves memory allocation and copying. The line profiler shows the original code spent 4.3% of time just on the join operation.
-
Regex on smaller strings is faster: Running the regex on individual lines (typically short) is much faster than running it on one large joined string. The optimized version processes 5,579 individual line searches (8.2ms total) versus 1,070 searches on joined strings (11.4ms in original).
-
Early exit opportunity: For method signatures that appear on a single line (the common case), the optimized code returns immediately after finding the match, skipping the join entirely.
Performance Characteristics:
The annotated tests reveal the optimization excels when:
- Method signatures are on single lines (8-13% faster): Most real-world Java methods have their signature on one line
- Large input lists with early matches (100-300% faster): When the signature appears early in a large list, the line-by-line search finds it quickly without processing remaining lines or joining them
- Multiple valid signatures (311% faster for 1000 repetitions): Early exit prevents unnecessary work
The optimization performs slightly worse when:
- Signatures span multiple lines (50-75% slower): The line-by-line search fails, then falls back to the join approach, adding overhead
- Fallback pattern is needed (17-40% slower): Similar double-work scenario when primary pattern fails
- Input is all invalid (87% slower for 1000 empty strings): Must scan all lines before falling back
However, the runtime metric shows these edge cases are rare in practice—the overall 58% speedup indicates the common case (single-line signatures) dominates real workloads.
✅ Correctness verification report:
| Test | Status |
|---|---|
| ⚙️ Existing Unit Tests | 🔘 None Found |
| 🌀 Generated Regression Tests | ✅ 1070 Passed |
| ⏪ Replay Tests | 🔘 None Found |
| 🔎 Concolic Coverage Tests | 🔘 None Found |
| 📊 Tests Coverage | 100.0% |
🌀 Click to see Generated Regression Tests
import pytest # used for our unit tests
from codeflash.languages.java.instrumentation import _extract_test_method_name
def test_basic_public_void_signature():
# A simple, common Java test method signature should extract the method name.
method_lines = ["public void testSomething() {"]
# Call the function with a typical signature and assert the expected name.
codeflash_output = _extract_test_method_name(method_lines) # 3.24μs -> 2.88μs (12.5% faster)
def test_modifiers_and_return_type_with_parameters():
# Signatures with multiple modifiers and a primitive return type should work.
method_lines = ["private static final int computeSum(int a, int b) throws Exception {"]
# The regex captures the method name "computeSum".
codeflash_output = _extract_test_method_name(method_lines) # 3.15μs -> 2.90μs (8.67% faster)
def test_array_return_type_and_annotation_above():
# An annotation line followed by a protected array return type should still match.
method_lines = ["@Override", "protected String[] getArray() {"]
# Verify the array-returning method name is correctly extracted.
codeflash_output = _extract_test_method_name(method_lines) # 8.01μs -> 4.90μs (63.4% faster)
def test_generic_return_type_falls_back_to_simple_pattern():
# Generic return types (e.g., List<String>) contain '<' and won't match the primary pattern.
# The fallback pattern should still find the method name.
method_lines = ["public List<String> listMethod() {"]
codeflash_output = _extract_test_method_name(method_lines) # 10.7μs -> 16.0μs (33.1% slower)
def test_no_modifiers_only_name_and_params():
# A minimal signature with only the method name and parentheses should be caught by fallback.
method_lines = ["doWork()"]
codeflash_output = _extract_test_method_name(method_lines) # 3.92μs -> 4.75μs (17.5% slower)
def test_underscore_and_digits_in_method_name():
# Method names can include underscores and digits; ensure such names are handled.
method_lines = ["int _do_42() {"]
codeflash_output = _extract_test_method_name(method_lines) # 2.87μs -> 2.61μs (9.60% faster)
def test_empty_input_returns_unknown():
# An empty list yields an empty string after joining; neither regex will match.
method_lines: list[str] = []
codeflash_output = _extract_test_method_name(method_lines) # 1.23μs -> 1.20μs (2.41% faster)
def test_whitespace_only_returns_unknown():
# Input that is only whitespace should also return "unknown".
method_lines = [" ", "\t"]
codeflash_output = _extract_test_method_name(method_lines) # 1.36μs -> 1.67μs (18.5% slower)
def test_missing_parentheses_returns_unknown():
# If there is no '(' in the text, the fallback cannot match, so result is "unknown".
method_lines = ["public void noParen"]
codeflash_output = _extract_test_method_name(method_lines) # 12.9μs -> 20.5μs (36.9% slower)
def test_multiple_methods_first_is_taken():
# If multiple method signatures are present, the first match should be returned.
method_lines = ["public void first() { } public void second() { }"]
# Expect the first method name, not the second.
codeflash_output = _extract_test_method_name(method_lines) # 3.02μs -> 2.71μs (11.4% faster)
def test_multidimensional_array_return_uses_fallback():
# The primary pattern supports a single [] only; int[][] won't match it.
# The fallback should still capture the method name.
method_lines = ["public int[][] multiArray() {"]
codeflash_output = _extract_test_method_name(method_lines) # 9.65μs -> 14.1μs (31.7% slower)
def test_method_signature_split_across_lines():
# Signatures split across multiple lines should be joined and still match.
method_lines = ["public", "void", "splitAcrossLines(", ") {"]
# Despite splitting, the join produces a single string that the primary regex can parse.
codeflash_output = _extract_test_method_name(method_lines) # 3.46μs -> 7.44μs (53.6% slower)
def test_large_input_many_filler_lines_and_one_signature():
# Build 1000 filler lines deterministically to test scalability.
filler = [f"int fillerVar{i};" for i in range(499)]
# Insert a target signature in the middle.
target_signature = ["public boolean largeScaleMethod(int x) {"]
filler2 = [f"// comment line {i}" for i in range(499)]
method_lines = filler + target_signature + filler2
# The function should find the method name that was placed in the middle.
codeflash_output = _extract_test_method_name(method_lines) # 2.01ms -> 1.97ms (1.66% faster)
def test_many_calls_loop_for_determinism_and_performance():
# Call the function 1000 times with predictable varying method names to ensure determinism.
for i in range(1000):
# Construct a unique but deterministic signature for each iteration.
name = f"repeatedMethod_{i}"
lines = [f"public void {name}() {{"] # simple signature per iteration
# Each call must return the exact method name we encoded.
codeflash_output = _extract_test_method_name(lines) # 788μs -> 781μs (0.864% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.import pytest
from codeflash.languages.java.instrumentation import _extract_test_method_name
def test_basic_public_method():
"""Test extraction of a simple public method name."""
method_lines = ["public void testExample() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.62μs -> 3.29μs (10.1% faster)
def test_basic_private_method():
"""Test extraction of a simple private method name."""
method_lines = ["private void testHelper() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.22μs -> 2.88μs (11.5% faster)
def test_basic_protected_method():
"""Test extraction of a simple protected method name."""
method_lines = ["protected void testProtected() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.33μs -> 2.99μs (11.4% faster)
def test_static_method():
"""Test extraction of a static method name."""
method_lines = ["public static void testStatic() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.16μs -> 2.81μs (12.1% faster)
def test_final_method():
"""Test extraction of a final method name."""
method_lines = ["public final void testFinal() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.90μs -> 2.83μs (2.48% faster)
def test_static_final_method():
"""Test extraction of a static final method name."""
method_lines = ["public static final void testStaticFinal() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.10μs -> 2.74μs (13.1% faster)
def test_method_with_parameters():
"""Test extraction of method name when method has parameters."""
method_lines = ["public void testWithParams(String arg1, int arg2) {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.99μs -> 2.76μs (8.35% faster)
def test_method_with_string_return_type():
"""Test extraction of method name with String return type."""
method_lines = ["public String getTestName() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.93μs -> 2.79μs (5.06% faster)
def test_method_with_int_return_type():
"""Test extraction of method name with int return type."""
method_lines = ["public int calculateValue() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.92μs -> 2.74μs (6.91% faster)
def test_method_with_boolean_return_type():
"""Test extraction of method name with boolean return type."""
method_lines = ["public boolean isValid() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.94μs -> 2.71μs (8.87% faster)
def test_method_with_custom_return_type():
"""Test extraction of method name with custom return type."""
method_lines = ["public MyClass testCustom() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.15μs -> 2.96μs (6.10% faster)
def test_method_with_array_return_type():
"""Test extraction of method name with array return type."""
method_lines = ["public int[] testArray() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.53μs -> 3.22μs (9.67% faster)
def test_method_with_multiple_spaces():
"""Test extraction when method signature has multiple spaces."""
method_lines = ["public void testMultiSpace() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.98μs -> 2.79μs (6.86% faster)
def test_method_split_across_lines():
"""Test extraction when method signature is split across multiple lines."""
method_lines = ["public void", "testMultiLine(", "String arg) {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.33μs -> 12.8μs (74.0% slower)
def test_method_with_no_visibility_modifier():
"""Test extraction of method without visibility modifier."""
method_lines = ["void testNoModifier() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.91μs -> 2.56μs (13.3% faster)
def test_method_with_leading_whitespace():
"""Test extraction when method line has leading whitespace."""
method_lines = [" public void testIndented() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.10μs -> 2.87μs (8.06% faster)
def test_method_with_trailing_whitespace():
"""Test extraction when method line has trailing whitespace."""
method_lines = ["public void testTrailing() "]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.03μs -> 2.69μs (12.3% faster)
def test_method_with_underscore_in_name():
"""Test extraction of method with underscore in name."""
method_lines = ["public void test_method_name() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.05μs -> 2.81μs (8.59% faster)
def test_method_with_numbers_in_name():
"""Test extraction of method with numbers in name."""
method_lines = ["public void test123Method456() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.99μs -> 2.79μs (7.22% faster)
def test_method_with_long_parameter_list():
"""Test extraction when method has many parameters."""
method_lines = ["public void testLongParams(String a, int b, boolean c, double d, float e) {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.88μs -> 2.76μs (4.32% faster)
def test_method_name_only_lowercase():
"""Test extraction of method with all lowercase name."""
method_lines = ["public void testmethod() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.96μs -> 2.75μs (7.69% faster)
def test_method_name_mixed_case():
"""Test extraction of method with mixed case name."""
method_lines = ["public void TeStMeThOd() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.94μs -> 2.71μs (8.54% faster)
def test_empty_method_lines_list():
"""Test extraction from empty method lines list."""
method_lines = []
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 1.26μs -> 1.29μs (2.40% slower)
def test_single_empty_string():
"""Test extraction from list with single empty string."""
method_lines = [""]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 1.14μs -> 1.37μs (16.8% slower)
def test_method_lines_with_only_whitespace():
"""Test extraction from lines containing only whitespace."""
method_lines = [" ", " ", " "]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 1.37μs -> 1.85μs (25.9% slower)
def test_no_parentheses():
"""Test extraction when method line has no parentheses."""
method_lines = ["public void testNoParens"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 15.9μs -> 26.0μs (38.9% slower)
def test_no_method_signature():
"""Test extraction when line doesn't contain valid method signature."""
method_lines = ["This is just random text without method"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 18.9μs -> 31.8μs (40.7% slower)
def test_fallback_pattern_simple_case():
"""Test fallback pattern when primary pattern doesn't match."""
method_lines = ["someMethod(arg) {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 5.17μs -> 7.21μs (28.3% slower)
def test_fallback_pattern_with_spaces():
"""Test fallback pattern with spaces before parentheses."""
method_lines = ["someFunction (param)"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 7.07μs -> 10.6μs (33.4% slower)
def test_method_line_with_comments():
"""Test extraction when method line contains comments."""
method_lines = ["public void testWithComment() { // this is a comment"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.05μs -> 2.83μs (7.44% faster)
def test_method_with_throws_clause():
"""Test extraction when method has throws clause."""
method_lines = ["public void testThrows() throws IOException {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.95μs -> 2.65μs (11.0% faster)
def test_method_with_annotation_on_same_line():
"""Test extraction when annotation is on same line as method."""
method_lines = ["@Test public void testAnnotated() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 6.45μs -> 6.16μs (4.72% faster)
def test_very_long_method_name():
"""Test extraction with very long method name."""
long_name = "a" * 1000
method_lines = [f"public void {long_name}() {{"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 5.52μs -> 5.33μs (3.58% faster)
def test_method_name_with_dollar_sign():
"""Test extraction when method name contains dollar sign."""
method_lines = ["public void test$Method() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 12.9μs -> 20.2μs (36.3% slower)
def test_multiple_method_signatures_in_line():
"""Test extraction when line contains multiple method patterns."""
method_lines = ["void method1() { void method2() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.96μs -> 2.67μs (11.3% faster)
def test_method_with_double_array():
"""Test extraction with double array return type."""
method_lines = ["public int[][] testDoubleArray() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 10.1μs -> 15.2μs (33.8% slower)
def test_method_with_tab_characters():
"""Test extraction when method uses tabs instead of spaces."""
method_lines = ["public\tvoid\ttestTabs() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.85μs -> 2.75μs (3.63% faster)
def test_method_with_newlines_in_list():
"""Test extraction with newlines distributed across list elements."""
method_lines = [
"public",
"void",
"testNewlines",
"(",
")",
"{"
]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.25μs -> 7.51μs (56.8% slower)
def test_fallback_with_underscore_prefix():
"""Test fallback pattern with underscore-prefixed method name."""
method_lines = ["_privateHelper(x) {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 5.14μs -> 7.26μs (29.3% slower)
def test_empty_parameter_list():
"""Test method with empty parameter list."""
method_lines = ["public void testEmpty() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.11μs -> 2.77μs (12.3% faster)
def test_method_with_generic_return_type():
"""Test extraction with generic return type (treated as custom type)."""
method_lines = ["public List testGeneric() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.20μs -> 2.92μs (9.60% faster)
def test_multiple_visibility_keywords():
"""Test line with multiple visibility keywords (malformed)."""
method_lines = ["public private void testBadFormat() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 8.66μs -> 8.53μs (1.51% faster)
def test_method_line_with_javadoc_marker():
"""Test extraction when line has javadoc end marker."""
method_lines = ["*/ public void testAfterJavadoc() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.09μs -> 2.85μs (8.47% faster)
def test_single_character_method_name():
"""Test extraction with single character method name."""
method_lines = ["public void x() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.86μs -> 2.79μs (2.55% faster)
def test_method_name_all_caps():
"""Test extraction with all uppercase method name."""
method_lines = ["public void TEST() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 3.02μs -> 2.73μs (10.3% faster)
def test_very_large_method_lines_list():
"""Test extraction with very large number of lines."""
# Create a list with 1000 lines, method definition at the beginning
method_lines = ["public void testLargeList() {"] + [" // line of code" for _ in range(999)]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 11.0μs -> 2.71μs (304% faster)
def test_very_long_joined_string():
"""Test extraction when lines are joined into a very long string."""
# Create 1000 lines that will be joined
method_lines = ["public void testLongJoin("] + [f"arg{i}," for i in range(999)] + ["arg999) {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 11.4μs -> 2.92μs (290% faster)
def test_method_with_many_parameters():
"""Test extraction from method with large number of parameters."""
params = ", ".join([f"param{i}" for i in range(100)])
method_lines = [f"public void testManyParams({params}) {{"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.94μs -> 2.75μs (7.29% faster)
def test_repeated_pattern_in_method_lines():
"""Test extraction when similar patterns are repeated many times."""
method_lines = ["public void foo() {"] * 1000
# Should match the first occurrence
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 10.8μs -> 2.62μs (311% faster)
def test_very_long_return_type():
"""Test with very long custom return type name."""
long_type = "VeryLongCustomTypeNameWith" + "X" * 900
method_lines = [f"public {long_type} testLongType() {{"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 5.48μs -> 5.15μs (6.45% faster)
def test_large_list_with_method_at_end():
"""Test extraction when method definition appears at end of large list."""
method_lines = ["random text"] * 999 + ["public void testAtEnd() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 4.19ms -> 2.07ms (102% faster)
def test_large_list_with_method_in_middle():
"""Test extraction when method definition is in middle of large list."""
method_lines = ["random text"] * 500 + ["public void testInMiddle() {"] + ["random text"] * 499
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 2.10ms -> 1.03ms (104% faster)
def test_thousand_element_list_all_empty():
"""Test extraction from list with 1000 empty strings."""
method_lines = [""] * 1000
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 8.06μs -> 65.5μs (87.7% slower)
def test_thousand_different_method_names():
"""Test extraction correctly identifies first method among many patterns."""
method_lines = [f"void method{i}() {{" for i in range(1000)]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 11.4μs -> 2.90μs (292% faster)
def test_pattern_at_very_end_of_very_long_string():
"""Test extraction when pattern appears at the very end of a large joined string."""
# Create lines that when joined will have method signature at the very end
method_lines = ["x"] * 999 + ["public void testAtEnd() {"]
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 950μs -> 60.6μs (1468% faster)
def test_alternating_valid_invalid_lines():
"""Test extraction from alternating valid and invalid lines."""
method_lines = []
for i in range(500):
method_lines.append("invalid line")
method_lines.append(f"public void method{i}() {{")
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 17.1μs -> 6.63μs (158% faster)
def test_many_false_positive_patterns():
"""Test extraction when many similar patterns exist but most are malformed."""
method_lines = []
# Add many lines with word( pattern but no valid method signature
for i in range(500):
method_lines.append(f"word{i}(param)")
# Add valid method at end
method_lines.append("public void testValid() {")
codeflash_output = _extract_test_method_name(method_lines); result = codeflash_output # 687μs -> 644μs (6.62% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.To test or edit this optimization locally git merge codeflash/optimize-pr1580-2026-02-20T06.12.37
Suggested change
| def _extract_test_method_name(method_lines: list[str]) -> str: | |
| def _extract_test_method_name(method_lines: list[str]) -> str: | |
| for line in method_lines: | |
| match = _METHOD_SIG_PATTERN.search(line) | |
| if match: | |
| return match.group(1) |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Comment on lines
248
to
253
| # Check common submodule locations | ||
| for submodule_name in ["test", "tests", "src/test", "testing"]: | ||
| submodule_pom = project_root / submodule_name / "pom.xml" | ||
| if submodule_pom.exists(): | ||
| logger.debug(f"Checking submodule pom at {submodule_pom}") | ||
| sub_junit5, sub_junit4, sub_testng = _detect_test_deps_from_pom(project_root / submodule_name) |
Contributor
There was a problem hiding this comment.
✅ Resolved — the recursion is now bounded to specific hardcoded directory names (["test", "tests", "src/test", "testing"]) with early break on first match. Practical recursion depth is at most 1 level.
Contributor
PR Review SummaryPrek ChecksPassed — Mypy523 errors across 42 files — these are largely pre-existing issues in the codebase, not introduced by this PR. Many stem from the new Java support module lacking type stubs and from existing untyped patterns across the codebase. Code ReviewThis is a substantial PR adding Java JUnit 4/5 support improvements, complex expression handling in instrumentation, and multiple bug fixes. Outstanding issues from previous reviews (still apply to latest code):
Bug fix verified: Test Coverage35 test failures observed (10 tracer tests, 2 Java e2e tests, others). Java e2e failures are expected due to Maven dependency resolution issues in CI. Tracer test failures appear pre-existing. New Files (added in this PR)
Modified Files
Optimization PRs4 open codeflash optimization PRs targeting this branch (#1584, #1586, #1594, #1595) — all have CI failures, none eligible for merge. Last updated: 2026-02-20T10:00Z |
… with vintage engine ConsoleLauncher runs both JUnit 4 (via vintage engine) and JUnit 5 tests. The detection now correctly distinguishes between JUnit 5 projects (have junit-jupiter on classpath) and JUnit 4 projects using ConsoleLauncher as the runner. Previously, the injected console-standalone JAR falsely triggered "JUnit 5 detected" for all projects. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
⚡️ Codeflash found optimizations for this PR📄 16% (0.16x) speedup for
|
The main optimization here is eliminating the `max(0, idx)` call by handling the edge case directly. Since `bisect_right` returns 0 when `byte_offset` is less than all elements, subtracting 1 gives -1, which we can catch with a simple comparison. This avoids the function call overhead of `max()`.
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Convert f-string logging to lazy % formatting (G004) and replace try-except-pass with contextlib.suppress (SIM105). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment on lines
698
to
722
| if is_junit4: | ||
| # Use JUnit 4's JUnitCore runner | ||
| cmd = [ | ||
| str(java), | ||
| # Java 16+ module system: Kryo needs reflective access to internal JDK classes | ||
| "--add-opens", | ||
| "java.base/java.util=ALL-UNNAMED", | ||
| "--add-opens", | ||
| "java.base/java.lang=ALL-UNNAMED", | ||
| "--add-opens", | ||
| "java.base/java.lang.reflect=ALL-UNNAMED", | ||
| "--add-opens", | ||
| "java.base/java.io=ALL-UNNAMED", | ||
| "--add-opens", | ||
| "java.base/java.math=ALL-UNNAMED", | ||
| "--add-opens", | ||
| "java.base/java.net=ALL-UNNAMED", | ||
| "--add-opens", | ||
| "java.base/java.util.zip=ALL-UNNAMED", | ||
| "-cp", | ||
| classpath, | ||
| "org.junit.runner.JUnitCore", | ||
| ] | ||
| # Add test classes | ||
| cmd.extend(test_classes) |
Contributor
There was a problem hiding this comment.
Bug: The JUnit 4 / JUnitCore path does not handle reports_dir — XML reports will silently not be generated for JUnit 4 projects using JUnitCore. The ConsoleLauncher branch (line 758) correctly uses --reports-dir, but the JUnitCore branch ignores it entirely. If downstream code relies on XML reports from reports_dir for parsing test results, this will silently fail.
| has_console_launcher = "console-standalone" in classpath or "ConsoleLauncher" in classpath | ||
| # Use ConsoleLauncher if available (works for both JUnit 4 via vintage and JUnit 5). | ||
| # Only use JUnitCore when ConsoleLauncher is not on the classpath at all. | ||
| is_junit4 = not has_console_launcher |
Contributor
There was a problem hiding this comment.
✅ Resolved in latest commit — the variable now has extensive inline documentation explaining the logic (lines 679-689), making the intent clear despite the name.
Comment on lines
+121
to
+131
| if current.type in { | ||
| "cast_expression", | ||
| "ternary_expression", | ||
| "array_access", | ||
| "binary_expression", | ||
| "unary_expression", | ||
| "parenthesized_expression", | ||
| "instanceof_expression", | ||
| }: | ||
| logger.debug("Found complex expression parent: %s", current.type) | ||
| return True |
Contributor
There was a problem hiding this comment.
Potential over-filtering: Including parenthesized_expression in the complex expression set may skip valid instrumentation targets. In Java tree-sitter, a simple (func()) is parsed as a parenthesized_expression. Patterns like assertTrue((calculator.add(1, 2)) > 0) would cause the target call to be silently skipped for instrumentation. Consider whether parenthesized_expression should be excluded from this check, or add a depth-limited check that only triggers for parenthesized expressions that are themselves inside another complex expression.
| String _cf_outputFile1 = System.getenv("CODEFLASH_OUTPUT_FILE"); | ||
| String _cf_testIteration1 = System.getenv("CODEFLASH_TEST_ITERATION"); | ||
| if (_cf_testIteration1 == null) _cf_testIteration1 = "0"; | ||
| String _cf_test1 = "testNegativeInput_ThrowsIllegalArgumentException"; |
Contributor
There was a problem hiding this comment.
i still think this identifier isn't enough. we need a unique identifier that if we see a given marker, we know a completely unique identity for that run. so uniqueness for a test invocation is defined by
test_file -> test_class -> test_function -> test_line_node ->run_counter.
This is largely how python instrumentation works. currently i see that the module name isn't unique (since it lacks the __perfinstrumented section - it may be fine to skip this but then look at what the python instrumentation does for a reference implementation).
There are other things we will fix shortly, the line numbers (needed for runtime annotations on the tests we display on PRs). will test more
Contributor
|
we would also need the same test function name in the performance instrumentation |
The optimization achieves a **17% runtime improvement** (from 1.05ms to 894μs) by caching the `current.type` attribute access in a local variable (`t` or `current_type`) inside the loop. This seemingly small change reduces repeated attribute lookups on the same object during each iteration. **What Changed:** Instead of accessing `current.type` twice per iteration (once for each conditional check), the optimized version stores it in a local variable and reuses that value. This transforms two attribute lookups into one per iteration. **Why This Improves Performance:** In Python, attribute access involves dictionary lookups in the object's `__dict__`, which carries overhead. By caching the attribute value in a local variable, the code performs this lookup once per iteration instead of twice. Local variable access in Python is significantly faster than attribute access because it's a simple array index operation at the bytecode level (LOAD_FAST) versus a dictionary lookup (LOAD_ATTR). **Key Performance Characteristics:** The line profiler shows the optimization is particularly effective for the common case where both conditions need to be checked. The time spent on the two conditional checks decreased from 28% + 23.4% = 51.4% of total time to 22.4% + 15.3% = 37.7%, demonstrating measurable savings from the reduced attribute access overhead. **Test Case Performance:** - The optimization shows the most significant gains in **large-scale traversal scenarios** (1000-node chains), with 4-5% speedups in `test_long_chain_with_lambda_at_top_large_scale` and `test_long_chain_with_method_declaration_earlier_large_scale` - Shorter chains show slight regressions (1-6% slower) in individual test cases, likely due to measurement noise and the overhead of the additional variable assignment being more noticeable in very short executions - The overall **17% improvement** across the full workload confirms the optimization is beneficial when amortized across realistic usage patterns with varying tree depths This optimization is particularly valuable when traversing deep AST structures, where the function may iterate many times before finding a lambda or method declaration, making the cumulative savings from reduced attribute access substantial.
Contributor
⚡️ Codeflash found optimizations for this PR📄 17% (0.17x) speedup for
|
This optimization achieves a **15% runtime improvement** (10.2ms → 8.81ms) by replacing recursive AST traversal with iterative stack-based traversal in two critical functions: `collect_test_methods` and `collect_target_calls`. ## Key Changes **1. Iterative AST Traversal (Primary Speedup)** - Replaced recursive tree walking with explicit stack-based iteration - In `collect_test_methods`: Changed from recursive calls to `while stack` loop with `stack.extend(reversed(current.children))` - In `collect_target_calls`: Similar transformation using explicit stack management - **Impact**: Line profiler shows `collect_test_methods` dropped from 24.2% to 3.8% of total runtime (81% reduction in that function) **2. Why This Works in Python** - Python function calls have significant overhead (frame creation, argument binding, scope setup) - Recursive traversal compounds this overhead across potentially deep AST trees - Iterative approach uses a simple list for the stack, avoiding repeated function call overhead - The `reversed()` call ensures children are processed in the same order as recursive traversal, preserving correctness **3. Performance Characteristics** Based on annotated tests: - **Large method bodies** (500+ lines): 23.8% faster - most benefit from reduced recursion overhead - **Many test methods** (100 methods): 9.2% faster - cumulative savings across many traversals - **Simple cases**: 2-5% faster - overhead reduction still measurable - **Empty/no-match cases**: Minor regression (8-9% slower) due to negligible baseline times (12-40μs) ## Impact on Workloads The function references show `_add_timing_instrumentation` is called from test instrumentation code. This optimization particularly benefits: - **Java projects with large test suites** containing many `@Test` methods - **Complex test methods** with deep AST structures and multiple method invocations - **Batch instrumentation operations** where the function is called repeatedly The iterative approach scales better than recursion as AST depth and method count increase, making it especially valuable for large Java codebases where instrumentation is applied across hundreds of test methods.
Contributor
⚡️ Codeflash found optimizations for this PR📄 16% (0.16x) speedup for
|
Contributor
|
in behavior instrumentation we are still not printing the test class +fn name in stdout markers
|
Contributor
⚡️ Codeflash found optimizations for this PR📄 17% (0.17x) speedup for
|
The optimized code achieves a **196% speedup** (from 13.3ms to 4.49ms) primarily through two focused optimizations that target the hottest paths identified by the line profiler:
## Key Optimizations
### 1. Early Exit in `wrap_target_calls_with_treesitter` (Primary Driver)
The profiler shows that in the original code, 55.5% of `wrap_target_calls_with_treesitter`'s time (9.7ms out of 17.5ms) was spent in `_collect_calls`, which parses Java code with tree-sitter. The optimization adds:
```python
body_text = "\n".join(body_lines)
if func_name not in body_text:
return list(body_lines), 0
```
This simple string membership check avoids expensive tree-sitter parsing when the target function isn't present in the test method body. Since many test methods don't call the function being instrumented, this provides massive savings. The annotated tests confirm this pattern - tests with empty or simple bodies (no function calls) show the largest speedups: 639% for large methods and 1018% for complex expressions.
### 2. Optimized `_is_test_annotation` (Secondary Improvement)
The profiler shows `_is_test_annotation` being called 1,950 times, spending 100% of its time (1.21ms) on regex matching. The optimization replaces the regex with direct string checks:
```python
if not stripped_line.startswith("@test"):
return False
if len(stripped_line) == 5: # exactly "@test"
return True
next_char = stripped_line[5]
return next_char == " " or next_char == "("
```
This avoids regex overhead for the 1,737 non-`@Test` annotations that can be rejected immediately with `startswith()`. The profiler shows this reduced time from 1.21ms to 0.91ms (25% faster in this function).
## Performance Impact by Test Type
The annotated tests reveal optimization effectiveness varies by workload:
- **Empty/simple methods**: 107-154% faster (early exit dominates)
- **Methods with complex expressions**: 396-1018% faster (avoids parsing large expression trees)
- **Large methods with many statements**: 510-639% faster (early exit + reduced AST traversal)
- **Methods with actual function calls**: 111-152% faster (smaller benefit since tree-sitter must run)
## Context and Production Impact
Based on `function_references`, this function is called from test discovery in `test_instrumentation.py`, specifically for behavior instrumentation that captures return values. The early exit optimization is particularly valuable here because:
1. Test discovery processes many test methods, but typically only a subset call the target function
2. The function operates on the hot path during test suite instrumentation
3. Large test suites with 100+ test methods (see test case showing 154% speedup for 150 methods) benefit significantly
The optimization maintains correctness - all test cases pass with identical output, confirming the early exit safely bypasses work that produces no changes when the function isn't present.
Contributor
⚡️ Codeflash found optimizations for this PR📄 197% (1.97x) speedup for
|
…2026-02-20T10.00.27 ⚡️ Speed up function `_add_behavior_instrumentation` by 197% in PR #1580 (`fix/java-direct-jvm-and-bugs`)
Contributor
|
This PR is now faster! 🚀 @misrasaurabh1 accepted my optimizations from: |
…2026-02-20T09.26.51 ⚡️ Speed up function `_add_timing_instrumentation` by 16% in PR #1580 (`fix/java-direct-jvm-and-bugs`)
Contributor
|
This PR is now faster! 🚀 @claude[bot] accepted my optimizations from: |
…2026-02-20T09.12.25 ⚡️ Speed up function `_is_inside_lambda` by 17% in PR #1580 (`fix/java-direct-jvm-and-bugs`)
Contributor
|
This PR is now faster! 🚀 @claude[bot] accepted my optimizations from: |
…2026-02-20T06.34.48 ⚡️ Speed up function `_byte_to_line_index` by 41% in PR #1580 (`fix/java-direct-jvm-and-bugs`)
Contributor
|
This PR is now faster! 🚀 @claude[bot] accepted my optimizations from: |
…imization The base class stubs for remove_test_functions_from_generated_tests() and add_runtime_comments_to_generated_tests() return None, causing an AttributeError crash in function_optimizer.py when iterating generated_tests.generated_tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ation Java stdout markers now include the test method name in the class field (e.g., "TestClass.testMethod") matching the Python marker format. The parser extracts the test method name from this combined field. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…all mode The module-level _test_file_path_cache persists across optimization iterations, which can cause negative cache entries to mask test files generated in later iterations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Author
Update: Fixed all outstanding issues from verificationFour additional commits pushed to address issues found during E2E verification and review feedback: 1.
|
| Run | Target | Result |
|---|---|---|
Phase 2 - Fibonacci (--no-pr) |
fibonacci |
PASSED — 476,142% speedup, no AttributeError crash |
| Phase 2 - Aerospike | Utf8.encodedLength |
PASSED — No optimization found (expected), pipeline clean |
Phase 3+4 - Fibonacci (--no-pr) |
fibonacci |
PASSED — 153,136% speedup, mark-as-success |
| Phase 3+4 - Aerospike | Utf8.encodedLength |
PASSED — 23% speedup found, PR created |
Contributor
Author
|
Hey @misrasaurabh1 — noticed your Your
So instead of returning tests unchanged, it now actually decorates them with original vs optimized runtime info — same as JS does. Let us know if you have any concerns with that approach. |
Contributor
Author
Addressed in commit // Before:
System.out.println("!$######" + _cf_mod5 + ":" + _cf_cls5 + ":" + _cf_fn5 + ":" + _cf_loop5 + ":" + _cf_iter5 + "######$!");
// After:
System.out.println("!$######" + _cf_mod5 + ":" + _cf_cls5 + "." + _cf_test5 + ":" + _cf_fn5 + ":" + _cf_loop5 + ":" + _cf_iter5 + "######$!");The parser in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes 5 out of 9 critical bugs found during Java E2E optimization testing with the aerospike-client-java project. It builds on and supersedes the previous PR #1552, which was reverted due to accumulated complexity. This branch was rebuilt incrementally with E2E validation at each step.
Key results:
Problems Fixed
Bug #7 (Critical): JUnit Version Detection Failure — 64% of all failures
Problem: The pom.xml parser only checked
<dependencies>, missing test dependencies declared in<dependencyManagement>(common in multi-module Maven projects like Aerospike). This caused JUnit 4 projects to be incorrectly identified as JUnit 5, generating incompatible test code.Root cause:
_detect_test_deps_from_pom()inconfig.pydidn't parse<dependencyManagement>sections or check submodule pom.xml files.Fix:
_detect_test_deps_from_pom()to parse both<dependencies>and<dependencyManagement>sections using a sharedcheck_dependencies()helpertest/,tests/,src/test/,testing/)Bug #3 (High): Direct JVM Execution Always Failing
Problem: Direct JVM benchmarking never worked — it always fell back to Maven. Three separate issues:
ConsoleLauncher not on classpath:
junit-platform-console-standaloneis a separate artifact not included in the normal dependency tree.mvn dependency:build-classpathdoesn't output it, soConsoleLauncherclass was not found at runtime.False JUnit 4 detection: The old code ran
java -cp ... JUnitCore -versionto detect JUnit 4. But JUnit 5 projects include JUnit 4 classes viajunit-vintage-engine, so this check always returned true. JUnitCore then ran against JUnit 5 tests, found 0 tests, and triggered the Maven fallback.Missing perf_stdout: After direct JVM execution,
perf_stdoutwasn't populated from the subprocess result, so the throughput calculation pipeline had no timing data.Fixes:
_find_junit_console_standalone()to locate the JAR in~/.m2/repository. If not present, downloads viamvn dependency:get. Appends to classpath in_get_test_classpath().junit-jupiter,junit-platform, orconsole-standalonein the classpath. This is deterministic, instant, and not fooled by vintage-engine compatibility classes. Addresses PR #1552 review comment about avoiding per-execution detection overhead.target/classesfrom sibling modules for projects where test modules depend on other modules.perf_stdoutcapture inparse_test_output.pyto extract stdout from subprocess results for Java performance tests.Bug #6 (High): Instrumentation Breaking Complex Expressions
Problem: Timing markers were inserted inside cast expressions, ternary operators, array access, and other complex expressions, causing "not a statement" compilation errors (e.g.,
(Long)list.get(2)broken by instrumentation).Fix:
_is_inside_complex_expression()that walks the tree-sitter AST upward to detect problematic parent types:cast_expression,ternary_expression,array_access,binary_expression,unary_expression,parenthesized_expression,instanceof_expression.in_complexflag to the call collection pipeline.Bug #2 (Medium-High): Extremely Slow rglob Calls
Problem:
resolve_test_file_from_class_path()was called for every timing marker without caching, causing 43+rglobcalls per optimization on large projects.Fix:
_test_file_path_cache: dict[tuple[str, Path], Path | None]module-level cachePre-existing Bug: TestConfig.test_framework Uncached Property
Problem:
TestConfig.test_frameworkwas an uncached@propertythat re-detected the framework on every access. During test result parsing, each test result accessed this property, causing 303,000+ calls to_detect_test_deps_from_pom()and 2.3M lines of output per optimization run. This bug exists onomni-javaand was not introduced by this branch.Fix:
_test_framework: Optional[str] = Nonecache field toTestConfig_detect_java_test_framework()fromjunit5tojunit4Behavior Test Method Name Fix
Problem: SQLite
setString(3, ...)in behavior instrumentation used a hardcoded"{class_name}Test"string instead of the actual test method name. This meant all test results mapped to the same identifier, losing per-test granularity.Fix:
_extract_test_method_name()with two regex patterns:_METHOD_SIG_PATTERN(full Java method signature) and_FALLBACK_METHOD_PATTERN(simple name extraction)String _cf_test{N} = "{methodName}"and uses it in the SQLite insertbisectoptimization for_byte_to_line_index()(O(log n) vs O(n) per call)Code Changes
codeflash/languages/java/config.py<dependencyManagement>, check submodule pom.xml files, change default to JUnit 4codeflash/languages/java/test_runner.py_find_junit_console_standalone(), classpath string JUnit detection, multi-module classpath,--add-opensfor JUnit 4 pathcodeflash/languages/java/instrumentation.py_is_inside_complex_expression(),_extract_test_method_name(),in_complexflag,bisectfor line index, correct SQLite test namecodeflash/verification/verification_utils.pytest_frameworkproperty, change default fallback to JUnit 4codeflash/verification/parse_test_output.pyperf_stdoutcapture for Java performance teststests/test_languages/test_java/test_instrumentation.py_cf_testvariable andsetString(3, ...)Other files with minor formatting changes (from pre-commit):
codeflash/cli_cmds/console.py,codeflash/cli_cmds/logging_config.py,codeflash/context/code_context_extractor.py,codeflash/languages/java/context.py,codeflash/optimization/function_optimizer.py,codeflash/verification/parse_line_profile_test_output.pyTesting
E2E Validation (Fibonacci — JUnit 5, single-module)
cd code_to_optimize/java/ CODEFLASH_CFAPI_SERVER=local CODEFLASH_AIS_SERVER=local \ uv run codeflash --file src/main/java/com/example/Fibonacci.java --function fibonacci --verbose --no-prE2E Validation (BubbleSort — exercises instrumentation + measurement)
Unit Tests
tests/test_languages/test_java/test_instrumentation.py)_cf_testvariable andsetString(3, _cf_test{N})Performance Impact
Known Issues Not Addressed
<includes>bypass: Behavior mode still uses Maven (not direct JVM). Aerospike's custom Surefire<includes>config blocks-Dtest=...filtering. Plan documented but not implemented in this PR.Relationship to PR #1552
This PR supersedes #1552. The original branch had 12 commits that were reverted due to accumulated debug logging and regressions. This branch was reset to the last known-good commit and rebuilt with:
🤖 Generated with Claude Code