⚡️ Speed up function _filter_new_declarations by 12% in PR #1546 (follow-up-reference-graph)

[codeflash-ai]

⚡️ This pull request contains optimizations for PR #1546

If you approve this dependent PR, these changes will be merged into the original PR branch follow-up-reference-graph.

This PR will be automatically closed if the original PR is merged.

---

📄 12% (0.12x) speedup for _filter_new_declarations in codeflash/languages/javascript/code_replacer.py

⏱️ Runtime : 783 microseconds → 698 microseconds (best of 250 runs)

📝 Explanation and details

The optimization achieves a 12% runtime improvement (783μs → 698μs) by replacing the lambda function in the sorting key with operator.attrgetter("start_line") and eliminating an unnecessary intermediate variable.

Key optimization:
Using attrgetter("start_line") instead of lambda d: d.start_line reduces per-item function call overhead during sorting. In Python, lambda functions incur dispatch costs for each element, while attrgetter is implemented in C and provides faster attribute access. The line profiler shows the sorting operation dropped from 967μs (24.7% of runtime) to 756μs (22.5%), a reduction of ~22% in sorting time alone.

Why this works:

• attrgetter is a specialized built-in operator that bypasses Python's function call mechanism
• Removing the sorted_declarations intermediate variable eliminates a list reference assignment and slightly reduces memory allocation overhead
• The optimization directly iterates over sorted(), which is more idiomatic and marginally more efficient

Performance characteristics from tests:

• Small inputs (< 10 items): Minor slowdown of 1-15% due to the overhead of importing attrgetter outweighing benefits
• Large inputs (100-1000 items): Significant speedup of 8-27%, with the best gains (27.7%) on the 1000-item test case where sorting overhead dominates
• The optimization particularly excels when filtering many declarations with duplicate sources, as seen in tests with 100-500 items showing 6-26% improvements

This is a textbook micro-optimization that pays dividends in hot paths processing many declarations, trading a negligible cost on trivial inputs for substantial gains on realistic workloads.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 40 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Click to see Generated Regression Tests

from collections import \
    namedtuple  # used to construct lightweight real instances with attributes

# imports
import pytest  # used for our unit tests
from codeflash.languages.javascript.code_replacer import \
    _filter_new_declarations

# We create a simple Declaration class using namedtuple from the standard library.
# This produces real instances (not mocks) that have the attributes the function expects:
# start_line, name, and source_code.
Declaration = namedtuple("Declaration", ["start_line", "name", "source_code"])

def test_includes_new_declarations_sorted_by_start_line():
    # Create three declarations with unsorted start_line values.
    d1 = Declaration(start_line=10, name="a", source_code="code_a")  # later in file
    d2 = Declaration(start_line=5, name="b", source_code="code_b")   # earlier in file
    d3 = Declaration(start_line=20, name="c", source_code="code_c")  # latest

    # existing_names does not include any of the declaration names, so all unique sources should be kept.
    existing = set()

    # Call the function under test
    codeflash_output = _filter_new_declarations([d1, d2, d3], existing); result = codeflash_output # 3.24μs -> 3.63μs (10.8% slower)

def test_duplicate_source_only_first_by_start_line_kept():
    # Two declarations share the same source_code "shared". They have different start_line values.
    d_early = Declaration(start_line=1, name="n_early", source_code="shared")
    d_mid = Declaration(start_line=2, name="n_mid", source_code="unique")
    d_late = Declaration(start_line=3, name="n_late", source_code="shared")

    # No existing names: the earliest declaration for "shared" should be kept (d_early),
    # and d_late should be discarded because its source_code was already seen.
    codeflash_output = _filter_new_declarations([d_late, d_early, d_mid], existing_names=set()); result = codeflash_output # 3.42μs -> 3.47μs (1.44% slower)

def test_existing_names_prevent_inclusion_but_do_not_block_same_source_for_later():
    # Two declarations share the same source_code "dup_source".
    # The earlier declaration's name is present in existing_names, so it will be skipped.
    # Because it was skipped, its source_code won't be added to seen_sources,
    # so the later declaration (with the same source_code) can still be included.
    d_skipped_first = Declaration(start_line=1, name="present", source_code="dup_source")
    d_allowed_later = Declaration(start_line=2, name="new_name", source_code="dup_source")

    existing = {"present"}  # causes the first declaration to be omitted

    codeflash_output = _filter_new_declarations([d_skipped_first, d_allowed_later], existing_names=existing); result = codeflash_output # 2.91μs -> 3.00μs (2.97% slower)

def test_empty_optimized_declarations_returns_empty_list():
    # When no declarations are provided, the result must be an empty list regardless of existing names.
    codeflash_output = _filter_new_declarations([], existing_names={"anything"}); result = codeflash_output # 1.81μs -> 2.07μs (12.6% slower)

def test_special_characters_and_none_name_behavior():
    # Declarations may contain unusual characters in names/source_code and even None for name.
    # The function only uses membership in the existing_names set for filtering,
    # so None is treated like any other hashable object.
    d_unicode = Declaration(start_line=1, name="naïve✓", source_code="𝓈𝑜𝓾𝓇𝒸𝑒")
    d_none_name = Declaration(start_line=2, name=None, source_code="none_source")
    d_emoji = Declaration(start_line=3, name="🚀", source_code="rocket_source")

    # existing_names only contains strings "something", doesn't contain None or the special ones.
    existing = {"something"}

    codeflash_output = _filter_new_declarations([d_emoji, d_none_name, d_unicode], existing_names=existing); result = codeflash_output # 3.26μs -> 3.43μs (4.96% slower)

def test_sort_stability_for_equal_start_lines_preserves_input_order():
    # Two declarations with the same start_line should preserve the original relative order
    # because Python's sorted is stable. We'll construct them in a specific order and
    # verify that order is kept.
    d_first = Declaration(start_line=5, name="first", source_code="s1")
    d_second = Declaration(start_line=5, name="second", source_code="s2")

    # The input order is [d_first, d_second], both have same key for sorting.
    codeflash_output = _filter_new_declarations([d_first, d_second], existing_names=set()); result = codeflash_output # 2.79μs -> 3.01μs (7.02% slower)

def test_large_scale_uniqueness_and_expected_count():
    # Create 1000 declarations with start_line = i to keep deterministic ordering.
    # source_code cycles among 10 values: "code_0" ... "code_9".
    # Names are "name_i". We'll add all even names into existing_names so they are skipped.
    total = 1000
    decls = []
    for i in range(total):
        src = f"code_{i % 10}"  # only 10 unique source_code values
        name = f"name_{i}"
        decls.append(Declaration(start_line=i, name=name, source_code=src))

    # existing_names contains every even-indexed name (deterministic)
    existing = {f"name_{i}" for i in range(0, total, 2)}

    # Call the function
    codeflash_output = _filter_new_declarations(decls, existing_names=existing); result = codeflash_output # 135μs -> 106μs (27.7% faster)

    # Ensure each returned declaration has a source_code among the expected odd remainders and
    # also that none of their names are in existing.
    expected_sources = {f"code_{r}" for r in range(10) if r % 2 == 1}
    returned_sources = {r.source_code for r in result}

    # Verify that returned declarations have names that were not in existing (i.e., odd indices)
    for r in result:
        pass

    # Also verify they are the earliest occurrences (smallest start_line) for their source_code
    # By construction, the earliest start_line for remainder r is r itself.
    expected_start_lines = {r for r in range(10) if r % 2 == 1}
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

from dataclasses import dataclass
from typing import Optional

# imports
import pytest
from codeflash.languages.javascript.code_replacer import \
    _filter_new_declarations

# Create a real Declaration class to use in tests
@dataclass
class Declaration:
    """Real Declaration class for testing."""
    name: str
    source_code: str
    start_line: int

def test_empty_optimized_declarations():
    """Test with empty optimized_declarations list."""
    codeflash_output = _filter_new_declarations([], set()); result = codeflash_output # 1.62μs -> 1.70μs (4.70% slower)

def test_empty_existing_names():
    """Test with empty existing_names set."""
    decls = [
        Declaration(name="func1", source_code="def func1(): pass", start_line=1),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.37μs -> 2.45μs (3.63% slower)

def test_single_new_declaration():
    """Test with single declaration that is new."""
    decls = [Declaration(name="newFunc", source_code="def newFunc(): pass", start_line=1)]
    existing = set()
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 1.94μs -> 2.28μs (14.9% slower)

def test_single_existing_declaration():
    """Test with single declaration that already exists."""
    decls = [Declaration(name="oldFunc", source_code="def oldFunc(): pass", start_line=1)]
    existing = {"oldFunc"}
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 1.80μs -> 2.01μs (10.5% slower)

def test_mixed_new_and_existing():
    """Test with mix of new and existing declarations."""
    decls = [
        Declaration(name="new1", source_code="def new1(): pass", start_line=1),
        Declaration(name="old1", source_code="def old1(): pass", start_line=2),
        Declaration(name="new2", source_code="def new2(): pass", start_line=3),
    ]
    existing = {"old1"}
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 2.67μs -> 2.81μs (4.99% slower)

def test_duplicate_source_code_filtered():
    """Test that duplicate source codes are filtered."""
    decls = [
        Declaration(name="func1", source_code="const x = 1;", start_line=1),
        Declaration(name="func2", source_code="const x = 1;", start_line=2),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.36μs -> 2.54μs (7.11% slower)

def test_maintains_line_order():
    """Test that result maintains line order from optimized code."""
    decls = [
        Declaration(name="func3", source_code="def func3(): pass", start_line=30),
        Declaration(name="func1", source_code="def func1(): pass", start_line=10),
        Declaration(name="func2", source_code="def func2(): pass", start_line=20),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.69μs -> 2.94μs (8.49% slower)

def test_preserves_declaration_attributes():
    """Test that declaration attributes are preserved."""
    decl = Declaration(name="myFunc", source_code="function myFunc() {}", start_line=42)
    codeflash_output = _filter_new_declarations([decl], set()); result = codeflash_output # 1.98μs -> 2.29μs (13.5% slower)

def test_multiple_existing_names():
    """Test filtering with multiple existing names."""
    decls = [
        Declaration(name="func1", source_code="def func1(): pass", start_line=1),
        Declaration(name="func2", source_code="def func2(): pass", start_line=2),
        Declaration(name="func3", source_code="def func3(): pass", start_line=3),
    ]
    existing = {"func1", "func3"}
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 2.34μs -> 2.60μs (9.67% slower)

def test_none_in_declaration_names():
    """Test behavior with special characters in names."""
    decls = [
        Declaration(name="$var", source_code="var $var = 1;", start_line=1),
        Declaration(name="_private", source_code="var _private = 2;", start_line=2),
        Declaration(name="CamelCase", source_code="var CamelCase = 3;", start_line=3),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.54μs -> 2.73μs (6.97% slower)

def test_empty_string_name():
    """Test with empty string as declaration name."""
    decls = [Declaration(name="", source_code="var x = 1;", start_line=1)]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.01μs -> 2.24μs (10.2% slower)

def test_empty_source_code():
    """Test with empty source code string."""
    decls = [Declaration(name="func", source_code="", start_line=1)]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.02μs -> 2.23μs (9.40% slower)

def test_unicode_in_names():
    """Test with unicode characters in names."""
    decls = [
        Declaration(name="función", source_code="def función(): pass", start_line=1),
        Declaration(name="переменная", source_code="var переменная = 1;", start_line=2),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.42μs -> 2.75μs (11.7% slower)

def test_zero_line_number():
    """Test with zero line number."""
    decls = [Declaration(name="func", source_code="def func(): pass", start_line=0)]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 1.99μs -> 2.24μs (11.1% slower)

def test_negative_line_number():
    """Test with negative line number."""
    decls = [Declaration(name="func", source_code="def func(): pass", start_line=-1)]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 1.97μs -> 2.25μs (12.5% slower)

def test_large_line_numbers():
    """Test with very large line numbers."""
    decls = [
        Declaration(name="func1", source_code="def func1(): pass", start_line=999999),
        Declaration(name="func2", source_code="def func2(): pass", start_line=999998),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.44μs -> 2.62μs (6.54% slower)

def test_same_name_different_source():
    """Test declarations with same name but different source code."""
    decls = [
        Declaration(name="func", source_code="def func(): pass", start_line=1),
        Declaration(name="func", source_code="def func(): return 1", start_line=2),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.48μs -> 2.63μs (5.69% slower)

def test_different_names_same_source():
    """Test declarations with different names but same source code."""
    decls = [
        Declaration(name="alias1", source_code="const shared = 1;", start_line=1),
        Declaration(name="alias2", source_code="const shared = 1;", start_line=2),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.21μs -> 2.49μs (11.2% slower)

def test_whitespace_in_source_code():
    """Test that whitespace differences in source code matter."""
    decls = [
        Declaration(name="func1", source_code="def func(): pass", start_line=1),
        Declaration(name="func2", source_code="def func():  pass", start_line=2),  # extra space
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.25μs -> 2.51μs (10.7% slower)

def test_multiline_source_code():
    """Test with multiline source code strings."""
    source1 = "def func():\n    return 1"
    source2 = "def func():\n    return 2"
    decls = [
        Declaration(name="func1", source_code=source1, start_line=1),
        Declaration(name="func2", source_code=source2, start_line=5),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.22μs -> 2.42μs (7.91% slower)

def test_case_sensitive_name_matching():
    """Test that name matching is case-sensitive."""
    decls = [
        Declaration(name="Func", source_code="def Func(): pass", start_line=1),
        Declaration(name="func", source_code="def func(): pass", start_line=2),
    ]
    existing = {"Func"}
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 2.19μs -> 2.40μs (8.39% slower)

def test_case_sensitive_source_matching():
    """Test that source code matching is case-sensitive."""
    decls = [
        Declaration(name="func1", source_code="DEF FUNC(): PASS", start_line=1),
        Declaration(name="func2", source_code="def func(): pass", start_line=2),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.30μs -> 2.52μs (8.39% slower)

def test_special_characters_in_source():
    """Test with special characters in source code."""
    decls = [
        Declaration(name="func1", source_code="const regex = /test[\\w]+/g;", start_line=1),
        Declaration(name="func2", source_code='const str = "quote\\"test";', start_line=2),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.27μs -> 2.54μs (10.6% slower)

def test_large_number_of_declarations():
    """Test with many declarations to assess scalability."""
    # Create 1000 unique declarations
    decls = [
        Declaration(name=f"func{i}", source_code=f"def func{i}(): pass", start_line=i)
        for i in range(1000)
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 173μs -> 154μs (12.2% faster)

def test_large_existing_names_set():
    """Test filtering with a large existing_names set."""
    # Create 500 declarations, 500 of which exist
    decls = [
        Declaration(name=f"func{i}", source_code=f"def func{i}(): pass", start_line=i)
        for i in range(500)
    ]
    existing = {f"func{i}" for i in range(250, 500)}
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 73.5μs -> 63.7μs (15.4% faster)

def test_all_declarations_existing():
    """Test when all declarations already exist."""
    decls = [
        Declaration(name=f"func{i}", source_code=f"def func{i}(): pass", start_line=i)
        for i in range(100)
    ]
    existing = {f"func{i}" for i in range(100)}
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 11.5μs -> 9.03μs (26.9% faster)

def test_many_duplicate_source_codes():
    """Test with many duplicate source codes."""
    # Create declarations where many share the same source code
    decls = []
    for i in range(100):
        # Every 10 declarations share the same source code
        source_idx = i // 10
        decls.append(
            Declaration(
                name=f"func{i}",
                source_code=f"shared_source_{source_idx}",
                start_line=i
            )
        )
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 15.4μs -> 13.3μs (16.2% faster)

def test_sorting_performance_large_input():
    """Test that sorting works correctly with large unsorted input."""
    decls = [
        Declaration(name=f"func{i}", source_code=f"def func{i}(): pass", start_line=999 - i)
        for i in range(500)
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 92.2μs -> 83.9μs (9.79% faster)
    # Verify results are in sorted order
    for i in range(len(result) - 1):
        pass

def test_mixed_large_dataset():
    """Test large dataset with mixed new and existing declarations."""
    decls = [
        Declaration(name=f"func{i}", source_code=f"def func{i}(): pass", start_line=i)
        for i in range(500)
    ]
    # Mark every other one as existing
    existing = {f"func{i}" for i in range(0, 500, 2)}
    codeflash_output = _filter_new_declarations(decls, existing); result = codeflash_output # 75.8μs -> 65.9μs (14.9% faster)

def test_large_source_code_strings():
    """Test with very large source code strings."""
    large_source = "def func():\n" + "    x = 1\n" * 100
    decls = [
        Declaration(name=f"func{i}", source_code=large_source if i % 10 == 0 else f"def func{i}(): pass", start_line=i)
        for i in range(100)
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 21.5μs -> 19.8μs (8.18% faster)

def test_many_declarations_same_name():
    """Test many declarations that all have the same name."""
    decls = [
        Declaration(name="same", source_code=f"def same_{i}(): pass", start_line=i)
        for i in range(100)
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 20.5μs -> 19.2μs (6.78% faster)

def test_sequential_duplicate_filtering():
    """Test proper duplicate filtering with sequential declarations."""
    decls = [
        Declaration(name="func1", source_code="source_A", start_line=1),
        Declaration(name="func2", source_code="source_A", start_line=2),
        Declaration(name="func3", source_code="source_B", start_line=3),
        Declaration(name="func4", source_code="source_B", start_line=4),
        Declaration(name="func5", source_code="source_C", start_line=5),
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 2.91μs -> 3.09μs (5.83% slower)

def test_performance_unsorted_large_declarations():
    """Test performance with large unsorted declaration list."""
    # Create 500 declarations in reverse order
    decls = [
        Declaration(name=f"func{i}", source_code=f"def func{i}(): pass", start_line=500 - i)
        for i in range(500)
    ]
    codeflash_output = _filter_new_declarations(decls, set()); result = codeflash_output # 92.7μs -> 84.4μs (9.75% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-pr1546-2026-02-19T11.06.50 and push.

[claude]

PR Review Summary

Prek Checks

• Import sorting issue found in codeflash/languages/javascript/code_replacer.py (I001: unsorted-imports) — the from operator import attrgetter import was placed after from codeflash.cli_cmds.console import logger instead of before it.
• Auto-fixed and committed in 0250d59e.
• Verified: prek passes cleanly after fix.
• mypy: 5 pre-existing errors in the file (missing type params, Language export, untyped def) — all present on the base branch follow-up-reference-graph, none introduced by this PR.

Code Review

No critical issues found.

This is a straightforward micro-optimization:

• Replaces lambda d: d.start_line with operator.attrgetter("start_line") (C-implemented, lower per-call overhead)
• Removes intermediate sorted_declarations variable, iterating directly over sorted()
• Functionally equivalent — no logic, API, or behavioral changes

Test Coverage

File	Stmts	Miss	Coverage	Status
codeflash/languages/javascript/code_replacer.py	81	19	77%	✅ ≥75%

• This file is new relative to main (exists on base branch follow-up-reference-graph).
• Coverage meets the 75% threshold for new files.
• The optimization changes no logic paths, so coverage is unchanged from the base branch.
• Test suite: 2410 passed, 57 skipped, 9 failed (all failures are in test_tracer.py, pre-existing and unrelated to this PR).

---

Last updated: 2026-02-19