Fix removeUsage for backups

[abh1sar]

Description

• Due to a bug in UsageBackupDaoImpl.removeUsage, entry in the cloud_usage.backup_usage doesn't get removed even after all the backups have been deleted and backup offering removed.
• Backup usage doesn't get updated to 0 size when all backups are deleted.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI
• test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Before Fix:

10:03:37.297 [Usage-Job-1] ERROR com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21 - Exception caught while removing UsageBackupVO: Remove backup usage exception: No value specified for parameter 4
com.cloud.exception.CloudException: Remove backup usage exception: No value specified for parameter 4                                                                                     
        at com.cloud.usage.dao.UsageBackupDaoImpl.removeUsage(UsageBackupDaoImpl.java:76) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]                                  
        at com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21.CGLIB$removeUsage$0(<generated>) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]           
        at com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21_FastClassByCloudStack_515fb514.invoke(<generated>) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[cglib-nodep-3.3.0.jar:?]                                                                                    
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:123) ~[cloud-utils-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21.removeUsage(<generated>) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]                   
        at com.cloud.usage.UsageManagerImpl.createBackupEvent(UsageManagerImpl.java:2077) ~[classes/:?]                                                                                   
        at com.cloud.usage.UsageManagerImpl.createHelperRecord(UsageManagerImpl.java:986) ~[classes/:?]                                                                                   
        at com.cloud.usage.UsageManagerImpl.parse(UsageManagerImpl.java:716) ~[classes/:?]                                                                                                
        at com.cloud.usage.UsageManagerImpl.runInContextInternal(UsageManagerImpl.java:432) ~[classes/:?]                                                                      
        at com.cloud.usage.UsageManagerImpl$1.runInContext(UsageManagerImpl.java:369) ~[classes/:?]                                                                                       
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]                                                                                                                                                                                           
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at com.cloud.usage.UsageManagerImpl.run(UsageManagerImpl.java:371) ~[classes/:?]                                                                                                  
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]                                                                                                
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]                                                                                                               
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]                                                                                                                                                                                                                                                   
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]                                                                                                                                                                                                                                                                                  
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]                                                                                                                                                                                                                                                                                  
        at java.lang.Thread.run(Thread.java:829) ~[?:?]                                                                                                                                  
Caused by: java.sql.SQLException: No value specified for parameter 4                                                                                                                     
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:130) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                                      
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                              
        at com.mysql.cj.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:555) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                        
        at com.mysql.cj.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:339) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                      
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1061) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                               
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1009) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                               
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1320) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                  
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:994) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                        
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) ~[HikariCP-5.1.0.jar:?]                                                                                                                                                                                                                                                     
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) ~[HikariCP-5.1.0.jar:?]                                                                                                                                                                                                                                            
        at com.cloud.usage.dao.UsageBackupDaoImpl.removeUsage(UsageBackupDaoImpl.java:72) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]             

After Fix:

1. Created two backups for a vm (id=8) with a backup_offering (id=1)
   (Ignore usage_backup id 20 as it from a previous testing)

2. Deleted one backup 3. Deleted another backup 4. Removed backup offering

How did you try to break this feature and the system with this change?

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[codecov]

Codecov Report

❌ Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.42%. Comparing base (ec533cd) to head (0a9b232).
⚠️ Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
...rg/apache/cloudstack/backup/BackupManagerImpl.java	50.00%	1 Missing and 1 partial ⚠️
...n/java/com/cloud/usage/dao/UsageBackupDaoImpl.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #11522   +/-   ##
=========================================
  Coverage     17.42%   17.42%           
- Complexity    15336    15339    +3     
=========================================
  Files          5892     5892           
  Lines        526521   526524    +3     
  Branches      64293    64294    +1     
=========================================
+ Hits          91767    91771    +4     
+ Misses       424401   424399    -2     
- Partials      10353    10354    +1     

Flag	Coverage Δ
uitests	3.61% <ø> (ø)
unittests	18.48% <40.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[weizhouapache]

code lgtm

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14738

[winterhazel]

CLGTM. Did not test

[harikrishna-patnala]

code LGTM

[sureshanaparti]

clgtm

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15187

[borisstoyanov]

@blueorangutan test keepEnv

[blueorangutan]

@borisstoyanov a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-14494)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 56786 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11522-t14494-kvm-ol8.zip
Smoke tests completed. 147 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[borisstoyanov]

LGTM. manually checked the fix and it seems to be working as expected