Skip to content

Add interactive mode to user agent#4497

Open
simonfaltum wants to merge 8 commits intomainfrom
add-interactive-mode-to-user-agent
Open

Add interactive mode to user agent#4497
simonfaltum wants to merge 8 commits intomainfrom
add-interactive-mode-to-user-agent

Conversation

@simonfaltum
Copy link
Member

@simonfaltum simonfaltum commented Feb 12, 2026

Changes

This PR adds terminal capability detection to the user agent string to track what proportion of CLI invocations support interactive features.
Having a better understanding of how the CLI is being used, and which commands are called in which context, can help guide us to improve the UX without over-investing in commands that are never used in a certain context

The user agent will include one of three values:

  • interactive/full: Both interactive output (spinners, colors) and user prompts are supported. Requires stderr to be a TTY, color enabled, stdin to be a TTY, and not running in Git Bash.
  • interactive/output_only: Interactive output is supported but prompts are not. This occurs when stdin is not a TTY (e.g., input piped) or when running in Git Bash (which has broken readline/ANSI support).
  • interactive/none: Non-interactive environment where neither spinners nor prompts work. This includes CI/CD pipelines, cron jobs, and cases where stderr is redirected.

Implementation

  • Add InteractiveMode() method to Capabilities struct in libs/cmdio
  • Add GetInteractiveMode(ctx) public function for context-safe access
  • Create cmd/root/user_agent_interactive_mode.go to integrate with user agent
  • Add withInteractiveModeInUserAgent() call in root.go
  • Comprehensive test coverage for all three interactive modes

Why User Agent Instead of Telemetry

As discussed with @pietern, this should be added to the user agent rather than telemetry because:

  • Broader coverage: User agent data is captured on every authenticated API call across all commands, while telemetry only fires on bundle deploy
  • No latency impact: Telemetry is sent synchronously which adds latency; user agent has no performance impact

Use Case

This telemetry helps inform decisions about when the CLI can prompt users for missing information versus requiring explicit flags. By measuring the proportion of invocations in each mode, we can understand how often interactive prompts would actually be shown to users.

This will also help track the impact of changes that affect interactive behavior.

Tests

Added comprehensive unit tests covering:

  • All three interactive modes (full, output_only, none)
  • Various TTY configurations (stdin, stderr combinations)
  • Git Bash detection
  • User agent integration

This change adds terminal capability detection to the user agent string
to track what proportion of CLI invocations support interactive features.

The field captures one of three values in the user agent:
- "interactive/full": Both interactive output (spinners, colors) and user
  prompts are supported. Requires stderr to be a TTY, color enabled, stdin
  to be a TTY, and not running in Git Bash.
- "interactive/output_only": Interactive output is supported but prompts
  are not. This occurs when stdin is not a TTY (e.g., input piped) or when
  running in Git Bash (which has broken readline/ANSI support).
- "interactive/none": Non-interactive environment where neither spinners
  nor prompts work. This includes CI/CD pipelines, cron jobs, and cases
  where stderr is redirected.

Changes:
- Add InteractiveMode() method to Capabilities struct in libs/cmdio
- Add GetInteractiveMode(ctx) public function for context-safe access
- Create user_agent_interactive_mode.go to integrate with user agent
- Add withInteractiveModeInUserAgent() call in root.go

Why user agent instead of telemetry:
- User agent data is captured on every authenticated API call across all
  commands, while telemetry only fires on bundle deploy
- No latency impact (telemetry is sent synchronously)
- Data goes to eng_deco_usage.deco_usage_logs_normalized for broader
  visibility

This will help inform decisions about when the CLI can prompt users for
missing information versus requiring explicit flags.
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Feb 12, 2026

Commit: 0dbe649

Run: 21968476535

Env 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 8 8 448 732 23:27
💚​ aws windows 8 8 417 741 22:31
🟨​ aws-ucws linux 2 4 10 5 702 585 69:18
🟨​ aws-ucws windows 2 11 5 668 596 54:30
💚​ azure linux 2 10 448 731 25:15
💚​ azure windows 2 10 417 740 21:46
💚​ azure-ucws linux 6 8 668 595 42:55
💚​ azure-ucws windows 6 8 631 606 37:21
🔄​ gcp linux 2 10 437 737 21:51
💚​ gcp windows 2 10 406 746 19:04
21 interesting tests: 9 RECOVERED, 5 SKIP, 5 flaky, 2 KNOWN
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 💚​R 💚​R 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R
🙈​ TestAccept/bundle/deployment/bind/alert 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/generate/alert 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/bundle/invariant/migrate 🙈​s 🙈​s 🔄​f ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s
🔄​ TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🔄​f 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
🔄​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl 🔄​f ✅​p ✅​p ✅​p
🙈​ TestAccept/bundle/resources/alerts/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/alerts/with_file 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🟨​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
10:18 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
8:25 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
8:11 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
8:05 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
8:04 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
7:25 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
7:22 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:22 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
6:49 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
6:43 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
6:33 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
6:24 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
6:15 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
6:11 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:54 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:52 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
5:50 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:49 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:44 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:35 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:34 azure-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:33 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:32 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:26 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:25 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:24 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:21 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:11 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:10 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:51 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
4:49 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:16 azure linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
3:54 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
3:46 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:46 azure linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:44 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:41 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
3:40 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:39 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
3:34 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
3:34 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
3:28 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:27 azure-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
3:20 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
3:14 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
3:12 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
3:12 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:09 azure-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
3:09 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
3:07 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

- Replace testify assertions with standard Go testing in
  user_agent_interactive_mode_test.go (new code should not use testify)
- Add test for interactive/none mode in user agent
- Add comment explaining why GetInteractiveMode returns "" when cmdio
  is not initialized in the context
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
// happen early in command setup before cmdio is configured in the context.
// The caller is expected to treat unknown as a no-op and skip adding the mode.
func GetInteractiveMode(ctx context.Context) InteractiveMode {
c, ok := ctx.Value(cmdIOKey).(*cmdIO)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not fromContext like the other accessors?

IMO, it is a feature to panic if initialization hasn't happened.

const (
InteractiveModeUnknown InteractiveMode = "" // cmdio not initialized in context
InteractiveModeFull InteractiveMode = "full" // Both interactive output and prompts supported
InteractiveModeOutputOnly InteractiveMode = "output_only" // Interactive output only, no prompts (stdin not TTY or Git Bash)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can rename to just "output" to save a few chars, implies the same.

ua := useragent.FromContext(ctx)

if !strings.Contains(ua, "interactive/") {
t.Errorf("expected user agent to contain 'interactive/', got %s", ua)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert.Contains?

const interactiveModeKey = "interactive"

func withInteractiveModeInUserAgent(ctx context.Context) context.Context {
// mode is empty when cmdio is not initialized in the context (e.g., early startup).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When does this happen? Per the other fromContext comment I think we can omit this case entirely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants