Repository navigation
fix(dev_env): restart xmc-bridge and ingestion-gateway in gke-upgrade restart step - #988
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds xmc-bridge and ingestion-gateway to the list of services restarted during the development environment upgrade, updating the corresponding unit tests to verify the new service count and ordering. Feedback on these changes highlights that because kubectl rollout restart is asynchronous, simply ordering the services in a loop will not guarantee that xmc-bridge finishes updating before ingestion-gateway starts; the script should instead wait for the rollout to complete. Additionally, the test helper _restart_cmd should dynamically reference self.config.namespace rather than hardcoding the namespace string.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request adds 'xmc-bridge' and 'ingestion-gateway' to the list of services restarted during a rolling restart in the development environment upgrade script, ensuring 'xmc-bridge' is triggered first. It also updates the unit tests to reflect the new service count and adds test cases to verify the restart order and the conditional skipping of these optional deployments. There are no review comments, so no further feedback is provided.
… restart step step_restart used a hard-coded list that omitted xmc-bridge and ingestion-gateway, so images pushed with --only images (e.g. a new xmc_bridge embedding updated component charts) were never rolled out. Deployments that are not present are already skipped. BUG=b/548435582 TAG=agy
Address review: restart the bridge first so it runs the new image before component releases roll, and assert on full kubectl commands in tests. BUG=b/548435582 TAG=agy
…pace in test kubectl rollout restart is asynchronous, so document that xmc-bridge is only triggered first rather than fully rolled first. The restart-order test now derives the namespace from the test config. BUG=b/548435582 TAG=agy CONV=baca6192-74d3-4579-9f73-0a3263d9a2a7
468e551 to
391f40b
Compare
…ntics rollout restart is asynchronous and step_restart needs no rollout ordering, so assert only that both restarts are triggered. Comment link: #988 (comment) BUG=b/548435582 TAG=agy CONV=baca6192-74d3-4579-9f73-0a3263d9a2a7
Summary
gke-upgrade --only restartnow also restartsxmc-bridgeandingestion-gateway, withxmc-bridgegoing first. Before this, images for those services that were pushed with--only images(:latesttag) never rolled out.Motivation & Context
step_restartused a hard-coded list of services that left outxmc-bridgeandingestion-gateway. During live testing of the ingestion-gateway 1P component, a rebuiltxmc_bridgeimage (with updated embedded component charts) was pushed, but the running pod kept the old image digest. The bridge restarts first because it may own component releases, such as the ingestion-gateway workload.Key Changes
tools/dev_env/upgrade_dev_env.py: addxmc-bridge(first) andingestion-gatewayto the restart list.resolve_deployment_namealready skips deployments that aren't deployed, e.g. when components or metrics are disabled.tools/dev_env/upgrade_dev_env_test.py: updated counts, plus tests for bridge-before-gateway ordering and for skipping when absent. The tests assert on the full kubectl commands.Assumptions
resolve_deployment_namereturnsNone.Verification & Testing
bazel test //tools/dev_env/...(14/14 pass),./.githooks/pre-commit(pass)....,gke-upgrade --only restartrankubectl rollout restarton bothxmc-bridge-deploymentandingestion-gateway-deployment.Related Issues / Bugs
Found while working on bug 548435582.
TAG=agy
CONV=baca6192-74d3-4579-9f73-0a3263d9a2a7