Back to build
Raw log | Switch to full mode | Login

vpython -u /b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py --blacklist-file /b/rr/tmp_OE0ZV/w/src/out/bad_devices.json --known-devices-file /home/chrome-bot/.android/known_devices.json --adb-path /b/rr/tmp_OE0ZV/w/src/third_party/android_tools/sdk/platform-tools/adb -v in dir /b/rr/tmp_OE0ZV/w: allow_subannotations: False base_name: 'device_recovery' cmd: ['vpython', '-u', '/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py', '--blacklist-file', '/b/rr/tmp_OE0ZV/w/src/out/bad_devices.json', '--known-devices-file', '/home/chrome-bot/.android/known_devices.json', '--adb-path', '/b/rr/tmp_OE0ZV/w/src/third_party/android_tools/sdk/platform-tools/adb', '-v'] env: {'PATH': '/b/rr/tmp_OE0ZV/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmp_OE0ZV/w/src/build/android:%(PATH)s'} env_prefixes: {'PATH': '/b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin'} env_suffixes: {} infra_step: True name: 'device_recovery' nest_level: 0 ok_ret: frozenset([0]) trigger_specs: () full environment: AWS_CREDENTIAL_FILE: /b/build/site_config/.boto BOTO_CONFIG: /b/build/site_config/.boto BUILDBOT_BLAMELIST: [u'Tom.Tan@microsoft.com', u'agl@chromium.org', u'alancutter@chromium.org', u'amaralp@chromium.org', u'amithi@chromium.org', u'amusbach@chromium.org', u'anthonysp@google.com', u'atotic@chromium.org', u'bnc@chromium.org', u'boliu@chromium.org', u'bpastene@chromium.org', u'brucedawson@chromium.org', u'chcunningham@chromium.org', u'chromium-autoroll@skia-public.iam.gserviceaccount.com', u'chromium-internal-autoroll@skia-corp.google.com.iam.gserviceaccount.com', u'cricke@chromium.org', u'davidben@chromium.org', u'dgozman@chromium.org', u'dnicoara@chromium.org', u'droger@chromium.org', u'ejoe@google.com', u'estade@chromium.org', u'eyor@google.com', u'falken@chromium.org', u'fs@opera.com', u'ftang@chromium.org', u'garykac@chromium.org', u'janewman@microsoft.com', u'jeffish@microsoft.com', u'jimmyxgong@chromium.org', u'jkrcal@chromium.org', u'kristipark@chromium.org', u'lgcheng@google.com', u'mmoss@chromium.org', u'monufriienko@chromium.org', u'nektar@chromium.org', u'nicolaso@chromium.org', u'oysteine@google.com', u'penghuang@chromium.org', u'pkasting@chromium.org', u'rdevlin.cronin@chromium.org', u'rkjnsn@chromium.org', u'samans@chromium.org', u'sdefresne@chromium.org', u'sonnysasaka@chromium.org', u'stgao@chromium.org', u'thakis@chromium.org', u'thegreenfrog@chromium.org', u'tonikitoo@igalia.com', u'treib@chromium.org', u'updowndota@chromium.org', u'vakh@chromium.org', u'wpt-autoroller@chops-service-accounts.iam.gserviceaccount.com', u'wutao@chromium.org', u'xingliu@chromium.org', u'xiyuan@chromium.org', u'yliuyliu@google.com', u'yusufo@google.com', u'zhongyi@chromium.org'] BUILDBOT_BRANCH: master BUILDBOT_BUILDBOTURL: https://build.chromium.org/p/chromium.android/ BUILDBOT_BUILDERNAME: Android Cronet Builder (dbg) BUILDBOT_BUILDNUMBER: 11967 BUILDBOT_CLOBBER: BUILDBOT_GOT_REVISION: None BUILDBOT_MASTERNAME: chromium.android BUILDBOT_REVISION: 0f2bc98d529a5d88659798c53fec4cfe4e1e6894 BUILDBOT_SCHEDULER: chromium_commits BUILDBOT_SLAVENAME: build69-b1 CHROME_HEADLESS: 1 CIPD_CACHE_DIR: /b/c/cipd DISPLAY: :0.0 GIT_USER_AGENT: git/2.11.0 linux2 build69-b1.labs.chromium.org HOME: /home/chrome-bot INFRA_BUILDBOT_MASTER_CLASS_NAME: ChromiumAndroid INFRA_BUILDBOT_SLAVE_ACTIVE_SUBDIR: INFRA_BUILDBOT_SLAVE_NAME: build69-b1 LANG: en_US.UTF-8 LOGDOG_COORDINATOR_HOST: logs.chromium.org LOGDOG_STREAM_PREFIX: bb/chromium.android/Android_Cronet_Builder__dbg_/11967 LOGDOG_STREAM_PROJECT: chromium LOGDOG_STREAM_SERVER_PATH: unix:/b/build/rr/tmpHENzod/butler.sock LOGNAME: chrome-bot PAGER: cat PATH: /b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/rr/tmp_OE0ZV/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmp_OE0ZV/w/src/build/android:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/cipd_client:/home/chrome-bot/slavebin:/b/depot_tools:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin PWD: /b/build/slave/Android_Cronet_Builder__dbg_/build PYTHONIOENCODING: UTF-8 PYTHONUNBUFFERED: 1 SHELL: /bin/bash USER: chrome-bot VPYTHON_VIRTUALENV_ROOT: /b/c/vpython I 0.014s Main Expected devices: I 0.014s Main 0aa6b19043e4b976 I 0.014s Main 06596466005cab1a I 0.014s Main 05c34aa83444c162 I 0.014s Main 0accc2af43e4affc I 0.875s TimeoutThread-1-for-MainThread condition '<lambda>' met (0.0s) I 4.769s blacklisting_device_status(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp_OE0ZV/w/src/out/bad_devices.json for reason: offline I 5.001s Main Resetting blacklist /b/rr/tmp_OE0ZV/w/src/out/bad_devices.json I 5.018s Main kill 15 31846 (adb [adb -L tcp:5037 fork-server server --reply-fd 4]) I 5.018s Main kill 15 31864 (adb [/b/rr/tmp_OE0ZV/w/src/third_party/android_tools/sdk/platform-tools/adb -s 05c34aa83444c162 logcat -v threadtime]) I 5.018s Main kill 15 31865 (adb [/b/rr/tmp_OE0ZV/w/src/third_party/android_tools/sdk/platform-tools/adb -s 06596466005cab1a logcat -v threadtime]) I 5.018s Main kill 15 31866 (adb []) I 5.029s Main kill 3 31864 (adb []) I 5.029s Main kill 3 31865 (adb []) I 5.029s Main kill 3 31866 (adb []) I 5.039s Main kill 9 31864 (adb []) I 5.039s Main kill 9 31865 (adb []) I 5.039s Main kill 9 31866 (adb []) E 5.049s Main Unable to kill 31864 (adb []) E 5.049s Main Unable to kill 31865 (adb []) E 5.049s Main Unable to kill 31866 (adb []) W 8.074s Main USB reset disabled for 0accc2af43e4affc (crbug.com/642914) E 37.150s RecoverDevice(0accc2af43e4affc) Failure while waiting for 0accc2af43e4affc. Attempting to recover. Traceback (most recent call last): File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 75, in RecoverDevice device.WaitUntilFullyBooted(retries=0) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout E 37.159s RecoverDevice(0accc2af43e4affc) Failed to reboot 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 83, in RecoverDevice device.Reboot(block=False, timeout=5, retries=0) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 859, in Reboot self.adb.Reboot() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 913, in Reboot self._RunDeviceAdbCmd(cmd, timeout, retries) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 296, in _RunAdbCmd args, output, status, device_serial) AdbCommandFailedError: (device: 0accc2af43e4affc) adb reboot: failed with exit status 1 and output: - error: device offline I 37.159s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp_OE0ZV/w/src/out/bad_devices.json for reason: reboot_failure I 97.213s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 157.263s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 217.311s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 277.362s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) E 322.479s RecoverDevice(0accc2af43e4affc) Timed out while waiting for 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 111, in RecoverDevice retries=0, timeout=device.REBOOT_DEFAULT_TIMEOUT) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmp_OE0ZV/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout I 322.479s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp_OE0ZV/w/src/out/bad_devices.json for reason: reboot_timeout