Back to build
Raw log | Switch to full mode | Login

vpython -u /b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py --blacklist-file /b/rr/tmpbxEGi8/w/src/out/bad_devices.json --known-devices-file /home/chrome-bot/.android/known_devices.json --adb-path /b/rr/tmpbxEGi8/w/src/third_party/android_tools/sdk/platform-tools/adb -v in dir /b/rr/tmpbxEGi8/w: allow_subannotations: False base_name: 'device_recovery' cmd: ['vpython', '-u', '/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py', '--blacklist-file', '/b/rr/tmpbxEGi8/w/src/out/bad_devices.json', '--known-devices-file', '/home/chrome-bot/.android/known_devices.json', '--adb-path', '/b/rr/tmpbxEGi8/w/src/third_party/android_tools/sdk/platform-tools/adb', '-v'] env: {'PATH': '/b/rr/tmpbxEGi8/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmpbxEGi8/w/src/build/android:%(PATH)s'} env_prefixes: {'PATH': '/b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin'} env_suffixes: {} infra_step: True name: 'device_recovery' nest_level: 0 ok_ret: frozenset([0]) trigger_specs: () full environment: AWS_CREDENTIAL_FILE: /b/build/site_config/.boto BOTO_CONFIG: /b/build/site_config/.boto BUILDBOT_BLAMELIST: [u'beaufort.francois@gmail.com', u'bnc@chromium.org', u'bsheedy@chromium.org', u'chcunningham@chromium.org', u'chlily@chromium.org', u'chromium-autoroll@skia-public.iam.gserviceaccount.com', u'chromium-internal-autoroll@skia-corp.google.com.iam.gserviceaccount.com', u'davidben@chromium.org', u'dgozman@chromium.org', u'dnicoara@chromium.org', u'drott@chromium.org', u'ender@google.com', u'etiennep@chromium.org', u'falken@chromium.org', u'jeffish@microsoft.com', u'karandeepb@chromium.org', u'kristipark@chromium.org', u'mmoss@chromium.org', u'nicolaso@chromium.org', u'rkjnsn@chromium.org', u'robertogden@chromium.org', u'samans@chromium.org', u'sdy@chromium.org', u'sebmarchand@chromium.org', u'siyua@chromium.org', u'sugoi@google.com', u'svaldez@chromium.org', u'takumif@chromium.org', u'tby@chromium.org', u'thakis@chromium.org', u'tonikitoo@igalia.com', u'v8-ci-autoroll-builder@chops-service-accounts.iam.gserviceaccount.com', u'vakh@chromium.org', u'wangxianzhu@chromium.org', u'wzang@chromium.org', u'yliuyliu@google.com', u'yusufo@google.com'] BUILDBOT_BRANCH: master BUILDBOT_BUILDBOTURL: https://build.chromium.org/p/chromium.android/ BUILDBOT_BUILDERNAME: Android Cronet Builder Asan BUILDBOT_BUILDNUMBER: 9247 BUILDBOT_CLOBBER: BUILDBOT_GOT_REVISION: None BUILDBOT_MASTERNAME: chromium.android BUILDBOT_REVISION: 7fa8ccb5428e25f1ece3efe4d2560a1346c06512 BUILDBOT_SCHEDULER: chromium_commits BUILDBOT_SLAVENAME: build69-b1 CHROME_HEADLESS: 1 CIPD_CACHE_DIR: /b/c/cipd DISPLAY: :0.0 GIT_USER_AGENT: git/2.11.0 linux2 build69-b1.labs.chromium.org HOME: /home/chrome-bot INFRA_BUILDBOT_MASTER_CLASS_NAME: ChromiumAndroid INFRA_BUILDBOT_SLAVE_ACTIVE_SUBDIR: INFRA_BUILDBOT_SLAVE_NAME: build69-b1 LANG: en_US.UTF-8 LOGDOG_COORDINATOR_HOST: logs.chromium.org LOGDOG_STREAM_PREFIX: bb/chromium.android/Android_Cronet_Builder_Asan/9247 LOGDOG_STREAM_PROJECT: chromium LOGDOG_STREAM_SERVER_PATH: unix:/b/build/rr/tmp1yZcVq/butler.sock LOGNAME: chrome-bot PAGER: cat PATH: /b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/rr/tmpbxEGi8/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmpbxEGi8/w/src/build/android:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/cipd_client:/home/chrome-bot/slavebin:/b/depot_tools:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin PWD: /b/build/slave/Android_Cronet_Builder_Asan/build PYTHONIOENCODING: UTF-8 PYTHONUNBUFFERED: 1 SHELL: /bin/bash USER: chrome-bot VPYTHON_VIRTUALENV_ROOT: /b/c/vpython I 0.016s Main Expected devices: I 0.016s Main 0aa6b19043e4b976 I 0.016s Main 06596466005cab1a I 0.017s Main 05c34aa83444c162 I 0.017s Main 0accc2af43e4affc I 0.877s TimeoutThread-1-for-MainThread condition '<lambda>' met (0.0s) I 4.769s blacklisting_device_status(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmpbxEGi8/w/src/out/bad_devices.json for reason: offline I 5.001s Main Resetting blacklist /b/rr/tmpbxEGi8/w/src/out/bad_devices.json I 5.014s Main kill 15 4578 (adb [adb -L tcp:5037 fork-server server --reply-fd 4]) I 5.015s Main kill 15 4596 (adb [/b/rr/tmpbxEGi8/w/src/third_party/android_tools/sdk/platform-tools/adb -s 05c34aa83444c162 logcat -v threadtime]) I 5.015s Main kill 15 4597 (adb []) I 5.015s Main kill 15 4598 (adb []) I 5.025s Main kill 3 4596 (adb []) I 5.025s Main kill 3 4597 (adb []) I 5.025s Main kill 3 4598 (adb []) I 5.035s Main kill 9 4596 (adb []) I 5.035s Main kill 9 4597 (adb []) I 5.035s Main kill 9 4598 (adb []) E 5.045s Main Unable to kill 4596 (adb []) E 5.045s Main Unable to kill 4597 (adb []) E 5.045s Main Unable to kill 4598 (adb []) W 8.084s Main USB reset disabled for 0accc2af43e4affc (crbug.com/642914) E 37.166s RecoverDevice(0accc2af43e4affc) Failure while waiting for 0accc2af43e4affc. Attempting to recover. Traceback (most recent call last): File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 75, in RecoverDevice device.WaitUntilFullyBooted(retries=0) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout E 37.175s RecoverDevice(0accc2af43e4affc) Failed to reboot 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 83, in RecoverDevice device.Reboot(block=False, timeout=5, retries=0) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 859, in Reboot self.adb.Reboot() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 913, in Reboot self._RunDeviceAdbCmd(cmd, timeout, retries) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 296, in _RunAdbCmd args, output, status, device_serial) AdbCommandFailedError: (device: 0accc2af43e4affc) adb reboot: failed with exit status 1 and output: - error: device offline I 37.175s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmpbxEGi8/w/src/out/bad_devices.json for reason: reboot_failure I 97.232s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 157.287s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 217.346s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 277.404s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) E 322.520s RecoverDevice(0accc2af43e4affc) Timed out while waiting for 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 111, in RecoverDevice retries=0, timeout=device.REBOOT_DEFAULT_TIMEOUT) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmpbxEGi8/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout I 322.521s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmpbxEGi8/w/src/out/bad_devices.json for reason: reboot_timeout