Back to build
Raw log | Switch to full mode | Login

vpython -u /b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py --blacklist-file /b/rr/tmp7T5A8o/w/src/out/bad_devices.json --known-devices-file /home/chrome-bot/.android/known_devices.json --adb-path /b/rr/tmp7T5A8o/w/src/third_party/android_tools/sdk/platform-tools/adb -v in dir /b/rr/tmp7T5A8o/w: allow_subannotations: False base_name: 'device_recovery' cmd: ['vpython', '-u', '/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py', '--blacklist-file', '/b/rr/tmp7T5A8o/w/src/out/bad_devices.json', '--known-devices-file', '/home/chrome-bot/.android/known_devices.json', '--adb-path', '/b/rr/tmp7T5A8o/w/src/third_party/android_tools/sdk/platform-tools/adb', '-v'] env: {'PATH': '/b/rr/tmp7T5A8o/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmp7T5A8o/w/src/build/android:%(PATH)s'} env_prefixes: {'PATH': '/b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin'} env_suffixes: {} infra_step: True name: 'device_recovery' nest_level: 0 ok_ret: frozenset([0]) trigger_specs: () full environment: AWS_CREDENTIAL_FILE: /b/build/site_config/.boto BOTO_CONFIG: /b/build/site_config/.boto BUILDBOT_BLAMELIST: [u'alph@chromium.org', u'atotic@chromium.org', u'beccahughes@chromium.org', u'bpastene@chromium.org', u'bradhall@google.com', u'bratell@opera.com', u'caseq@chromium.org', u'changwan@google.com', u'chouinard@chromium.org', u'chromium-autoroll@skia-public.iam.gserviceaccount.com', u'chromium-internal-autoroll@skia-corp.google.com.iam.gserviceaccount.com', u'ctzsm@chromium.org', u'dfried@chromium.org', u'domenic@chromium.org', u'ellyjones@chromium.org', u'ericorth@chromium.org', u'erikchen@chromium.org', u'etiennep@chromium.org', u'finnur@chromium.org', u'gangwu@chromium.org', u'gmx@chromium.org', u'ios-autoroller@chromium.org', u'jam@chromium.org', u'jamescook@chromium.org', u'justincohen@google.com', u'kojii@chromium.org', u'kylechar@chromium.org', u'mahmoudi@chromium.org', u'manukh@chromium.org', u'mohsen@chromium.org', u'mrefaat@chromium.org', u'nasko@chromium.org', u'nator@chromium.org', u'nburris@chromium.org', u'noel@chromium.org', u'npm@chromium.org', u'pmonette@chromium.org', u'qingsi@google.com', u'rkgibson@google.com', u'rohitrao@chromium.org', u'rsesek@chromium.org', u'sdefresne@chromium.org', u'sky@chromium.org', u'slangley@google.com', u'tmartino@chromium.org', u'tommycli@chromium.org', u'torne@google.com', u'uwyiming@google.com', u'v8-ci-autoroll-builder@chops-service-accounts.iam.gserviceaccount.com', u'vollick@chromium.org', u'weidongg@chromium.org', u'wfh@chromium.org', u'xingliu@chromium.org', u'yusufo@google.com'] BUILDBOT_BRANCH: master BUILDBOT_BUILDBOTURL: https://build.chromium.org/p/chromium.android/ BUILDBOT_BUILDERNAME: Android Cronet Builder (dbg) BUILDBOT_BUILDNUMBER: 11909 BUILDBOT_CLOBBER: BUILDBOT_GOT_REVISION: None BUILDBOT_MASTERNAME: chromium.android BUILDBOT_REVISION: a7529e7731c9f1f54154399db806edf6cc0d39f0 BUILDBOT_SCHEDULER: chromium_commits BUILDBOT_SLAVENAME: build69-b1 CHROME_HEADLESS: 1 CIPD_CACHE_DIR: /b/c/cipd DISPLAY: :0.0 GIT_USER_AGENT: git/2.11.0 linux2 build69-b1.labs.chromium.org HOME: /home/chrome-bot INFRA_BUILDBOT_MASTER_CLASS_NAME: ChromiumAndroid INFRA_BUILDBOT_SLAVE_ACTIVE_SUBDIR: INFRA_BUILDBOT_SLAVE_NAME: build69-b1 LANG: en_US.UTF-8 LOGDOG_COORDINATOR_HOST: logs.chromium.org LOGDOG_STREAM_PREFIX: bb/chromium.android/Android_Cronet_Builder__dbg_/11909 LOGDOG_STREAM_PROJECT: chromium LOGDOG_STREAM_SERVER_PATH: unix:/b/build/rr/tmpX0riu0/butler.sock LOGNAME: chrome-bot PAGER: cat PATH: /b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/rr/tmp7T5A8o/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmp7T5A8o/w/src/build/android:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/cipd_client:/home/chrome-bot/slavebin:/b/depot_tools:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin PWD: /b/build/slave/Android_Cronet_Builder__dbg_/build PYTHONIOENCODING: UTF-8 PYTHONUNBUFFERED: 1 SHELL: /bin/bash USER: chrome-bot VPYTHON_VIRTUALENV_ROOT: /b/c/vpython I 0.009s Main Expected devices: I 0.009s Main 0aa6b19043e4b976 I 0.009s Main 06596466005cab1a I 0.009s Main 05c34aa83444c162 I 0.009s Main 0accc2af43e4affc I 0.899s TimeoutThread-1-for-MainThread condition '<lambda>' met (0.0s) I 4.766s blacklisting_device_status(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp7T5A8o/w/src/out/bad_devices.json for reason: offline I 5.063s Main Resetting blacklist /b/rr/tmp7T5A8o/w/src/out/bad_devices.json I 5.078s Main kill 15 15753 (adb [adb -L tcp:5037 fork-server server --reply-fd 4]) I 5.079s Main kill 15 15771 (adb [/b/rr/tmp7T5A8o/w/src/third_party/android_tools/sdk/platform-tools/adb -s 05c34aa83444c162 logcat -v threadtime]) I 5.079s Main kill 15 15772 (adb [/b/rr/tmp7T5A8o/w/src/third_party/android_tools/sdk/platform-tools/adb -s 06596466005cab1a logcat -v threadtime]) I 5.079s Main kill 15 15773 (adb []) I 5.089s Main kill 3 15771 (adb []) I 5.089s Main kill 3 15772 (adb []) I 5.089s Main kill 3 15773 (adb []) I 5.099s Main kill 9 15771 (adb []) I 5.100s Main kill 9 15772 (adb []) I 5.100s Main kill 9 15773 (adb []) E 5.110s Main Unable to kill 15771 (adb []) E 5.110s Main Unable to kill 15772 (adb []) E 5.110s Main Unable to kill 15773 (adb []) W 8.135s Main USB reset disabled for 0accc2af43e4affc (crbug.com/642914) E 37.210s RecoverDevice(0accc2af43e4affc) Failure while waiting for 0accc2af43e4affc. Attempting to recover. Traceback (most recent call last): File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 75, in RecoverDevice device.WaitUntilFullyBooted(retries=0) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout E 37.228s RecoverDevice(0accc2af43e4affc) Failed to reboot 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 83, in RecoverDevice device.Reboot(block=False, timeout=5, retries=0) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 859, in Reboot self.adb.Reboot() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 913, in Reboot self._RunDeviceAdbCmd(cmd, timeout, retries) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 296, in _RunAdbCmd args, output, status, device_serial) AdbCommandFailedError: (device: 0accc2af43e4affc) adb reboot: failed with exit status 1 and output: - error: device offline I 37.228s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp7T5A8o/w/src/out/bad_devices.json for reason: reboot_failure I 97.288s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 157.348s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 217.407s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 277.461s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) E 322.579s RecoverDevice(0accc2af43e4affc) Timed out while waiting for 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 111, in RecoverDevice retries=0, timeout=device.REBOOT_DEFAULT_TIMEOUT) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmp7T5A8o/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout I 322.579s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp7T5A8o/w/src/out/bad_devices.json for reason: reboot_timeout