Back to build
Raw log | Switch to full mode | Login

vpython -u /b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py --blacklist-file /b/rr/tmp1ZXd2n/w/src/out/bad_devices.json --known-devices-file /home/chrome-bot/.android/known_devices.json --adb-path /b/rr/tmp1ZXd2n/w/src/third_party/android_tools/sdk/platform-tools/adb -v in dir /b/rr/tmp1ZXd2n/w: allow_subannotations: False base_name: 'device_recovery' cmd: ['vpython', '-u', '/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py', '--blacklist-file', '/b/rr/tmp1ZXd2n/w/src/out/bad_devices.json', '--known-devices-file', '/home/chrome-bot/.android/known_devices.json', '--adb-path', '/b/rr/tmp1ZXd2n/w/src/third_party/android_tools/sdk/platform-tools/adb', '-v'] env: {'PATH': '/b/rr/tmp1ZXd2n/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmp1ZXd2n/w/src/build/android:%(PATH)s'} env_prefixes: {'PATH': '/b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin'} env_suffixes: {} infra_step: True name: 'device_recovery' nest_level: 0 ok_ret: frozenset([0]) trigger_specs: () full environment: AWS_CREDENTIAL_FILE: /b/build/site_config/.boto BOTO_CONFIG: /b/build/site_config/.boto BUILDBOT_BLAMELIST: [u'abhijeet@igalia.com', u'acomminos@fb.com', u'akalugin@yandex-team.ru', u'akihiroota@chromium.org', u'binji@chromium.org', u'bsheedy@chromium.org', u'chromium-autoroll@skia-public.iam.gserviceaccount.com', u'chromium-internal-autoroll@skia-corp.google.com.iam.gserviceaccount.com', u'collinbaker@chromium.org', u'ctzsm@chromium.org', u'danakj@chromium.org', u'danan@chromium.org', u'delphick@chromium.org', u'dgozman@chromium.org', u'dmblack@google.com', u'dpranke@chromium.org', u'drubery@chromium.org', u'dtapuska@chromium.org', u'dvadym@chromium.org', u'engedy@chromium.org', u'enne@chromium.org', u'estade@chromium.org', u'estevenson@chromium.org', u'etienneb@chromium.org', u'fdoray@chromium.org', u'fs@opera.com', u'gab@chromium.org', u'hhli@chromium.org', u'iwells@chromium.org', u'jdufault@google.com', u'jsbell@chromium.org', u'kbr@chromium.org', u'khorimoto@chromium.org', u'kristipark@chromium.org', u'lgrey@chromium.org', u'manucornet@chromium.org', u'marcin@mwiacek.com', u'mek@chromium.org', u'mlamouri@chromium.org', u'mmenke@chromium.org', u'nicolaso@chromium.org', u'oysteine@google.com', u'palmer@chromium.org', u'pcc@chromium.org', u'qiyuh@google.com', u'raul@tambre.ee', u'rayankans@chromium.org', u'rohitrao@chromium.org', u'sammiequon@google.com', u'sdefresne@chromium.org', u'sebsg@chromium.org', u'semenzato@chromium.org', u'shubag@amazon.com', u'svenzheng@google.com', u'thakis@chromium.org', u'tikuta@chromium.org', u'torne@google.com', u'v8-ci-autoroll-builder@chops-service-accounts.iam.gserviceaccount.com', u'vakh@chromium.org', u'vmpstr@chromium.org', u'wangxianzhu@chromium.org', u'wpt-autoroller@chops-service-accounts.iam.gserviceaccount.com', u'xiaochengh@chromium.org', u'yigu@chromium.org'] BUILDBOT_BRANCH: master BUILDBOT_BUILDBOTURL: https://build.chromium.org/p/chromium.android/ BUILDBOT_BUILDERNAME: Android Cronet Builder (dbg) BUILDBOT_BUILDNUMBER: 11824 BUILDBOT_CLOBBER: BUILDBOT_GOT_REVISION: None BUILDBOT_MASTERNAME: chromium.android BUILDBOT_REVISION: 1e5f4ba9592084985e40ad7cbebfe4c901e9562b BUILDBOT_SCHEDULER: chromium_commits BUILDBOT_SLAVENAME: build69-b1 CHROME_HEADLESS: 1 CIPD_CACHE_DIR: /b/c/cipd DISPLAY: :0.0 GIT_USER_AGENT: git/2.11.0 linux2 build69-b1.labs.chromium.org HOME: /home/chrome-bot INFRA_BUILDBOT_MASTER_CLASS_NAME: ChromiumAndroid INFRA_BUILDBOT_SLAVE_ACTIVE_SUBDIR: INFRA_BUILDBOT_SLAVE_NAME: build69-b1 LANG: en_US.UTF-8 LOGDOG_COORDINATOR_HOST: logs.chromium.org LOGDOG_STREAM_PREFIX: bb/chromium.android/Android_Cronet_Builder__dbg_/11824 LOGDOG_STREAM_PROJECT: chromium LOGDOG_STREAM_SERVER_PATH: unix:/b/build/rr/tmp5_hguM/butler.sock LOGNAME: chrome-bot PAGER: cat PATH: /b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/rr/tmp1ZXd2n/w/src/third_party/android_tools/sdk/platform-tools:/b/rr/tmp1ZXd2n/w/src/build/android:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/cipd_client:/home/chrome-bot/slavebin:/b/depot_tools:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin PWD: /b/build/slave/Android_Cronet_Builder__dbg_/build PYTHONIOENCODING: UTF-8 PYTHONUNBUFFERED: 1 SHELL: /bin/bash USER: chrome-bot VPYTHON_VIRTUALENV_ROOT: /b/c/vpython I 0.016s Main Expected devices: I 0.016s Main 0aa6b19043e4b976 I 0.016s Main 06596466005cab1a I 0.016s Main 05c34aa83444c162 I 0.016s Main 0accc2af43e4affc I 0.949s TimeoutThread-1-for-MainThread condition '<lambda>' met (0.0s) I 4.815s blacklisting_device_status(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp1ZXd2n/w/src/out/bad_devices.json for reason: offline I 5.119s Main Resetting blacklist /b/rr/tmp1ZXd2n/w/src/out/bad_devices.json I 5.139s Main kill 15 4865 (adb [adb -L tcp:5037 fork-server server --reply-fd 4]) I 5.139s Main kill 15 4883 (adb [/b/rr/tmp1ZXd2n/w/src/third_party/android_tools/sdk/platform-tools/adb -s 05c34aa83444c162 logcat -v threadtime]) I 5.139s Main kill 15 4884 (adb [/b/rr/tmp1ZXd2n/w/src/third_party/android_tools/sdk/platform-tools/adb -s 06596466005cab1a logcat -v threadtime]) I 5.140s Main kill 15 4885 (adb []) I 5.150s Main kill 3 4883 (adb []) I 5.150s Main kill 3 4884 (adb []) I 5.150s Main kill 3 4885 (adb []) I 5.160s Main kill 9 4883 (adb []) I 5.160s Main kill 9 4884 (adb []) I 5.160s Main kill 9 4885 (adb []) E 5.170s Main Unable to kill 4883 (adb []) E 5.170s Main Unable to kill 4884 (adb []) E 5.170s Main Unable to kill 4885 (adb []) W 8.184s Main USB reset disabled for 0accc2af43e4affc (crbug.com/642914) E 37.261s RecoverDevice(0accc2af43e4affc) Failure while waiting for 0accc2af43e4affc. Attempting to recover. Traceback (most recent call last): File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 75, in RecoverDevice device.WaitUntilFullyBooted(retries=0) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout E 37.269s RecoverDevice(0accc2af43e4affc) Failed to reboot 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 83, in RecoverDevice device.Reboot(block=False, timeout=5, retries=0) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 859, in Reboot self.adb.Reboot() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 913, in Reboot self._RunDeviceAdbCmd(cmd, timeout, retries) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 296, in _RunAdbCmd args, output, status, device_serial) AdbCommandFailedError: (device: 0accc2af43e4affc) adb reboot: failed with exit status 1 and output: - error: device offline I 37.269s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp1ZXd2n/w/src/out/bad_devices.json for reason: reboot_failure I 97.331s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 157.382s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 217.434s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) I 277.494s RecoverDevice(0accc2af43e4affc) Still working on WaitUntilFullyBooted(0accc2af43e4affc, retries=0, timeout=300) E 322.597s RecoverDevice(0accc2af43e4affc) Timed out while waiting for 0accc2af43e4affc. Traceback (most recent call last): File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 111, in RecoverDevice retries=0, timeout=device.REBOOT_DEFAULT_TIMEOUT) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/device_utils.py", line 831, in WaitUntilFullyBooted self.adb.WaitForDevice() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 862, in WaitForDevice self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File "/b/rr/tmp1ZXd2n/w/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 425, in GetCmdStatusAndOutputWithTimeout raise TimeoutError(output.getvalue()) CommandTimeoutError: Timeout I 322.597s RecoverDevice(0accc2af43e4affc) Adding 0accc2af43e4affc to blacklist /b/rr/tmp1ZXd2n/w/src/out/bad_devices.json for reason: reboot_timeout