Back to build
Raw log | Switch to full mode | Login

/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py --blacklist-file /b/s/w/ir/cache/builder/src/out/bad_devices.json --known-devices-file /home/chrome-bot/.android/known_devices.json --adb-path /b/s/w/ir/cache/builder/src/third_party/android_tools/sdk/platform-tools/adb -v in dir /b/s/w/ir/cache/builder: allow_subannotations: False base_name: 'device_recovery' cmd: ['/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py', '--blacklist-file', '/b/s/w/ir/cache/builder/src/out/bad_devices.json', '--known-devices-file', '/home/chrome-bot/.android/known_devices.json', '--adb-path', '/b/s/w/ir/cache/builder/src/third_party/android_tools/sdk/platform-tools/adb', '-v'] cwd: '/b/s/w/ir/cache/builder' env: {'CHROME_HEADLESS': '1', 'PATH': '/b/s/w/ir/cache/builder/src/third_party/android_tools/sdk/platform-tools:/b/s/w/ir/cache/builder/src/build/android:%(PATH)s'} env_prefixes: {} env_suffixes: {} infra_step: True name: 'device_recovery' nest_level: 0 ok_ret: frozenset([0]) trigger_specs: () full environment: ADDRFAM: inet BOTO_CONFIG: /b/s/w/ir/tmp/gsutil_task/.boto BUILDBUCKET_EXPERIMENTAL: FALSE CHROME_HEADLESS: 1 CIPD_CACHE_DIR: /b/s/cipd_cache/cache CIPD_PROTOCOL: v2 DEVSHELL_CLIENT_PORT: 45649 DOCKER_CONFIG: /b/s/w/ir/tmp/docker_cfg_task DOCKER_TMPDIR: /b/s/w/ir/tmp/docker_tmp_task GIT_CONFIG_NOSYSTEM: 1 GIT_TERMINAL_PROMPT: 0 HOME: /home/chrome-bot IFACE: em1 INFRA_GIT_WRAPPER_HOME: /b/s/w/ir/tmp/git_home_task LANG: en_US.UTF-8 LANGUAGE: en_US: LOGDOG_COORDINATOR_HOST: logs.chromium.org LOGDOG_STREAM_PREFIX: buildbucket/cr-buildbucket.appspot.com/8930807046052815008 LOGDOG_STREAM_PROJECT: chromium LOGDOG_STREAM_SERVER_PATH: unix:/b/s/w/ir/tmp/ld.sock LOGICAL: em1 LOGNAME: chrome-bot LUCI_CONTEXT: /b/s/w/itPDhEE9/luci_context.526013266 MAC_CHROMIUM_TMPDIR: /b/s/w/ir/tmp/t MAIL: /var/mail/chrome-bot METHOD: dhcp NO_GCE_CHECK: False PATH: /b/s/w/ir/cache/builder/src/third_party/android_tools/sdk/platform-tools:/b/s/w/ir/cache/builder/src/build/android:/b/s/w/ir/cipd_bin_packages:/b/s/w/ir/cipd_bin_packages/bin:/b/s/cipd_cache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games PWD: /b/s/w/ir PYTHONIOENCODING: UTF-8 PYTHONUNBUFFERED: 1 SHELL: /bin/bash SHLVL: 2 SWARMING_BOT_ID: build4-b9 SWARMING_HEADLESS: 1 SWARMING_SERVER: https://chromium-swarm.appspot.com SWARMING_TASK_ID: 40f664f5b7364411 TEMP: /b/s/w/ir/tmp/t TEMPDIR: /b/s/w/ir/tmp/t TERM: linux TMP: /b/s/w/ir/tmp/t TMPDIR: /b/s/w/ir/tmp/t UPSTART_EVENTS: local-filesystems net-device-up UPSTART_INSTANCE: UPSTART_JOB: swarming USER: chrome-bot VPYTHON_VIRTUALENV_ROOT: /b/s/w/ir/cache/vpython XDG_RUNTIME_DIR: /run/user/1000 XDG_SESSION_ID: c1 _: /b/s/w/ir/cipd_bin_packages/vpython I 0.086s Main Expected devices: I 0.086s Main 0acc25be25995b8a I 0.087s Main 0d88ca9443e4bb01 I 0.087s Main 06c14b05003b815c I 0.087s Main 06ad9a16003bb372 I 0.087s Main 0aa167a825995d01 I 0.087s Main 06a9b452003beda2 I 0.087s Main 0acca87043e4ba6d I 0.087s Main 06b32a05003b9923 I 1.337s TimeoutThread-1-for-MainThread condition '<lambda>' met (0.0s) I 3.193s Main Resetting blacklist /b/s/w/ir/cache/builder/src/out/bad_devices.json I 3.203s Main kill 15 5349 (adb [adb -L tcp:5037 fork-server server --reply-fd 12]) I 3.204s Main kill 15 5364 (adb [/b/s/w/ir/cache/builder/src/third_party/android_tools/sdk/platform-tools/adb -s 0acca87043e4ba6d logcat -v threadtime]) I 3.204s Main kill 15 5365 (adb [/b/s/w/ir/cache/builder/src/third_party/android_tools/sdk/platform-tools/adb -s 0aa167a825995d01 logcat -v threadtime]) I 3.204s Main kill 15 5366 (adb []) I 3.204s Main kill 15 5367 (adb []) I 3.204s Main kill 15 5368 (adb []) I 3.215s Main kill 3 5364 (adb []) I 3.215s Main kill 3 5365 (adb []) I 3.215s Main kill 3 5366 (adb []) I 3.216s Main kill 3 5367 (adb []) I 3.216s Main kill 3 5368 (adb []) I 3.226s Main kill 9 5364 (adb []) I 3.226s Main kill 9 5365 (adb []) I 3.226s Main kill 9 5366 (adb []) I 3.227s Main kill 9 5367 (adb []) I 3.227s Main kill 9 5368 (adb []) E 3.237s Main Unable to kill 5364 (adb []) E 3.237s Main Unable to kill 5365 (adb []) E 3.238s Main Unable to kill 5366 (adb []) E 3.238s Main Unable to kill 5367 (adb []) E 3.238s Main Unable to kill 5368 (adb []) W 6.304s Main USB reset disabled for 0acc25be25995b8a (crbug.com/642914) W 6.304s Main USB reset disabled for 06b32a05003b9923 (crbug.com/642914) W 6.304s Main USB reset disabled for 06a9b452003beda2 (crbug.com/642914) C 36.323s RecoverDevice(0acc25be25995b8a) Timed out. Dumping threads. C 36.324s RecoverDevice(06a9b452003beda2) Timed out. Dumping threads. C 36.324s RecoverDevice(0acc25be25995b8a) ******************************************************************************** C 36.324s RecoverDevice(06a9b452003beda2) ******************************************************************************** C 36.324s RecoverDevice(0acc25be25995b8a) Stack dump for thread 'TimeoutThread-1-for-RecoverDevice(0acc25be25995b8a)' C 36.324s RecoverDevice(06a9b452003beda2) Stack dump for thread 'TimeoutThread-1-for-RecoverDevice(06a9b452003beda2)' C 36.324s RecoverDevice(0acc25be25995b8a) ******************************************************************************** C 36.324s RecoverDevice(06a9b452003beda2) ******************************************************************************** C 36.325s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 774, in __bootstrap C 36.325s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 774, in __bootstrap C 36.325s RecoverDevice(06a9b452003beda2) self.__bootstrap_inner() C 36.325s RecoverDevice(0acc25be25995b8a) self.__bootstrap_inner() C 36.325s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 801, in __bootstrap_inner C 36.325s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 801, in __bootstrap_inner C 36.326s RecoverDevice(06a9b452003beda2) self.run() C 36.326s RecoverDevice(06b32a05003b9923) Timed out. Dumping threads. C 36.326s RecoverDevice(0acc25be25995b8a) self.run() C 36.326s RecoverDevice(06b32a05003b9923) ******************************************************************************** C 36.326s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run C 36.326s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run C 36.326s RecoverDevice(06b32a05003b9923) Stack dump for thread 'TimeoutThread-1-for-RecoverDevice(06b32a05003b9923)' C 36.326s RecoverDevice(06a9b452003beda2) self._ret = self._func(*self._args, **self._kwargs) C 36.326s RecoverDevice(0acc25be25995b8a) self._ret = self._func(*self._args, **self._kwargs) C 36.326s RecoverDevice(06b32a05003b9923) ******************************************************************************** C 36.326s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> C 36.326s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> C 36.326s RecoverDevice(06a9b452003beda2) child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), C 36.326s RecoverDevice(0acc25be25995b8a) child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), C 36.326s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 36.326s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 36.326s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 774, in __bootstrap C 36.326s RecoverDevice(06a9b452003beda2) return f(*args, **kwargs) C 36.327s RecoverDevice(0acc25be25995b8a) return f(*args, **kwargs) C 36.327s RecoverDevice(06b32a05003b9923) self.__bootstrap_inner() C 36.327s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 815, in WaitUntilFullyBooted C 36.327s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 815, in WaitUntilFullyBooted C 36.327s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 801, in __bootstrap_inner C 36.327s RecoverDevice(06a9b452003beda2) self.adb.WaitForDevice() C 36.327s RecoverDevice(0acc25be25995b8a) self.adb.WaitForDevice() C 36.327s RecoverDevice(06b32a05003b9923) self.run() C 36.327s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 866, in WaitForDevice C 36.327s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 866, in WaitForDevice C 36.327s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run C 36.327s RecoverDevice(06a9b452003beda2) self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) C 36.327s RecoverDevice(0acc25be25995b8a) self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) C 36.327s RecoverDevice(06b32a05003b9923) self._ret = self._func(*self._args, **self._kwargs) C 36.327s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd C 36.327s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd C 36.327s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> C 36.327s RecoverDevice(06a9b452003beda2) ensure_logs_on_timeout=ensure_logs_on_timeout) C 36.327s RecoverDevice(0acc25be25995b8a) ensure_logs_on_timeout=ensure_logs_on_timeout) C 36.327s RecoverDevice(06b32a05003b9923) child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), C 36.327s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper C 36.327s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper C 36.327s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 36.328s RecoverDevice(06a9b452003beda2) return impl() C 36.328s RecoverDevice(0acc25be25995b8a) return impl() C 36.328s RecoverDevice(06b32a05003b9923) return f(*args, **kwargs) C 36.328s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 36.328s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 36.328s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 815, in WaitUntilFullyBooted C 36.328s RecoverDevice(06a9b452003beda2) return f(*args, **kwargs) C 36.328s RecoverDevice(0acc25be25995b8a) return f(*args, **kwargs) C 36.328s RecoverDevice(06b32a05003b9923) self.adb.WaitForDevice() C 36.328s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 273, in _RunAdbCmd C 36.328s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 273, in _RunAdbCmd C 36.328s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 866, in WaitForDevice C 36.328s RecoverDevice(06a9b452003beda2) timeout, env=cls._ADB_ENV) C 36.328s RecoverDevice(0acc25be25995b8a) timeout, env=cls._ADB_ENV) C 36.328s RecoverDevice(06b32a05003b9923) self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) C 36.328s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 415, in GetCmdStatusAndOutputWithTimeout C 36.328s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 415, in GetCmdStatusAndOutputWithTimeout C 36.328s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd C 36.328s RecoverDevice(06a9b452003beda2) for data in _IterProcessStdout(process, timeout=timeout): C 36.328s RecoverDevice(0acc25be25995b8a) for data in _IterProcessStdout(process, timeout=timeout): C 36.328s RecoverDevice(06b32a05003b9923) ensure_logs_on_timeout=ensure_logs_on_timeout) C 36.328s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 277, in _IterProcessStdoutFcntl C 36.328s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 277, in _IterProcessStdoutFcntl C 36.328s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper C 36.328s RecoverDevice(06a9b452003beda2) [child_fd], [], [], iter_aware_poll_interval) C 36.329s RecoverDevice(0acc25be25995b8a) [child_fd], [], [], iter_aware_poll_interval) C 36.329s RecoverDevice(06b32a05003b9923) return impl() C 36.329s RecoverDevice(06a9b452003beda2) ******************************************************************************** C 36.329s RecoverDevice(0acc25be25995b8a) ******************************************************************************** C 36.329s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl E 36.329s RecoverDevice(06a9b452003beda2) Failure while waiting for 06a9b452003beda2. Attempting to recover. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 71, in RecoverDevice device.WaitUntilFullyBooted(retries=0) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 163, in _JoinAll (len(alive_threads), len(self._threads))) CommandTimeoutError: Timed out waiting for 1 of 1 threads. E 36.329s RecoverDevice(0acc25be25995b8a) Failure while waiting for 0acc25be25995b8a. Attempting to recover. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 71, in RecoverDevice device.WaitUntilFullyBooted(retries=0) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 163, in _JoinAll (len(alive_threads), len(self._threads))) CommandTimeoutError: Timed out waiting for 1 of 1 threads. C 36.329s RecoverDevice(06b32a05003b9923) return f(*args, **kwargs) C 36.329s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 273, in _RunAdbCmd C 36.329s RecoverDevice(06b32a05003b9923) timeout, env=cls._ADB_ENV) C 36.331s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 415, in GetCmdStatusAndOutputWithTimeout C 36.331s RecoverDevice(06b32a05003b9923) for data in _IterProcessStdout(process, timeout=timeout): C 36.332s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 277, in _IterProcessStdoutFcntl C 36.332s RecoverDevice(06b32a05003b9923) [child_fd], [], [], iter_aware_poll_interval) C 36.332s RecoverDevice(06b32a05003b9923) ******************************************************************************** E 36.332s RecoverDevice(06b32a05003b9923) Failure while waiting for 06b32a05003b9923. Attempting to recover. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 71, in RecoverDevice device.WaitUntilFullyBooted(retries=0) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 163, in _JoinAll (len(alive_threads), len(self._threads))) CommandTimeoutError: Timed out waiting for 1 of 1 threads. E 36.393s RecoverDevice(06a9b452003beda2) Failed to reboot 06a9b452003beda2. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 79, in RecoverDevice device.Reboot(block=False, timeout=5, retries=0) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 843, in Reboot self.adb.Reboot() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 917, in Reboot self._RunDeviceAdbCmd(cmd, timeout, retries) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd ensure_logs_on_timeout=ensure_logs_on_timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 289, in _RunAdbCmd raise device_errors.DeviceUnreachableError(device_serial) DeviceUnreachableError: 06a9b452003beda2 I 36.393s RecoverDevice(06a9b452003beda2) Adding 06a9b452003beda2 to blacklist /b/s/w/ir/cache/builder/src/out/bad_devices.json for reason: reboot_failure E 36.395s RecoverDevice(0acc25be25995b8a) Failed to reboot 0acc25be25995b8a. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 79, in RecoverDevice device.Reboot(block=False, timeout=5, retries=0) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 843, in Reboot self.adb.Reboot() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 917, in Reboot self._RunDeviceAdbCmd(cmd, timeout, retries) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd ensure_logs_on_timeout=ensure_logs_on_timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 289, in _RunAdbCmd raise device_errors.DeviceUnreachableError(device_serial) DeviceUnreachableError: 0acc25be25995b8a I 36.395s RecoverDevice(0acc25be25995b8a) Adding 0acc25be25995b8a to blacklist /b/s/w/ir/cache/builder/src/out/bad_devices.json for reason: reboot_failure E 36.401s RecoverDevice(06b32a05003b9923) Failed to reboot 06b32a05003b9923. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 79, in RecoverDevice device.Reboot(block=False, timeout=5, retries=0) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 170, in _JoinAll thread.ReraiseIfException() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run self._ret = self._func(*self._args, **self._kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 843, in Reboot self.adb.Reboot() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 917, in Reboot self._RunDeviceAdbCmd(cmd, timeout, retries) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd ensure_logs_on_timeout=ensure_logs_on_timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 289, in _RunAdbCmd raise device_errors.DeviceUnreachableError(device_serial) DeviceUnreachableError: 06b32a05003b9923 I 36.401s RecoverDevice(06b32a05003b9923) Adding 06b32a05003b9923 to blacklist /b/s/w/ir/cache/builder/src/out/bad_devices.json for reason: reboot_failure I 96.431s RecoverDevice(06a9b452003beda2) Still working on WaitUntilFullyBooted(06a9b452003beda2, retries=0, timeout=300) I 96.433s RecoverDevice(0acc25be25995b8a) Still working on WaitUntilFullyBooted(0acc25be25995b8a, retries=0, timeout=300) I 96.444s RecoverDevice(06b32a05003b9923) Still working on WaitUntilFullyBooted(06b32a05003b9923, retries=0, timeout=300) I 156.468s RecoverDevice(06a9b452003beda2) Still working on WaitUntilFullyBooted(06a9b452003beda2, retries=0, timeout=300) I 156.470s RecoverDevice(0acc25be25995b8a) Still working on WaitUntilFullyBooted(0acc25be25995b8a, retries=0, timeout=300) I 156.480s RecoverDevice(06b32a05003b9923) Still working on WaitUntilFullyBooted(06b32a05003b9923, retries=0, timeout=300) I 216.506s RecoverDevice(06a9b452003beda2) Still working on WaitUntilFullyBooted(06a9b452003beda2, retries=0, timeout=300) I 216.508s RecoverDevice(0acc25be25995b8a) Still working on WaitUntilFullyBooted(0acc25be25995b8a, retries=0, timeout=300) I 216.516s RecoverDevice(06b32a05003b9923) Still working on WaitUntilFullyBooted(06b32a05003b9923, retries=0, timeout=300) I 276.543s RecoverDevice(06a9b452003beda2) Still working on WaitUntilFullyBooted(06a9b452003beda2, retries=0, timeout=300) I 276.546s RecoverDevice(0acc25be25995b8a) Still working on WaitUntilFullyBooted(0acc25be25995b8a, retries=0, timeout=300) I 276.553s RecoverDevice(06b32a05003b9923) Still working on WaitUntilFullyBooted(06b32a05003b9923, retries=0, timeout=300) C 336.481s RecoverDevice(06a9b452003beda2) Timed out. Dumping threads. C 336.481s RecoverDevice(06a9b452003beda2) ******************************************************************************** C 336.481s RecoverDevice(06a9b452003beda2) Stack dump for thread 'TimeoutThread-1-for-RecoverDevice(06a9b452003beda2)' C 336.481s RecoverDevice(06a9b452003beda2) ******************************************************************************** C 336.481s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 774, in __bootstrap C 336.481s RecoverDevice(06a9b452003beda2) self.__bootstrap_inner() C 336.481s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 801, in __bootstrap_inner C 336.481s RecoverDevice(06a9b452003beda2) self.run() C 336.481s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run C 336.481s RecoverDevice(06a9b452003beda2) self._ret = self._func(*self._args, **self._kwargs) C 336.481s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> C 336.481s RecoverDevice(06a9b452003beda2) child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), C 336.481s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 336.481s RecoverDevice(06a9b452003beda2) return f(*args, **kwargs) C 336.481s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 815, in WaitUntilFullyBooted C 336.481s RecoverDevice(06a9b452003beda2) self.adb.WaitForDevice() C 336.481s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 866, in WaitForDevice C 336.482s RecoverDevice(06a9b452003beda2) self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) C 336.482s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd C 336.482s RecoverDevice(06a9b452003beda2) ensure_logs_on_timeout=ensure_logs_on_timeout) C 336.482s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper C 336.482s RecoverDevice(06a9b452003beda2) return impl() C 336.482s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 336.482s RecoverDevice(06a9b452003beda2) return f(*args, **kwargs) C 336.482s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 273, in _RunAdbCmd C 336.482s RecoverDevice(06a9b452003beda2) timeout, env=cls._ADB_ENV) C 336.482s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 415, in GetCmdStatusAndOutputWithTimeout C 336.482s RecoverDevice(06a9b452003beda2) for data in _IterProcessStdout(process, timeout=timeout): C 336.482s RecoverDevice(06a9b452003beda2) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 277, in _IterProcessStdoutFcntl C 336.482s RecoverDevice(06a9b452003beda2) [child_fd], [], [], iter_aware_poll_interval) C 336.482s RecoverDevice(06a9b452003beda2) ******************************************************************************** E 336.482s RecoverDevice(06a9b452003beda2) Timed out while waiting for 06a9b452003beda2. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 107, in RecoverDevice retries=0, timeout=device.REBOOT_DEFAULT_TIMEOUT) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 163, in _JoinAll (len(alive_threads), len(self._threads))) CommandTimeoutError: Timed out waiting for 1 of 1 threads. I 336.482s RecoverDevice(06a9b452003beda2) Adding 06a9b452003beda2 to blacklist /b/s/w/ir/cache/builder/src/out/bad_devices.json for reason: reboot_timeout C 336.483s RecoverDevice(0acc25be25995b8a) Timed out. Dumping threads. C 336.483s RecoverDevice(0acc25be25995b8a) ******************************************************************************** C 336.483s RecoverDevice(0acc25be25995b8a) Stack dump for thread 'TimeoutThread-1-for-RecoverDevice(0acc25be25995b8a)' C 336.483s RecoverDevice(0acc25be25995b8a) ******************************************************************************** C 336.483s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 774, in __bootstrap C 336.483s RecoverDevice(0acc25be25995b8a) self.__bootstrap_inner() C 336.483s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 801, in __bootstrap_inner C 336.483s RecoverDevice(0acc25be25995b8a) self.run() C 336.483s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run C 336.483s RecoverDevice(0acc25be25995b8a) self._ret = self._func(*self._args, **self._kwargs) C 336.483s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> C 336.483s RecoverDevice(0acc25be25995b8a) child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), C 336.483s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 336.484s RecoverDevice(0acc25be25995b8a) return f(*args, **kwargs) C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 815, in WaitUntilFullyBooted C 336.484s RecoverDevice(0acc25be25995b8a) self.adb.WaitForDevice() C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 866, in WaitForDevice C 336.484s RecoverDevice(0acc25be25995b8a) self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd C 336.484s RecoverDevice(0acc25be25995b8a) ensure_logs_on_timeout=ensure_logs_on_timeout) C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper C 336.484s RecoverDevice(0acc25be25995b8a) return impl() C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 336.484s RecoverDevice(0acc25be25995b8a) return f(*args, **kwargs) C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 273, in _RunAdbCmd C 336.484s RecoverDevice(0acc25be25995b8a) timeout, env=cls._ADB_ENV) C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 415, in GetCmdStatusAndOutputWithTimeout C 336.484s RecoverDevice(0acc25be25995b8a) for data in _IterProcessStdout(process, timeout=timeout): C 336.484s RecoverDevice(0acc25be25995b8a) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 277, in _IterProcessStdoutFcntl C 336.484s RecoverDevice(0acc25be25995b8a) [child_fd], [], [], iter_aware_poll_interval) C 336.484s RecoverDevice(0acc25be25995b8a) ******************************************************************************** E 336.484s RecoverDevice(0acc25be25995b8a) Timed out while waiting for 0acc25be25995b8a. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 107, in RecoverDevice retries=0, timeout=device.REBOOT_DEFAULT_TIMEOUT) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 163, in _JoinAll (len(alive_threads), len(self._threads))) CommandTimeoutError: Timed out waiting for 1 of 1 threads. I 336.484s RecoverDevice(0acc25be25995b8a) Adding 0acc25be25995b8a to blacklist /b/s/w/ir/cache/builder/src/out/bad_devices.json for reason: reboot_timeout C 336.490s RecoverDevice(06b32a05003b9923) Timed out. Dumping threads. C 336.490s RecoverDevice(06b32a05003b9923) ******************************************************************************** C 336.490s RecoverDevice(06b32a05003b9923) Stack dump for thread 'TimeoutThread-1-for-RecoverDevice(06b32a05003b9923)' C 336.490s RecoverDevice(06b32a05003b9923) ******************************************************************************** C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 774, in __bootstrap C 336.490s RecoverDevice(06b32a05003b9923) self.__bootstrap_inner() C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cipd_bin_packages/lib/python2.7/threading.py", line 801, in __bootstrap_inner C 336.490s RecoverDevice(06b32a05003b9923) self.run() C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 93, in run C 336.490s RecoverDevice(06b32a05003b9923) self._ret = self._func(*self._args, **self._kwargs) C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> C 336.490s RecoverDevice(06b32a05003b9923) child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 336.490s RecoverDevice(06b32a05003b9923) return f(*args, **kwargs) C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/device_utils.py", line 815, in WaitUntilFullyBooted C 336.490s RecoverDevice(06b32a05003b9923) self.adb.WaitForDevice() C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 866, in WaitForDevice C 336.490s RecoverDevice(06b32a05003b9923) self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries) C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 315, in _RunDeviceAdbCmd C 336.490s RecoverDevice(06b32a05003b9923) ensure_logs_on_timeout=ensure_logs_on_timeout) C 336.490s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper C 336.490s RecoverDevice(06b32a05003b9923) return impl() C 336.491s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl C 336.491s RecoverDevice(06b32a05003b9923) return f(*args, **kwargs) C 336.491s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 273, in _RunAdbCmd C 336.491s RecoverDevice(06b32a05003b9923) timeout, env=cls._ADB_ENV) C 336.491s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 415, in GetCmdStatusAndOutputWithTimeout C 336.491s RecoverDevice(06b32a05003b9923) for data in _IterProcessStdout(process, timeout=timeout): C 336.491s RecoverDevice(06b32a05003b9923) File: "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/cmd_helper.py", line 277, in _IterProcessStdoutFcntl C 336.491s RecoverDevice(06b32a05003b9923) [child_fd], [], [], iter_aware_poll_interval) C 336.491s RecoverDevice(06b32a05003b9923) ******************************************************************************** E 336.491s RecoverDevice(06b32a05003b9923) Timed out while waiting for 06b32a05003b9923. Traceback (most recent call last): File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 107, in RecoverDevice retries=0, timeout=device.REBOOT_DEFAULT_TIMEOUT) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper retry_if_func=retry_if_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/timeout_retry.py", line 158, in Run error_log_func=error_log_func) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 198, in JoinAll self._JoinAll(watcher, timeout) File "/b/s/w/ir/cache/builder/src/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 163, in _JoinAll (len(alive_threads), len(self._threads))) CommandTimeoutError: Timed out waiting for 1 of 1 threads. I 336.491s RecoverDevice(06b32a05003b9923) Adding 06b32a05003b9923 to blacklist /b/s/w/ir/cache/builder/src/out/bad_devices.json for reason: reboot_timeout