Task 19039686

Name	hadam3p_pnw_xf75_2005_1_010348507_1
Workunit	10169432
Created	26 Oct 2015, 17:58:59 UTC
Sent	26 Oct 2015, 17:59:07 UTC
Report deadline	7 Oct 2016, 23:19:07 UTC
Received	10 Jan 2016, 22:38:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1075479
Run time	4 days 13 hours 54 min 32 sec
CPU time	3 days 1 hours 42 min 4 sec
Validate state	Invalid
Credit	2,509.66
Device peak FLOPS	3.67 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v7.27 windows_intelx86
Stderr	<core_client_version>7.2.47</core_client_version> <![CDATA[ <stderr_txt> Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7192, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2768, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7852, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8080, selfPID=7444, iMonCtr=1 Model crash detected, will try to restart... GGlobal Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4312, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8120, selfPID=7624, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4364, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6308, selfPID=7772, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7872, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8128, selfPID=7064, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7668, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7012, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6148, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8092, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7608, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7960, selfPID=6992, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7652, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7980, selfPID=7124, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7316, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7368, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7688, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7260, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7920, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5600, selfPID=7404, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7836, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8044, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7504, iMonCtr=2 Model crash detected, will try to restart... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7264, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7096, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7736, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7580, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8172, iMonCtr=2 Controller:: CPDN process is nos not running, exiting, bRetVal = 1, c running, exiting, bRetVal = 1, checkPID=0, selfPID=7452, iMonCtr=2 Model crash detected, will try to restart... heckPID=0, selfPID=7932, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7612, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7776, selfPID=6576, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7456, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7112, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8044, selfPID=7332, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8116, selfPID=7324, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6940, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8016, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7616, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7764, selfPID=7172, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3260, selfPID=7160, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7248, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7756, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8076, selfPID=7008, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6384, selfPID=7920, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6176, iMonCtr=2 GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8640, selfPID=8412, iMonCtr=1 Model crash detected, will try to restart... GSuspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1104, selfPID=7976, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1500, selfPID=8076, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... forrtl: Not enough storage is available to process this command. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5084, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 15:32:01 (5320): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_xf75_2005_1_010348507_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_xf75_2005_1_010348507_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Jan 2016 18:02:43	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	115,499	244,825	2.1197
29 Dec 2015 15:15:46	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	103,979	219,555	2.1115
18 Dec 2015 17:29:45	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	92,459	195,235	2.1116
14 Dec 2015 19:34:09	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	80,939	170,480	2.1063
08 Dec 2015 02:41:26	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	69,419	146,365	2.1084
03 Dec 2015 21:33:18	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	57,899	121,686	2.1017
28 Nov 2015 15:05:58	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	46,379	97,058	2.0927
21 Nov 2015 17:57:04	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	34,859	73,600	2.1114
18 Nov 2015 15:01:59	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	23,339	49,652	2.1274
05 Nov 2015 14:23:37	1075479	19039686	hadam3p_pnw_xf75_2005_1_010348507_1	11,819	25,246	2.1361