climateprediction.net home page
CPDN not sync whith BOINCstats

CPDN not sync whith BOINCstats

Message boards : Number crunching : CPDN not sync whith BOINCstats
Message board moderation

To post messages, you must log in.

AuthorMessage
USTL-FIL (Lille Fr)

Send message
Joined: 16 Jun 05
Posts: 10
Credit: 20,676,311
RAC: 0
Message 41455 - Posted: 9 Jan 2011, 14:51:32 UTC

Hi,
CPDN not reporting credit to BOINCstats for last 4 days.
Boincstats admin say CPDN not exporting data.

Mike
ID: 41455 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 41456 - Posted: 9 Jan 2011, 20:03:30 UTC - in response to Message 41455.  

Probably just a script failed.
There's been no project people since Milo started his new job at the start of December, and it looks like the new people won't arrive for another few weeks.

Until then, no credits on external sites.


Backups: Here
ID: 41456 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 41457 - Posted: 9 Jan 2011, 21:22:49 UTC

Luckily I spotted this and running the script manually is not really any trouble.
Although the main stats script has been running nicely, the boinc_stats script failed due to being unable to connect to the database.
ID: 41457 · Report as offensive     Reply Quote
Belfry

Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 0
Message 41465 - Posted: 12 Jan 2011, 19:43:38 UTC

Still not working as of today.
ID: 41465 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 41466 - Posted: 12 Jan 2011, 19:55:27 UTC - in response to Message 41465.  

Still no project programmers/trouble shooters either. :)


Backups: Here
ID: 41466 · Report as offensive     Reply Quote
old_user559274

Send message
Joined: 4 Mar 09
Posts: 1
Credit: 100,990
RAC: 0
Message 41470 - Posted: 12 Jan 2011, 22:50:34 UTC

I think this is an issue for all stats sites, not just BOINCstats. Free-DC has the same issue.
ID: 41470 · Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 13 Jul 05
Posts: 125
Credit: 11,778,421
RAC: 0
Message 41473 - Posted: 13 Jan 2011, 16:51:47 UTC - in response to Message 41470.  

Yes,it is for all stats sites, and is Climate specific and has persisted for nearly a week.
ID: 41473 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 41479 - Posted: 14 Jan 2011, 19:16:30 UTC

An update:

The current problems with this forum not being available for a few hours each day, as well as the missing credits exports, is apparently due to the daily backups. These are taking so long now, with the terabytes of stored data, that the other things, as mentioned above, as well as general access to the database, is impossible at times.



Backups: Here
ID: 41479 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 41480 - Posted: 14 Jan 2011, 20:10:47 UTC

Les' point about data is a good one - CPDN now has about 70TB of stored data files, with multiple database records (file, parameters, workunit details &c.) for each of these.

It seems that the staff are willing to order another database server, which can be configured as a slave to the main BOINC database, and also to consider archiving more results. Meanwhile, I have at least got the stats dump to run, so hopefully boincstats will pick this up.
ID: 41480 · Report as offensive     Reply Quote
Berserker

Send message
Joined: 17 Feb 06
Posts: 8
Credit: 2,825,821
RAC: 0
Message 41483 - Posted: 14 Jan 2011, 22:31:02 UTC
Last modified: 14 Jan 2011, 22:31:25 UTC

Thanks Milo for the status update. My site has already accepted the stats update so BOINCstats should catch up soon.
ID: 41483 · Report as offensive     Reply Quote
USTL-FIL (Lille Fr)

Send message
Joined: 16 Jun 05
Posts: 10
Credit: 20,676,311
RAC: 0
Message 41486 - Posted: 15 Jan 2011, 14:03:49 UTC

I confirm , boincstats get an update!
Thanks
ID: 41486 · Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 13 Jul 05
Posts: 125
Credit: 11,778,421
RAC: 0
Message 41493 - Posted: 17 Jan 2011, 19:32:32 UTC - in response to Message 41480.  

Milo, how much of that 70TB is active? Perhaps a large piece of the data could be handled as archived data (near storage rather than live storage), thus simplifying and shortening the backup process and perhaps improve over all performance/


Les' point about data is a good one - CPDN now has about 70TB of stored data files, with multiple database records (file, parameters, workunit details &c.) for each of these.

It seems that the staff are willing to order another database server, which can be configured as a slave to the main BOINC database, and also to consider archiving more results. Meanwhile, I have at least got the stats dump to run, so hopefully boincstats will pick this up.


ID: 41493 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 41495 - Posted: 17 Jan 2011, 20:07:27 UTC - in response to Message 41493.  

That is one of several options that will be considered, once the new people arrive.
For now, backups are being made on odd numbered days, to allow the other programs a chance to run.


Backups: Here
ID: 41495 · Report as offensive     Reply Quote
Belfry

Send message
Joined: 19 Apr 08
Posts: 179
Credit: 4,306,992
RAC: 0
Message 41497 - Posted: 17 Jan 2011, 21:47:03 UTC

MySQL cluster.
ID: 41497 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 41499 - Posted: 18 Jan 2011, 9:29:40 UTC - in response to Message 41493.  

Milo, how much of that 70TB is active? Perhaps a large piece of the data could be handled as archived data (near storage rather than live storage), thus simplifying and shortening the backup process and perhaps improve over all performance/


The problem is not really the size of data files (although that causes problems in other ways) but the number of database records referring to them. We can archive the older records, and have done so twice, but then they become inaccessible to users who want to look at past results. Neither Tolu nor I ever had any time to write code to let users examine archived data.

I used to have a database slave which could be used for backups (the hardware has since been retired) and I have suggested to the CPDN staff that they set up a new one.
ID: 41499 · Report as offensive     Reply Quote

Message boards : Number crunching : CPDN not sync whith BOINCstats

©2024 climateprediction.net