Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 450
Posts: 450   Pages: 45   [ Previous Page | 6 7 8 9 10 11 12 13 14 15 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 787662 times and has 449 replies Next Thread
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out


My computer started downloading ten ARP1 tasks this morning. More than three hours later, all files had not downloaded. I restarted my computer, and the downloads were completed in a few seconds. I don't know why the "backing off" and repeated attempts to download did not work as expected.

The downloads completing after you rebooted was just luck/coincidence. 503 is a server-side error, and no action taken by any remote user will affect it.

The backoff behavior isn't a client-side fix either, and also isn't intended to be a direct solution, but rather a mitigation strategy. In networking, a backoff is when retries happen at longer and longer intervals rather than happening in a loop as quickly as possible. The intent there is to give an overloaded server some breathing room (because all well-behaved clients will begin backing off).

If the problem is caused by some internal factor rather than too many incoming requests, then a client backoff probably won't be any help -- but when writing netcode, it's always good practice to implement one anyway.
----------------------------------------

----------------------------------------
[Edit 1 times, last edit by mdxi at Jul 19, 2022 2:47:57 PM]
[Jul 19, 2022 2:43:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
hchc
Veteran Cruncher
USA
Joined: Aug 15, 2006
Post Count: 746
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

I'm curious about the IT team at Krembil; i.e., how many sysadmins and how many DBAs are there?
----------------------------------------
  • i3-8100 (Coffee Lake, 4C/4T) @ 3.6 GHz
  • i5-4590 (Haswell, 4C/4T) @ 3.3 GHz
  • E5800 (Wolfdale, 2C/2T) @ 3.2 GHz

[Jul 19, 2022 4:12:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Unixchick
Veteran Cruncher
Joined: Apr 16, 2020
Post Count: 835
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

While an official answer about the IT team would be nice...Here's what I can tell from the forums. It is important to note that it is highly likely a sysadmin wouldn't post to the forums, but just lurk.

https://www.worldcommunitygrid.org/forums/wcg/viewmember?member=Cyclops Cyclops is the Communications intern. (this should be in their title on the forum)

https://www.worldcommunitygrid.org/forums/wcg/viewmember?member=chiara.p World Community Grid Admin, Mapping Cancer Markers Scientist

https://www.worldcommunitygrid.org/forums/wcg/viewmember?member=Jurisica World Community Grid Admin, Mapping Cancer Markers and Help Conquer Cancer Scientist

I'm a loudmouth on these forums as a user, but if I was admin I'd probably lurk. I tried searching the user db, but it is only in order up to a point,
[Jul 19, 2022 5:18:19 PM]   Link   Report threatening or abusive post: please login first  Go to top 
supdood
Senior Cruncher
USA
Joined: Aug 6, 2015
Post Count: 333
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

I'm curious about the IT team at Krembil; i.e., how many sysadmins and how many DBAs are there?

From what I've gathered from previous Krembil team responses to similar questions, the answer is zero. It's likely that they had some support from their IT for the initial data migration, but it certainly sounds like the Krembil BOINC team and the Krembil science team are one and the same. A bold move by them to step up and try to save the project, knowing that they would have to deal with an IBM-customized BOINC instance.

On a related note, others have suggested that Krembil should have negotiated a better support agreement with IBM. Clearly, the transition could have gone better, but do you really think that Krembil had any leverage to negotiate a better deal when IBM essentially just dumped the project? Pretty hard to get a corporation to care about something that they consider trash.
----------------------------------------
Crunch with BOINC team USA
www.boincusa.com

[Jul 19, 2022 5:51:36 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

Clearly, the transition could have gone better, but do you really think that Krembil had any leverage to negotiate a better deal when IBM essentially just dumped the project?

Yes, I do. Do I think it would have been highly favorable to Krembil? No, I don't. If you don't have IT people then what were you thinking would happen when IBM left? There was no reason Krembil couldn't have asked for: (1) IBM resources related to WCG on an as needed basis by phone for a period of time (say 1 year). If IBM said NO, we are not putting anymore resources into WCG. then (2) Why not ask for a special pricing deal on a support contract tailored to the WCG environment through their Global Services group? IBM does that now for many customers. Maybe a time and materials contract so that if you don't need the help you don't pay. IBM isn't going to turn down an opportunity to make a buck. If you don't have an IT group you are probably paying, or going to pay, somebody anyway sooner or later. Since this is no longer an IBM project, did Krembil buy licenses for WebSphere and DB2(those aren't cheap)? If so, there are maintenance and support agreements that go along with those licenses. What about the RedHat products like Ansible? Are they paying for RedHat support for those?

So, yes, there isn't any reason why Krembil couldn't have sat down with IBM and said, We're willing to take the project but we are going to need to get up to speed. We are going to need a,b,c,d.... If IBM says no to some of it, then Krembil needs to determine if they still want to take it or ask for something else on less favorable terms but better than nothing. Another thing, did anyone approach Scripps or Delft and ask if they had any IT folks that might be willing to help out temporarily? They are benefiting from this thing staying active. Never hurts to ask. If you don't ask, you don't get...

I don't believe IBM just came up to them and said here it is take it or leave it. If they did, then why didn't Krembil just build a BOINC project from scratch without all the IBM customizations? Probably would have taken the same amount of time.
----------------------------------------
[Edit 1 times, last edit by Former Member at Jul 19, 2022 7:21:52 PM]
[Jul 19, 2022 7:20:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
debs3759
Cruncher
Joined: Jun 10, 2021
Post Count: 20
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

I must say, I'm surprised at how many people are so upset at how things are progressing. I'm not always the most patient person, but I can see how hard it must be for a new company to take on project as big as WCG, with so much work and so many volunteers.

Of course, I'd love to see more work, but I'm glad they are fixing as much as possible before going into full production more.
[Jul 19, 2022 10:02:40 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Robokapp
Senior Cruncher
Joined: Feb 6, 2012
Post Count: 248
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

yes, we've been clear-cut into the camps of people upset and the people surprised that others are upset.

honestly, I'm kinda fine with where we are now. Forums are up, work comes through sometimes, the ones in charge won't talk to us... it's not perfect but it's good enough.
[Jul 19, 2022 10:34:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
hchc
Veteran Cruncher
USA
Joined: Aug 15, 2006
Post Count: 746
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

Unixchick said:
While an official answer about the IT team would be nice...Here's what I can tell from the forums. It is important to note that it is highly likely a sysadmin wouldn't post to the forums, but just lurk.

https://www.worldcommunitygrid.org/forums/wcg/viewmember?member=Cyclops Cyclops is the Communications intern. (this should be in their title on the forum)

https://www.worldcommunitygrid.org/forums/wcg/viewmember?member=chiara.p World Community Grid Admin, Mapping Cancer Markers Scientist

https://www.worldcommunitygrid.org/forums/wcg/viewmember?member=Jurisica World Community Grid Admin, Mapping Cancer Markers and Help Conquer Cancer Scientist

I'm a loudmouth on these forums as a user, but if I was admin I'd probably lurk. I tried searching the user db, but it is only in order up to a point,


Are you the same person as "NixChick"?

I noticed that some of the WCG/Krembil updates on Twitter and the main homepage were *very* technical and gave some fantastic details on exactly what things were wrong, whether certificates or load balancer, etc.

So who knows...

A big research institute would likely have full-time IT staff who are separate from full-time academic/scientific researchers. There's usually millions of dollars in grant money, and some of that is allocated towards sysadmins, developers, security, database, etc. instead of pure PhD types. I'm curious about the relationship between the Jurisica Lab and Krembil Research Institute. In other words, IBM is massive and barely supported WCG with resources, so I really hope KRI supports Jurisica Lab with the acquisition of WCG.

I wish Krembil/WCG the best of luck, for everybody's sake. I know what it's like to be understaffed during big mergers & acquisitions despite working around the clock.
----------------------------------------
  • i3-8100 (Coffee Lake, 4C/4T) @ 3.6 GHz
  • i5-4590 (Haswell, 4C/4T) @ 3.3 GHz
  • E5800 (Wolfdale, 2C/2T) @ 3.2 GHz

----------------------------------------
[Edit 1 times, last edit by hchc at Jul 21, 2022 1:54:31 PM]
[Jul 21, 2022 1:40:51 PM]   Link   Report threatening or abusive post: please login first  Go to top 
hchc
Veteran Cruncher
USA
Joined: Aug 15, 2006
Post Count: 746
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

Entity said:
I don't believe IBM just came up to them and said here it is take it or leave it. If they did, then why didn't Krembil just build a BOINC project from scratch without all the IBM customizations? Probably would have taken the same amount of time.


I worked at a bank holding company in a brand new IT department, and executives/bankers had a billion USD to buy failed banks from the 2008 crisis, whether on the open market or FDIC auctions (it's a strange world and playground when bankers might get a call at 3 AM saying whether or not their bid for a $300M bank went through)...

Anyway instead of 30-70 people in IT for an organization of our size, we started with maybe 7-15 people, so even though people worked long hours.... with one network engineer, one Windows Server guy, one e-mail guy, one phone network guy, couple desktop guys, etc., it's just exhausting no matter how talented. There's so many hundreds of applications and systems to integrate into the existing environment.

I imagine that WCG is likely a bit smaller in technical scope than an entire national bank, but I won't sneeze at the complexity. I'm sure there's insane load balancers, dozens of applications, proprietary IBM web servers and back-end databases, and lots of ancient legacy systems (e.g. this forum software that is likely full of bugs and security vulnerabilities).

It took years for IBM WCG to customize the BOINC back-end. I didn't really agree with lots of the changes 15 years ago, since customizations tend to become permanent.

I still wish we could merge device IDs and delete device IDs, and I wish statistics on each project had more detail on the number of work units/results currently out and the backlog.

I think Krembil's level of effort (6-8 months or so) so far has been reasonable, especially if they are understaffed. There's no way to put on a migration as well as re-architecture of all the BOINC customizations: That's just asking for failure from all the moving pieces.

I'm somewhat impatient and frustrated as a distributed computing addict since 1997 (with distributed.net then United Devices then WCG), but I'd rather Krembil do things right.

I'd love to see them go live in July or August 2022, but whatever. it's summer and electricity is expensive, so I'll work on other tech projects, and if I want to scratch the itch, I can do some Folding@home work units or distributed.net RC5-72/OGR-28 work units.
----------------------------------------
  • i3-8100 (Coffee Lake, 4C/4T) @ 3.6 GHz
  • i5-4590 (Haswell, 4C/4T) @ 3.3 GHz
  • E5800 (Wolfdale, 2C/2T) @ 3.2 GHz

[Jul 21, 2022 1:51:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Unixchick
Veteran Cruncher
Joined: Apr 16, 2020
Post Count: 835
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Workunits are being sent out

hchc - I'm just unixchick. I was a sysadmin/network person a long time ago. I worked at a university and helped set up the first servers and wiring to the desktop at the medical school for the university. This was of course a long time ago. I'm familiar with the lack of staffing and lack of sleep this entails. The IT dept started with just 2 of us and grew to 50 by the time I left.

I too am a distributed computing addict from the 90s when I was doing SETI WUs.

I'm so glad to see so many people return to the WCG project, ready to get crunching. It is too hot for me to do much crunching right now, so I'm just hanging out on the boards being annoying doing the odd WU that gets sent to me from WCG.
[Jul 21, 2022 2:12:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 450   Pages: 45   [ Previous Page | 6 7 8 9 10 11 12 13 14 15 | Next Page ]
[ Jump to Last Post ]
Post new Thread