Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 12
Posts: 12   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3452 times and has 11 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Make Project-Applications Open Source

Hello,
It would be great to have the Source-Code of the Project-Applications, So that the Users can compile it for there specific Architecture & OS.
That can give a significant Performacne-increase on some Architectures and give you a wider range of supported Systems.

For example at Einstein@home I was able to halfe the runtime on AARCH64 by recompile it and modify some parts of code to take full Advantage of the SIMD(NEON).
One Task of there original App needs ~31ksec on my Odroid-C2, My recompiled Version Needs ~13,5ksec. So the outpout is more than doubled.

WCG is my favorite Project but ist restricted to a few Operating-Systems & Architectures that I decided to Switch to another Project.
[Aug 12, 2016 6:21:20 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

There's quite a few reasons why this is never going to happen, in amongst x-validation. Your 'optimized' results wont validate against a results from any other random machine on the grid doing the wingman job.
[Aug 12, 2016 6:29:55 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

There's quite a few reasons why this is never going to happen, in amongst x-validation. Your 'optimized' results wont validate against a results from any other random machine on the grid doing the wingman job.

Yes the Validation can be a Problem, but If you investigate some Time it is possible. Einstein@home uses Cross-Validation too, At the Beginning of my Project I had some Problems with that... But my last Version have contributed many(>10000) valid Results on many Devices(not only my Devices).
If you do the Job properly it is possible to get valid results.
[Aug 12, 2016 6:51:25 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

There's the crux... this grid is contributed to by many large companies and colleges. It's very hard to convince these to participate here, much harder if WCG has no total control over the science applications. But anyway, you've gone on record... maybe the times they are a-changin. smile
[Aug 12, 2016 7:09:26 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

Okay, I understand that point of view. But maybe you should see it from the other side:
It don't have to be a open-source-project. You can only publish the source code(Maybe only on Request). That can be helpful to convince Companys. If they don't trust you they can prove the Code(or let there IT-specialists do that), so they really know whats going on there. Even in the case of colleges: The students can learn from the Code of the scientific-Apps or maybe improve it and send it back to you. And users like I can port the App to any specific Architecture or OS and send the Makefile, Build-script, etc back to you. Then the WCG can decide to take the changes or not. You wouldn't lose control over the code but getting a wider range of supported devices and maybe improved(faster) Applications. That can be really helpful to increase the throughput.

I think, if you do it the right way it can be a win-win-situation.


Hopefully you understand what I mean. I know my english isn't the best.
[Aug 12, 2016 1:15:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
koschi
Cruncher
Joined: Dec 16, 2007
Post Count: 5
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

Hello grid,

I bet all the mentioned companies and institutions use Linux, Apache or Samba to some extend, did the open approach of these make them any more insecure? Of course no college or other large entity would have to run "untrusted" compiled code from "someone on the internet". The optimized code could flow back to the code owner, must be reviewed and understood and if appropriate can be integrated to the master branch. If you don't like it, don't integrate it.
I wouldn't worry that much about the validation. Of course no one wants to waste compute time. Not the user of optimized apps, nor their wing men (which actually never loses, just waits longer for the resend to come back). Obviously those running apps through the app_info mechanism also want 100% revalidation successes as well. Of course there can be an initial time when results don't match yet, but with thorough testing this can be minimized. The numbers of invalid WUs and hence resends would be insignificant against what you need to resend every day due to failures from overclocking, disk crashes, lost devices, etc...

With so many active WCG projects, I'd understand that this or that one can't provide the source because they would have licensing issues or fear their code would be used by competing institutions or like that. But hey, is that really the case for all of them? After all quite a few BOINC projects are open these days, often run by universities as well.

Open code would give coders like N30dG the opportunity to provide applications to a widened user base. In my case I purchased 8 quad core ARM boards that I wanted to use for WCG. Back then I had no idea that the Android BOINC client is just a shadow of the real BOINC client, making a farm of Android devices (especially with Android > 5) hardly manageable remotely. So I'm using Linux on these, donating their time to Einstein@home and other projects that either provide ARM binaries themselves or release the code for others to create binaries. I'm convinced that Linux@ARM (not Android) is the next big thing. Scaling out with cheap ARM chips is perfect for todays big data, cloud and IOT business. Lets not ignore all that power (as is unfortunately done with GPU as well :-/ ). Lets be ready and have an app for these and other platforms, that have enough skilled users willing to port, optimize and contribute...
[Sep 9, 2016 11:05:25 PM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1671
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

Hi Koschi,
I agree with you about the necessity for WCG projects to support ARM architecture using Linux: Raspbian or Ubuntu-Mate.
I advocate for this move since the release of Raspberry Pi 2 and even more since Raspberry Pi 3. It would be wonderful - and better for the electricity bill - to be able to build "RPi farms" for WCG.
Regarding the initial question: even if it could look fine to make possible, for advanced crunchers, optimization and to compile a project, the way to secure the result reliability makes such a "freedom" impossible.
The question is not only to provide results, but to ensure the result correctness.
If you are septic about the critical importance of this point, you have only to take a look on two news released during the last couple of weeks regarding brain research and publications about genome biology.
Concerning the brain research, over 70% of the published results during the last 15 years are not reproducible (i.e. possibly invalidated) because of possible failures in the IRM software. 40'000 publications are impacted. See http://www.sciencealert.com/a-bug-in-fmri-sof...earch-scientists-discover
Concerning genomics publications, people "misuse" excel for managing gene list and finally they do not control the list integrity. Finally, at least 704 published articles in scientific publications rely on erroneous gene lists because excel misinterpret gene names as dates !!! See http://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1044-7
Just remember that WCG is not only a hobby or a kind of playful and exciting challenge for contributor. WCG help to generate fundamental research results. If the science does not compute correctly, the scientists will maybe take the wrong decision regarding possible therapy: rejecting a good product or approach, following inefficient product or approach.
Happy crunching,
Yves
----------------------------------------
[Sep 10, 2016 8:17:00 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

Yes, stumbled many times on these issues (not only excel, any spreadsheet programs), last week found HST1 was interpreted as a sheet cell (CEP2 is another), not a project moniker. IF you're aware it's not an issue and can code around, but...
----------------------------------------
[Edit 1 times, last edit by SekeRob* at Sep 10, 2016 8:26:04 AM]
[Sep 10, 2016 8:24:36 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Stiwi
Advanced Cruncher
Joined: May 19, 2012
Post Count: 73
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

autodock/autodock vina are open source but I don't know if there are any changes for the wcg application.
[Sep 10, 2016 10:19:38 AM]   Link   Report threatening or abusive post: please login first  Go to top 
KLiK
Master Cruncher
Croatia
Joined: Nov 13, 2006
Post Count: 3108
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Make Project-Applications Open Source

autodock/autodock vina are open source but I don't know if there are any changes for the wcg application.

if I was on his place, I'd contact the SRI (which is responsible for OET, OZ & FA@H, all ran on Androids) & offer their service to them!

but you can also go to SETi@home & improve their Android app!
wink
----------------------------------------
oldies:UDgrid.org & PS3 Life@home


non-profit org. Play4Life in Zagreb, Croatia
[Sep 11, 2016 7:49:40 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 12   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread