C.M

[Discussion] BOINC whitelist monitoring

566 posts in this topic

Hey,

 

We should really begin working together on a BOINC project whitelist membership & monitoring process - BOINC whitelist membership could rapidly change depending on project reliability, security and work availability.

 

Important links:

I've compiled a list of all boinc projects from boincstats in order of available work (descending) & have highlighted dead/offline projects:

https://docs.google.com/spreadsheets/d/1MRNTNGSKykYbJDhrSMuv-P_GjmqkQBCKspaa6CzVeVA/edit?usp=sharing

 

Note: You'll only be able to comment & view the spreadsheet. In order to start editing it yourself, you'll need to pm me your email address. Make a new gmail account for this sole purpose. I don't want to know your boinc email address!

 

An overview of all BOINC project's current work statistics: http://boincstats.com/en/page/projectStatusInfo

 

BOINC forum topic keeping on top of dead/offline BOINC projects: https://boinc.berkeley.edu/dev/forum_thread.php?id=8633&sort_style=&start=280

 

Projects IMO that should be removed:

We should immediately remove "boinc_projects[71]="http://slinca.com/|slinca@home";" as the url has been hijacked & no longer/never did represent a real boinc project.

 

IMO, the following projects should be removed from the whitelist purely because they have 0 available work. Perhaps just disable them until they become active again?

RALPH@Home, DistrRTgen, iGEM@home, AlmereGrid Boinc Grid, MindModeling@Home, Najmanovich Research Group, Primaboinca, Rioja Science, SETI@Home Beta, Spatiotemporal Quality of service.

 

OProject@Home and Beal@Home both appear to be completely offline/the projects are dead, so might want to consider removing them.

 

SLinCA's website appears to be completely offline - might be a safe option to disable it incase it's currently under attack & malware gets uploaded as work units. Thoughts?

 

Projects IMO that should be added:

 

The following projects aren't in the whitelist, but have work:

FiND@Home, Universe@Home, ATLAS@Home, vLHCathome, Citizen Science Grid, TN-Grid, RADIOACTIVE@HOME, YAFU, Climateprediction.net Beta.

 

Whitelist Observations:

Is there any issue with whitespace between the end of the boinc project's URL and the | symbol? Because some have the space, and some don't.

 

We've got duplicate BOINC whitelist entries in the code:

boinc_projects[42]="http://boinc.almeregrid.nl/|AlmereGridBoinc Grid";
boinc_projects[64]="http://boinc.almeregrid.nl/|almeregridboinc grid";
boinc_projects[61]="http://dg.imp.kiev.ua/slinca/|SLinCA";
boinc_projects[71]="http://slinca.com/|slinca@home";

 

Several boinc projects appear offline, but the last contact is within the last 6 days (since the new year) - I believe these are just temp issues since there's available work for most of them.

 

Potential whitelist process improvements:

Within the linked whitelist above, you can see the quantity of work units being worked on & can deduce potentially deduce whether or not it'd be fair to include it as a whitelisted project. If a BOINC project doesn't have enough work units available, then a small pool of users might gain an unfair advantage over others.

Should we have a minimum quantity of work units available before being considered viable as a project?

 

Should we take into account the BOINC project's current project status for whitelist status? (alpha/beta/production)

 

Should a BOINC project that has 10x the quantity of work units than another BOINC project have a higher magnitude cap than other whitelisted projects?

 

How fast can we adapt the BOINC whitelist to decisions made in this thread regarding whitelist status?

 

We should regularly be scanning every whitelisted BOINC project's work units with virustotal to verify that no malware is being distributed to our users.

 

Let's use this thread and a similar spreadsheet format for keeping ontop of BOINC project whitelist worthiness. If anyone has any suggestions regarding process improvements for whitelisting projects, please do speak up.

 

Regards,

CM.

Edited by C.M

^ Smash that upvote button! ;D


Follow me on:


Github


Twitter


Steemit


SoundCloud (Hangouts)

Share this post


Link to post
Share on other sites

Cross post from the announcement thread:

 

 

Hey,

 

Just noticed on my main computer that my antivirus (Bitdefender) had been blocking all YAFU BOINC project based files.

I went to the official website and it was blocked by bitdefender - don't think it's a false positive, I trust bitdefender quite a bit.

 

Can anyone vouch for their validity, or are we looking at a potentially hijacked BOINC project?

It's not the first time such a thing has happened..

 

CM.


^ Smash that upvote button! ;D


Follow me on:


Github


Twitter


Steemit


SoundCloud (Hangouts)

Share this post


Link to post
Share on other sites

I remember I once reported a wrong url for LHC2 but can't find the original post.

Share this post


Link to post
Share on other sites

C.M - I really appreciate you putting this together.  I just got my jimmies rustled when projects were added with no discussion out in the open after myself and The Drake tried to bring attention to modifying the BOINC whitelist a few months back.  Anyways, I hope that is all water under the bridge.

 

It is my opinion that we shouldn't add any beta projects to the whitelist (especially when there is a production BOINC project - i.e. SETI@home and Climateprediction).  The projects that have 0 work available right now, I believe should be temporarily disabled.  I do think that it is a good idea to require that projects have a minimum quantity of work units available before being considered viable.  Projects that are included in the magnitude report should have work available for gridcoiners to crunch, otherwise I think it gives a temporary advantage to someone who has credit in that project when work is unavailable to others.

 

I will have to do some research into the other projects that you would like to add, but I can certainly say I'd like to see vLHCathome added.  I am not sure about YAFU since you reported that it might be malicious, and we definitely need to take that into a consideration whenever adding a new project.

 

Projects that I think should be removed immediately: Beal@home, SLINCA@home, and boincsimap (no more work units available ever for this project).  I personally don't like wanless2 being enabled because it is 32 bit only as you cannot run it on a 64 bit OS at all.  It would be very easy for me to setup a 32 bit virtual machine just to run the project (or anyone that cares to), but I just feel like a project that won't run on most people's OS (I mean come on.. who doesn't run a 64 bit OS these days) - if enabled helps those that really want to min/max their magnitude.  I am happy with the new magnitude system, as min/maxing is very hard but one project stands out to me - WUProp@home.  It is impossible to get the required 100 RAC with a single node, but if you have 2-3 nodes on this project you can add a decent amount of magnitude to your CPID and an additional minable CPU block on classic.  I have always believed this to be an unfair project that has been included since classic and would like to see it removed.

C.M, sebastian-- and RTMoney like this

Share this post


Link to post
Share on other sites

Working on the block explorer today and noticed something with the project whitelist: There are 59 projects on the whitelist (using "list projects") and 36 projects on the network including "investor" (using "list network").  There are a few projects on the whitelist that do have RAC on BOINCStats but aren't listed in the network.  

 

Below is the output from the two tables with this sql query on 2 tables I have named projects and network in the research database:  (this output is going to be available on the block explorer webpage when I'm done)

SELECT research.projects.Project, research.network.RAC, research.network.AvgRAC FROM research.projects LEFT JOIN research.network on research.projects.Project=research.network.project order by research.network.RAC DESC;

Project RAC AvgRAC

milkyway@home     1165306540 332848.67509661

gpugrid     346309348 863592.79818458

collatz conjecture 84435895     195902.40365653

moo! wrapper     68728573      149406.69333275

primegrid     38016920     80713.61542218

einstein@home     4599216     6864.39903882

rosetta@home     3323895     1690.68061709

seti@home     1885022     2891.09369488

albert@home     1794497     22712.27692697

world community grid 970760     1128.77757235

poem@home     857171     2574.0097895

asteroids@home     743707     2600.28320688

malariacontrol.net 583160     884.90311224

sat@home     464455    1265.51047655

enigma@home     438365    1605.67378484

cosmology@home     420314    1449.30864453

constellation     412212    928.38449584

theskynet pogs     360721    1830.97812294

numberfields@home 334537    1137.84225026

nfs@home     298259    1498.71363248

yoyo@home     183278    1078.04246809

climateprediction.net 136861    743.76936036

edges@home     61438    451.71678553

lhc@home 1.0     57411    247.45054092

leiden classical 46556    443.34825255

simap     41848    337.45665672

wuprop@home     38398    241.48166782

sztaki desktop grid 27717    1978.37259101

convector     20592    214.47765858

burp     16801    559.84671776

distributed data mining    2898    578.44311377

beal@home     2840    177.38913179

primaboinca     879    870.2970297

mindmodeling@beta 239    118.90547264

investor     0    0

almeregrid boinc grid        

cas@home        

distributed rainbow table generator        

distrrtgen        

docking        

eon2        

gerasim@home        

gridcoin finance        

ibercivis        

igem@home        

mindmodeling@home        

najmanovich research        

oproject@home        

quake-catcher network sensor monitoring        

rioja science        

simone@home        

slinca        

slinca@home        

subsetsum@home        

test4theory@home        

the lattice project        

vgtu project@home        

volpex        

wanless2        

Can you look into why we have so many projects showing 0 credits?

Share this post


Link to post
Share on other sites

No worries, typhoon :)

 

So, i've put together a more thorough spreadsheet.

I'm suggesting that we remove 16 whitelisted projects, add the following projects (Universe@Home, vLHCathome, TN-Grid, SRBase), knock back 14 boinc project applicants, etc.

 

Can someone peer review what I've put together and comment on whether you agree with my suggestions?

 

What boinc task availability should we consider the minimum?

 

Download:

https://mega.co.nz/#!yshXxaqC!rcrpe8Lp12MCjTittGSG-JPs-2tb7c9HHBIlHFcg3nQ

https://www.virustotal.com/en/file/81e372177e3706022579e93977a1afeaac6167ed7657b3a85baae3dd62a3ad67/analysis/1422226753/


^ Smash that upvote button! ;D


Follow me on:


Github


Twitter


Steemit


SoundCloud (Hangouts)

Share this post


Link to post
Share on other sites

It is my opinion that we shouldn't add any beta projects to the whitelist (especially when there is a production BOINC project - i.e. SETI@home and Climateprediction).

+1

Share this post


Link to post
Share on other sites

Why don't we use Google Docs for the spreadsheet collaboration?... wait weren't we using it already at some point?

Share this post


Link to post
Share on other sites

Why don't we use Google Docs for the spreadsheet collaboration?... wait weren't we using it already at some point?

If we use google docs, everyone will have to make sure they're not using the same email address on the collaboration as they do with boinc - otherwise we'll be risking our gridcoin security.

TyphooN likes this

^ Smash that upvote button! ;D


Follow me on:


Github


Twitter


Steemit


SoundCloud (Hangouts)

Share this post


Link to post
Share on other sites

In regards to the doc C.M posted - I would like all of the projects under the "To Remove" tab to be removed for the reasons C.M pointed out besides Collatz Conjecture.  Collatz is back up, and it has been a project that has had a ton of work (and the Gridcoin team has racked up a ton of credits at that project).  I don't particularly care for the science behind the project, but historically it has been one with plenty of work to go around.

 

Currently Collatz Conjecture is showing:

Work # Tasks ready to send 1,583

Tasks in progress 60,305

 

 

I believe we shouldn't add TN-Grid because it requires an invite.  If everyone cant easily sign up, that is not fair game in my book.

 

I don't see any issues with adding the following projects:

* SRBase

* vLHCathome

* Universe@Home

Edited by TyphooN

Share this post


Link to post
Share on other sites

In regards to the doc C.M posted - I would like all of the projects under the "To Remove" tab to be removed for the reasons C.M pointed out besides Collatz Conjecture.  Collatz is back up, and it has been a project that has had a ton of work (and the Gridcoin team has racked up a ton of credits at that project).  I don't particularly care for the science behind the project, but historically it has been one with plenty of work to go around.

 

Currently Collatz Conjecture is showing:

Work # Tasks ready to send 1,583

Tasks in progress 60,305

 

 

I believe we shouldn't add TN-Grid because it requires an invite.  If everyone cant easily sign up, that is not fair game in my book.

 

I don't see any issues with adding the following projects:

* SRBase

* vLHCathome

* Universe@Home

I've converted it back to google docs, and made a lot of changes:

We've now got a changelog/notes page.

I've moved Collatz Conjecture and MindModeling@Home back to the whitelist (I made a mistake with mindmodeling. I thought it had no work units, turns out it's an active BOINC project).

I've added more info throughout the spreadsheet. I'll add each project's individual aims so that everyone knows what the BOINC projects do.

 

https://docs.google.com/spreadsheets/d/1MRNTNGSKykYbJDhrSMuv-P_GjmqkQBCKspaa6CzVeVA/edit?usp=sharing

 

Note: You'll only be able to comment & view the spreadsheet. In order to start editing it yourself, you'll need to pm me your email address.

Make a new gmail account for this sole purpose. I don't want to know your BOINC email address!

 

We should discuss the following whitelist applicants (they've not been decided on yet):

WEP-M+2 Project

Citizen Science Grid

BOINC@Fiit

Edited by C.M

^ Smash that upvote button! ;D


Follow me on:


Github


Twitter


Steemit


SoundCloud (Hangouts)

Share this post


Link to post
Share on other sites

Just to clarify my previous comment (in regards to mindmodeling specifically).  I actually like mindmodeling quite a bit, and run it on all my CPIDs but I just notice that work frequently isn't available.  I am personally very interested in the scientific research it wants to produce, but I have just seen several periods of time where work hasn't been available.  We haven't really come up with a definitive list of standards that all projects need to adhere to, but I just wanted to bring up that point (which is why I voted to have it removed).  Obviously we can't guarantee that work is available from projects that we don't control, but I just thought I'd clarify why left it lumped in with the other to be removed projects even though I support the project.

Edited by TyphooN

Share this post


Link to post
Share on other sites

I think FiND@Home is a worthy project but it just has a low number of tasks ready to send.  Do you think that tasks available to send is a good metric for enabled projects?  Is there a way that we could disable a project if it goes under 500 (or another arbitrary number) tasks available to send?  I just don't like the fact that there are times that I have had RAC on some projects when work is unavailable to others. If we want a project enabled but don't want to allow a temporary advantage to certain users when work is unavailable this seems like a sensible solution to me.

 

WEP-M+@ Project (wanless2) does not have enough work, can only run on 32 bit OSes, and like you said isn't even sending you work in your VM.  Don't add it.

I also don't believe we should add BOINC@Fiit.  I can't even load the message board page, and it is unsettling to me when I can't even read about the science behind a project.

I am unable to load http://volunteer.cs.und.edu/- until we can research this project I would say that we shouldn't add it.

Share this post


Link to post
Share on other sites

I think FiND@Home is a worthy project but it just has a low number of tasks ready to send.  Do you think that tasks available to send is a good metric for enabled projects?  Is there a way that we could disable a project if it goes under 500 (or another arbitrary number) tasks available to send?  I just don't like the fact that there are times that I have had RAC on some projects when work is unavailable to others. If we want a project enabled but don't want to allow a temporary advantage to certain users when work is unavailable this seems like a sensible solution to me.

This is the thing I'm not sure about - what if a BOINC project has a constant buffer of 200ish (for example) work units ready at any one time as to save resources on their systems?

It'd be really handy to keep a history of available tasks for each BOINC project - incase projects go through work availability phases, etc.

I don't believe it's possible for the gridcoin client to dynamically change the whitelist accoring to task availability.

 

There's either two things we could do:

1. Directly contact each BOINC project that has a low quantity of work units & enquire into future BOINC work unit availability.

2. Remove BOINC projects from the whitelist if there are insufficient work units available for the entire team. We'll have to determine what that quantity is, as you say it could be 500+ considering we've got 200+ users..

 

 

WEP-M+@ Project (wanless2) does not have enough work, can only run on 32 bit OSes, and like you said isn't even sending you work in your VM.  Don't add it.

I also don't believe we should add BOINC@Fiit.  I can't even load the message board page, and it is unsettling to me when I can't even read about the science behind a project.

Seconded, I created 2 dedicated VMs for this and it didnt work, gutted.

 

 

I am unable to load http://volunteer.cs.und.edu/- until we can research this project I would say that we shouldn't add it.

I can load it, but I think I previously had a similar issue.. we can put its addition off until others verify whether or not they can access it. http://volunteer.cs.und.edu/csg/ is it the missing /csg/ that's the issue?


^ Smash that upvote button! ;D


Follow me on:


Github


Twitter


Steemit


SoundCloud (Hangouts)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.