🔒 Locked

distributed processing doubts

Started by matias suarez Jan 24, 2004 at 2:48 AM 2 replies 400+ views

Original Post

Jan 24, 2004 at 2:48 AM

I''m doing some research that is computational expensive, it does a lot of ANN and GA stuff so i decided to use 3 machines to do it (2 personal ones and my company provided laptop

) It is also possible for me to use some computers abroad (10k kilometers from here) I started writting some java classes to split the ANN work between computers with one ''server'' that assigns work units to each connected ''client'', the basic stuff seems to work well. Since my machines are linux and one is windows i decided to use java and sockets as the more portable solution. I was wondering if there''s some software already made for this kind of stuff and how a linux cluster will differ from this (i don''t really know what is a linux cluster and my googling didn''t provide clear info) Any help is appreciated! -Mat

oliii

Jan 24, 2004 at 7:23 AM

I''m not too hot on parallel computing, but I remember using PVM (Paralle Virtual Machine) in my studies, a long time ago (well, in 1998), it worked seamlessly on Win32 and SGI platforms (Unix). I''m sure it came a long way since then, and you could probably check out the competition.

anyhow, this is what (''parallel computing libraries'') came up with google. Look for PVM, it should bring up some interesting results.

good luck.

ironfroggy

Jan 24, 2004 at 1:31 PM

You could also use something like DCOM (im sure someone implements it for linux), which would allow you to write the code natively.

The best advice I would give is to just be careful where you split up the ANN, because you definately want the least ammount of communication between systems possible. Maybe your work distribution code should account for that, even monitor module communication bandwidth and adjust as needed.

matias suarez

Jan 24, 2004 at 2:23 PM

But why I would use dcom if i''m coding java? i could move it to c or interface the dcom with java but java already has it''s own rmi.
Even so dcom or RMI the configuration, access rights and update is the most tedious and complicated part, i get some extra machines so now is a total of 6 machines.
That PVM stuff sounds quite interesting and worth to check it, thanks for the info!
Now i''ll start spreading my ANN around the planet!

-Mat

PS: yes, each net is completely isolated from the others and the GA part mutates the net parameters to find the optimal(s) ones for the problem, since each work unit travels with the the net (java code) and the data is always at the machine the communication requirements are low.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.