MAPPERS: A cluster to cut the time of map compiling

[Psychcf]

Right now I'm trying to set up a linux cluster of computers for myself. I'll be running a few nexuiz servers on this, yeah, but I also know how map compile times can cripple how fast a map can be posted. On the cluster I will have one VM for the nexuiz servers, one VM for anything I need, and one VM for mappers who are on the nexuiz development team and only mappers who are on the nexuiz development team.

Some info on the computers in these pictures:

Each computer has a pentium 3 processor, not sure how much ram it has yet
There are 15 in the pictures, I am donating somewhere around 7 of them to a local organization, which is why I got them in the first place
I got them from a local school district that was getting rid of them
They will be behind a DSL line with 1500kbps download, and 360kbps upload
They may or may not be up 100% of the time due to power usage. If you ask me to turn them on, I'll turn them on though.

I've done some research, I'm still not sure how I should approach this. Things like what distro to use, what patches (if any) I should install. Any help with setting them up would be highly appreciated.

[fefern29]

Hey thats pretty nice sounding but you might want to do some more looking at the computers and research because you should probably get a good understanding of the differences between the patches and such.

[fefern29]

So Psych will i be able to join or what is going on?

[:)]

interesting
maybe try clusterknoppix?

[tZork]

cool, however running nexuiz on a cluster is pointless since its singlethreaded afaik. map compiles should be good tho.

[KadaverJack]

before you get too excited: can q3map2 even run on a cluster? correct me if i'm wrong, but i don't think so...

[tZork]

before you get too excited: can q3map2 even run on a cluster? correct me if i'm wrong, but i don't think so...

Cluster i dont know, however it does do very well on multi core/cpu systems. close to double speed on two cores vs one.

[KadaverJack]

Yes, but a cluster is quite a bit different than a smp system.
And since the nodes are probably connected via 100mbit lan it's more likely that it will get slower the more nodes you use...

[John Galt]

And when you compare a 7 node x 700mhz PIII cluster with a modern dual core proc with enough RAM, I'm not sure you'll see much improvement in time at all. Although it would free up developer CPU time...

Good suggestion from Dave. There are other out-of-the-box clustering distros out there as well. I would forget about patching the kernel until you have proven to yourself that the maps will build well in a cluster environment at all.

I'm not sure how well your plan to run VMs on the cluster will work out, either. VMware uses their high range enterprise ESX server (and VMware Infrastructure 3) to do this sort of thing (like bouncing VMs to and from other machines without interruption), and clustering in this sense isn't like creating a single machine from many. The nodes have to be relayed the information to be processed.

Now, it still sounds like a great idea, you just might want to research the architecture a bit. Maybe get a clustered source compile working first, and then look at the mapping tools to see if they're cluster-able. They should parallelize well, but the tools might not exist to do this across nodes.

[:)]

before you get too excited: can q3map2 even run on a cluster? correct me if i'm wrong, but i don't think so...

Our friend mikeeusa is an expert q3map2 hacker, I'm sure help you here.