Why parallel?
First of all, I just realized that it has been a while since I posted something here. It is a bit odd since I was not busy at all so maybe my business has a high correlation with me posting stuff :S
I am running many experiments and some of them are taking a lot of time. No, ... A LOT! you might say: "Come on you guys are in CS and should have the best computers and stuff to run programs". Well the answer might be true, but corresponding to that demands are higher as well. If you want to say my experiments are statistically significant, you should run them at least 30 times, which in my case it can translate into 7 days! I finally decided to implement a script to take advantage of many CPUs on the network at the same time to run these separate parts in parallel so I can see my results 30 times faster! It was a lot of pain and involved wrestling with Linux commands but it was finished today, thanks to the help of my lab mates! Is it working that fast? well due to my lack of luck our main cluster is down, but I am aiming to run it as soon as it is back. Wish me luck! :)
I am running many experiments and some of them are taking a lot of time. No, ... A LOT! you might say: "Come on you guys are in CS and should have the best computers and stuff to run programs". Well the answer might be true, but corresponding to that demands are higher as well. If you want to say my experiments are statistically significant, you should run them at least 30 times, which in my case it can translate into 7 days! I finally decided to implement a script to take advantage of many CPUs on the network at the same time to run these separate parts in parallel so I can see my results 30 times faster! It was a lot of pain and involved wrestling with Linux commands but it was finished today, thanks to the help of my lab mates! Is it working that fast? well due to my lack of luck our main cluster is down, but I am aiming to run it as soon as it is back. Wish me luck! :)
0 Comments:
Post a Comment
<< Home