Anyone tried running ROMS on HPC Cloud (e.g. Amazon)?

[rsignell]

ROMS Folk,
Has anyone tried running ROMS on HPC Cloud (e.g. Amazon)? I was looking at the web page for High Performance Computing (HPC) on Amazon Web Services (AWS) and then watched this video on How to set up a 64 Core HPC system on AWS in 10 minutes. Looks easy and cool! Seems like it would be interesting to spec out performance and cost. Was tempted to try a test, but thought I'd see whether anyone else had already done it and has some experience to report.
-Rich

[kate]

Not yet, but Rob has tried some cloud computing tests. He says he'll do it for you for consultant rates.

[andres]

For what I understand, AWS is a bunch of dual Quadcore-CPUs with a lot of memory and disk space. A problem is that they are connected with Gigabit cables and not Infiniband.

It would be good to know if the chance to use a lot of cores compensates the "slow" communication between computing nodes.

[cermak]

ROMS Folk,
Has anyone tried running ROMS on HPC Cloud (e.g. Amazon)? I was looking at the web page for High Performance Computing (HPC) on Amazon Web Services (AWS) and then watched this video on How to set up a 64 Core HPC system on AWS in 10 minutes. Looks easy and cool! Seems like it would be interesting to spec out performance and cost. Was tempted to try a test, but thought I'd see whether anyone else had already done it and has some experience to report.
-Rich

ROMS compiles nicely with Fedora 16 with mpich2 and netcdf rpms (virtually right out of the box). There is a boatload of Fedora 16 images at AWS.

It would be interesting to compare the individual nodes (unfortunate that they use Gb networking). One could hope that they use better networking between the Cluster Compute units.

The Free Tier provides some nice resources during the first year after signing up.

[iof]

Anyone tried AWS or Azure more recently? I have a grant from Azure, working on one from Amazon. On both cases they have offered their architect services to build the set-up. This service is not part of the regular (paid) support services, so I'm reaching out

1) If anyone has succesfully used either service: what set-up did you use?
2) If you have wanted to try it out but didn't, what did you find discouraging? (Both Azure and AWS are interested in offering a solution for this).
3) Would having a ready made template with HPC-storage-pc setup make it easier for you?
4) What other info would you be interested on?

[stef]

Yes I'm trying AWS. At the moment I'm doing some benchmark testing, see https://github.com/poidl/awsroms and https://poidl.github.io/awsroms/. I still have few results, and they are not interpreted properly. I'd like to become an independent (sub-) contractor, so I'm also very interested in estimating financial cost. If anybody knows good alternatives to AWS, please let me know, I have no good overview of the market. At the moment I'm thinking that it may be cheapest for a private contractor to apply for funding from public agencies/institutions and use their equipment.

[Tomasz]

You can also try Microsoft Azure

[johnluick]

A few years ago we got interested in this and we simply copied the oceanM, ocean.in, and input data files up to Amazon and had it running in about ten minutes from go to whoa. The oceanM was an mpich2/mpif90/ifort/netcdf4 compile on an HPC system. The next step would be to re-compile on Amazon in order to get the optimisations. We never got that far because it was a proof of concept test and we had access to a "free" machine (as opposed to Amazon's rather costly one). But it nice to know that it was available and could be done fairly painlessly.

[rsignell]

I just discovered this paper https://www.geosci-model-dev-discuss.ne ... 17-270.pdf in which the authors use the AWS Elastic Network Adaptor (ENA) to deliver up to 20 Gbps with low latency. They were able to obtain similar performance compared to their 128 core HPC Cluster with Infiniband (up to 40 Gbps with very low latency), for their model grid sizes of roughly 200x200x40 up to 800x800x40. In addition, they point out that the Azure Cloud has Infiniband. I've been going around saying that we can't effectively run models like ROMS in the Cloud due to networking performance, but no longer!