Entering edit mode
5.7 years ago
m98
▴
420
I am making a private UCSC data hub and I am wondering what is the best location to host my hub . As I understand, it has a to be a place that is accessible via the web.
My question is: Is it possible to host the hub on an HPC cluster (like a company cluster or a university cluster?) If so, it is 'safe'? What kind of path would my data files have? (https? ftp?)
Thanks
I did. From what I understand, any Internet accessible place is fine ("an Internet-enabled web server or ftp server"). But my question was more: has anyone ever tried hosting it on an hpc cluster? I am not very experienced in ucsc data hubs (in particular I am not entirely sure how the bigDataUrl parameter would be set if my data is indeed on a cluster) so I was wondering if anyone out there had tried it.
Is your cluster accessible from internet (chances of that not being accessible are greater)? If you do decide to host your hub on a cluster it will have to be on an accessible node (all cluster nodes will rarely have directly inbound public connections). Your cluster admins would need to allow UCSC servers to reach in and access the data. A lot of this will depend on local security constraints/setup. You should talk with your system admins about this or use one of the public locations mentioned to host your data (again keep security of data in mind).
BTW: There is no benefit of a HPC cluster here since all the data is pre-computed.
If you just want to be able to view your data in UCSC Genome Browser then you may want to install a local copy of it.
Thanks, that makes sense! The only reason I was wondering about the cluster as a location was because I have a lot data files for the hub, all of which were made on the cluster. But you're right, doesn't seem to be a safe option.
See if your admins would install a local copy of UCSC browser so you can view the data locally/securely.
A web server is a web server. It's just another process, whether it is run from a cluster node or from any other host. Mainly, whatever files you want to make accessible through your hub should be visible to your web server — by visible, I mean accessible through normal, local file access on whatever machine is your web server.
You'll likely gain no more benefit from running a web server on a cluster node than on any other similar hardware.
Clusters are mainly used for computation, and you're not doing computation here, just serving up raw data from files. Web servers are generally not doing CPU-intensive work. It is likely that you do not want to use a cluster for running web services, for that reason.
Running this is as "safe" as the web server being used (Apache, nginx, etc.) and the network you're running this on. I'd keep a cluster firewalled away from open, public internet access, myself, and I suspect the people maintaining your cluster may prefer that, as well.