This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Submitting RNA-seq to Supercomputer

Hi,

I'm totally new to running scripts, and I'm struggling to get going with submitting a job to our university supercomputer.

Basically, I want to index a Fasta reference genome before I perform my mapping, using BWA to do this.

The script provided by the university, to submit jobs to the supercomputer, is not that clear to me, and I was hoping that someone might be able to help. The documentation gives the following, which I can follow:

#!/bin/sh
# Grid Engine options (lines prefixed with #$)
#$ -N hello              
#$ -cwd                  
#$ -l h_rt=00:05:00 
#$ -l h_vmem=1G
#  These options are:
#  job name: -N
#  use the current working directory: -cwd
#  runtime limit of 5 minutes: -l h_rt
#  memory limit of 1 Gbyte: -l h_vmem

However, I have zero idea of where to insert the following script: 

bwa index -a bwtsw Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz

Also, this is probably really daft, but I also have no idea of how to reference the location of the file, so that the script knows where it is...

Sorry if this is all very basic stuff, and thanks for any help!

Matthew

rna-seq fasta next-gen assembly
• 2,217 views • link
updated by Biostar Biostar • written by matthew.sinton • matthew.sinton

You would add bwa command line at the end of the script above (keeping in mind $PATH considerations). Initial part with # is setting up parameters for your job scheduling system. You would need to provide correct parameters. This is just a skeleton example.

I recommend that you follow basic UNIX tutorial here. Investing sometime in it will be forever useful.

• 0 views • link
written by GenoMax GenoMax

Thanks for the quick reply! So when I add the bwa command, when I put in the file name, do I just have to ensure that I include the full path so that it knows where the file is?

• 0 views • link
written by matthew.sinton • matthew.sinton

You could include full path for now as you learn about unix file system and relative path concepts.

• 0 views • link
written by GenoMax GenoMax

A runtime limit of 5 minutes is very short.

• 0 views • link
written by Michael Dondrup Michael Dondrup

1 answer

As @genomax said, this is just an example. The example script given by your University give you access to 1Go of RAM for a 5 minutes running time.

You need first to upload your data to the server, something like this :

scp /home/matthew/Documents/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz metthew_login@University_adress:/home/matthew_login/work/genome/

Then, log in on the server and create a bash script ( index_human.sh ) where you'll write something like this

#!/bin/sh
# Grid Engine options (lines prefixed with #$)
#$ -N create_human_index_GRCh38     
#$ -cwd                  
#$ -l h_vmem=30G
#  These options are:
#  job name: -N
#  use the current working directory: -cwd
#  memory limit of 1 Gbyte: -l h_vmem

bwa index -a bwtsw /home/matthew_login/work/genome/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz

Adapt the location of Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz

Seems like your cluster is SGE managed

Launch your bash script :

qsub index_human.sh

Check the ouput of your script in your current directory to see how is goes

• 0 views • link
written by Bastien Hervé Bastien Hervé

Adding on this, I recommend against the use of scp, better use rsync instead. Reasons why, read this.

• 1 views • link
written by ATpoint ATpoint

Log in to answer this question.

More posts like this