MG-RAST API - Changing outputs when downloading same data

Question

[fibar]

For a specific sample, I want to download the sequences that have been classified as phylum Bacteroidetes. This is the command I use:

curl -X GET -H "auth: pgsDxzBeMk287a8EUwinE9Vrk" "http://api.metagenomics.anl.gov/1/annotation/sequence/mgm4444444.4?evalue=10&identity=80&source=RefSeq&type=organism&filter_level=phylum&filter=Bacteroidetes" > Bacteroidetes_sequences

I've repeated it a couple of times. The thing is that for some samples I get significantly varying sizes in the different downloading trials.

e.g.

Sample01 - Bacteroidetes_sequences - Size 2 MB

Sample01 - Bacteroidetes_sequences - Size 30 MB

Any hint? Thanks in advance. Cheers!

Answer 1

[5heikki]

Make curl more talkative:

-v, --verbose
              Makes the fetching more verbose/talkative. Mostly useful for debugging. A line starting with '>' means "header data" sent by curl, '<' means "header data" received by curl that is  hidden  in
              normal cases, and a line starting with '*' means additional info provided by curl.

              Note that if you only want HTTP headers in the output, -i, --include might be the option you're looking for.

              If you think this option still doesn't give you enough details, consider using --trace or --trace-ascii instead.

              This option overrides previous uses of --trace-ascii or --trace.

              Use -s, --silent to make curl quiet.

Direct stderr to file:

2>stderr.log

Study the log. Perhaps your connection is timing out or closed from the server side or something..