This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Basic computer question: why does my computer slow AFTER running a script?

Hi All,

This is a basic computer question that could help enlighten a biologist turned data analyst.

I have a script that forks into 6 processes, my computer has 8 cores and 8GB of RAM.

After running the script and quitting my terminal, my computer is extremely slow. What's the reason behind this? How can I fix it?

I know this is probably a stupid question, but I'm curious.

Thanks

computer ram procs memory
• 3,925 views • link
updated by Ram Ram • written by QVINTVS_FABIVS_MAXIMVS QVINTVS_FABIVS_MAXIMVS

Hi,

I think we'd need more data on the problem, like if any external resources are being used (files, databases etc). Also, what does this script do?

• 0 views • link
written by Ram Ram

Sure,

The perl script pulls certain binary files from a parent directory and then forks into 6 processes .Next, the binary files are converted to .txt files using an outside script which it calls using system().

.txt files are opened and read, parsed out for data, and then deleted while the parsed out data is printed to a file.

I assumed this is a common problem with forking, I never had this issue before I started writing scripts that fork processes.

• 0 views • link
updated by Ram Ram • written by QVINTVS_FABIVS_MAXIMVS QVINTVS_FABIVS_MAXIMVS

Hi,

Like you, I am a biologist who has started doing some data analysis, and I too use a forking perl script that reads data from external files. I like stupid questions, I ask them often myself.

Do you close the files after you have read them? It is not essential to close files, the script will still work, but I guess the open file will use up some computer resources.

Are you using Parallel::ForkManager perl module from cpan? Avoid forking a forked process. Try forking 4 processes rather than 6

How big are the files that you are reading from? are they compressed, and how big are the files that you write too and subsequently delete?

It may be useful to run your script and have a second terminal window and type htop to see how much memory is being used, and which cores are being used.

• 0 views • link
updated by Ram Ram • written by rbagnall rbagnall

I agree. Check memory usage, ensure you are closing all handles, check usage guidelines of external program, do not explicitly split into multiple threads because sometimes, a single thread is better or the splitting algorithm you use might not gel well with the way the program works.

• 0 views • link
written by Ram Ram
  1. Yes I close my files.
  2. Yes I used ForkManager. I might use 6 but I'm writing this script for a computer with 16 processors and will be expanding to 12 or 14
  3. Around 250MB are the .txt files, they are not compressed
  4. Interesting. But it's still slow after 5 or 10 minutes the process has completed running.
• 0 views • link
updated by Ram Ram • written by QVINTVS_FABIVS_MAXIMVS QVINTVS_FABIVS_MAXIMVS

A trivial answer apart form what others have already said: perhaps your CPU is heated up!!

• 0 views • link
written by Bharat Iyengar Bharat Iyengar

I was thinking that it could be a disk read/write thing, especially if OP doesn't have a SSD.

• 0 views • link
written by 5heikki 5heikki

Assuming you're running Linux, what happens if you try the following (after a script runs to completion and your computer runs slowly):

$ sudo su -
...
# sync; echo 3 > /proc/sys/vm/drop_caches
• 0 views • link
written by Alex Reynolds Alex Reynolds

Hello QVINTVS_FABIVS_MAXIMVS!

We believe that this post is does not fit the main topic of this site.

Whilst you may well be doing bioinformatics with your scripts, this is not technically a bioinformatics question and would probably be better off posted on the SuperUsers StackExchange site.

For this reason we have closed your question. This allows us to keep the site focused on the topics that the community can help with.

If you disagree please tell us why in a reply below, we'll be happy to talk about it.

Cheers!

• 0 views • link
written by User 59 User 59

1 answer

This might answer it, I asked last 2 years back on SuperUser.

http://superuser.com/questions/511470/why-system-still-hangs-slows-after-process-is-finished

• 0 views • link
written by Sukhi Singh Sukhi Singh

8 GB RAM and 8 cores. I'm pretty sure OP's not running into swap. A htop should shine some light in the situation for us.

• 0 views • link
written by Ram Ram

Log in to answer this question.

More posts like this