There is not a lot to go on here but one thing to check is whether your account has write permissions to the working directory. Sounds like muscle may not be able to write the output file.

I think the best way to troubleshoot this is by first doing one of the examples that come with MEGA. If they work, at least you would eliminate any installation problems brought up by @genomax. That would mean that your problem is dataset-specific. For example, the sequences may be too long, or not properly formatted.