Is it correct that I report them as genes that are a little up-regulated within cells?

I don't think it is a matter of correct or incorrect in this case. You can report whatever the data shows, and some people will believe that log2FC = 0.4 (and whatever p-adj) is upregulated, while others will not. In case you care, I'd be most likely in the latter category (tough to know for sure without additional information).

I think it also matters whether you are just reporting this as potentially upregulated, or trying to make a big case in your study for this particular gene. If the former, I would let that go in the spirit of information transparency. But if you are making a big case for the "upregulation" of this gene being important without any additional data, I'd have a problem accepting that.