Thank you for pointing me in the right direction! This is how I solved it and applied it to every VCF in dir. I could not quite get bcftools annotate to work, likely due to a local server issue, but vcf-annotate works fine as well.
for i in *.vcf; do echo "annotating $i";
bcftools query -f '%CHROM\t%POS\t%REF\t%ALT\t[%AF]\n' $i > tmp.tsv
bgzip -f tmp.tsv
tabix -f -b 2 -s 1 -e 2 tmp.tsv.gz
cat $i | vcf-annotate -a tmp.tsv.gz -c CHROM,POS,REF,ALT,INFO/AF -d key=INFO,ID=AF,Number=1,Type=Float,Description='Variant allelic fraction' > "${i//.vcf}"_vaf.vcf
done