If I run this:

separate_rows(data, ALT, AF, convert = TRUE)

where data is your 1st data frame I obtain the second data frame.

How did you run the function separate_rows()?

You could use a Python script to do this easily:

#!/usr/bin/env python

import sys

headers = None
idx = 0
for line in sys.stdin:
    elems = line.rstrip().split('\t')
    if idx == 0:
        headers = elems
        sys.stdout.write(line)
    else:
        items = {x:y for x,y in zip(headers, elems)}
        alleles = items['ALT'].split(',')
        afs = items['AF'].split(',')
        for ai in range(len(alleles)):
            items['ALT'] = alleles[ai]
            items['AF'] = afs[ai]
            sys.stdout.write('{}\n'.format('\t'.join([items[x] for x in headers])))
    idx += 1

For example:

$ ./split.py < variants.txt
CHROM   POS REF ALT TYPE    AF
chr1    1   A   T   MISSENSE    0.23
chr2    1   A   T   MISSENSE    0.17
chr2    1   A   G   MISSENSE     0.09

Write it out to a file and bring that back into R:

$ ./split.py < variants.txt > variants.split.txt

Using data.table:

library(data.table)

x <- fread("CHROM POS REF ALT TYPE AF
chr1 1 A T MISSENSE 0.23
chr2 1 A T,G MISSENSE 0.17,0.09")

x[, lapply(.SD, function(x) unlist(tstrsplit(x, ",", fixed = TRUE))),
    by = .(CHROM, POS, REF, TYPE)
  ][, .(CHROM, POS, REF, ALT, TYPE, AF = as.numeric(AF))]
#    CHROM POS REF ALT     TYPE   AF
# 1:  chr1   1   A   T MISSENSE 0.23
# 2:  chr2   1   A   T MISSENSE 0.17
# 3:  chr2   1   A   G MISSENSE 0.09

Below should work with auto type conversion, but it fails, as the first value "T" gets converted as logical "TRUE", next "T,G" as character, then when binding it, unfortunately, errors out:

x[, lapply(.SD, function(x) unlist(tstrsplit(x, ",", fixed = TRUE, type.convert = TRUE))),
    by = .(CHROM, POS, REF, TYPE)]
# Error in `[.data.table`(x, , lapply(.SD, function(x) unlist(tstrsplit(x,  : 
#   Column 1 of result for group 2 is type 'character' but expecting type
#   'logical'. Column types must be consistent for each group.

Related SO post with other alternative solutions:

• Split comma-separated strings in a column into separate rows