Here is one solution:

$ ls -1 *.csv
file.345.csv
file1.10.csv
file2.87.csv

$ for i in *.csv; do name=$(echo ${i} | cut -f1 -d "." ); echo ${name} ; echo mv ${i} ${name}.csv; done
file
mv file.345.csv file.csv
file1
mv file1.10.csv file1.csv
file2
mv file2.87.csv file2.csv

Remove echo before mv when you are sure things look ok.

My goal is to remove the second "column", so the 10 in the first case, the 87 in the second and the 345 in the third.

This statement is confusing. On one hand, you are (OP) saying you want to remove second column and on the other hand, you are saying 10 in first case. What is 10, 87 and 345 in first, second, third cases? Do you want to rename files or do you want to remove the second column in all the files?

If it is renaming and pattern is exactly same, try following dry-run:

$ tree .
.
├── file1.10.csv
├── file2.87.csv
└── file3.345.csv

$ rename -n 's/(.*)\..*\.(.*)/$1\.$2/' *.csv

'file1.10.csv' would be renamed to 'file1.csv'
'file2.87.csv' would be renamed to 'file2.csv'
'file3.345.csv' would be renamed to 'file3.csv'

Remove -n if you are okay with dummy run.

bash loop:

$ for i in *.csv; do echo $i ${i%%\.*}.csv;done

Replace echo with mv if you are okay with output

with parallel

$ parallel --dry-run --col-sep "\." mv {1}\.{2}\.{3} {1}\.{3}  ::: *.csv

Remove --dry-run if you are okay with output