Biostar Beta. Not for public use.
What makes a good bionformatics prediction service from the point of user experience?
0
Entering edit mode
11 months ago
aquaq • 10

Most of us tried to use one or more bioinformatics prediction services. We probably all have came across good and bad ones. How do you decide which one to use? What features make it more/less probable that you would stick to one? Let's assume that the underlying algorithms have the same performance.

Some viewpoints that could be discussed:

  • design
  • description of the algorithm and server
  • input and output format, downloading
  • error handling
  • batch prediction
  • reliability of server availability
  • date of last update

... any other criteria that you could think of?

And if we do not assume that the performances of the algorithms are the same: have you ever chosen a service that you knew have lower accuracy than an other? Why?

EDIT - I see, I wasn't clear enough, this is one of my answer for a comment.

My question is not about the performance of the predictions but rather how to make their usage user-friendly by setting up webpages that are easily usable, stable and provide all the features that would help the work of a user (both other bioinformaticians and experimentalist researchers).

As for me, I do now want to create such a service/serve/web page just for myself to play with and create a paper, but something that are really useful for others. I think many of us stumbled across pages with poor documentation, impossible input/output format or visualization, etc. I have also heard and read complaints from experimentalist of services that may be straightforward for bioinformaticians, but hard to use for them. I would like to assess the needs of potential users and experts who have already have experience using these things and collect good and bad practices besides what I had seen.

I have just found the expression , it's user experience design.

ADD COMMENTlink
4
Entering edit mode

To me, the question is not clear. What do you want to predict...? (The weather? The stock market? Protein structure? My future choice for lunch? DNA alignments? )

In addition: Are you sure you talk about "servers" and don't mean "service"?

ADD REPLYlink
0
Entering edit mode

Thanks for the feedback, I will try to make myself clearer: bioinformatics researchers often make prediction services for protein structure/function/subcellular localizations/interactions/PTMs/glycolization/disorder/transmembrane topology whatever (same things apply for DNA/RNA predictions but I am less familiar with these). For most of these task there are at least a few different services that can be used (e.g https://omictools.com/protein-sequence-analysis-category).

I am interested in how others choose from these services the ones they actually use.

ps. services may better describe what I wanted to ask, thank you

ADD REPLYlink
0
Entering edit mode

"prediction of protein properties"

Add words above (or something similar) to the title of this post to clarify your question.

ADD REPLYlink
0
Entering edit mode

I've edited the post title and added that in.

ADD REPLYlink
0
Entering edit mode

I'm sorry, I have to revert it, I have already added 'bioinformatics'. Protein properties were only examples for a question above, I am interested in all kinds of bioinformatics prediction services in general.

ADD REPLYlink
1
Entering edit mode

This is a bioinformatics forum, so that part on adding 'bioinformatics' adds absolutely nothing of relevance. Asking "what makes a prediction service good" makes for an over-generalization that assumes that one metric (or even a finite set of metrics) can be used to evaluate all prediction services. The field does not work that way. Each prediction service has a set of metrics which taken together can give you a picture of how close they get to certainty, but there is no global goldilocks standard.

Think of it this way - the data you're gathering here will help you make a service that predicts which service is best. But do I want which service gets updates the most frequently or which gets the largest coverage of the underlying entities? How can you tell me which is the "best" when defining a quantitative scale is not straightforward? Whether I use snpEff or VEP is based more on what I want to do with the annotations downstream (which depends on the biologists that will look at the annotated variants) than how frequently they're updated or how well they adhere to Sequence Ontology terms. This manual component can skew even the best metrics that try to evaluate a service.

ADD REPLYlink
0
Entering edit mode

My question is not about the performance of the predictions but rather how to make their usage user-friendly by setting up webpages that are easily usable, stable and provide all the features that would help the work of a user (both other bioinformaticians and experimentalist researchers).

As for me, I do now want to create such a service/serve/web page just for myself to play with and create a paper, but something that are really useful for others. I think many of us stumbled across pages with poor documentation, impossible input/output format or visualization, etc. I have also heard and read complaints from experimentalist of services that may be straightforward for bioinformaticians, but hard to use for them. I would like to assess the needs of potential users and experts who have already have experience using these things and collect good and bad practices besides what I had seen.

I have just found the expression , it's user experience design.

ADD REPLYlink
0
Entering edit mode

Ah, UX design - that would be wonderful, seeing good UX on bioinformatics resources. I mean, I do find plain simple HTML tables the best way to represent data. Sometimes even plain text output works, but there are definitely good websites and weird ones.

In my experience, batch input is super important. If you don't let me batch input using a textarea of a file upload, odds are I'll switch to a command line version of your tool if it exists, or abandon your tool altogether.

ADD REPLYlink
0
Entering edit mode

Thank you!

I will make a mental not to myself - finding the right expression may worth a lot more than dozens of words. Just to enhance other's "forum experience".

ADD REPLYlink
0
Entering edit mode

user experience and user interface design are not the same. They need to work together seamlessly for user nirvana. No two users are the same and you can't please everyone no matter how good you are or how hard you try :)

ADD REPLYlink
3
Entering edit mode
9 months ago
EMBL Heidelberg, Germany

A good prediction service would be one that has a documented high probability of outputting a true/useful result under real-life conditions. Unfortunately most predictions are tested by one or more of cross-validation, use of simulated data or use of test data that are not independent from the training data, e.g. GO annotations and protein-protein interactions are highly linked data because annotations are derived from the same papers as the interactions. On top of this, one can only publish a service if it provides numbers that say it's doing better than other similar services which I think biases reporting on true performance. So in my view, the real life performance of many prediction services is unknown. This might be fine if you're using the predictions in some sort of data integration or statistical framework in which you don't care about a particular prediction instance but when you want to use a prediction tool to design experiments, you want to have reliable confidence measures on individual predictions.
Efforts like CASP, in which prediction methods are tested in a double-blind approach on previously unseen experimental data, are important in building confidence in the tools.
In the end, people will use what they are comfortable with: the service is easy to use, the process leading to the predictions is clearly explained and can be understood intuitively and the output is clear and presented in a usable way. In my experience, users will test a new service with something for which they know what the result should be. When the service doesn't give the expected result, it is considered unreliable and usually never looked at again.

ADD COMMENTlink

Login before adding your answer.

Similar Posts
Loading Similar Posts
Powered by the version 2.1