Creating a simple weighting and scoring system

[Greg Bernhardt]

TL;DR Summary

Looking for how to create a score of three unequal metrics

Say I want to create/chart a single time series score for 3 unequal metrics.

Let's say those metrics are PF-related:

1. New registrations that turn into PF Advisors
2. New registrations
3. Page views

The first thing I'd think to do is try to ballpark a multiplier for how much more valuable one metric is over the one below it. New registrations that turn into PF Advisors are the most valuable but also the least likely to occur so maybe that needs a large multiplier. Then you have a common metric like page views, this may not even need a multiplier because it will be high.

So how do I approach creating a weighted score between them that will communicate progress accurately?

 

[Dale]

For a single metric the key is getting a sensible weighting. So you can ask questions like “if I gained 1 registration and lost 10 page views, would I consider it progress?”

Do you have some other (independent) variables that you think drive changes in these three (dependent) variables? If so then you could also do something like a multivariate anova or a principal component analysis

 

[Frabjous]

You could do multiplicative instead of additive. Or take functions of the numbers.

I am not sure that becoming a PF advisor is a good metric. I think it is too rare an event and will look like noise. How about still active after a certain period of time? Or reach a number of posts?

 

[Greg Bernhardt]

I am not sure that becoming a PF advisor is a good metric.

True, we could say members that reach 10 posts

 

[WWGD]

I suspect we may be able to use Persistent Homology here, in the sense of seeing how many people persist beyond a basic initial point , like that of registering, then seeing whether they evolve over time into , like @Frabjous said, a certain number of points or advisor status, number of upvotes, or a certain ratio of posts per day, upvotes per post, etc. Those who , say, have a total of fewer than 10 posts in a year would be consider noise, others that somehow progress along in time would be part of the signal



Persistent Homology is intended to separate signal; the traits that persist through several levels of resolution, from noise ; those traits/classes that do not persist. Here, persisting would go from signing up as a user , then having a total number of posts, through maybe becoming an advisor, etc. , over time. Again, still at a basic level .

Just throwing it in here, in case someone knows more about it or has a better idea on how to set up the bar code. Maybe someone with strong skills in both theory and the applied, like @BvU , or @Mark44 ?

I'll try to turn it into a project , as time allows.