# scoring-rules's questions - English 1answer

105 scoring-rules questions.

### Using Stats to provide a score for a daily rating with low sample size

Context We have a register of people in our database, with a record of teams and leaders. Every day we send out a survey to each team asking them to rate their leader on whether they performed a ...

### How to construct a weighted Continuous Rank Probability Score Rule

0 answers, 11 views self-study scoring-rules
I am wanting to weight the continuous rank probability scoring rule so I can focus on the left side of the tail. Could someone please tell me how I would do this? I don't need code as such, just an ...

### 25 Error metrics for cross-validating Poisson models

I'm cross validating a model that's trying to predict a count. If this was a binary classification problem, I'd calculate out-of-fold AUC, and if this was a regression problem I'd calculate out-of-...

### Procedure for rescaling and Centering complex data

I have a question for which i requrest your comments and recommendations: In my research, I use a National level entrance test, which has high signficance for the students in their pursuit of higher ...

### 2 How to compare model fit of OLS and Poisson regression?

I have built two regression models to predict sales of different products based on a number of explanatory variables, with an offset term for the number of days each product was on sale. One is a ...

### What is null distribution of brier's score or logarithmic score? How to test them properly?

I would like to get some p-values for predictions in an independent test set according to proper scoring rules. So I would like to know if my brier score/logarithmic score is statistically ...

### 75 Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...

### 2 Estimate event probability from yes/no data

I need to predict (estimate) probabilities of (rare) events when the training data only contains the yes/no indicator. I.e., my target (dependent) variable is binary (logical). What I need is not ...

### 1 OK to use residual sum of squares for cross-validation of binary outcome?

For an OLS model the mean squared error can be used to assess the fit of the trained model on the validation data. What is the equivalent for a logistic regression model? Can I simply use the ...

### pseudo R2 as xgboost objective function

I want to use a custom objective function with xgboost: 1 - (log(y) - log(p)) / (log(y) - log(q))y = true value, p = my probabilities, q = some other base ...

### Comparing goodness of fit of quasi-Poisson regression model predictions vs. unknown forecast

0 answers, 20 views scoring-rules quasi-likelihood
I have built a quasi-Poisson regression to predict sales of different products based on a number of explanatory variables, with an offset term for the number of days each product was on sale. To ...

### 17 How to choose between ROC AUC and F1 score?

6 answers, 17.778 views machine-learning modeling roc scoring-rules
I recently completed a Kaggle competition in which roc auc score was used as per competition requirement. Before this project, I normally used f1 score as the metric to measure model performance. ...

### 1 comparing two estimators [closed]

0 answers, 57 views estimation estimators scoring-rules
I tried to solve the problem of comparing two estimators for soccer matches. The "estimators" are actually two punters trying two predict games results. The predicted value is between 0 and 1. The ...

### 6 Is accuracy an improper scoring rule in a binary classification setting?

1 answers, 198 views probability accuracy scoring-rules
I have recently been learning about proper scoring rules for probabilistic classifiers. Several threads on this website have made a point of emphasizing that accuracy is an improper scoring rule and ...

### 1 Interpreting score function in Cox model

0 answers, 24 views forecasting scoring-rules

### 6 How to learn Bayesian Network Structure from the dataset?

2 answers, 988 views bayesian bayesian-network scoring-rules
I need to learn a Bayesian Network Structure from a dataset. I read the book titled "Learning Bayesian Networks" written Neapolitan and Richard but I have no clear idea. According to the book from ...