# scoring-rules's questions - English 1answer

99 scoring-rules questions.

### 1 OK to use residual sum of squares for cross-validation of binary outcome?

For an OLS model the mean squared error can be used to assess the fit of the trained model on the validation data. What is the equivalent for a logistic regression model? Can I simply use the ...

### pseudo R2 as xgboost objective function

I want to use a custom objective function with xgboost: 1 - (log(y) - log(p)) / (log(y) - log(q))y = true value, p = my probabilities, q = some other base ...

### Comparing goodness of fit of quasi-Poisson regression model predictions vs. unknown forecast

0 answers, 13 views scoring-rules quasi-likelihood
I have built a quasi-Poisson regression to predict sales of different products based on a number of explanatory variables, with an offset term for the number of days each product was on sale. To ...

### 15 How to choose between ROC AUC and F1 score?

6 answers, 15.601 views machine-learning modeling roc scoring-rules
I recently completed a Kaggle competition in which roc auc score was used as per competition requirement. Before this project, I normally used f1 score as the metric to measure model performance. ...

### 64 Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...

### 1 comparing two estimators [closed]

0 answers, 39 views estimation estimators scoring-rules
I tried to solve the problem of comparing two estimators for soccer matches. The "estimators" are actually two punters trying two predict games results. The predicted value is between 0 and 1. The ...

### 4 Is accuracy an improper scoring rule in a binary classification setting?

1 answers, 95 views probability accuracy scoring-rules
I have recently been learning about proper scoring rules for probabilistic classifiers. Several threads on this website have made a point of emphasizing that accuracy is an improper scoring rule and ...

### 1 Interpreting score function in Cox model

0 answers, 21 views forecasting scoring-rules

### 6 How to learn Bayesian Network Structure from the dataset?

2 answers, 946 views bayesian bayesian-network scoring-rules
I need to learn a Bayesian Network Structure from a dataset. I read the book titled "Learning Bayesian Networks" written Neapolitan and Richard but I have no clear idea. According to the book from ...

### 1 Bayesian Network: Scoring functions for structure learning?

1 answers, 1.072 views bayesian-network scoring-rules
Which are the widely used scoring functions for structural learning? More, specifically I am interested in scoring function that favours the random variables which have binary possible states. For an ...

### 2 What is an intuitive explanation of the Bayesian score?

0 answers, 280 views bayesian-network scoring-rules
I am familiar with the Bayesian score, which is used to compare competing structures of a BN. However, I have difficulty in understanding how the Bayesian score formula below is derived. 1) What is ...

### different variable distributions in train vs. score datasets - keep or drop them?

we work in such a cycle : every month we create a new predictive model, based on data known about half a year before. Then we make a prediction on the newest data available, and that scoring goes to ...

### 1 What is the benefit of developing different scores for LGD modelling?

In the LGD Model flow presented in the figure 4.13 in the book "Developing Credit Risk Models Using SAS Enterprise Miner and SAS/STAT: Theory and Application" which is partially available on the web: ...

### 1 Verification on probabilities and interpretation of evaluation scores

my situation is the following: I have a matrix consists of purchase probabilities for different products per user. Retrospectively I have another matrix consists of real-purchases. Now my task is to ...

### 1 How to combine purchase and click data togehter in sparse matrix

my problem is the following: I have purchase probability estimations of different products. The model behind don't take care of any inter-correlations through these products. So my task is to re-...

### 6 Using proper scoring rule to determine class membership from logistic regression

I am using logistic regression to predict likelihood of an event occurring. Ultimately, these probabilities are put into a production environment, where we focus as much as possible on hitting our "...