r/pythonhelp • u/Minute-Raspberry9401 • Sep 16 '22
Am I understanding this process correctly?
Hello. I am going through a machine learning class online (just a series of videos, no instructors with feedback) and am trying to ensure I understand the process of building and using a model correctly.
The class has a project at the end in which I am to build a classifier. The goal of the project is to build a model that has a Recall score of at least 50%.
My approach is as follows:
* Clean / prep dataset
* fix datatypes
* remove unwanted columns
* remove NA's
* For each type classifier I want to consider:
* Build a pipeline:
* Scale Data
* Use Feature Selection
* Classify
(e.g.):
```
Pipeline([
('Scale Data', StandardScaler()),
('Select Features', SelectKBest(chi2, k=MY_COLS)),
('Classify Data', LogisticRegression())
])
```
(at this point, if I wanted to compare a NaiveBayes with a LogisticRegression, I have two pipelines, one with a NaiveBayes classifier and one with a LogisticRegression classifier)
* Tune hyperparameters of each model to find optimal version of each model:
* I'm using GridSearch with scoring set to 'Recall'. Since I did not split my dataset into Training/Test beforehand, I am using:
StratifiedKFold for cross-validation
* Compare scores of each model's optimal parameters to decide which model is best
* Save optimal model for reuse later (e.g. in Pickle)
My questions are:
1. Does this process seem correct?
2. If doing this process manually without pipelines:
do I normalize labels as well as features (if for regression)?
3. Once my model is fitted and saved, to use it on a new unlabelled dataset is it as simple as:
```
## load trained model from pickle as model
## load unlabelled data as unlabelled_X
predictions = model.predict(unlabelled_X)
```
4. If this is how I use the model, how can I be sure that it goes through the entire pipeline I created with the optimal model (e.g. scaling and feature selection prior to predicting)?
5. Once time passes and the previously unlabelled data can be 'labeled', can I improve the model by training it with this new data using:
model.fit(newly_labelled_X, newly_labelled_Y)
Any and all guidance and answers to my questions are greatly appreciated. Thank you in advance for your time and help.
2
Upvotes