r/pythonhelp • • Sep 16 '22

Am I understanding this process correctly?

Hello. I am going through a machine learning class online (just a series of videos, no instructors with feedback) and am trying to ensure I understand the process of building and using a model correctly.

The class has a project at the end in which I am to build a classifier. The goal of the project is to build a model that has a Recall score of at least 50%.

My approach is as follows:

* Clean / prep dataset
    * fix datatypes
	* remove unwanted columns
	* remove NA's


* For each type classifier I want to consider:
	* Build a pipeline:
	     * Scale Data 
	     * Use Feature Selection
	     * Classify

	     (e.g.):  

	     ```
		Pipeline([
			('Scale Data', StandardScaler()),
			('Select Features', SelectKBest(chi2, k=MY_COLS)),
			('Classify Data', LogisticRegression())
		])
	     ```  


(at this point, if I wanted to compare a NaiveBayes with a LogisticRegression, I have two pipelines, one with a NaiveBayes classifier and one with a LogisticRegression classifier)

* Tune hyperparameters of each model to find optimal version of each model:
    * I'm using GridSearch with scoring set to 'Recall'.  Since I did not split my dataset into Training/Test beforehand, I am using:

        StratifiedKFold for cross-validation


* Compare scores of each model's optimal parameters to decide which model is best


* Save optimal model for reuse later (e.g. in Pickle)

My questions are:

 1. Does this process seem correct?

 2. If doing this process manually without pipelines:  
     do I normalize labels as well as features (if for regression)?

 3. Once my model is fitted and saved, to use it on a new unlabelled dataset is it as simple as:  

     ```
      ## load trained model from pickle as model
      ## load unlabelled data as unlabelled_X
      predictions = model.predict(unlabelled_X)
     ```

 4.  If this is how I use the model, how can I be sure that it goes through the entire pipeline I created with the optimal model (e.g. scaling and feature selection prior to predicting)?  


 5.  Once time passes and the previously unlabelled data can be 'labeled', can I improve the model by training it with this new data using:
     model.fit(newly_labelled_X, newly_labelled_Y)

Any and all guidance and answers to my questions are greatly appreciated. Thank you in advance for your time and help.

2 Upvotes

0 comments sorted by