Creating an Inference Script: A Practical Guide for Production

Previously in this course, we explored Exporting Trained Models: Serialization with Pickle and Joblib to save our progress. Now that you have a serialized model file, the next logical step in the The Machine Learning Workflow: From Data to Deployment is to build an inference script.

An inference script is the bridge between your static model file and the real world. It turns raw input data into actionable insights by handling the loading, pre-processing, and prediction logic in a reproducible way.

Why You Need a Dedicated Inference Script

You shouldn't perform predictions inside your training notebook. Notebooks are for experimentation; scripts are for reliability. A production-grade inference script ensures that:

1. Consistency: The same preprocessing steps (scaling, encoding) applied during training are applied to new data.
2. Standardization: Your model outputs are formatted in a way that downstream systems (like a web API or a database) expect.
3. Error Handling: You can catch malformed input data before it crashes your model.

Designing the Inference Function

A professional inference script should be modular. Instead of writing one giant block of code, we encapsulate the logic in a function that takes raw input and returns a prediction.

Worked Example: Building the Predictor

Assuming you have already saved your pipeline.joblib file, here is how you structure a clean, production-ready script.

PYTHON
import joblib
import pandas as pd
import logging

# Set up logging for production visibility
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def generate_prediction(input_data, model_path=CE9178">'model/pipeline.joblib'):
    CE9178">"""
    Loads a model and returns predictions for the given input.
    
    Args:
        input_data(dict or pd.DataFrame): The raw features for prediction.
        model_path(str): Path to the serialized pipeline.
        
    Returns:
        dict: A dictionary containing the prediction and status.
    """
    try:
        # 1. Load the model
        model = joblib.load(model_path)
        
        # 2. Ensure input is a DataFrame
        if isinstance(input_data, dict):
            input_data = pd.DataFrame([input_data])
            
        # 3. Predict (The pipeline handles all scaling/encoding)
        prediction = model.predict(input_data)
        
        return {
            "status": "success",
            "prediction": float(prediction[0]),
            "model_version": "v1.0"
        }
        
    except Exception as e:
        logger.error(f"Inference failed: {e}")
        return {"status": "error", "message": str(e)}

# Example Usage
new_sample = {"feature_1": 0.5, "feature_2": 1.2}
result = generate_prediction(new_sample)
print(result)

Integrating with the Project

For our ongoing project, we will create a file named predict.py in our root directory. This script will import our pipeline and expose the generate_prediction function. By keeping this logic separate, we can easily import this function later when we build a web interface or a scheduled batch job.

Hands-on Exercise

1. Create a file named predict.py in your project folder.
2. Copy the generate_prediction function above into the file.
3. Update the model_path to point to the location where you saved your pipeline from the previous lesson.
4. Write a small if __name__ == "__main__": block at the bottom of your script that calls the function with a dummy dictionary of features and prints the output.
5. Run the script from your terminal using python predict.py to verify it loads and executes without errors.

Common Pitfalls to Avoid

• Feature Mismatch: The most common error is providing a dictionary with different keys than what the model was trained on. Ensure your input keys match the column names of your training data exactly.
• Hardcoding Paths: Avoid hardcoding absolute file paths (like C:/Users/Name/...). Use relative paths or environment variables so your code works on any machine.
• Ignoring Data Types: If your model expects a float, passing a string will cause an error during the predict step. Always validate your input types before passing them to the pipeline.
• Over-processing: Remember that if you saved a Scikit-Learn Pipeline, it already contains the encoders and scalers. Do not manually scale your data before passing it to the pipeline, or you will "double-scale" it and ruin your predictions.

Recap

An inference script is your model's public face. By wrapping the loading and prediction logic in a single function, you ensure that your ML system is modular, error-resistant, and ready for deployment. We’ve moved from training models in a notebook to creating a functional, repeatable tool.

Up next: Building a Simple Web Interface.