Understanding the Essential First Step in Model Building
Written on
Chapter 1: The Importance of Asking the Right Question
When embarking on model creation, it’s vital to begin with the right inquiry. Are you focused on the lemonade or the lemons?
Courtesy of Rod Long on Unsplash
Currently, numerous individuals are engaged in the process of model building. This spans a wide array of purposes, including business applications, academic research, or personal projects. The use of mathematics to represent real-world scenarios has surged, providing valuable insights and guiding decision-making in response to various phenomena.
With advancements in computing capabilities, the complexity of modeling has significantly increased. Rather than relying solely on simple Excel spreadsheets, today’s models are developed across various programming languages and platforms. Some models utilize small datasets, while others handle vast amounts of information. The time invested in creating these models can range from a few hours to extensive projects that last months or even years.
However, many model creators often overlook the importance of posing critical questions before diving into the work. They may hastily gather data and apply formulas without adequate consideration. Through years of experience in mathematics and statistics, I’ve discovered that the effectiveness of a model heavily relies on the preliminary thought processes undertaken before engaging with the data.
A crucial question that I consistently emphasize at the outset is: Is the goal of my model explanatory or predictive?
This distinction is fundamental. An explanatory model aims to clarify why a certain phenomenon occurs, addressing inquiries like: Why does a particular illness affect specific demographics? What factors contribute to temperature fluctuations? Conversely, a predictive model focuses on delivering accurate forecasts, answering questions such as: How many customers are likely to visit a shopping mall tomorrow? What will be the vote distribution among political parties in the upcoming election?
To illustrate this distinction, consider the analogy of a lemonade stand owner. If her intention is to understand customer preferences and the timing of sales, she would employ an explanatory model. On the other hand, if she needs to ensure sufficient lemons for the week ahead, a predictive model would serve her better.
Achieving success in both explanatory and predictive aspects within a single model is rare. In my experience, I have never developed a model that excels in both domains simultaneously. Several factors contribute to this challenge, which I will explore further, starting with the initial data choices all the way to evaluating model performance.
Section 1.1: Input Data Considerations
When constructing an explanatory model, the emphasis is on acquiring a profound understanding of the question at hand. This often means that the model is created only once or intermittently in the future. As such, no data source is disregarded. Even poorly formatted datasets requiring significant cleaning may be included to ensure thoroughness. Old records, still stored in filing cabinets, might be digitized to maximize available information. Conversely, certain variables may be excluded to highlight deeper explanatory factors. For example, in a medical model, age could be omitted to prevent overshadowing other significant variables influencing disease susceptibility.
In contrast, a predictive model is built for repeated execution. Thus, data selection is driven by future accessibility, often favoring connected sources that are readily formatted for use. The primary goal is to enhance prediction accuracy, raising discussions about the balance between accuracy and inductive bias.
Section 1.2: Modeling Techniques
When it comes to techniques, an explanatory model benefits from methods that are easy to interpret. Insight control is paramount. For instance, Logistic Regression provides odds ratios that clarify how input variables affect outcomes. Simpler decision trees can also serve a valuable explanatory function by revealing the impact of specific decision points.
Predictive modeling, however, often embraces complex methodologies that prioritize predictive power over interpretability. You may have encountered the term “black box model,” which refers to models that are intricate and difficult to dissect, such as neural networks. These models operate on numerous interconnected neurons, making decisions based on learned behaviors from training datasets.
Chapter 2: Evaluating Model Performance
The effectiveness of explanatory models is assessed based on the insights generated and the overall goodness of fit. Goodness of fit measures how closely the predicted values align with observed outcomes. It is common for explanatory models to yield valuable insights even if they lack a strong overall fit, particularly in social sciences.
In contrast, predictive models are evaluated solely on accuracy. Metrics such as mean absolute error, root mean squared error, precision, and recall are used to quantify how well a model performs in making predictions.
Over the years, I have adopted the mindset of the lemonade stand owner: Am I more interested in the lemonade or the lemons? I encourage you to cultivate this habit as well.
Do you have additional insights on model design? Please share your thoughts in the comments.
The first video titled "Mental Model #3: The First Question to Ask" explores the foundational questions essential for effective model building.
The second video titled "Beginners Guide to Model Building" provides a comprehensive overview of the basics of creating effective models.