AI Glossary
The complete dictionary of Artificial Intelligence
LASSO Regression
L1 regularization method that penalizes the absolute coefficients of features, forcing some coefficients to zero to perform automatic variable selection.
Mutual Information
Statistical measure quantifying the dependency between two variables, used to evaluate the relevance of features relative to the target variable.
Chi-Square Test
Statistical test evaluating the independence between categorical features and the target variable, used to filter irrelevant variables.
ANOVA F-test
Statistical method comparing variances between groups to evaluate the importance of numerical features relative to a categorical target variable.
Boruta Algorithm
Feature selection algorithm based on random forests that compares the importance of real features with randomly generated shadow features.
SelectKBest
Univariate selection method choosing the k features with the highest statistical scores according to a specific test (chi2, f_classif, mutual_info_classif).
Variance Threshold
Basic filtering technique eliminating features whose variance is below a predefined threshold, considered uninformative.
Sequential Feature Selection
Greedy method sequentially adding or removing features to optimize a model performance metric according to a forward or backward strategy.
Genetic Algorithm for Feature Selection
Metaheuristic approach using natural selection principles to explore the feature subset space and find a quasi-optimal solution.
SHAP Values
Interpretability method based on game theory quantifying the impact of each feature on individual model predictions.
Correlation-based Feature Selection
Method evaluating feature relevance by analyzing their correlation with the target variable while minimizing redundancy between features.
Information Gain
Measure quantifying the entropy reduction of the target variable when a feature is known, used to evaluate variable relevance.
Relief Algorithm
Filter feature selection algorithm evaluating variable relevance by comparing distances between similar and dissimilar instances.
Auto Feature Selection
Automated process combining multiple selection techniques to identify the optimal feature subset without manual intervention.
Embedded Methods
Feature selection approaches integrated directly into the model training process, such as decision trees or regularization methods.
Wrapper Methods
Selection techniques using a machine learning model to evaluate feature subset quality through cross-validation or performance metrics.