Total members 11806 |It is currently Thu Nov 21, 2019 4:48 am Login / Join Codemiles

Java

C/C++

PHP

C#

HTML

CSS

ASP

Javascript

JQuery

AJAX

XSD

Python

Matlab

R Scripts

Weka





This example illustrates Feature selection using techniques. The first technique is Univariate Selection which a statistical test is applied to the features and dependent variable (class). The second approach is Recursive feature elimination (RFE) which a recursive method of dropping features and measuring performance is applied.

python code
#https://jupyter.org/try
#Demo7 - part1
#M. S. Rakha, Ph.D.
# Post-Doctoral - Queen's University
#
# Feature Selection#1 RandomForest
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

from mpl_toolkits.mplot3d import Axes3D
import pandas as pd
from sklearn.cluster import KMeans
from sklearn import datasets
from sklearn.preprocessing import scale
import sklearn.metrics as sm
from sklearn.metrics import confusion_matrix,classification_report
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import chi2


#Univariate Selection
#Statistical tests can be used to select those features that have the strongest relationship with the output variable.
# example below uses the chi squared (chi^2) statistical test for non-negative features
np.random.seed(5)
breastCancer = datasets.load_breast_cancer()

list(breastCancer.target_names)

#Only two features
X = breastCancer.data
y = breastCancer.target



# feature extraction
test = SelectKBest(score_func=chi2, k=4)
fit = test.fit(X, y)
# summarize scores
np.set_printoptions(precision=3)
print(fit.scores_)
features = fit.transform(X)
# summarize selected features
print(features[0:5,:])



## Recursive Feature Elimination
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

from mpl_toolkits.mplot3d import Axes3D
import pandas as pd
from sklearn.cluster import KMeans
from sklearn import datasets
from sklearn.preprocessing import scale
import sklearn.metrics as sm
from sklearn.metrics import confusion_matrix,classification_report
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import RFE
np.random.seed(5)
breastCancer = datasets.load_breast_cancer()

list(breastCancer.target_names)

X = breastCancer.data
y = breastCancer.target


X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.50, random_state=42)

X_train[:,0].size
X_train[:,0].size

varriableNames= breastCancer.feature_names

#Feature extraction
#Recursive feature elimination (RFE) is a feature selection method that fits a model
#and removes the weakest feature (or features) until the specified number of features is reached.
#Features are ranked by the model’s coef_ or feature_importances_

randomForestModel = RandomForestClassifier(n_estimators=100, max_depth=2, random_state=0)
rfe = RFE(randomForestModel, 3)
fit = rfe.fit(X, y)
print("Num Features: %d" % fit.n_features_)
print("Selected Features: %s" % fit.support_)
print("Feature Ranking: %s" % fit.ranking_)


The output of this example is :
Code:
[2.661e+02 9.390e+01 2.011e+03 5.399e+04 1.499e-01 5.403e+00 1.971e+01
1.054e+01 2.574e-01 7.431e-05 3.468e+01 9.794e-03 2.506e+02 8.759e+03
3.266e-03 6.138e-01 1.045e+00 3.052e-01 8.036e-05 6.371e-03 4.917e+02
1.744e+02 3.665e+03 1.126e+05 3.974e-01 1.931e+01 3.952e+01 1.349e+01
1.299e+00 2.315e-01]
[[1001.    153.4   184.6  2019.  ]
[1326.     74.08  158.8  1956.  ]
[1203.     94.03  152.5  1709.  ]
[ 386.1    27.23   98.87  567.7 ]
[1297.     94.44  152.2  1575.  ]]
Num Features: 3
Selected Features: [False False False False False False False False False False False False
False False False False False False False False False False  True  True
False False False  True False False]
Feature Ranking: [ 9 13  7  4 28 14  5  3 22 21 18 19 16  8 25 27 20 23 24 26  2 10  1  1
15 11  6  1 12 17]




_________________
M. S. Rakha, Ph.D.
Queen's University
Canada


Author:
Mastermind
User avatar Posts: 2715
Have thanks: 74 time
Post new topic Reply to topic  [ 1 post ] 

  Related Posts  to : Feature selection: Statistical and Recursive examples
 Inverse Transformation for some Statistical Distributions (C     -  
 problem, recursive function using array_merge()     -  
 problem, recursive function using array_merge()     -  
 recursive function to delete directories     -  
 examples with c     -  
 Examples in C#     -  
 few examples of c using ascii     -  
 Examples of heading tags     -  
 Matlab basics examples     -  
 Asp.net insert update delete Examples     -  



Topic Tags

Machine Learning, Python, Artificial Intelligence






Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
All copyrights reserved to codemiles.com 2007-2011
mileX v1.0 designed by codemiles team
Codemiles.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com