Data Science #4.2: Intro Machine Learning Problem - Random Forest


It can be overwhelming to get your feet with Machine Learning, between determining what algorithm you might want to try and even finding a data set to play with. Hopefully this post will make you feel better about giving ML a shot in Python and maybe even applying it to your own problems.

Imagine if buying a Titanic Ticket had come with a 'Change of Survival' 
I'm going to go ahead an use a publicly available data set - The Titanic Survivors - that's popular because of the ongoing Kaggle Competition (which is a GREAT place to start if you're looking for problems to work on while applying for jobs, etc.).

Things you'll need:

Titanic Data Set

Python

Scikit learn and the Scipy stack installed (no idea what I'm talking about? https://www.scipy.org/install.html)

Some time, patience, and a drive to predict the future

Let's get cracking!

Goal: Could we have determined Titanic passenger survival based on information known at the time of departure? 


First things first, make sure you have the data set in an accessible location and let's open it up into a pandas dataframe.

df=pd.read_csv('Titanic.csv')

Sweet! Now let's check out what's in it:

df.columns.values
Out[1]: array(['Unnamed: 0', 'Name', 'PClass', 'Age', 'Sex', 'Survived', 'SexCode'], dtype=object)

Those are our variables that we're going to try to predict who survives and who doesn't survive. Well, except for the 'Survived' column - that's our solution. The 'Unnamed' column is just where pandas read the file in and didn't know what to call the index. Open up this data set and take a peek - there are a lot of gaps and holes. We're going to keep things simple (note: this is not always the best answer!!) and remove all rows in our data frame that have NaNs.

# first, let's clean up our data set
df=df.dropna()

Great! We reduced our data set from 1312 rows to 756 rows (yikes!). You can interpolate to fill some of these NaNs, but let's start easy knowing we're sacrificing accuracy. Next step is to reduce the number of columns, setting aside our Survived column and reducing the duplicates 'Sex' and 'Sex Code' down to just 'Sex' which is a numerical value. We also drop the name column because it's a text column and (in this case) will not enter our algorithm well (plus, a little intuition, should name length matter? Maybe... but that'd be weird). 

# then let's put the solution off to the side
# and drop the labeled 'male/female' column in favor of the
# sex code one (female = 1, male = 0)
SolutionCol = df['Survived']
df=df.drop(['Survived', 'Sex', 'Name', 'Unnamed: 0'], axis=1)

Unfortunately data cleaning is just part of the job. We also have to change the PClass column from '1st, 2nd, etc.' to '1, 2, etc.'

# then we have to fix PClass from 1st, 2nd, etc. to just 1, 2, 3
# once again, we need numerical values 
df['PClass'] = df['PClass'].map(lambda x: str(x)[0])

Now we're down to 756 rows and 3 columns. But before we build our model, it's imperative that we 'split' our data into test/train sets. The idea is how do you tell someone how good your model is if you use all the data? If the model is genuinely predictive, you should be able to use some of the data to predict the remaining. However, it's a fine line... using too much for testing means that you have less to train on. A good place to start is 3/4 train, 1/4 test.

# now split up train-test data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
     df, SolutionCol, test_size=0.25, random_state=1)

The random state variable allows you to start from this exact configuration again if you so choose. To really verify that our model is/isn't consistent, we'll try different states. Now... let's actually set up the model using sci-kit learn in Python. We're going to use a classifier here because we want a yes/no answer - will a person survive or not? We don't need a regressor - a 60% dead person isn't actually informative for us. I chose Random Forest because it's one of my favorites, but maybe we'll play around with this a bit more with another classifier in the future.

# setup the model
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier()
clf.fit(X_train, y_train)
predictions=clf.predict(X_test)

It's that easy, really. I mean, the devil is in the details, but the initial set up is that ridiculously easy - you just built a high powered statistical model. BAM!

Next step? Score it! How good is our model! Let's use that test data set!

# score the model
# how accurate was the model overall
from sklearn.metrics import accuracy_score
acc=accuracy_score(y_test, predictions)

# what was it's precision
from sklearn.metrics import precision_score
precision=precision_score(y_test, predictions)

# what is the TP, FP, FN, TN rate? (true positive, false positive, etc.) 
from sklearn.metrics import confusion_matrix
confusionMatrix=confusion_matrix(y_test, predictions)

Results????

accuracy is: 79%
precision is: 0.83606557377
confusion matrix: [[99 10]
 [29 51]]

The confusion matrix should be read as "Correctly Predicted True, Incorrectly True, Incorrectly False, Correctly False". We're doing pretty well for a first run! However, I'd personally never submit this to Kaggle... we can tune our features to slowly increase our accuracy and improve our model. But this is a good start :)

Speaking of which - what mattered the most?

# what were the most important features?
list(zip(df.columns.values, clf.feature_importances_))

'PClass', 0.18,
'Age', 0.44, 
'SexCode', 0.38











Age was the biggest determiner of survival in the Titanic accident, followed by male/female, and then your fare class. Fascinating right? Based on my 4th grade readings about the Titanic, I would have thought it was all the rich who survived...

Just as a quick test, remember how we set the random state = 1 for our train/test random splitting? Let's set it to 2 and see how the accuracy changes:


accuracy is: 75%
precision is: 0.686567164179
confusion matrix: [[97 21]
 [25 46]]
Uh oh, accuracy changed a bit, and precision quite a bit. This means depending on what train/test combination we use, the more sensitive our model is to favoring certain answers. In this second test, we had way more false positives (everyone died!). Depending on what your intended outcome is, this can be good/bad, but the large changes based on the random state aren't great.

So we have some work to do on this model. Let's call this a blog post, but work on tuning our model next time.

Want to see the script? Don't Hesitate to Create A New Branch :)

and the next in this series is here: http://www.innocentheroine.com/2017/01/data-science-43-improving-random-forest.html

Lo

48 comments:

Anonymous said...

Great post.

Lois Keller said...

Thank you :) I'm always taking requests too !

Unknown said...

Thanks so much, great job keep posting we will wait for new one :), your new student Wathela

allison jaynes said...

(Note: this is actually a comment on your previous post, 'Aware', but I couldn't find a place to comment there...)

Yes, I feel you. I'm the same way with the waste guilt. And I was also overjoyed the day I found out that air bag packing could be recycled!

Sometimes plastic is useful, but is it worth it? Like my bento lunchbox that I love and will probably use for years, saving tons of packaged food waste - but for long-term impact, would it have just been better to use paper bags every day? Who knows. These are the thoughts that keep me up at night. :-)

Renjith Madhavan said...
This comment has been removed by the author.
Renjith Madhavan said...

Great blog. I think I forgot how I stumbled upon this :) . Its superbly organized.

Lois Keller said...

Exactly! It's a weird combination of guilt from constant waste (wrappers... they're the worst) and then joy at realizing there's another way to reduce personal impact (i.e. challenging myself to take a shorter shower by 2 minutes a week) One realization at a time...

Lois Keller said...

Thank you Renjith! I'm glad you enjoyed it :)

SFJ Business Solution Training said...

Great Information. Thank You Author, for sharing your valuable information about Machine Learning with us. People who are reading this blog can continue your knowledge which you gained with us and know how to apply this practically along with our Machine Learning Course

Komal said...

thank you for the valuable information giving on data science it is very helpful.
Data Science Training in Hyderabad

markson said...

In the event that you need to turn into an ace in Big Data investigation and need to turn into an expert Data Scientist you should learn Hadoop Training Chennai and you have to accomplish more research with the enormous measure of information. data science course in pune

digitaltucr said...


Attend The Digital Marketing Courses in Bangalore From ExcelR. Practical Digital Marketing Courses in Bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Digital Marketing Courses in Bangalore.
Digital Marketing training in Bangalore

seoexpert said...

Nice Post...I have learn some new information.thanks for sharing.
ExcelR data analytics course in Pune | business analytics course | data scientist course in Pune

Priyanka said...

Attend The Data Analytics Courses From ExcelR. Practical Data Analytics Courses Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analytics Courses.
ExcelR Data Analytics Courses

for ict 99 said...

Great Article
final year projects on machine learning
Final Year Project Centers in Chennai


JavaScript Training in Chennai
JavaScript Training in Chennai

seoexpert said...

Such a very useful article. I have learn some new information.thanks for sharing.
data scientist course in mumbai

ameer said...


Excelr is providing emerging & trending technology training, such as for data science, Machine learning, Artificial Intelligence, AWS, Tableau, Digital Marketing. Excelr is standing as a leader in providing quality training on top demanding technologies in 2019. Excelr`s versatile training is making a huge difference all across the globe. Enable ?business analytics? skills in you, and the trainers who were delivering training on these are industry stalwarts. Get certification on "hyderabad data science" and get trained with Excelr.

ExcelR Solutions said...

Thanks for sharing your valuable information to us, it is very useful.
pmp certification training in bangalore

Maneesha said...

I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page!
data scientist certification

Manikanta said...

Nice Article...Very interesting to read this article. I have learn some new information.thanks for sharing.
Click here

Tyler Worgan said...

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading.
tree service near me in west palm beach

Tyler Worgan said...

Very nice bro, thanks for sharing this with us. Keep up the good work and Thank you for sharing information
bathroom remodel albuquerque nm

Tyler Worgan said...

I think this is one of the most significant information for me. And i’m glad reading your article. But should remark on some general things, The web site style is perfect, the articles is really great : D. Good job, cheers.
tree pruning port st lucie

David Bryns said...

Thanks for your post.
Click Here

pamelapeterson said...

I think this is one of the most significant information for me. And i’m glad reading your article. But should remark on some general things, The web site style is perfect, the articles is really great : D. Good job, cheers

Click here

neomacooper said...

Superbly written article, if only all bloggers offered the same content as you, the internet would be a far better place.
ac installation palm beach county

Lee Davis said...

Great article and a nice way to promote online. I’m satisfied with the information that you provided phoenix remodeling contractors

haroonullah said...

I think this is one of the most significant information for me. And i’m glad reading your article. But should remark on some general things, The web site style is perfect, the articles is really great : D. Good job, cheers

Haroon Ullah

Tyler Worgan said...

Very nice bro, thanks for sharing this with us. Keep up the good work and Thank you for sharing information
bathroom remodelers reno

jasminegates said...

Nice post.commercial ac unit wellington

Tyler Worgan said...

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading.
wake county bathroom remodeling

Dana Milton said...

Hello, I have browsed most of your posts. This post is probably where I got the most useful information for my research. Thanks for posting, maybe we can see more on this. Are you aware of any other websites on this subject. hvac repair royal palm beach

Shop Sous Vide said...

Great post shared.
tree stump removal west palm beach

Clark said...

thanks for sharing.
kitchen remodel fort lauderdale fl

Shop Sous Vide said...

Useful sharing. termite pest control west palm beach

Clark said...

Great updates sharing.
kitchen and bath remodeling miami-dade

Excelrsolutions said...

I am looking for and I love to post a comment that "The content of your post is awesome" Great work! excelr data science

Shop Sous Vide said...

Great blog post. port st lucie tree removal services

Clark said...

Great.
kitchen and bath remodeling palm beach

Clark said...

Great post.
restoration company st petersburg fl

Clark said...

Great sharing.
land cleaning services san diego

Clark said...

Awesome post.
bathtub reglazers nashville

Clark said...

Great post.
commercial concrete contractors san diego ca

Clark said...

Nice one.
tree stump removal hillsborough county

Komal said...

thank you for the valuable information giving on data science it is very helpful.
Data Science Training in Hyderabad

data science training in hyderabad said...

your article on data science is very interesting thank you so much.
Data Science Training in Hyderabad

Priyanka said...

Attend The Data Analytics Course From ExcelR. Practical Data Analytics Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analytics Course.
ExcelR Data Analytics Course

ExcelR Pune said...

Awesome..I read this post so nice and very imformative information...thanks for sharing
Click here for data science course

Instagram