Skip to main content

Posts

Fit vs. Transform

  Fit vs. Transform in SciKit libraries for Machine Learning We have seen methods such as fit(), transform(), and fit_transform() in a lot of SciKit’s libraries. And almost all tutorials, including the ones I’ve written, only tell you to just use one of these methods. The obvious question that arises here is, what do those methods mean? What do you mean by fit something and transform something? The transform() method makes some sense, it just transforms the data, but what about fit()? In this post, we’ll try to understand the difference between the two. To better unders t and the meaning of these methods, we’ll take the Imputer class as an example, because the Imputer class has these methods. But before we get started, keep in mind that fitting something like an imputer is different from fitting a whole model. You use an Imputer to handle the missing value in dataset. Imputer gives you easy methods to replace NaNs and blanks with something like the mean of the column or even median. Bu
Post a Comment
Read more
Recent posts

Standardization

What is Standardization? Standardization is another scaling technique where the values are centered around the mean with a unit standard deviation. This means that the mean of the attribute becomes zero and the resultant distribution has a unit standard deviation. Here’s the formula for standardization: μ   is the mean of the feature values and  σ  is the standard deviation of the feature values.  Note that in this case, the values are not restricted to a  particular range
Post a Comment
Read more

Normalization

 What is Normalization? Normalization is a scaling technique in which values are shifted and rescaled so that they end up ranging  between 0 and 1. It is also known as Min-Max scaling. Here’s the formula for normalization: Here, Xmax and Xmin are the maximum and the minimum values of the feature respectively. When the value of X is the minimum value in the column, the numerator will be 0, and hence X’ is 0. On the other hand, when the value of X is the maximum value in the column, the numerator is equal to  the denominator and thus the value of X’ is 1 If the value of X is between the minimum and the maximum value, then the value of X’ is between 0 and  1
Post a Comment
Read more

Feature Scaling

Why Should We Do Feature Scaling? The first question we need to address – why do we need to scale the variables in our dataset? Some machine learning algorithms are sensitive to feature scaling while others are virtually invariant to it. Let me explain that in more detail. Gradient Descent Based Algorithms : Machine learning algorithms like linear regression , logistic regression , neural network , etc. that use gradient descent as an optimization technique require data to be scaled. Distance-Based Algorithms :  Distance algorithms like KNN , K-means , and SVM are most affected by the range of features. This is because behind the scenes they are using distances between data points to determine their similarity. Tree-Based Algorithms :  Tree-based algorithms , on the other hand, are fairly insensitive to the scale of the features. Think about it, a decision tree is only splitting a node based on a single feature. The decision tree splits a node on a feature that increases the homoge
Post a Comment
Read more

Standard Deviation And Variance

   Standard Deviation :  Standard deviation is a number that describes how spread out the values are. A low standard deviation means that most of the numbers are close to the mean (average) value. A high standard deviation means that the values are spread out over a wider range. Example: This time we have registered the speed of 7 cars: speed = [ 86 , 87 , 88 , 86 , 87 , 85 , 86 ] The standard deviation is:  0.9 Meaning that most of the values are within the range of 0.9 from the mean value, which is 86.4. Let us do the same with a selection of numbers with a wider range: speed = [ 32 , 111 , 138 , 28 , 59 , 77 , 97 ] The standard deviation is:  37.85 Meaning that most of the values are within the range of 37.85 from the mean value, which is 77.4. As you can see, a higher standard deviation indicates that the values are spread out over a wider range. The NumPy module has a method to calculate the standard deviation:  import  numpy speed = [ 86 , 87 , 88 , 86 , 87 , 85 , 86 ] x = numpy.
Post a Comment
Read more

Batch and Online Learning

  It is the criterion used to classify Machine Learning systems is whether or not the system can learn incrementally from a stream of incoming data. Batch learning In batch learning , the system is incapable of learning incrementally: it must be trained using all the available data. This will generally take a lot of time and computing resources, so it is typically done offline. First the system is trained, and then it is launched into production and runs without learning anymore; it just applies what it has learned. This is called offline learning . If you want a batch learning system to know about new data (such as a new type of spam), you need to train a new version of the system from scratch on the full dataset (not just the new data, but also the old data), then stop the old system and replace it with the new one. Fortunately, the whole process of training, evaluating, and launching a Machine Learning system can be automated fairly easily (as shown in Figure 1-3 ), so even a batch
Post a Comment
Read more

What is text.similar() & text.common_contexts() of nltk

Let's first define our input text, I will just Copy/Paste the first paragraph of  Game of Thrones Wikipedia page : input_text = "Game of Thrones is an American fantasy drama television series \ created by David Benioff and D. B. Weiss for HBO. It is an adaptation of A Song \ of Ice and Fire, George R. R. Martin's series of fantasy novels, the first of \ which is A Game of Thrones. The show was filmed in Belfast and elsewhere in the \ United Kingdom, Canada, Croatia, Iceland, Malta, Morocco, Spain, and the \ United States.[1] The series premiered on HBO in the United States on April \ 17, 2011, and concluded on May 19, 2019, with 73 episodes broadcast over \ eight seasons. Set on the fictional continents of Westeros and Essos, Game of \ Thrones has several plots and a large ensemble cast, and follows several story \ arcs. One arc is about the Iron Throne of the Seven Kingdoms, and follows a web \ of alliances and conflicts among the noble dynasties either vying to claim the
Post a Comment
Read more