Text Preprocessing Techniques You Must Know for NLP
Introduction What is Text Preprocessing? Text Preprocessing is a method that converts text into numerical vector. Why we used text preprocessing? Machine learning algorithms cannot understand raw text directly . Therefore, we use text preprocessing to convert text into numerical or mathematical values Steps of text preprocessing Step 1: Lowercasing - Convert all text in to lower case Example: "Hi My Name Is Rajat" → "hi my name is rajat" Step 2: Stop Words- It Remove common words which add little meaning (e.g., "is" , "the" , "and") Example: "hi my name is rajat and i am learning AI/ML" → "my name rajat learning AI/ML" Step 3: Stemming- It is used for reducing words to their root form from which it helps group similar words together so that the machine treats them as the same feature. "Taste ,Tasty ,Tasteful" → "Tasty(Root word)" Example: Original: "I like the taste of this tast...