The Importance of Data Processing in Machine Learning

Published Date

Machine Learning and Artificial Intelligence has often been used interchangeably but they are not the same. So, let me begin with what is machine learning and settle that misconception.

Machine Learning vs Artificial Intelligence

Machine Learning takes vast amounts of data (hence Big Data) to learn from the patterns. It creates self-learning algorithms so that machines can learn from themselves.

If you are looking for Machine learning examples, we need go only as far as Amazon. They got to be the world leaders in inventory management and customer satisfaction index because they brought in Machine Learning in all their systems. The company ships about 1.6 million packages a day all because they have optimized inventory by using trained algorithms.

Artificial Intelligence is a step over ML, in that it uses Big Data but it mimics human intelligence to apply knowledge it has gained to solve complex problems. Machine learning on the other hand will work on a specific task for accuracy and does not go beyond the problem it is set to work on.

To put the difference into perspective, let’s look at it vis a vis the application of ML and AI on a video game. Let’s say your avatar in the video game has to find a way out of a maze, littered with traps that pull you in and that’s it, the game is over. Machine learning would collect data of where the traps are located and then allow you to reach the end safely. But what happens if the traps position changes or new elements are introduced? It has to relearn what it must do. That is all well and good but then new patterns require new algorithms to be set for Machine Learning.

AI on the other hand would look at the new problem differently, mimicking human intelligence it would look at signals that indicate a problem and codify its own new rules and find new paths that will deliver success.

Related article:6 Things to know before planning a data migration

Data preparation for Machine Learning

Whether it is AI or ML, the foundation is data and lots of it. Every business has this data in one form or the other but it has to be cleaned and prepared into a state that can be used by Machine Learning to learn from.

These are the various stages in data preparation for Machine Learning

Data Collection: The amount of data that is needed is directly dependent on the complexity of the problem that needs to be solved.