{"id":11654,"date":"2022-09-21T07:30:35","date_gmt":"2022-09-21T07:30:35","guid":{"rendered":"https:\/\/predictly.se\/getting-started-machine-learning-at-gcp-part-2-making-data-clean-and-useful\/"},"modified":"2024-05-08T11:46:41","modified_gmt":"2024-05-08T11:46:41","slug":"getting-started-machine-learning-at-gcp-part-2-making-data-clean-and-useful","status":"publish","type":"post","link":"https:\/\/predictly.se\/en\/getting-started-machine-learning-at-gcp-part-2-making-data-clean-and-useful\/","title":{"rendered":"Getting started: Machine Learning at GCP – Part 2: Making data clean and useful"},"content":{"rendered":"

In the last article<\/a> we covered how to make our data accessible and available. Now that we have our raw data in a BigQuery table we can begin exploring and getting to know our data.<\/p>\n

Data exploration is a highly iterative process. It helps to work in an interactive environment, where the data is kept in memory. This allows the user to make multiple transformations, visualizations and test hypothesis without needing to query the raw data multiple times.<\/p>\n

For this purpose we are going to use a GCP hosted Jupyter Notebook<\/a>, a well known tool for most data scientists.<\/p>\n

\"Jupyter<\/p>\n

Data mining<\/h6>\n

The notebook allows the data scientist to use the tools already known to them, like pandas or numpy for data transformations, or seaborn and matplotlib for data visualizations.<\/p>\n

The notebook environment also allows the data scientist to work from anywhere without having to setup a local environment, opening connections to fire walled source systems, etc., tasks that usually comes with both great headaches and long delays.<\/p>\n

We can instead focus on enabling more and faster data innovation, since our data, setup and installation of the environment and computations are entirely managed by GCP.<\/p>\n

To start a new notebook instance, in you cloud console, navigate to AI platform -> Notebooks<\/em> and start a new notebook.
\nFor our purposes we will use a Standard Python2 and 3 notebook with default settings.<\/p>\n

When the instance is ready click the link to open JupyterLab.<\/p>\n

Data exploration is an iterative process where you make multiple passes over the data, making new findings as you go. This process is hard to show in an article, so for the ones of you who are interested in how I came to the conclusions ahead, here is a gist with the notebook where I explored the data.<\/p>\n