pip install modAL scikit-learn numpy pandas joblib<\/code><\/code><\/pre>\n\n\n\nThis installs the libraries in the environment so they are not rerun in the code.<\/p>\n\n\n\n
These libraries provide a solid foundation for the project, allowing efficient data handling and building robust machine learning models.<\/p>\n\n\n\n
Creating and Saving the Initial Dataset<\/h3>\n\n\n\n
With the libraries installed, it was time to create some data. I used make_classification<\/code> from scikit-learn to generate a synthetic dataset that simulates a binary classification problem, such as whether a picture is or is not a dog, or whether sentiment in text is positive or negative.<\/p>\n\n\n\nThis dataset served as the foundation for training and testing the model. Imports required:<\/p>\n\n\n\n
import numpy as np\nimport joblib\nfrom sklearn.datasets import make_classification\nfrom sklearn.model_selection import train_test_split\n<\/code><\/code><\/pre>\n\n\n\nThe next portion of code generates a dataset that simulates a binary classification problem as mentioned earlier.<\/p>\n\n\n\n
# Generate a synthetic dataset\nX, y = make_classification(n_samples=1000, n_features=20, n_informative=2, n_redundant=10, random_state=42)\n<\/code><\/code><\/pre>\n\n\n\nBy saving the dataset using joblib<\/code>, I ensured that it could be easily loaded and used later in the project.<\/p>\n\n\n\n# Save the dataset\njoblib.dump((X, y), 'X_y_data.pkl'<\/code><\/code><\/pre>\n\n\n\nThe data was then split into training and pool sets, where the training set would initialize the model, and the pool set would be used for querying.<\/p>\n\n\n\n
# Split the dataset\nX_train, X_pool, y_train, y_pool = train_test_split(X, y, test_size=0.75, random_state=42)<\/code><\/code><\/pre>\n\n\n\nLet’s break down what the previous block of code does in a way that’s easier to understand, because I struggled with it. <\/p>\n\n\n\n
Imagine you have a big list of data (like numbers, images, or text), and you want to use this data to teach a computer to do something, like recognizing cats in pictures or predicting how much your favorite sports team will score in a game. This dataset<\/em><\/strong> has x “input” data (like pictures of animals) and y labels (such as “dog” or “cat”). <\/p>\n\n\n\nYou can’t use all your data to teach the computer (train it). You need to keep some data aside to test if the computer has learned correctly.<\/p>\n\n\n\n
This code splits your data into two parts: one part for training and one part for testing or other purposes.<\/p>\n\n\n\n
\n- The Code Breakdown<\/strong>\n
\ntrain_test_split<\/code> is a function that does the splitting for you.<\/li>\n\n\n\nX_train<\/code> and y_train<\/code> are the parts of your data that you’ll use to train the computer.<\/li>\n\n\n\nX_pool<\/code> and y_pool<\/code> are the parts of your data that you’ll keep aside for later (like testing).<\/li>\n<\/ul>\n<\/li>\n\n\n\n- Parameters in the Code<\/strong>\n
\nX<\/code> and y<\/code>: Your whole dataset.<\/li>\n\n\n\ntest_size=0.75<\/code>: This means 75% of your data will be kept aside (in X_pool<\/code> and y_pool<\/code>), and 25% will be used for training (in X_train<\/code> and y_train<\/code>).<\/li>\n\n\n\nrandom_state=42<\/code>: This is like setting a “seed” so that if you run this code again, you get the same split every time. It’s useful for consistency.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n
\n\n\n\nInitializing the Active Learner Model<\/h3>\n\n\n\n
Even though I didn’t realize what I’d done at the time, I’d generated a synthetic dataset and split the data into training and testing data. The next step was to initialize the active learner model. <\/p>\n\n\n\n
I chose the RandomForestClassifier<\/code> from scikit-learn as the base estimator for my active learner.<\/p>\n\n\n\nfrom modAL.models import ActiveLearner\nfrom sklearn.ensemble import RandomForestClassifier<\/code><\/code><\/pre>\n\n\n\nKeeping with our cats\/dogs categorizing example, a learner<\/em><\/strong> is like a student that learns from these examples. Active learning is a special way of teaching the computer. Instead of showing it all the pictures at once, you start by showing it a few, and then it asks for more examples of things it is unsure about. This way, it learns more efficiently.<\/p>\n\n\n\n# Initialize the learner\nlearner = ActiveLearner(\n estimator=RandomForestClassifier(),\n X_training=X_train, y_training=y_train\n)<\/code><\/code><\/pre>\n\n\n\nI saved the model and data.<\/p>\n\n\n\n
# Save the model and the pool data\njoblib.dump(learner, 'active_learner_model.pkl')\njoblib.dump((X_pool, y_pool), 'X_y_pool.pkl')<\/code><\/code><\/code><\/pre>\n\n\n\nThe model was initialized with the training data and saved for future use. This step marked the completion of our initial setup, providing a solid base for building the interactive elements of our project.<\/p>\n\n\n\n
Challenges and Reflections<\/h3>\n\n\n\n
One of the initial challenges I faced was ensuring that all libraries were correctly installed and compatible with each other. That was mostly due to my misunderstanding about how “pip” works. I initially had everything in my notebook, but once I realized the libraries should be installed through the terminal\/command line, I was able to make progress.<\/p>\n\n\n\n
Once the environment was set up, the process of generating and saving data, as well as model initialization, went smoothly. This part of my adventure was crucial for understanding the basic components needed for the project and building a strong foundation. It took me days to figure out…<\/p>\n\n\n\n
, I realized the importance of patience and attention to detail. Patience is definitely not a large part of my personality or temperament, but I realized through this endeavor that it (and attention to detail) are vitally important. Setting up the environment might seem like a straightforward task, but it’s the backbone of the entire project. Ensuring that everything is correctly installed and configured saves a lot<\/em><\/strong> of time and frustration down the line.<\/p>\n\n\n\nIn the next post, we’ll dive into building the interactive web application using Flask. This is where the project starts to come to life, allowing us to interact with the model in real-time. Stay tuned!<\/p>\n\n\n\n
\n<\/div>","protected":false},"excerpt":{"rendered":"
My Machine Learning Adventure (A Series) Welcome back (previous post)! My journey to understand machine learning began a year ago, but as someone who has always been curious about the capabilities of AI, I\u2019ve only recently made any real beginner breakthroughs. My new approach is to interactively work with various GPT models and formats to…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"mf2_syndication":[],"venue_id":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-49","post","type-post","status-publish","format-standard","hentry","category-uncategorized","kind-"],"kind":false,"_links":{"self":[{"href":"https:\/\/ragecognito.digital\/index.php?rest_route=\/wp\/v2\/posts\/49","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ragecognito.digital\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ragecognito.digital\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ragecognito.digital\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ragecognito.digital\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=49"}],"version-history":[{"count":5,"href":"https:\/\/ragecognito.digital\/index.php?rest_route=\/wp\/v2\/posts\/49\/revisions"}],"predecessor-version":[{"id":59,"href":"https:\/\/ragecognito.digital\/index.php?rest_route=\/wp\/v2\/posts\/49\/revisions\/59"}],"wp:attachment":[{"href":"https:\/\/ragecognito.digital\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=49"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ragecognito.digital\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=49"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ragecognito.digital\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}