Image recognition is an extremely complex process. One of the hardest parts of this process is obtaining data and accurately classifying it into different categories. After a fruitless search for a more pragmatic classification method that could save time and resources, we developed our own solution. Enter, Flokka, a bot for Slack.
Tried and tested at this year’s Myriad Festival, Flokka aimed to crowdsource the classification and training of a simple image recognition model. Using the Flokka slackbot, users were able to quickly sort files from unordered folders or lists into a sorted file structure — ideal for training a machine learning model.
A slackbot is a type of program, exclusive to the Slack platform that allows a user to interact with an interface to a program through a Slack chatroom. This can be done through natural language, for example “Please fetch me the latest headlines for the last 24 hours”, or done through commands: “/fetch news 10”. This helps users who would otherwise not know how to use certain programs/programming interfaces to easily use them.
We chose to develop Flokka as a Slackbot due to the integral role that Slack plays in the Max Kelsen team. The multi-platform nature of Slack allows us to easily classify images anytime, anywhere; during short breaks from other projects or even while drinking our morning coffees. In the office, this accessibility helps to streamline the process of image classification by removing the need for hours of manual sorting.
Flokka was built using the SlackClient Python API (specifically the Events API) to create slash commands and handle interactive messages.
Using the /classify [number] command, the slackbot takes [number] of images from the training folder and displays them in the designated Slack channel. Once displayed, users can then select the appropriate classification using the button inputs below each image. The images are then sorted into the appropriate file structure ready to be used in a machine learning model. Flokka saves people the tedious task of dragging images into folders on desktops and creates a more engaging and enjoyable process overall. Even if categories and images become more complex as the slackbot develops, the ability to classify bit by bit should increase the accuracy of individuals performing the task.
I will be writing a more technical breakdown of Flokka in the coming weeks.
Myriad was a three-day technology and innovation festival that ran from the 29th — 31st of March at the Brisbane Powerhouse. The event encouraged collaboration and networking among entrepreneurs, investors, business owners and tech-lovers. In addition, Myriad provided us with the opportunity to showcase Flokka and have like-minded individuals develop an understanding and test the slackbot. We invited attendees to assist us in labelling an unsorted collection of four exotic animals; a bear, an orca, a tiger and a toucan. The results of this labelling would then be used as training set to construct an Image Classification model.
Over the course of two sessions, 2000 images were classified into; 712 bears, 790 toucans, 154 tigers and 286 orca whales. In addition, 58 images contained unrelated animals.
Once the images had been classified we used them to train an image recognition model to detect and categorise new photos of each animal. This model was created by retraining a tensor flow model (for a tutorial see here).
We then ran a blind set (50 images from each category) through the model and the results from this model have been displayed in the graphic below:
The 200 blind images were sorted by the model into the four categories with an average confidence interval for each category of over 99.8%. This statistic reflects the ability of the program to perform a task more efficiently and accurately than manual methods. We acknowledge that this accuracy is likely attributed to the fact that the image categories have a large variance in terms of colour, texture, shape and patterns for the model to discern from. Acknowledging this, we sought to test the model further and ran images of subspecies of the categories through the model including a black bear, panda bear, polar bear and white tiger.
The results from those images are shown below:
The results from this further encouraged the notion that patterns and shapes were considered more crucial elements for recognition than colour. This was concluded because the white tiger, polar bear and black bear images vary in colour but share similar patterns and shapes to their corresponding category ‘original’, and maintained high accuracy rates in the model’s image recognition. However, the model was found to be slightly less accurate in identifying the panda bear image, which had the greatest differentiation in shape.
Flokka has created an image classification process that is both quicker and more accessible for users than previous techniques of manually sorting images. Although we note that individuals seemed to enjoy the game aspect of Flokka, we recognise that this may have been amplified by the nature of the vivid and exotic sample images used for this study. The process was also made easier due to the simplistic nature of a four category system, in comparison to a system of ten or more categories which may reduce Flokka’s productivity.
It is also important to consider that conference goers were able to classify quickly because of the significant visual difference between the images. Smaller difference would present greater difficulty and individuals may struggle to sort images with the same level of speed and accuracy. Despite this, the accessibility of Flokka through Slack platforms and ease of organisation would still produce an overall efficiency gain in the process. This is an exciting time for Max Kelsen as we are constantly developing methods such as Flokka to improve our platforms, techniques and overall productivity.
The Max Kelsen team are currently working to improve upon and develop Flokka to be used in any workplace and image recognition model. Stay tuned for an Open Source version of Flokka coming soon and subscribe to the Flokka newsletter at flokka.io to keep updated on details of the project.
