December 20, 2018
|
7 min read
Smart Smarties: How to sort M&Ms using Internet of Things, Raspberry Pi and Lego
by
Eike Zeller
Head of IoT Product Development
Earlier in the year — September 2018 — the Sheda team got to attend the IoT Impact conference in Sydney. We wanted to create an engaging and practical demo of what Industrial IoT or Industry 4.0 can be used for. After much brainstorming and deliberation we decided to create an Internet of Things demo that would show how you could use Industry 4.0 technology to deliver value in a food manufacturing, in this case using these technologies to sort M&Ms by their colour.
In this post I’m going to walk you through how we used Sheda’s delivery process, a Raspberry Pi with computer vision, an iOS app, Google Cloud technologies to create a scaled down example of an Industry 4.0 application built with Lego pieces because..Lego is awesome that’s why.
Before I get into the details what is IoT and Industry 4.0?
The Internet of Things or short IoT basically refers to all devices and sensors collectively known as “things” connected to the internet. That can be anything from your smartwatch used to track your health, your internet connected car finding the quickest route using realtime traffic, or to the smart bulb and smart speakers in your home. Things usually refer to any device that is not a desktop computer, laptop or smartphone that is sending and/or receiving data to/from the internet, all these things are part of the Internet of Things.
The examples I gave above are related to consumer based IoT, but IoT doesn’t stop at the consumer level. The World Economic Forum estimates that by 2020 more than 50 billion things, ranging from cranes to coffee machines, will be connected to the internet.
It’s clear that sooner rather than later IoT will enter almost every area of our lives, as more machines in manufacturing and production industries are getting connected to each other over virtual networks there will be a shift to having whole production lines powered by machines communicating with each other in real-time to create value and deliver products and services. This type of IoT is commonly referred to as Industrial IoT (IIoT) or Industry 4.0.
From speaking to our customers in manufacturing especially in the food and agriculture space, we found that most people have heard of the buzz word Industry 4.0 but couldn’t envision how it would apply in their industry. We wanted to put together a demo that showcases automation powered by internet connected devices aka Industry 4.0.
Since a few of Sheda’s IoT customers are in the agriculture space, we decided that our demo should have some relevance to agriculture and food processing, like these crazy tomato sorting machines.
The team agreed that whatever we came up with had to be fun, spark the imagination, be engaging, easy to assemble, disassemble, be accessible and most importantly be able to be completed within our constraints of time and budget.
In order for our colour sorter to be a truly interactive IoT setup, we created a native iOS app that you could use to control what colours get sorted and used Google Cloud’s RealTime database to develop the backend. To cut down development time, we picked the best rapid prototyping material commercially available: LEGO!
A bit of research in Lego Great Ball Contraptions brought up some inspiration like this fantastic post.
Here are a few other M&M sorters other people built:
Sheda’s IoT M&M sorter was designed to be an M&M contraption with a twist. A camera connected to a Raspberry Pi will use computer vision to detect the colours of M&Ms and trigger a servo motor connected to a lever. If the M&M matches the colour we’ve set the machine to sort, the lever will move and extract the M&Ms from the system.
Being a true IoT device the sorter would receive the current colour from our internet connected cloud database in real time. The companion iOS app would be used to change the colour of the M&M’s to be sorted.
The sorter consisted of
Once we got our Lego parts, we assembled a prototype to test how the sorter will work. Here are our first trials.
Pro tip: Crispy M&Ms have the most spherical shape, so they roll the best.
We got the team together and started brainstorming ideas on how to assemble our sorter.
We opted for a conveyor belt system to transport the M&Ms to the top of the contraption, at the top they M&Ms will roll pass the camera and selection gate. After a few hours of tinkering, we had our first M&M contraption going.
While this structure worked we had to make lots of tweaks around the feeder to make sure M&Ms don’t get stuck inside the conveyor belt and don’t block each other. The picture below shows the final structure we devised to allow a smooth ride for the M&Ms to go up the conveyor belt and ensure that not too many M&M were picked up at the same time.
The Raspberry Pi and camera were mounted onto the sorter and aim at the rail to detect the colour of the M&Ms rolling down the from the conveyor belt.
Lachi Agnew one of Sheda’s software engineers developed the computer vision colour detection and servo triggering mechanism. Read how he developed a computer vision system to detect M&M colours in the linked post.
After we got the sorter able to select M&M of a specified colour it was time to turn it into an actual IoT device. We set up a Firebase database on Google Cloud to store the current colour to be selected.
The iOS Lego Sorter mobile app which we deployed to an iPad is the primary means users choose the colour of the M&M they want, the current colour is sent up to Firebase and the Raspberry Pi periodically queries Firebase to see if the colour has been changed. A more elegant solution would be to use a cloud function to alert the Raspberry Pi when the current colour has changed but we ran out of time to do that.
If you’re interested in learning how to create an iOS app that uses Firebase as a backend check out Lachi’s quick guide on Rapid Prototyping with Firebase and iOS
Once that was all done, we were good to go, check out the little video we created on how to use our IoT M&M sorter sort in conjunction with the iOS app.
The beautiful thing about Lego is that you can take it apart and set it up again really quickly. So we packed up the sorter at Sheda HQ and flew with it from Melbourne to Sydney and we were able to set it up again within minutes. Here is the Sheda IoT M&M sorter at IoT Impact sorting out yellow M&Ms.
We got a lot of interests and people stopping by our stand, conference attendees enjoyed getting a hands-on Industry 4.0 experience with some sweets to take away.
This was quite a fun project and we’ll be creating more like these.
and if you are interested in keeping up with what we get up to at Sheda, follow me and Sheda’s publication or get in touch at iot@sheda.ltd.
You can also reach me on LinkedIn.
Would love to help you explore how you can use IoT to increase the impact of your business activities.
