***Machine Vision towards Reducing Household Food Waste***, with thanks to my supervisor, Dr David Boyle. ![[gif.gif]] ### The UK's Food Waste Problem Of all anthropogenic Greenhouse Gas emissions, **9%**[1] come from food waste. In the UK, **70%** of food waste **comes from homes**, and of that **70% is edible**[2]! Wasted food is estimated to cost an individual **£210 per year.**[2] During the first Covid lockdown, **domestic food waste decreased by a massive 43%**.[3] The Waste and Resources Action Programme (WRAP) investigated why and found that 79% of respondents had taken up: - Pre-shop inventory planning - Creative cooking with any available ingredients - Increased awareness of use-by dates [3] So I cooked up a plan to *use the insights from this research* to intervene: >[!question] > How might we use **automation** to reinforce behaviours that **reduce domestic food waste**? #### What's already out there? Some research explores uses physical sensors like ultrasonic sensors, RFID tags and barcode scanning to track items in the fridge, but these always required the user to change the way they use their fridge. I'm looking for minimal barriers to entry and therefore as few behavioural changes as possible! There were also some existing models for recognising food but many were large, computationally expensive and meal-based rather than ingredient-based. Decay recognition had not been incorporated into generalised food models. #### So, I proposed a system that... - Accurately recognises ingredients and store-bought items in real-time with machine vision. - Integrates recognition of 'freshness' for fresh food items into inventory management. ### Building Pocket Kitchen ![[finalprot (1).png]] I used a Raspberry Pi 4 with an HQ camera to log food coming in and out of the fridge to help people use food they already have, buy more efficiently and waste less. This involved: - Taking **2740 photos** of 12 food items in 16 sub-classes and labelling all of them with bounding boxes over 48 hours. - Creating my own **fine-tuned** version of EffificientDet0-Lite with corresponding metadata for **object detection**. - Integrating a MongoDB database and Heroku / Node.JS web app to present the user their fridge **inventory, recipe recommendations and expiry date tracking**. - Leveraging the vision domain, Pocket Kitchen could also **detect states of decay** for several food items! Here's the Pocket Kitchen UI - accessible wherever you are on-the-go, showing what you have at home, the status and a countdown to expiry! ![[Pasted image 20240110215123.png | 400]] I'm working on a new version of my Master's project using GPT4-Vision, [[🥕 Pocket Kitchen (PK2.0)]]. Click the link to find out how it's going! **Read my thesis below** to learn out more about Pocket Kitchen (v1.0). ![[PocketKitchen (1).pdf]] --- [![[Pasted image 20240110213631.png | 100]]](<https://github.com/myPocketKitchen>) --- 1. Monica Crippa, E Solazzo, D Guizzardi, F Monforti-Ferrario, FN Tubiello, and AJNF Leip. 2021. Food systems are responsible for a third of global anthropogenic GHG emissions. Nature Food 2, 3 (2021), 198–209. 2. WRAP. 2020. Food surplus and waste in the UK – key facts. Technical Report. WRAP. https://wrap.org.uk/sites/default/files/2020-11/Food-surplus-and-waste-in-the-UK-key-facts-Jan-2020.pdf 3. WRAP. 2020. CITIZEN RESPONSES TO THE COVID-19 LOCKDOWN – FOOD PURCHASING, MANAGEMENT AND WASTE. Technical Re- port. WRAP. https://wrap.org.uk/sites/default/files/2020-10/ WRAP-Citizen_responses_to_the_Covid-19_lockdown_0.pdf