catflap

The catflap for detecting prey in the cats snout. Based on a raspberry pi 2 with the raspicam and the catflap sureflap. The raspberry runs ubuntu with ros indigo, python 2.x opencv. The system has 4 sensors, two ir sensors to detect the presence of a cat in the catflap tunnel, a sensor to check, whether the status of the flap and a camera to take pictures of the cats profile.

I further think that the original idea was not created by the three youngsters or at least they were not the only ones. I was further inspired by joakim soderberg and the catcierge https://joakimsoderberg.github.io/catcierge/ (Nice work Joakim!) and the flow control project which can no longer be found on the web (origial url: http://www.quantumpicture.com/Flo_Control/flo_control.htm) but we can find some remaints of its impact on others http://www.openscience.org/blog/?p=109.

Some time after the flo control project, three germans participated in a contest called 'jugend forscht' http://www.neumarktonline.de/art.php?newsid=59507) with a simpler concept only using light barriers. The price was patent application http://www.freepatentsonline.com/DE202009002677.html which is currently owned by the Feldberglicht GmbH. They are still developing a commercial application or at least the say they are developing- check it out http://www.mouse-lock.de/. But don't expect a working catflap before a working fusion reactor ;)

usage so far

setup a raspi 2 with ubuntu, ros, opencv raspi cam and the sensors. adapt the GPIO Pins, if neccessary.
clone the repo into a ros workspace and build it
start the single nodes to check your hardware
if all is working fine, use the launch file for everything roslaunch catflap_training.launch to start the ros nodes to aquire images, when a cat passes the flap

i added the startup.sh via sudo crontab -e just add a line like @reboot /home/max/projects/catflap/ros_catkin_ws/src/startup.sh >>/home/max/projects/catflap/log_launch 2>&1 to start it at when the raspi boots up.

What it looks like:

charlie and her catflap — Charlie in front of her catflap

catflap seen from inside — View from inside, light on, door closed

catflap open - backlight on — door opened, backlight can be seen

peek through to the right - outer ir sensor — peeking through the door, looking at one of the IR sensors

peek through, look to the right raspi cam — peeking further trough, looking to the right, on the right side, the raspi cam housing. The raspi is mountet on a wooden board to the left. Reflections can be seen in the plexiglass plate that shall keep the wiring save :)

charlie inspecting the catflap — Charlie inspects the catflap

build phase - testing the backlight — backlight during build phase

Why I need a catflap and how i thought it should work

My work is based on a SureFlap catflap that I bought three years ago. It has the feature to learn the signature of certain RFID devices which are used to tag cats, and release the catflap lock only for known RFID signatures.

I bought the catflap as it has a convenient locking mechanism included. We had two cats at that time. They embraced their freedom to come and leave as they wanted and - as any decent cat would do - brought back prey from time to time as a present - as a proof of their love. I planned on stopping the - to me sometimes very disgusting - presents with a raspberry pi I had at home. The idea was to pictures of the cat before it enters the flap, and with some image recognition, release the lock only when there is no prey in the cats snout. I build a housing to mount it in front of my catflap, which contained the raspberry pi together with the raspi cam, two ir sensors mounted at the entrance of the housing an a IR LED spotlight behind the camera, lighting the cat that enters.

After a while I switched to a setup using backlight and modified my housing accordingly, as my plans were to work on black and white images of the cats contours. I want to show you two images with very nice presents:)

my cat Socke with a mouse — a small mouse, i don't remember if it was a living or a dead mouse

dont know which cat with gutted blackbird — gutted blackbird, definitely dead

Unfortunately, my cat Luigi was hit by a cat and died, and the other cat Socke lived with us for some more months. At some day she just didn't return home. We never found out what happened to her, hopefully she just found a new home. So this was the end of the catflap plans - prototype one.

The new cats on the block

A little down the road we decided to get two new cats. Since Socke and Luigi had a lot of bad luck, we did not want to let them go outdoors. Bad luck did not end here, one of the two, called Kitty died soon after we got her (FIP) So we were a one cat household with an indoor cat.

But a little further down the road our household got a new fellow lodger, the cutest baby in the world :) And things got worse with our cat Charlie. She simply has a loooooot of energy that she could not for her life shake off during her day or her night. So after some artistic high speed stunts involving rayor sharp claws very close to our baby we decided that Charlie should indeed be an outdoor cat again. This time the catflap would lead her to the garden, and not he front door, minimizing the need to cross the street there and the according risks of getting hit by a cat or something.

In her first night out our cat must have been replaced by someone or at least brainwashed. The standoffish cat left and a snuggly cat returned home. But it's ok for us, she still looks the same:)

Time for a new catflap prototype

Considering the energy Charlie displayed so far I expected her to bring home a small deer or one of the neighbours dogs at least on one of her next next nightly tours. we were lucky so far, she only brought in small insects.

charlie bringing home a small insect — Charlie bringing home a small insect

In the meantime a raspberry pi 2 was bought, a new housing was constructed and I heard about ros and wanted to try it out. And here we are with the catflap. Right now it can aquire images, when a cat enters the door and does not take pictures when the door is already open, or when a cat is leaving the house.

How the protoype works right now:

Two IR sensors detect when Charlie enters the housing. A door sensor detects, if the door is closed or open. When a the IR sensors are active and the door is closed and an internal timeout has elapsed, the backlight is turned on and an image is recorded, saved, and the backlight is turned off. A timeout is set to a certain short period then (under a second). If the IR sensors are still active, the door is still closed when the timeout has elapsed, the light is turned on again, image recorded ... timeout is set again ... and so on until the door is opened.

As long as the door is open no image will be recorded. When the door closes, a timeout is set to a certain long period to get a certain dead time (about 5 seconds).

The next steps are:

refactor the timeout and dead time. The camera should detect, when a cat leaves the house and only start recording images, when the cat has left the housing for a certain time. So far it starts recording when the door was closed for a certain time.
aquire lot of training images
train a classifier to detect the snout in the image
do something to detect prey in the snout - I don't have concrete plans here
atomically connect the sureflaps locking mechanism to the raspi or mount a servo on the locking mechanic.
install a second door before the RFID tunnel of the SureFlap - to keep the cat in the cameras field of view (The SureFlap has a RFID tunnel of about 10cm in length before the door. The camera can't see the cats head when she is standing right in front of the locked door, waiting for the lock to release. A second door, closing just with the end of the tunnel, mechanically connected with the inner door should do the job to keep the cat in the field of view)
create a new ros node for lock - release logic
create a new node for mailing a thumbnail picture plus lock/release decision
some more ideas will certainly come along

Update May 2017: I did work on some of the steps above:

refactor the timeout and dead time. - the timeout and dead time are quite ok now. If the door was opened once, the ir sensors must not be active for two consecutive message broadcasts (minimum 5 sec. each) before a new detection can be started
aquire lot of training images - Well i really have a lot now, but i might need them for training a second, better classifier
train a classifier to detect the snout in the image - Did that, on a catsnout, i will link to some valuable resources for that
do something to detect prey in the snout - I don't have concrete plans here - pray gets detected as I explain in details below
atomically connect the sureflaps locking mechanism to the raspi or mount a servo on the locking mechanic. - the servo is mounted and operating
install a second door before the RFID tunnel of the SureFlap - to keep the cat in the cameras field of view (The SureFlap has a RFID tunnel of about 10cm in length before the door. The camera can't see the cats head when she is standing right in front of the locked door, waiting for the lock to release. A second door, closing just with the end of the tunnel, mechanically connected with the inner door should do the job to keep the cat in the field of view)
create a new ros node for lock - release logic
create a new node for mailing a thumbnail picture plus lock/release decision
some more ideas will certainly come along

How detection and the door opening logic works: I trained a haar classifier on my cats snout. catsnout.xml is the haar classifier for catsnouts trained with opencv. (with the help of Thorsten Ball, here too and Trevor Sherrard) The classifier is used to look for catsnouts in each image taken. If a catsnout is found: The area of the catsnout (rectangle) and below (two times the catsnout rectangles height) is the region of interest for detection. The background is partly removed from the roi, then it is blurred and then thresholded with OTSUs method to get the biggest contour in this area (usually the catsnout and pray). If this contour is reaching down below the catsnout rectangle and takes up more pixels than a configurable threshold a pray is detected. If the contour does not reach down, or the area taken up is below the threshold, no pray is detected. Or there can be no catsnout found at all. The doorlock is operated on a trust value. The thre possible detection outcomes each have a configurable trust factor assigned. e.g.

catsnout detected, no pray - factor 2.0
catsnout detected, pray detected - factor 0.33
no catsnout detected - factor 0.95

The momentary trust value is multiplied with the detection outcome factor. Starting value is 1.0 The door opens if the trust value is above or equal to 4.0