Posts on Dino Fizzotti

Running a man-in-the-middle proxy on a Raspberry Pi 4

Sun, 24 Apr 2022 00:00:00 +0000

This post is an update to my 2019 page on Running a man-in-the-middle proxy on a Raspberry Pi 3, now revisited and rewritten to accommodate using a Raspberry Pi 4, the current version of mitmproxy (v8.0.0), Raspberry Pi OS (bullseye) as well as changes to how some of the software is installed and configured.

I have repeated much of the original content, especially the overview and explanations, so you do not need to refer back to the original 2019 post.

SiteMapper Part 2: Distributed crawling using Kubernetes, NATS and Cassandra

Tue, 04 Jan 2022 01:00:00 +0000

In Part 1 of this project series I created a stand-alone CLI tool written Go to build a sitemap of internal links for a given URL to a specified maximum depth. In Part 2 I describe I how achieved the same result of creating a sitemap, but by distributing the crawl activity using the Kubernetes API to schedule independent ephemeral crawl jobs for each link. I’m using NATS for pod-to-pod messaging and AstraDB (a managed Cassandra DB) for persistence. All of the application code is written in Go.

SiteMapper Part 1: Exploring concurrency in Go

Tue, 04 Jan 2022 00:00:00 +0000

$ ./sm -s https://google.com | jq
2021/12/31 14:57:26 Using mode: concurrent
2021/12/31 14:57:26 Crawling https://google.com with depth 1
2021/12/31 14:57:26 visiting URL https://google.com at depth 0 with parent https://google.com
2021/12/31 14:57:26 Elapsed milliseconds: 263
[
{
"URL": "https://google.com",
"Links": [
"https://accounts.google.com/ServiceLogin",
"https://drive.google.com/",
"https://mail.google.com/mail/",
"https://news.google.com/",
"https://play.google.com/",
"https://www.google.com/advanced_search",
"https://www.google.com/intl/en/about.html",
"https://www.google.com/intl/en/ads/",
"https://www.google.com/intl/en/policies/privacy/",
"https://www.google.com/intl/en/policies/terms/",
"https://www.google.com/preferences",
"https://www.google.com/search",
"https://www.google.com/services/",
"https://www.google.com/setprefdomain"
]
}
]

I’ve really been enjoying my journey in becoming more familiar with Go. One of Go’s strengths is its built-in concurrency primitives, namely goroutines and channels. I decided to practice my Go skills by building a tool which explores some of these features. SiteMapper accepts a root URL and a maximum depth, and then uses one of three modes of operation to crawl the site (synchronous, concurrent and concurrent-limited), building out a sitemap listing links to internal pages.

Part 1 of this project details the stand-alone CLI tool implementation written in Go. Part 2 details an implementation using Kubernetes, where site crawling is performed by ephemeral Kubernetes job pods.

FnRow v1: A configurable function-row-layout mechanical keyboard

Sun, 07 Feb 2021 00:00:00 +0000

FnRow is a mechanical keyboard I designed and built during the Christmas break. It features a single row of switches in the form of a “function row”. Each switch is configurable and can be programmed to perform as any key on a typical keyboard, or even combinations of key presses. FnRow is akin to a “macropad”, but instead of having a square or rectangular “pad”, the switches are stretched out in a single row. All my hardware and software source files are available on GitHub.

Raspberry Pi Cluster Part 3: Running Load Tests with Kubernetes and Locust

Sat, 04 Jul 2020 13:40:46 +0100

This post details how I used a Python based load test framework (Locust) to perform some simple tests on an HTTP API application using Kubernetes in my Raspberry Pi Cluster.

Raspberry Pi Cluster Part 2: ToDo API running on Kubernetes with k3s

Sat, 09 May 2020 11:26:00 +0100

In this post I go over how I set up my Kubernetes cluster across four Raspberry Pi 4 nodes using k3s, configured persistent storage using NFS, and then installed a simple “todo” API into the cluster using Helm.

Raspberry Pi Cluster Part 1: Provisioning with Ansible and temperature monitoring using Prometheus and Grafana

Fri, 10 Apr 2020 12:46:42 +0100

I decided to build a Raspberry Pi cluster to give me a platform with which I can practice distributed computing technologies without needing to rely on a cloud provider.

This first post details my hardware set-up as well as how I used Ansible to “remote control” the installation of monitoring software on each of the Pi hosts, with the goal to observe the Raspberry Pi CPU temperatures.

Diskplayer: Using 3.5" floppy disks to play albums on Spotify

Wed, 05 Feb 2020 22:13:04 +0000

Diskplayer is an audio device which uses physical media to play streaming music with 3.5” floppy disks and Spotify. You can find a GitHub repo with code here: https://github.com/dinofizz/diskplayer and a video showing playback and record activity here: https://youtu.be/1usBGe_ZiGc

Running a man-in-the-middle proxy on a Raspberry Pi 3

Wed, 09 Jan 2019 08:35:00 +0200

[2022-04-24] See my new post on running mitmproxy on a Raspberry Pi 4 here.

[2020-06-21] I have done another run through of this tutorial on my Rasperry Pi 3, this time with the latest Raspberry Pi OS. Changes made to the tutorial are indicated with a note featuring a timestamp “[2020-06-21]”. Let me know in the comments if you are unsuccessful. I try to re-run everything every 6 months or so.

[2019-08-03] I have since updated this post with new instructions for running mitmproxy on Raspbian Buster, which now includes Python 3.7.

In preparation for a training session I will be giving on public key infrastructure (with a focus on TLS and certificates) I wanted to demonstrate how a transparent “man-in-the-middle” (MITM) proxy works.

This post walks through the configuration of a Raspberry Pi 3 acting as a Wi-Fi access point, running a transparent man-in-the-middle proxy (mitmproxy), which can be used to sniff HTTP and https traffic on connected devices.

CarbAlert - Part 4: Deploying and Using CarbAlert

Sun, 14 Oct 2018 08:27:43 +0200

This is part 4 of a 4 part series of articles where I explain how I discovered and purchased my laptop by building a web application which scrapes a local PC parts forum and sends automated email alerts when posts featuring specific keywords appear:

Part 1: Let Your Next Laptop Find YOU!
Part 2: Django and Scrapy
Part 3: Celery, Mailgun and Flower
Part 4: Deploying and using CarbAlert (this page)

CarbAlert on GitHub: https://github.com/dinofizz/carbalert

Docker

As part of my quest to learn new things I wanted to deploy my CarbAlert solution using Docker and Docker Compose. Docker Compose is well suited to this application as it enables a group of related Docker containers to be built and deployed together.

CarbAlert - Part 3: Celery, Mailgun and Flower

Sun, 14 Oct 2018 08:20:49 +0200

This is part 3 of a 4 part series of articles where I explain how I discovered and purchased my laptop by building a web application which scrapes a local PC parts forum and sends automated email alerts when posts featuring specific keywords appear:

Part 1: Let Your Next Laptop Find YOU!
Part 2: Django and Scrapy
Part 3: Celery, Mailgun and Flower (this page)
Part 4: Deploying and using CarbAlert

CarbAlert on GitHub: https://github.com/dinofizz/carbalert

Celery

Celery is a distributed task queue framework. In conjunction with a message broker (in my case Redis) it can be used to process asynchronous tasks as well as schedule periodic tasks. I am using both of these features:

A periodic task is run every 5 minutes to initiate the Scrapy CarbSpider to scrape and scan the first page of the Carbonite Laptop forum index page for new threads featuring search phrases of interest.
From within the CarbPipeline activity I push asynchronous email tasks for Celery to handle. This separates the sending of my email notifications from the parsing of the thread metadata.

CarbAlert - Part 2: Django and Scrapy

Sun, 14 Oct 2018 08:13:05 +0200

This is part 2 of a 4 part series of articles where I explain how I discovered and purchased my laptop by building a web application which scrapes a local PC parts forum and sends automated email alerts when posts featuring specific keywords appear:

CarbAlert on GitHub: https://github.com/dinofizz/carbalert

Django

In order to manage the search phrases and email addresses I am using Django. Django is a Python web framework, and is known for including many extras right out of the box. I am taking advantage of two specific extras: Django’s built-in ORM and the Django admin console.

CarbAlert - Part 1: Let Your Next Laptop Find YOU!

Sun, 14 Oct 2018 08:07:42 +0200

This is part 1 of a 4 part series of articles where I explain how I discovered and purchased my laptop by building a web application which scrapes a local PC parts forum and sends automated email alerts when posts featuring specific keywords appear:

Part 1: Let Your Next Laptop Find YOU! (this page)
Part 2: Django and Scrapy
Part 3: Celery, Mailgun and Flower
Part 4: Deploying and using CarbAlert

CarbAlert on GitHub: https://github.com/dinofizz/carbalert

TL;DR

CarbAlert is a web application which scrapes a local (South African) second-hand computer parts forum for new posts offering laptops featuring keywords of interest (specifically the first page of the “laptops” forum) I’m using Django for the admin console and database/ORM integration, Scrapy for web-scraping, Celery for task management, Flower for task monitoring and Mailgun for sending out alert emails. I am using Docker to manage and run the containers which make up the CarbAlert application.

Find the code here: https://github.com/dinofizz/carbalert

NoiseBlanket: Arduino White Noise Player with IR Remote Control

Mon, 21 Aug 2017 00:00:00 +0000

Overview After a recent sinus/ear infection I began to experience tinnitus. In my case it presents itself as a constant high frequency static hiss/whine in my left ear. It’s been about a month since I first noticed it. Hopefully it will eventually disappear. During the day when I’m at work or around friends I don’t really notice it. However when I’m in a quiet room, such as when falling asleep or in the early morning after waking up, it is quite noticeable and distracting.

A Python wrapper for the Adafruit USB/Serial LCD Backpack

Sat, 10 Jun 2017 08:11:36 +0200

Overview I’m working on a project which requires a character LCD to work with a Raspberry Pi. A character LCD is thing displaying the text in the image above. The one I am currently using is a “green backlit 16x2” character LCD. This means that the display is capable of displaying 2 rows of 16 characters, and it features black characters on a green background. There are various sizes and colour combinations (text and backlight) available.

Adding Hugo version and commit information to a status page

Mon, 01 May 2017 18:25:44 +0200

As described in a previous post, I have set up a GitLab CI runner to build and deploy this blog. For obvious content changes it is easy to see if they have been applied - I can just visit the site itself. For changes that are a bit more “behind the scenes” it may not be so easy to determine if and what changes have been applied. For example: updating the version of Hugo used by the GitLab CI runner to generate the site.

Domain Name Change

Sun, 26 Mar 2017 09:04:34 +0200

FYI I have decided to “re-brand” this blog under my own name. I have 301’d all the server blocks for practicalmagic.co.za to point to dinofizzotti.com. I also promise to actually blog more often (sorry).

Automated blog posts with Hugo, GitLab CI and Docker

Tue, 18 Oct 2016 16:52:22 +0200

…or how this blog is built and deployed. Introduction I’m using the tools and methods described in this post because I wanted to learn more about Hugo, GitLab CI and Docker. I don’t claim this is the best way of combining these technologies, or that everyone should do it this way. I wanted to create a blog and publish content in a way that felt fun, and at the same time learn something new.