Generate RSS feeds for (almost) any website with mkfd
Some people don't like visiting websites. Yes, that's hard to believe in an age where you're never more than three shallow breaths away from a web browser at any moment, but it's easy to understand. Websites generally exist to make money for their owners, and while this can be as benign as sponsored links offering great value domain registration services, static banner ads or exhortations to buy the owner a coffee, it can also include tracking that follows your every move across the internet and in the real world.
The same kind of people don't usually like signing up for proprietary services or apps that are designed to capture your data with even more granularity.
In a world where every click is monitored, RSS offers an easy and private way to keep up with your preferred news sources, blogs, property prices, and semi-professional online magazines.
RSS feeds deliver content without tracking
RSS (which may or may not stand for Really Simple Syndication) is a web feed format that allows you to automatically receive updates from websites, such as news articles or blog posts, in a structured XML format.
If you subscribe to a website's RSS feed, every time it pushes a new article, the feed is updated, and when your RSS client next checks, it will pull down the details of the new article. At a minimum, this will include the title, URL, and maybe a brief preview of the story so you can decide whether to click through, or simply digest the headlines. Some websites - such as this one - provide the full article text and images in their RSS feed.
Our H2 isn't quite accurate in saying that RSS is tracking-free. There's no technical reason why analytics code can't be injected into an RSS feed, but it's not something The Crow has ever seen. Additionally, if the site owner has access to logs, they can see which RSS readers have been picking up feeds - along with associated IP addresses. These are small concerns though, and generally speaking, you're pretty safe.
The RSS landscape is a flaming mess
Once upon a time - possibly in the early 2000s - RSS ruled the web. Sites would display RSS-powered tickers at the top of every page, boldly flashing their latest headlines to casual visitors, and the only option to keep up with the latest news was to subscribe to a carefully curated and categorised RSS feed.
Those days are long long gone, and although every WordPress-powered website (about 65 per cent of all content) will generate an RSS feed automatically, almost every news organisation has its own aggressively marketed app, and readable articles are served up to users via Google's "Discover" page built into every Android device. We imagine there's an equivalent for Apple. You can express interest in topics, but you can't choose what you get.
Some websites don't offer RSS feeds at all. National Geographic doesn't bother, for instance. Occasionally, reputable feeds are abandoned and taken over by squatters.
And while in days of yore, a news organisation might split its feeds into categories such as News, Opinion, Sport, UK, and World, that's a rarity these days, and instead, RSS users are forced to swallow the entire firehose output in one gulp.
This will generally include SEO articles that are designed to answer specific user queries and won't appear on any "normal" site pages. Stuff you don't want to read, in short.
mkfd is the answer to all your RSS woes
We don't think we're overstating the case with this subheading. mkfd - a self-hosted service developed by Tim Barani - addresses all of the concerns listed above, and more besides, by helping you to create your own feeds for any publicly facing page on the internet.
It achieves this by scraping, then extracting specified elements from items on the page and combining them in a way that can be interpreted by your reader of choice.
We deployed mkfd back in March 2025, and so far, we've tried it on sites without feeds, sites whose own feeds are stuffed with SEO spam, and we've even used it create feeds of properties for sale within a certain area and price range. It's versatile, is what we're saying.
Installing mkfd is easy-peasy with Docker Compose
If you're using Linux - or even Windows Subsystem for Linux -mkfd offers a variety of installation methods, from piping a bash script via curl:
curl https://bun.sh/install | bash ...to running via Docker with:
docker pull tbosk/mkfd:latest
docker run -p 5000:5000 -v /local/mount/path:/app/configs -e PASSKEY=your_passkey -e COOKIE_SECRET=your_cookie_secret -e ENCRYPTION_KEY=your_encryption_key -e SSL=true/false tbosk/mkfd:latestHere at The Crow, we're heavily invested in a Docker plus Docker Compose stack, as it makes containers easy to orchestrate and manage. The mkfd repository doesn't yet have a docker-compose.yml file, so you'll have to create your own. We ran the Docker command once, and then used Docker Autocompose to create a fully functional Docker Compose file, then tinkered with it until it suited our needs. You can do this yourself, or simply copy ours.
First off, make sure you have Docker and Docker Compose installed on your system, then create a directory for mkfd, and another for its config files.
mkdir mkfd
mkdir mkfd/configsMove into the new mkfd directory:
cd mkfdand use the nano text editor to create a new config file for Docker Compose to use:
nano docker-compose.ymlIn it, paste the following:
services:
dreamy_tharp:
command:
- "bash"
- "-c"
- "bun run /app/index.ts"
container_name: "dreamy_tharp"
entrypoint:
- "/usr/local/bin/docker-entrypoint.sh"
environment:
- "COOKIE_SECRET=Your_Secret"
- "ENCRYPTION_KEY=Your_Very_Long_Encrytion_Key"
- "PASSKEY=Your_Secret_PassKey"
- "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bun-node-fallback-bin"
- "BUN_RUNTIME_TRANSPILER_CACHE_PATH=0"
- "BUN_INSTALL_BIN=/usr/local/bin"
hostname: "e959fb08db1a"
image: "tbosk/mkfd:latest"
ipc: "private"
labels:
org.opencontainers.image.created: "2025-03-27T10:36:35.624Z"
org.opencontainers.image.description: "Incredibly fast JavaScript runtime, bundler, test runner,\
\ and package manager – all in one"
org.opencontainers.image.licenses: "NOASSERTION"
org.opencontainers.image.revision: "5c0fa6dc214431a11cbd8279c6629f0fc562881d"
org.opencontainers.image.source: "https://github.com/oven-sh/bun"
org.opencontainers.image.title: "bun"
org.opencontainers.image.url: "https://github.com/oven-sh/bun"
org.opencontainers.image.version: "1.2.7"
logging:
driver: "json-file"
options: {}
mac_address: "02:42:ac:11:00:02"
network_mode: "bridge"
ports:
- "5000:5000/tcp"
volumes:
- "/home/Your_User_Name/mkfd/configs:/app/configs"
working_dir: "/app"
restart: always
version: "3.6"
You should probably change the values for COOKIE_SECRET, ENCRYPTION_KEY, and PASSKEY. Or don't - it's up to you. Also, make sure to set the paths correctly for working_dir. If you're running another sevice on port 5000, you'll want to change this, too.
When you're done, save and exit nano with Ctrl + O, then Ctrl + X.
Start the containers with:
docker-compose up -d...then go and walk the dogs while Docker Compose pulls images, and sets up mkfd for you.
When it's done and your animals are suitably exercised, open the browser of your choice and visit localhost:5000.
If you've installed mkfd on a local or remote server, go to your.server.ip.address:5000 instead.
How to create RSS feeds from any website
While mkfd offers a whole bunch of features, we're only going to show you the basics - enough to get you up and running.
Once mkfd is loaded in your browser, you'll be presented with cryptically titled text fields the purpose of some of these is obvious. Others, less so.
The first thing you'll want to pay attention to is the Preview button at the bottom of the page. As the name suggests, hitting this will generate a preview of the feed in its current state. You should check this after every change to make sure it's being generated as you expect.
Next, give a name to your feed by filling in the Feed Name field.
For the Target URL field, copy and paste the address of the page you want to scrape from your browser's URL bar.
As a demonstration, we've decided to go for the History and Culture section of National Geographic, so in this case, the Target URL is https://www.nationalgeographic.com/history .
Hit preview at this point, and your generated feed will be as follows:
<rss version="2.0">
<channel>
<title>Nat Geo </title>
<description>Nat Geo </description>
<link>https://www.nationalgeographic.com/history</link>
<generator>Generated by mkfd</generator>
<lastBuildDate>Sat, 26 Jul 2025 14:03:10 GMT</lastBuildDate>
<author>mkfd</author>
</channel>
</rss>That's great and all, but it doesn't contain any stories. You'll need to identify the Item Selector that each individual article has in common, then features unique to each one.
A good place to start is to expand the Link section, then add a as the Link Selector, and href as the Link Attribute. These are the same across most sites and will allow you to quickly validate your Item Selector.
mkfd has a Selector Playground that allows you to click on any any component of the page you're scraping and quickly see the relevant selector.
Click on the Selector Playground and click around. You're looking for something that all the stories have in common. For Nat Geo, this is ".PromoTile", so paste this into mkfd' Item Selector field.
You don't have to use the Selector Playground, and we find it a little clunky. Instead, you can open the page in FireFox, right-click on the area you're interested in, then click Inspect. You'll be able to see the tree and easily home in on what you're looking for. Right-click the relevant place in the inspection panel then Copy > CSS selector.
Once you've found the item selector, hit Preview again, and inspect the document tree. There should be a section for every article that has the ".Promotile" selector. If you pre-filled the link attributes, there should be a valid link in the following format:
<link>
https://www.nationalgeographic.com/culture/article/hurricane-katrina-anniversary-new-orleans-rebuilding
</link>So far so good. Next you need a Title Selector or a Title Attribute. The attribute can be something like an H2, but it's better to go for the selector itself. You don't really need both.
For the page we're looking at, the selector is ".PromoTile__Title--mobile", so plug it in and hit the preview button again. There should be a new line for each iterator along these lines:
<title>
20 years after Katrina, New Orleanians are redefining 'home'
</title>That's great! We have the right title and URL, and basically that's all we need. More to the point, it's all that the iterators on National Geographic are providing - it's enough to let you know whether you want to read the full article, anyway.
If the scraping target does have additional attributes such as description, image, video, author, and date for each article, you can add them in the relevant field in mkfd, or even misuse one of the existing fields.
Use mkfd to create shopping or Right Move alerts
mkfd isn't limited, and you can use it in unconventional ways, such as updating your feeds when items that match a premade search are added to a site.
Say, for instance, you're looking to buy a four-bedroom house in Liverpool. You don't want to pay more than £110,000 and you don't want a flat.
You want to be notified pretty quickly if one comes up, and you don't want to be visiting the website dozens of times per day.
If you visit Right Move and fill in the relevant fields, your address bar will look something like:
https://www.rightmove.co.uk/property-for-sale/find.html?searchLocation=Liverpool%2C+Merseyside&useLocationIdentifier=true&locationIdentifier=REGION%5E813&radius=0.0&maxPrice=110000&_includeSSTC=on&minBedrooms=4&index=0&sortType=2&channel=BUY&transactionType=BUY&displayLocationIdentifier=Liverpool.html&propertyTypes=detached%2Csemi-detached%2Cterraced%2Cbungalow... while the rest of the page is populated with pictures and descriptions.
Copy that URL to your clipboard and head back over to your mkfd instance.
Give the feed whatever title you want, and paste the URL into the Target URL field.
The Item selector is obfuscated, but will work with little wildcard magic :
div[class^="PropertyCard_propertyCard"]For the Title Selector and Link selector, go with ".propertyCard-link" and for the link attribute "href".
Right Move uses relative links, so check the relevant box, and put "https://www.rightmove.co.uk" as the base URL.
You'll want a description of the property as well:
[data-testid="property-description"]...works as a selector.
[data-testid="property-price"]... is obviously the price. mkfd doesn't have a field for this, so feel free to stick it in any unoccupied field, such as Author Selector.
Generating your mkfd feed
When you have all the information you want in your XML preview, it's almost time to generate the feed itself.
Before you do, consider how often you want to scrape the site in question. Too often and you'll likely be blocked.
By default, mkfd will refresh the feed every five minutes. To our mind this is a bit too frequent, but you can change the value in the Additional Options section.
Once you're happy, scroll down to Submit, and generate the feed.
Ideally, you're running mkfd on a machine on your local network, in which case the result will be something along the lines of "http://192.168.1.xxx:5000/public/feeds/0xxxc8-9xxx-4443-xxx-45431xxxxxxx.xml
As mentioned, we're running a FreshRSS instance on the same machine, and can simply add that URL manually and it will work. We can access it or Fresh feeds withan RSS client wherever we are in the world.
If you're running on a remote server, bear in mind that mkfd doesn't currently have any kind of authentication built in - meaning that internet randos may be able to access your instance and add unsavory feeds. Make sure that that you're behind a proxy, have ufw properly configured, and use htpasswd to restrict access.
RSS isn't for everyone
We get that some people prefer not to have to deal with RSS - especially when you have to create your own feeds to get the results you want. If you do go down the route of installing apps and signing up for services and newsletters, make sure you take necessary steps to protect your email inbox from spam.
