src/Blog/wwwroot/posts/atproto-personal-data-server-setup/index.md

@@ -0,0 +1,219 @@
+```json meta
+{
+  "title": "How to Run Your Own Personal Data Server",
+  "lead": "The idea and vision behind the At protocol really excites me as someone who has spent a lot of time over the last decade creating content on the web. At the core of that excitement is the ability to host your own data server where all the content you publish to the atmosphere is stored. I recently went through the process of migrating my existing atproto identity to my own PDS and wanted to share how I did it.",
+  "isPublished": true,
+  "publishedAt": "2026-08-28",
+  "openGraphImage": "posts/atproto-personal-data-server-setup/og-image.png"
+}
+```
+
+I've spent a good amount of my adult life making things on the internet. It started my senior year of high school with [this](https://intellectualfitness.weebly.com/) not so impressive Weebly website. At the time and still to this day I was a pretty big nerd about all things physical fitness and found it was cool to be able to share the things I was learning and studying with other people. Obviously I'd be lying too if I didn't also want it to be a way to establish myself as a bit of an - albeit tiny - authority on the topic to my friends and family. It wasn't too much earlier than that when I got introduced to social media on the internet with a Facebook account and not too much later before I started using it and other social platforms as again a way to share content about what I was passionate about at that time and build a bit of a personal brand. And while what I make and how I make it has changed over the last decade or so my enjoyment for creating content really hasn't ever left me.
+
+However, the internet of today isn't the same as it was back then and I imagine that will continue to be the case as time goes on. Platforms come, many go, and some stay and even those that stay change and evolve over time losing and increasing relevance depending on whatever else is going on in the world. A pain point for me as someone who has a regular life with a day job, kids, and a wife has always been that I've found it quite a bit exhausting to always have to be maintaining the different identities I have across platforms like [Twitter](https://x.com/stevanfreeborn), [Facebook](https://www.facebook.com/stevanreece), [Instagram](https://www.instagram.com/stevan.freeborn/), [YouTube](https://youtube.stevanfreeborn.com), [Twitch](https://twitch.stevanfreeborn.com), etc. Not to mention it is a pain that anything I share on one of those only exists there and anyone who follows me on one of those only follows me there. The social web and let's be honest for the most part today the social web is pretty much a bunch of siloed, walled gardens that sit next to each other. From a content creation perspective as well as just from a general user perspective this sucks. If I ever want to stop using one platform it means I stop getting the content from the people I enjoy following and the small subset who enjoy following me don't get any of my content either. Granted I know this isn't quite the way content gets distributed to folks on these platforms today, but I think you get my point. Our identities and our data on these apps exist only and exclusively to those apps.
+
+## What is a Personal Data Server (PDS)?
+
+It was because of this reason I got really excited about the [At protocol](https://atproto.com/) when I learned about it. I signed up for [Bluesky](https://bsky.app/) likely for a lot of the same reasons everyone else did. Twitter became some strange, lesser form of its former self and if you liked that style of social media Bluesky seemed to present a great alternative experience. At the time though I knew nothing about the underlying technology or how absolutely kickass the whole vision of it was. See the idea behind the At protocol is basically that the social web should be like the original web was - open, decentralized, interoperable. Users should be able to establish an identity on the network and then own and control the content they publish on that network. I'm a fairly new beginner when it comes to all things atproto and many smarter people have done a better job writing about what the At protocol is, how it works, and how you can try to conceptually think about it so I'll just link to those folks below.
+
+- [https://atproto.com/guides/understanding-atproto](https://atproto.com/guides/understanding-atproto)
+- [https://overreacted.io/open-social/](https://overreacted.io/open-social/)
+- [https://overreacted.io/a-social-filesystem/](https://overreacted.io/a-social-filesystem/)
+
+The point of this post though is to talk through a recent project I did related to the At protocol that was fun and also highlights one of the coolest parts about the protocol and network. Remember when I said users get to own and control their data? Well that is exactly the case and one of the most fundamental pieces of enabling that is the idea of a Personal Data Server (PDS). This is basically the home for all the data that you publish to the [atmosphere](https://atproto.com/guides/glossary#atmosphere) (a fun way to refer to the ecosystem of atproto). A PDS might sound like a new concept to you, but if you are on Bluesky you are using a PDS it is just likely one supplied by Bluesky.
+
+The [Personal Data Server](https://atproto.com/specs/account) is a part of the [At protocol specification](https://atproto.com/specs/atp) which means anyone can build their own implementation of it and then anyone is free to host those implementations and users are free to pick and choose where they specifically choose to host their data on one of these PDSs. You can even host your own on your own hardware as I've chosen to do and how I did it is what I want to share with you. This likely won't serve as a great step by step tutorial given the topology of my home lab probably is unique to me, but maybe some of what I share can help you and at the very least pique your interest into learning more about the At protocol and getting involved with it.
+
+## Choosing a PDS Implementation
+
+The first step in setting up a PDS is deciding on which implementation you are going to use. There are actually quite a few floating around in the wild, but the vast majority from what I looked at seem to be more of toy or hobby projects rather than anything serious that you'd actually want to deploy and run in production. There are though some options and I'll just share the ones I considered below:
+
+- [Bluesky PDS](https://github.com/bluesky-social/pds)
+- [Cocoon](https://github.com/haileyok/cocoon)
+- [Tranquil](https://tangled.org/tranquil.farm/tranquil-pds)
+
+Ultimately I chose to stick with what looks to be the oldest, most used, and probably best supported and maintained - which to no surprise - is Bluesky's own implementation. I know it is a pretty boring choice, but when it comes to picking technologies that I am actually going to trust to support my own personal needs for the foreseeable future boring and reliable sound wonderfully attractive to me. Not to mention it never hurts to be using something a lot of other people are also using and which likely means there is a lot of freely available helpful information about out on the web...because you know the web is awesome like that. Kind of illustrates the point why you'd want the social web to be similar.
+
+## Deployment Architecture & Reverse Proxy Setup
+
+With a PDS implementation selected it is now time to figure out the plan for deployment. This is where things are going to get uber specific to me so let me just give you the lay of the land a bit for myself when it comes to self-hosting things like this. I like to use VPSs either hosted in my homelab or in someone's cloud. I like to proxy traffic from the outside world to my servers through [Cloudflare](https://cloudflare.com). I like to use [Nginx](https://nginx.org/) as a webserver. And whenever possible I like to run things in containers on the host. I don't think any of that is edgy or really notable, but those technologies are ubiquitous, well-supported, and don't have much of a cost to them at all.
+
+I don't want to get into the details too much on the DNS and networking side of things here, but I do want to highlight what steps I took in preparation just to make sure I had things properly configured to make sure that traffic could reach my server. I started by making sure that there was a DNS record entry for the domain I want the PDS to be available at pointing to the machine where the PDS container will be running. Then I made sure I had an nginx configuration setup to properly route traffic received at the machine for this domain to the PDS which will be running in a container. Below is an example of what that config looks like:
+
+```conf
+map $http_upgrade $connection_upgrade {
+        default upgrade;
+        ''      close;
+}
+
+server {
+        server_name <proper_domain>;
+
+        listen 80;
+        listen [::]:80;
+
+        return 302 https://$server_name$request_uri;
+}
+
+server {
+        server_name <proper_domain>;
+
+        listen 443 ssl http2;
+        listen [::]:443 ssl http2;
+
+        ssl_certificate         /etc/ssl/cloudflare_cert.pem;
+        ssl_certificate_key     /etc/ssl/cloudflare_key.pem;
+        ssl_client_certificate  /etc/ssl/cloudflare.crt;
+        ssl_verify_client on;
+
+        location / {
+                client_max_body_size 512M;
+                proxy_pass http://localhost:3737;
+                proxy_set_header Connection $connection_upgrade;
+                proxy_set_header Upgrade $http_upgrade;
+                proxy_set_header Host $host;
+                proxy_set_header X-Real-IP $remote_addr;
+                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+                proxy_set_header X-Forwarded-Proto $scheme;
+        }
+}
+```
+
+Pretty standard typical Nginx reverse proxy stuff. Seriously I don't mean to skip over anything here just everyone's exact deployment environment is likely going to be different especially when we get into the self-hosting world that I don't want to belabor any of it more than what is necessary. If you have questions you can always hit me up on the atmosphere here. 
+
+## Running PDS in Docker
+
+The [Bluesky PDS](https://github.com/bluesky-social/pds) implementation already comes containerized and if you are starting with a fresh machine they've pretty much done all the heavy lifting for you with great install and setup [script](https://github.com/bluesky-social/pds#installing-on-ubuntu-200422042404-and-debian-111213) you can use to get up and running in no time aside from any potential networking or DNS management you might need to do on your own. In my case though I'm not deploying to a fresh machine. The machine I'm using is a shared host that I use for several other things and is already setup to handle traffic proxied through Cloudflare and has Nginx already running on it as a reverse proxy so using the out-of-the-box setup that Bluesky provides didn't work for me. The changes though I had to make were really minimal.
+
+I basically stole the necessary services configurations for their official docker compose file and created one for myself that looks like this:
+
+```yml
+services:
+  pds:
+    container_name: pds
+    image: ghcr.io/bluesky-social/pds:latest
+    restart: always
+    ports:
+      - "3737:3000"
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    volumes:
+      - type: bind
+        source: .
+        target: /pds
+    env_file:
+      - ./pds.env
+  watchtower:
+    container_name: watchtower
+    image: ghcr.io/nicholas-fedor/watchtower:latest
+    restart: always
+    volumes:
+      - type: bind
+        source: /var/run/docker.sock
+        target: /var/run/docker.sock
+    environment:
+      WATCHTOWER_CLEANUP: true
+      WATCHTOWER_SCHEDULE: "@midnight"
+```
+
+> [!NOTE]
+> If you aren't familiar with [watchtower](https://github.com/nicholas-fedor/watchtower) like I was it is basically a docker container that you can run in your stack that will manage automatically updating the version of the containers you are using without you having to manually do it yourself. It isn't a mandatory inclusion, but I figured why not?
+
+You'll likely notice that the `pds` container does expect to receive some configuration through environment variables which we are supplying with a `.env` file. These variables and what they do are explained thoroughly in [Bluesky's documentation](https://github.com/bluesky-social/pds#environment-variables). The values for some of them will also require you to run some commands to generate what you'll need.
+
+`PDS_JWT_SECRET`
+
+```sh
+openssl rand --hex 16
+```
+
+`PDS_ADMIN_PASSWORD`
+
+```sh
+openssl rand --hex 16
+```
+
+`PDS_PLC_ROTATION_KEY`
+
+```sh
+openssl ecparam --name secp256k1 --genkey --noout --outform DER | tail --bytes=+8 | head --bytes=32 | xxd --plain --cols 32
+```
+
+Below is what my final `.env` file looks like albeit I obviously populated the actual values with the values I obtained in the prior steps.
+
+```env
+PDS_HOSTNAME=<domain_where_you_expect_the_pds_to_be_reached>
+PDS_JWT_SECRET=<jwt_secret_you_generated>
+PDS_ADMIN_PASSWORD=<admin_password_you_generated>
+PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX=<rotation_key_you_generated>
+PDS_DATA_DIRECTORY=<path_to_where_you_want_data_stored>
+PDS_BLOBSTORE_DISK_LOCATION=<path_to_where_you_want_blobs_stored>
+PDS_BLOB_UPLOAD_LIMIT=314572800
+PDS_DID_PLC_URL=https://plc.directory
+PDS_BSKY_APP_VIEW_URL=https://api.bsky.app
+PDS_BSKY_APP_VIEW_DID=did:web:api.bsky.app
+PDS_REPORT_SERVICE_URL=https://mod.bsky.app
+PDS_REPORT_SERVICE_DID=did:plc:ar7c4by46qjdydhdevvrndac
+PDS_CRAWLERS=https://bsky.network
+LOG_ENABLED=true
+PDS_RATE_LIMITS_ENABLED=true
+PDS_INVITE_REQUIRED=true # !IMPORTANT: Prevents just any random from creating an account on your PDS
+PDS_EMAIL_SMTP_URL=smtp://<pds_user_on_my_host>:<password_for_pds_user_on_my_host>@host.docker.internal/?servername=<mail_domain>
+PDS_EMAIL_FROM_ADDRESS=<email_address_to_use>
+PDS_CONTACT_EMAIL_ADDRESS=<email_address_to_use>
+```
+
+Assuming you've got the compose and `.env` file created and they are situated in the proper places relative to where one expects the other all that is left to do to get the PDS up is run `docker compose up -d`. If all goes well the PDS container should start and be running ready for you to use. You can do some simple smoke testing to verify.
+
+```sh
+# Verify the container responds locally
+curl http://127.0.0.1:<proper_port>/xrpc/_health
+
+# Verify the container responds remotely
+curl https://<proper_domain>/xrpc/_health
+
+# Verify you can establish a websocket connection to the container
+wsdump "wss://<proper_domain>/xrpc/com.atproto.sync.subscribeRepos?cursor=0"
+```
+
+## Creating Your Account & Domain Handle
+
+If all those check out you can be pretty confident you now have your very own PDS that you can use for all your things atproto related. However first you'll need to establish an account on the PDS, perform the needful to claim the handle you want, and actually start using that account and identity to do things on the atmosphere.
+
+Lucky for us the pds container contains all the tools you need for the job, specifically this CLI tool called [`goat`](https://github.com/bluesky-social/goat). This is a tool that enables you to do all the necessary bits of administration you'll need to do for creating/updating accounts, creating invite codes, etc. In this case we can keep it pretty simple and get our first account created like so:
+
+```sh
+docker exec pds goat pds admin account create \
+  --handle <domain_you_want_to_use> \
+  --email <email_address_you_want_to_use_for_your_account> \
+  --password <password_for_the_account>
+```
+
+> [!NOTE]
+> One thing I did run into here is that the PDS expects the accounts you create to use a handle that is a subdomain of the domain your pds is located at. For example, if your PDS's domain is pds.open.social then the handles for new accounts need to be something like <username>.pds.open.social. You aren't restricted to this you can update the handle after you create the account to something different. Say change <username>.pds.open.social to <username>.open.social or what have you.
+
+After running that account creation command you should have received a DID for the new account something like this `did:plc:74yqep5iwq2kr2y6n7fn2e4c`. You'll want to keep that handy as you'll need it so that you can take the necessary steps to establish the connection between your chosen handle and this account you've created on your PDS. To do this all you really have to do is go to wherever you manage DNS records for the domain of your chosen handle and create a `TXT` record like the one below:
+
+```txt
+_atproto.<domain_used_for_the_handle_of_the_account_you_created>	TXT	did=did:plc:<your_account_DID>
+```
+
+You can verify that you've created the record correctly and that your handle is now properly resolving to your account by running the following command that again makes use of `goat` in the docker container.
+
+```sh
+docker exec pds goat did resolve-handle --handle <username>.open.social
+```
+
+If that was successful you can be confident you've got an account that is hosted on your PDS and whose data will be stored on your PDS, but whose identity can be used all around the atmosphere. Easiest way to test is just by signing in to your account using [Bluesky](https://bsky.app). You'll need to make sure that you specifically set the hosting provider to your pds using the domain you configured to reach it at.
+
+## Operations: Email Relay & Backups
+
+Couple of things I want to touch on related to this set up. The first is related to allowing my PDS to send emails. The PDS implementation supports this by supplying the necessary information as the following environment variables: `PDS_EMAIL_SMTP_URL`, `PDS_EMAIL_FROM_ADDRESS`, and `PDS_CONTACT_EMAIL_ADDRESS`. The underlying host I deployed my PDS to is already configured to send emails using a relay service so I just set up a local account for the PDS to use for this purpose, but you can obviously choose the solution that best fits your setup and needs. The second is that I opted to mount a TrueNAS dataset to the host machine and use a dead simple shell script and cron job to regularly back up the locations that the PDS is storing data in as specified by these environment variables: `PDS_DATA_DIRECTORY` and `PDS_BLOBSTORE_DISK_LOCATION`. I think this is an important consideration given that you are now 100% in control and thus responsible for the safety and retention of the data you publish to the atmosphere as it all ends up back here on your PDS. I can just hear Uncle Ben now...happy to share more specifics about these two things if you have questions. Reach out to me [here](https://bsky.app/profile/stevanfreeborn.com).
+
+## Migrating an Existing Account
+
+Last thing I want to touch on before wrapping up is on account migration because so far we've only really gone through the necessary steps for establishing a new PDS with a new account, but chances are if you are already on Bluesky you'll want to also migrate that existing account to this new PDS. Originally I thought this was going to be a topic for a whole separate blog post which it definitely is if you want to go the manual route. Lucky for us though the manual route isn't the only option anymore. There is this really wonderful tool called [PDS MOOver](https://pdsmoover.com/) created by a gentleman named [Bailey Townsend](https://bsky.app/profile/pds.dad) that will do all the work for you and if I had to guess with far more consideration for everything that can go wrong than you'll be able to muster yourself. The tool also has support for maintaining a backup of your PDS data for you as well which can help alleviate some of the pressure you may have felt from my comments above regarding being responsible for keeping your data safe. I'm using it myself. Again if you run into any troubles yourself using the tool feel free to reach out to me [here](https://bsky.app/profile/stevanfreeborn.com) and I'll be happy to try and steer you in the right direction.
+
+## Conclusion
+
+Alright I've taken up more than enough of your time today with my atproto excitement and retelling, but I hope I've sparked your interest even a little about what atproto has to offer both users and content creators and maybe even enough to get you to start getting involved with it yourself. I know things can look pretty grim right now for the general state of the internet, but to me atproto presents a really exciting future for what could be and I'd like to try to do my part to help make it so by using it, talking about it, and sharing it with others.