Step by step instructions on how to get up and running with Immich on a Proxmox VM. I'll also set up machine learning to be offloaded to another machine and use Intel Quicksync for video transcoding.

Fresh Install of Immich on a Proxmox VM


Immich is probably the best self hosted and local way to manage, store and view your photos. I've been using it for quite a while now to backup photos from my phone, but in the past I had the NUC doing all the heavy-lifting. In this guide I'm going to pass that off to a better equipped machine and also pass through Quicksync for transcoding videos. I'm also going to set up a dataset in TrueNAS to store all the photos. And of course I'll be doing all of this in Proxmox.

---

### Ubuntu Server Setup

Normally I would prefer to use an LXC instead of a full VM, but Immich suggests using Docker and I don't feel like setting up each service one by one, so that means I'll be doing this in a VM. I really don't like running Docker in an LXC.

Create a new VM in Proxmox, I'm using Ubuntu 22.04, 4 cores, 4GB of ram and 24GB HDD.

Once that's up and running go through the install process. When it asks if you want any packages pre installed, don't select any, just skip through.

Now you should be logged in and in the terminal.

Update the system
```
sudo apt update && sudo apt upgrade
```

The next two steps are optional, but I'll be using Vim as my text editor and Zsh as my shell.
Install some quality-of-life packages
```
sudo apt install neovim zsh
```

Install oh-my-zsh
```
sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
```

---

### Docker

Install Docker Stuff
```
sudo apt install apt-transport-https ca-certificates curl software-properties-common
```

Add GPG key
```
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
```

Add Docker repo to apt
```
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
```

Update
```
sudo apt update
```

Check to see that it's ready to go. If you see something like (Installed), you're good to go.
```
apt-cache policy docker-ce
```

Install
```
sudo apt install docker-ce
```

Check
```
sudo systemctl status docker
```

You should see the docker version.

---

### Docker Compose

Docker is cool, but Docker Compose is a better way of using it. Let's get that going now.

Create a directory for Docker Stuff
```
mkdir -p ~/.docker/cli-plugins/
```

Install Docker Compose
```
curl -SL https://github.com/docker/compose/releases/download/v2.3.3/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose
```

Set permissions
```
chmod +x ~/.docker/cli-plugins/docker-compose
```

Check version
```
sudo docker compose version
```

You should now see the Docker Compose version. If you've used Docker Compose before and you're used to the command being `docker-compose`, just know that they've updated it and the new command is now `docker compose`. Everything else should be the same, but there's just no hyphen (-) anymore.

---

This is where we start to deviate on our own path. My Immich server is going to be storing all the photos on my TrueNAS server, so we need to set up a dataset to hold everything and then link it up with our VM.

---

### TrueNAS

Create a dataset in TrueNAS and set up the permissions as you wish. When developing I tend to set read/write for everyone and only allow specific IP addresses in the NFS rules, but it's up to you.

Set up an NFS share for the dataset you just created and put in the IP address of your Immich VM in the allow list. Or you can just put you local subnet and allow all devices on your network, whatever works for you.

---

Back on the Immich VM.

### NFS

Install nfs-common
```
sudo apt install nfs-common
```

Make shared directory for images. I like to keep all my external/network shares in the `/mnt` folder.
```
sudo mkdir /mnt/immich-nas
```

Obviously replace the next line with the IP address/path of you TrueNAS dataset.
```
sudo mount 192.168.X.X:/mnt/<pool>/<dataset> /mnt/immich-nas
```

Check that it mounted
```
df -h
```

In the Filesystem column you should see the IP address of the NAS and the path of the dataset. The Mounted On column should have `/mnt/immich-nas`

Set the Immich server to mount this folder when it boots up
```
sudo nvim /etc/fstab
```

Add to the bottom of the file
```
192.168.X.X:/mnt/<pool>/<dataset> /mnt/immich-v2 nfs auto,nofail,noatime,nolock,intr,tcp,actimeo=1800 0 0
```

---

### Immich

Now that we're all set up it's finally time to install Immich.

Create a directory for the Docker files
```
mkdir ~/immich-app
```

Navigate to the new directory
```
cd ~/immich-app
```

Download the Docker Compose file
```
wget https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
```

Download the .env file
```
wget -O .env https://github.com/immich-app/immich/releases/latest/download/example.env
```

Download the hardware acceleration file (optional)
```
wget https://github.com/immich-app/immich/releases/latest/download/hwaccel.yml
```

Edit the .env file
- Change `UPLOAD_LOCATION` value to `/mnt/immich-nas`
- Update `DB_PASSWORD` to a random string

---

### Quicksync

Since my Proxmox server is an Intel NUC with Quicksync, I'm going to pass that through to the VM so Immich can use it to transcode. If you aren't going to use Quicksync or don't have it available you can skip this part.

Install some hardware packages
```
sudo apt install --install-recommends linux-generic-hwe-22.04
```

Download/install the Intel Compute-Runtime stuff
Follow the steps on this page.
```
https://github.com/intel/compute-runtime/releases
```

Log in to the Proxmox web ui and add the Intel driver to the VM
- Select the VM
- Go to the Hardware tab
- Click Add > PCI
- Select the Intel graphics driver option
- Stop and restart VM
- SSH back in

Once you're back in the VM, type
`ls -l /dev/dri`

You should see something like `renderD128` listed now.

Go in to the Immich folder with the config files
```
cd ~/immich-app
```

Edit the `hwaccel` file
```
nvim hwaccel.yml
```

Uncomment the Intel lines in `hwaccel.yml` to pass the folder to the container.
```yml
services:  
 hwaccel:  
   devices:  
     - /dev/dri:/dev/dri  # If using Intel QuickSync or VAAPI
```

Enable the `hwaccel` file in the docker-compose file
```yml
immich-microservices:  
   container_name: immich_microservices  
   image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}  
   extends:  
     file: hwaccel.yml  
     service: hwaccel  
   command: [ "start.sh", "microservices" ]
```

---

At this point the Immich server should be up and running and you should be able to log in/create the admin account by going to
```
http://192.168.X.X:2283
```

### Remote Machine Learning

The Intel NUC is a pretty great server, but I'm planning on throwing a bunch of photos in here and I want to use Immich's facial recognition features. This can really bog down the NUC so I'm going to offload this work to another machine I have in the house that I mainly use for AI/ML/LLM stuff. The Following steps will be done on that computer, which is also running Ubuntu 22.04, but the full desktop version with an RTX 3060 Ti graphics card.

This computer didn't already have Docker installed so I followed the steps above for installing Docker/Docker Compose.

Create a folder for the Docker Compose file
```
mkdir -p ~/immich-ml && cd ~/immich-ml
```

Create the compose file
```
touch docker-compose.yml
```

Edit the file
```
nvim docker-compose.yml
```

Add the Following
```yml
version: '3.8'

services:
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    restart: always
    ports:
      - 3003:3003

volumes:
  model-cache:
```
Save, exit and run the container
```
sudo docker compose up
```

You can also add the `-d` flag here, but since it's my first time running this I'm going to omit that so I can see the logs.

Open up the Immich web ui and log in as the administrator if you've already set that up. If not just go to `http://192.168.X.X:2283` and set up the admin account now.

Once logged in as the administrator, click the `Administration` button at the top right and go to `Settings > Machine Learning Settings`

In the URL textfield, put in the IP address of the machine running the `immich-machine-learning` container with the port `3003`.
```
http://192.168.X.X:3003
```

Save the settings.

---

In the past I've run in to issues where the Immich logs take up so much space it fills up the VM storage. So I like to set the Log Level to either Error or Fatal once I'm confident everything is working as it should.

And that's that. We've got Immich up and running with all the machine learning stuff offloaded to a computer that can handle it and Quicksync ready to transcode all those stupid hevc (or whatever) files from people with iPhones. Even if you're the only one using it, I still suggest making yourself a new account that isn't the admin account. Keep the admin account just for admin stuff.













An automated way to backup Home Assistant to a dataset on TrueNAS Scale.

Backing up Home Assistant to TrueNAS Scale


> __UPDATE:__ Since the release of Home Assistant 2023.6 you are able to add network storage directly to Home Assistant and the process for backing up to TrueNAS is ridiculously easy.

Directly below is the new way to do this, but I'm keeping the old way on here for reference in case I ever need to look back at it.

---

In Home Assistant, go to `Settings > System > Storage` and click the Add Network Storage button

Enter a name for the entity/folder. I'm calling mine `trnas_ha`

Select the `Backup` radio button

Enter the IP address of your TrueNAS device

Select `Network File Share (NFS)` radio button

Enter the path to your backup dataset. `/mnt/<Pool>/<dataset>/home-assistant`. (See the Home Assistant section below for more details on this)

Click the Connect button

Go to `Settings > System > Backups`

Click the three dots in the top right corner and select "Change default backup location"

There should be a dropdown and the NAS should be selected

Click the Create Backup button in the bottom corner

Give this backup a name. I'm just doing `trnas_test_backup.

I'm selecting to do a partial backup and just backing up Node-RED. I'm doing this because I just want to make sure it's connected and works.

Click the Create button

It should only take a second because it's a partial backup. I've got my dataset mounted to my computer and when I refresh I can see the backup I just created.

All is good. Now you can just set up a script/automation/whatever to do this at whatever interval you want and it'll always just go to the TrueNAS box.

---

__These are the original instructions that are outdated__

What I'm doing:
- Create a dataset in TrueNAS to store Home Assistant backups
- Share that dataset via NFS in TrueNAS
- Mount the share from TrueNAS to the `/backup` folder on the Home Assistant machine
- Set up an automation in Home Assistant to do regularly scheduled backups
- Backups should be saved to the shared folder, which is a dataset on TrueNAS

If you've looked up how to mount an NFS share in Home Assistant for backing up to a NAS you've probably figured out that it's not as simple as you would think. Yes, you can mount the drive, but it won't stay mounted after reboots and the automatic backup folder can't easily be changed. So that means we have to find a work-around. I think what I've got here is a pretty good solution and seems to be working quite well.

## TrueNAS

Create a new dataset to hold all the backups. I'll just call mine `home-assistant`

I just set the permissions to allow `Other` to read/write, but depending on your situation you might want to create a new user and give the permissions to them. Just make sure to set it to whatever user is active in the container Home Assistant is going to run the script in. I would suggest keeping it as the default `root` with all the permissions checked until it's working and then go back and make changes because it can get confusing when trying to debug if it doesn't work.

Create a new NFS share for the `home-assistant` dataset that was just created.

In the Hosts section, allow the IP address of your Home Assistant device

Save

## Home Assistant

### Create full backup
This is just a good idea before messing with stuff like this.
In Home Assistant go to Settings > System > Backups and click the "+ Create Backup" button at the bottom right of the screen

Give the backup a name and make sure the "Full Backup" radio button is selected
Click the Create button

Once it's done, click on the backup you just created, click the three dots at the bottom of the modal that appears and select Download. Save it somewhere just in case you need it.

Delete all the backups in this folder. This isn't necessary, but since this is going to be automated, I just want to start fresh.

---

This is where things start to get tricky.

The problem(s): Home Assistant runs several containers and each of them have different privileges and also different directories mapped. When you run a `shell_command` action, it runs inside a Home Assistant container that does _not_ have access to the `/backups` folder that the other ones do. This means we have to SSH in to a container that _does_ have access to the backups, mount the NAS drive and copy them over. Let's begin...

- Download and install the [SSH and Web Terminal](https://github.com/hassio-addons/addon-ssh) add-on.
- Download and install the [File Editor](https://github.com/home-assistant/hassio-addons/tree/master/configurator) add-on.

On the main screen for the SSH add-on, flip the switch to add it to the sidebar (optional) and __turn off Protection Mode__ then restart the add-on.

On the Configuration tab of the SSH plugin, take note of the username/password. We'll need these in a minute. I don't remember what the defaults are, but change them if you want. For the following steps, I'm using the username `hassio`.

Open up the Web Terminal and get in to the Docker container that Home Assistant uses for running shell scripts by typing: `docker exec -it homeassistant bash`

After running that you should be in a bash shell, not as pretty as the zsh one, but oh well...

Do an `ls -l /`. You shouldn't see the `/backup` folder in there. This is why we can't just do a simple mount/copy. The container that runs all the shell_commands can't see our backups for whatever reason.

Generate SSH keys for this container to use with the container the SSH add-on is using
`ssh-keygen -t rsa -C "keys-for-backup"`

The `-C "keys-for-backup"` flag is just a comment, this can be whatever you want

Press `Enter` and just skip though all the questions

Once that's done, make sure it generated the keys with: `cat ~/.ssh/id_rsa.pub`

You should see a string like `ssh-rsa AAAAIg2Wh47gx0VEoA...RQmDax93 keys-for-backup`

Copy this whole thing somewhere
> Sometimes copying from this web terminal doesn't work out so you can do:
>
> `mkdir /config/temp`
>
> `cp ~/.ssh/id_rsa.pub /config/temp/id_rsa.pub`
>
> And then open the file up in the File Editor plugin and copy it from there when you need it. 🤷‍♀️

(still in the bash shell here)
SSH in to the SSH/Web Terminal container `<add-on-username>@a0d7b954-ssh`. Since my username for the addon is `hassio`, I'm entering `hassio@a0d7b954-ssh`.

Type `yes` to add this host to the `known_hosts` file and enter your password (the one from the SSH add-on configuration page)

You should now be in a zsh shell and connected to the container that runs the SSH add-on

Do another `ls -l /` and you should see the `/backup` folder

Exit this container with `exit` and we'll be back in the bash shell (the homeassistant Docker container)

Open the configuration tab for the SSH/Web Terminal add-on.

In the field that says `authorized_keys: []` put the `ssh-rsa AA...` string from earlier. This will let us log in to this container without having to enter a password. We need to do this because when we run the `shell_script` action we won't be able to interact with the shell, it'll all just have to work in one go.

Save and restart the add-on

Open the File Editor add-on
It should automatically open up the `configuration.yml` file, but if not, open it.

In that file add:
```yml
shell_command:
  nas_backup: "ssh <add-on-username>@a0d7b954-ssh 'sleep 10 && sudo mkdir ~/backup_to_nas && sudo mount -t nfs4 <your-truenas-ip>:/mnt/<pool>/<dataset> ~/backup_to_nas && sudo cp /backup/$(ls -t /backup | head -n1) ~/backup_to_nas && sleep 5 && sudo umount ~/backup_to_nas && sudo rm -rf ~/backup_to_nas'"
```
__Change the <add-on-username>, <your-truenas-ip>, <pool> and <dataset> sections__

This is where the magic happens. I'll try to break this down:

`ssh hassio@a0d7b954-ssh`
- log in to the SSH/Web Terminal container because that container has access to the backups we need

The commands wrapped in single-quotes after that are commands that will get run as soon as the SSH connection is made. So they'll run on the SSH/Web Terminal container

`sleep 3`
- Basically just wait for three seconds for good measure. No harm in that.

`mkdir ~/temp_nas`
- Create a temporary folder to mount our TrueNAS dataset while we transfer the backup

`sudo mount -t nfs4 192.168.X.X:/mnt/<Pool>/<dataset> ~/temp_nas`
- This will mount our TrueNAS dataset to the `~/temp_nas` directory we just created. The `-t nsf4` tells it to use NFS4, I'm not 100% sure if this is always required, but I couldn't get it to mount without it. Make sure to __change the IP address and path__ to match your TrueNAS IP and the path to your dataset

`cp /backup/$(ls -t /backup | head -n1)`
- `cp` - Copy (`cp <file-to-copy> <where-to-copy>`)
	- In the `<file-to-copy>` spot, we don't know what the name of the file will be because it's random with every backup, but we do know it's in the `/backup` folder. So we do
	- `ls -t` - List the files by time
	- `|` Pass the result to the next command
	- `head -n1` Only get the first result, which would be the most recent one if they're sorted by time.
	- When we put that together it's the same as `cp /backup/<most-recent-file> ~/temp_nas`

`sleep 5`
- Just wait a few more seconds. The `&&` should mean that the next command only runs after the one before it is done, but I like to play it safe

`sudo umount ~/temp_nas`
- Unmount the shared folder so it doesn't get in our way next time. And because it doesn't need to be there if we're not using it anymore

`rm -rf ~/temp_nas`
- remove the temp folder we created

`exit`
- Exit the container.

Save the `configuration.yml` file
Restart Home Assistant. __Don't just reload the yaml__, make sure to do an actual restart.

### Automating the backup
In Home Assistant go to Settings > Automations & Scenes > Scripts
Click 'Add Script'
Give the script a name. Mine is just `Backup to TrueNAS`
Click the 'Add Action' button and select `Home Assistant Supervisor: Create a full backup`
Add another action and select `Shell Command: nas_backup`
Save

Go back and go to the Automations tab
Create a new automation
Set the Trigger to "Time" and set a time you want the backup to happen. I'm choosing `04:00:00` so it runs at 4am every day.
Add an action and choose `script.backup_to_nas`
Save

Now every day at 4:00am there should be a new backup in my TrueNAS dataset.

---

Apparently this is not persisting through Home Assistant updates. I'm pretty sure I had this working at some point, but anyway...

__After an update__:
- Open the web terminal
- `docker exec -it homeassistant bash`
- `ssh-keygen -t rsa -C "keys-for-backup"`
- `mkdir -p /config/temp && cp ~/.ssh/id_rsa.pub /config/temp/id_rsa.pub`
- `ssh hassio@a0d7b954-ssh`
- Type `yes`
- Open File Editor, copy the new key to the SSH plugin config and save/restart the plugin

Having to do this is obviously not ideal, but I'll figure out how I had it working and either update this or make a follow up on it.

Get up and running with a fresh Ubuntu install, set up LoRa training with kohya_ss and use them in Automatic1111's web-ui

Setting Up Ubuntu for Stable Diffusion and LoRa Training


After stepping away from Stable Diffusion for about five months I came back to it only to find a mess of my system. Things were all out of sync, packages were outdated and upgrading made it worse. I had multiple versions of Python conflicting with each other etc. Not to mention all the various venv's peppered throughout it all. Previously, my Stable Diffusion PC was running Manjaro, but this time I decided to go the "easy" route and set everything up from scratch on Ubuntu.

Just gonna get this out now, I'm not a fan of Ubuntu. I use Ubuntu Server for my LXC containers in Proxmox, but not much else. And I've never had any interest in running a DE on it, especially Gnome. Buuuuut those are just my opinions and apparently a lot of people really like it and that's why there's a ton of support/resources for it. So that's what made me decide to go with it for this.

For this setup I'm using:
- AMD Ryzen 5700x
- 1TB SSD
- 64GB DDR4 RAM
- Nvidia GeForce RTX 3060 TI (8GB)

---

I'm not going through the steps for installing Ubuntu, but basically:
- [Download Ubuntu Desktop](https://ubuntu.com/download/desktop)
- Flash it to the SSD with [Balena Etcher](https://etcher.balena.io/#download-etcher)
- Put the SSD in the computer and turn it on
- Go through the setup, connect WiFi etc

---

The steps below begin with a clean install of Ubuntu 22.04.2 LTS

## Update the system

`sudo apt update && sudo apt upgrade`

## Install/Setup Timeshift
This step is optional, but it was good to do because I messed this up several times and being able to restore from a snapshot was very nice.

`sudo apt install timeshift`

Timeshift settings:
-   Type: RSYNC
-   Location: Main SSD
-   Schedule: None    
-   Users: "Include All Files" for `/home/pywkt`
-   Filters: Default
-   Misc: Default

> Create backup with Timeshift

![First backup {500x400}](/images/posts/setting-up-sd-on-ubuntu/001_timeshift.png)

On my first attempt at doing this I tried to restore from a backup and something got super messed up and I had to reinstall Ubuntu. I was thinking it was because I was saving/restoring from a backup that was on the same HDD. Not sure if that's what caused the issue, but just to be safe, after each backup I'm manually sending them to a folder on my NextCloud server.

Restart

## Install Nvidia Drivers
-   Open up the Ubuntu Settings and select the About tab
-   Click Software Updates to open a new window
-   On the Additional Drivers tab, select the one with the highest number
-   Click Apply Changes
-   Restart

Before Nvidia drivers:

![nvidia drivers {800x300}{caption: Notice the Graphics row on the left has the default Linux drivers}](/images/posts/setting-up-sd-on-ubuntu/002_nvidia_driver_setup.png)

After Nvidia drivers:

![nvidia drivers installed {400x320}{caption: Now it has the Nvidia drivers}](/images/posts/setting-up-sd-on-ubuntu/003_after_nvidia_install.png)

> Backup with Timeshift

## Install ZSH and others
Again, optional, but why would we want to bother with Bash when we can ZSH?

`sudo apt install build-essential curl file git`

`sudo apt install git-core curl fonts-powerline`

`sh -c "$(wget https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh -O -)"`

Logout and log back in for the changes to take effect.

> Backup with Timeshift

## Setup Tensorflow

### Install Miniconda

`cd ~/Downloads`

`curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o Miniconda3-latest-Linux-x86_64.sh`

`chmod +x Miniconda3-latest-Linux-x86_64.sh`

`./Miniconda3-latest-Linux-x86_64.sh`

Press Enter a few times to confirm the TOS and default options. Close the terminal and open it back up. If you entered 'yes' for the last question, when you start the terminal you should see `(base)` on your terminal line.

### Setup a new environment for Tensorflow
`conda deactivate`

`conda create --name tf python=3.10`

`conda activate tf`

### Install CUDA and cuDNN
`conda install -c conda-forge cudatoolkit=11.8.0`

`pip install nvidia-cudnn-cu11==8.6.0.163`

### Configure system paths when starting tf environment
`mkdir -p $CONDA_PREFIX/etc/conda/activate.d`

`echo 'CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh`

`echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/:$CUDNN_PATH/lib' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh`

`conda deactivate`

`conda activate tf`

### Install TensorFlow
`pip install --upgrade pip`

`pip install tensorflow==2.12.*`

### Verify CPU
`python3 -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"`

### Verify GPU
`python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"`

To be totally honest, I'm not even sure if this is what I should expect to be seeing, but I'm getting this:

CPU:

![tf cpu {700x400}](/images/posts/setting-up-sd-on-ubuntu/004_tf_cpu_output.png)

GPU:

![tf gpu {700x400}](/images/posts/setting-up-sd-on-ubuntu/005_tf_gpu_output.png)

Based on the fact that all of the messages (except one) have the `I` at the beginning I think this is ok, but the one `W` has me wondering. I'm going to create another backup and try to install tensorRT with pip.

> Backup with TimeShift

## Install TensorRT
`pip install tensorrt`

After running this it seems like it's downloading `nvidia-x-x12 (instead of 11 like I did above). I might have just been able to do this from the start to get everything I needed, but either way, this seems to work.

Running the CPU/GPU commands from above seems to output the same result.

Moving on...

## Setup kohya_ss

### Clone the kohya_ss repository

`mkdir ~/stable_diffusion`

`cd ~/stable_diffusion`

`git clone https://github.com/bmaltais/kohya_ss.git`

`cd kohya_ss`

`sudo apt install python3.10-venv`

`sudo ./setup.sh`

`./gui.sh`

Open a web browser and go to `127.0.0.1:7860` and you should see the gui.

## Prepare training images

### Install Gimp
`sudo apt install gimp`

### Resize images
I have a folder of about 18 images of the person I want to train a LoRa model on. I opened up each one in Gimp, resized each one so that the smallest side was 512px and then cropped them to 512x512 and saved them in a new folder

### Caption images
In the kohya_ss web ui, go to the Utilities tab and then the BLIP Captioning tab inside that. Put in the directory of the images you just resized and click the Caption Images button. It should create one text file for each image in the folder of images we just resized. Go through each text file and replace the subject with the keyword you're going to use for the training. Mine is `jbov` so instead of a file that says:

> a man with glasses and a beard sitting in front of a brick wall

It now says:

> jbov with glasses and a beard sitting in front of a brick wall

### Create folder structure
Set up a folder structure that looks something like this. My subject is `jbov` so make sure to rename that to whatever your subject is called.

```
- /home/pywkt
  - jbov
    - model
    - log
    - image
      - 100_jbov
        - 001.jpg
        - 002.jpg
        - ...
```


Put the cropped/resized images and text files in the `100_jbov` folder.

Download these two files: [LoraBasicSettings](/var/LoraBasicSettings.json) and [LoraLowSettings](/var/LoraLowVRAMSettings.json).

Place them in the `kohya_ss` folder.

__Note:__ These files are just modified versions of the ones in [Aitrepreneur's video](https://www.youtube.com/watch?v=70H03cv57-o) description.

### Configure training
Start the kohya_ss gui and open it up in a browser Click the Dreambooth LoRa tab Expand the Configuration file dropdown Click Open and select one of the two json files from above Click the Load button. (not sure if this is necessary, but I do it anyway) On the Source Model tab, select the model to train against

![kohya model settings {800x120}](/images/posts/setting-up-sd-on-ubuntu/006_kohya_model_settings.png)

Select the Folders tab and set the Image, Output and Logging folders to the ones created earlier.

![kohya folder settings {800x300}](/images/posts/setting-up-sd-on-ubuntu/007_kohya_folder_settings.png)

**Note:** For the Image path, choose the `image` folder, not the actual `100_jbov` folder that has all the images.

Click the Train Model button at the bottom. If everything went as it should, the training should start with no issues.

![kohya success output {500x600}](/images/posts/setting-up-sd-on-ubuntu/008_kohya_terminal_output_success.png)

Once it's done there should be a `jbov.safetensors` file in the `model` folder that we created

> Backup with Timeshift

## Stable Diffusion WebUI

I was debating trying out the [vladmandic fork](https://github.com/vladmandic/automatic "https://github.com/vladmandic/automatic") of automatic1111's web ui, but for now I think I'm just sticking with what I know. So auto1111 it is...

### Set up automatic1111

`git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git`

`cd stable-diffusion-webui`

Copy/paste the `v1-5-pruned-emaonly.ckpt` file in to `stable-diffusion-webui/models/Stable-diffusion`

Open the `webui-user.sh` file, uncomment line 13 and change it to: `export COMMANDLINE_ARGS="--xformers"`

I'll probably be adding more params to that, but for now we'll try this out

**Note:** I'm not sure it matters at all, but I'm still in the `tf` venv.

`./webui.sh`

I ran in to an error where `open_clip` and packages related to that weren't installing correctly/just timing out. To solve this I changed the permissions on my `/tmp` folder and ran the `webui.sh` script again. Once it installed I just switched the permissions back to `root`. The next three steps explain how to solve that.

Allow current user to use it: `sudo chown pywkt:pywkt /tmp` (replace with your username/group)

Start the web-ui/install everything: `./webui.sh`

Set permissions back to root: `sudo chown root:root /tmp`

Once it's all done installing everything, the ui should be available at `127.0.0.1:7860`

> Backup with TimeShift

Everything should be all set up and Automatic1111's web ui should be working with the LoRa models we create with the kohya_ss gui.

_A++++++ Would reccommend_

Setting up Proxmox to use a dataset in TrueNAS for daily backups

Backing up Proxmox LXCs/VMs to TrueNAS Scale


One of the reasons I wanted to set up an actual NAS on my network was because I wasn't able to backup all my Proxmox containers the way I really wanted to. With the space I had allocated in Proxmox I was really only able to have one days worth of backups, with the exception of my "dev" container, which I was keeping about three days worth. My solution to this was to set up a TrueNAS instance and send all the Proxmox backups there.

---

These steps assume you already have Proxmox and TrueNAS set up and running on your local network

## Datasets

> I'm making this dataset at the top level of my pool. It will be the main folder for all the devices I save backups for, but right now I'm just setting it up for Proxmox.

### Creation
Log in to your TrueNAS server and click the "Datasets" option from the menu on the left
- Click the "Add Dataset" button at the top right of the screen
- Give your dataset a name. I'm calling mine `backup`
- Add a comment if you want: `Top level backup dataset`
- Click "Save"

Once that's done, you should see the "backup" folder as a child of your main pool.

- Click the "backup" dataset and add another dataset to that one. We're basically just making folders here. I'm calling this dataset `proxmox-backup` and the comment is just something like `Used for Proxmox backups`
- Click "Save"

Once again, we see this new folder as a child, but this time it's a child of the "backup" folder

You can probably see where I'm going with this. Eventually I'll have other folders like _phones, laptops etc..._ and each folder will have different permissions based on who/what is pushing data to them. And speaking of permissions...

### Permissions

- With the `proxmox-backup` dataset still selected, click the "Edit" button on the Permissions card
- Select all the checkboxes*
- Save

_*_ I feel like the _proper_ way to do this is by setting the permissions based on just the user or just the group, but I couldn't seem to get Proxmox to bring in the share like that. So to make up for that, in the next step we set it up so only the Proxmox IP address can access the share 🤷‍♀️

### Sharing

- Click the "Shares" button in the menu on the left
- On the UNIX (NFS) Shares row, click the "Add" button
- In the Path section, select the folder you just made for Proxmox backups
- Add a description if you want. It doesn't really matter.
- In the Hosts textfield, add the IP address of your Proxmox server `192.168.X.X`. This will make it so only the Proxmox container can write to this dataset
- Save

That's it for the TrueNAS part of it. Now we'll set up Proxmox.

## Configuring Proxmox

- In Proxmox, select your Datacenter and select the "Storage" tab
- Click the "Add" dropdown and select "NFS"
- Give it an ID. Mine will be `truenas-backup`
- Enter the IP address of your TrueNAS server
- Click the dropdown for the Export option and the path to the dataset we created in TrueNAS should be shown. Select that.

Since this is only going to be for backing up Proxmox, I'm going to deselect the default Disk Image option in Content and select `VZDump backup file` instead

Everything else can stay the same.

- Click "Add"

## Configure backups to use the TrueNAS dataset

Still on the Datacenter section
- Select "Backup" from the menu
- Click the "Add" button at the top
- In the Storage dropdown, select the `truenas-backup` we just created

For the schedule I set it to `04:00` so it runs the backup at 4am every night

For Selection Mode I keep it on "Include selected VMs" and select the ones I want to backup

On the "Retention" tab I have it set to Keep Last: 3 and Keep Daily: 3 so it will always have the last three days of backups

- Click the Create button

## Test it out

When you select the new backup task that we just created, the "Run now" button should become enabled. Click that to start the backup.

Go to your TrueNAS Datasets page and select the `proxmox-backup` dataset and refresh the page. If everything worked correctly, the Total Allocation value should now reflect the size of your backups.

Now that it's finished I can see that it wrote 28GiB to the dataset, which means if I'm keeping three days worth of backups, the total size it occupies should be around 84Gib

Now that I've got Prometheus and Grafana all set up to pull in metrics from Proxmox, I figured it might be cool to pull in all the data from the UniFi gear too.

Monitoring My UniFi Network with Prometheus and Grafana


Now that I've got Prometheus and Grafana all set up to pull in metrics from Proxmox, I figured it might be cool to pull in all the data from the UniFi gear too.

These instructions assume you already have Prometheus and Grafana set up and running.  If not, you can read about that setup [here](/blog/20230302-prometheus-and-grafana-on-proxmox).

I'm going to set up the UniFi Poller on the same LXC container I'm running prometheus-pve-exporter on. So this container will just be the one that pulls in all the data from different places. Once that's set up I'll configure my Prometheus container to pull in the new UniFi stuff and Grafana is already set up so it should just be ready to go.

> Before starting,  create a new user in the UniFiOS and give it the View Only role.

---

## Setup UniFi Poller

Log in to the container Prometheus will pull from

### Download and install UniFi Poller

`curl -s https://golift.io/repo.sh | sudo bash -s - unpoller`

### Edit the config file

`nano /etc/unpoller/up.conf`

Set `disable = true` in the InfluxDB section

Change the `url` in the `[unifi]` section to the IP address of UniFiOS `192.168.X.X` and remove the colon and port

Change the `user` to the view-only user that you created earlier

Save and exit

## Prometheus config

### Log in to the Prometheus container and edit the config file

`nano /etc/prometheus/prometheus.yml`

After the last entry in `scrape_configs` add:
```toml
  - job_name: 'unifi'
    static_configs:
      - targets: ["192.168.X.X:9130"] # The IP address of the container running UniPoll
```

Restart Prometheus

`systemctl restart prometheus`

## Grafana

### Install Grafana plugins
Log in to the container running Grafana and install the plugins needed

`grafana-cli plugins install grafana-clock-panel`

`grafana-cli plugins install natel-discrete-panel`

`grafana-cli plugins install grafana-piechart-panel`

Restart Grafana

`systemctl restart grafana-server.service`

### Add UniFi dashboards to Grafana

- Open up Grafana in the browser
- Click the Dashboards option from the left menu and select "+ Import"
- In the textfield labeled "Import via grafana.com"
- Enter in any of the UniPoller Prometheus dashboards listed below

![UnPoller dashboard options](/images/posts/unpoller/dashboards.jpg)

I mainly just look at the Clients one, but it's nice to have the Sites and Client DPI ones too sometimes.

---

References:
- [https://unpoller.com/](https://unpoller.com/)
- [https://grafana.com/grafana/dashboards/11315-unifi-poller-client-insights-prometheus/](https://grafana.com/grafana/dashboards/11315-unifi-poller-client-insights-prometheus/)

A step by step guide on how to set up Prometheus, proxmox-pve-exporter and Grafana on LXC containers and monitor everything in a pretty dashboard.

Setting up Prometheus and Grafana on Proxmox LXCs


What I'm doing: 
- Setting up three LXC containers in my Proxmox node.
  - Prometheus
  - prometheus-pve-exporter
  - Grafana.
- pve-exporter will pull data from the Proxmox server
- Prometheus will pull the data from the pve-exporter container
- Grafana will pull from the Prometheus container and display it nice and pretty.

I assume you're already wondering why I would use three separate containers when I could just do this on one or even two. There's a few reasons, but the main one is that I prefer to have a bunch of small containers each doing one thing rather than a few larger containers each doing several things. Personal preference, but it works for me 🤷‍♀️

---

The following instructions assume you already have a Proxmox server up and running and know how to create LXC containers.

All the containers created in these steps are created with the Ubuntu Server 22.04 (Jellyfin) image.

## Setting up the Prometheus container
2cpu, 2gb ram, 8gb storage. Calling this one 'prometheus'

> The steps in this section are taken directly from [samynitsche.de](https://samynitsche.de/3-monitoring-proxmox-with-prometheus-and-grafana)

### Create a group/user

`groupadd --system prometheus`

`useradd -s /sbin/nologin --system -g prometheus prometheus`

### Create directories
`mkdir /var/lib/prometheus`

`for i in rules rules.d files_sd; do mkdir -p /etc/prometheus/${i}; done`

### Download/install Prometheus
`mkdir -p /tmp/prometheus && cd /tmp/prometheus`

```sh
curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest \
  | grep browser_download_url \
  | grep linux-amd64 \
  | cut -d '"' -f 4 \
  | wget -qi -
```
### Extract files and move to correct directory

`tar xvf prometheus*.tar.gz`

`cd prometheus`

`mv prometheus promtool /usr/local/bin/`

`mv prometheus.yml  /etc/prometheus/prometheus.yml`

`mv consoles/ console_libraries/ /etc/prometheus/`

### Create systemd config

`cd /etc/systemd/system/ && touch prometheus.service`

`nano prometheus.service`

Paste in the following and save
```toml
[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.listen-address=0.0.0.0:9090 \
  --web.external-url=

SyslogIdentifier=prometheus
Restart=always

[Install]
WantedBy=multi-user.target
```

### Set permissions

```sh
for i in rules rules.d files_sd; do chown -R prometheus:prometheus /etc/prometheus/${i}; done
```

```bash
for i in rules rules.d files_sd; do chmod -R 775 /etc/prometheus/${i}; done
```

`chown -R prometheus:prometheus /var/lib/prometheus/`

### Start Prometheus
`systemctl daemon-reload`

`systemctl start prometheus`

`systemctl enable prometheus`

You should now be able to go to `http://<container-ip>:9090` and see the Prometheus instance running

---
## Installing proxmox-pve-exporter

The following steps are a modified version of [this post](https://blog.zwindler.fr/2020/01/06/proxmox-ve-prometheus/).

### Create a new container
1cpu, 1gb ram, 6gb storage. Calling it 'prom-export'

`prometheus-exporter`.

### Install pip3
`apt-get install python3-pip`

`pip3 install prometheus-pve-exporter`

### Make directory for pve_exporter
`mkdir -p /usr/share/pve_exporter/`

### Add Proxmox credentials
`nano /usr/share/pve_exporter/pve_exporter.yml`

Add the following, but change to your Proxmox info
```yml
default:
    user: proxmox_pve_user@pve
    password: secret_password
    verify_ssl: false
```

Save and exit

### Test that it works
`/usr/local/bin/pve_exporter /usr/share/pve_exporter/pve_exporter.yml`

After running that in the console you should see something like:
```bash
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://:9221
Press CTRL+C to quit
```

### Create systemd script

`nano /etc/systemd/system/pve_exporter.service`

Add the following to the file:
```toml
[Unit]
Description=Proxmox VE Prometheus Exporter
After=network.target
Wants=network.target

[Service]
Restart=on-failure
WorkingDirectory=/usr/share/pve_exporter
ExecStart=/usr/local/bin/pve_exporter /usr/share/pve_exporter/pve_exporter.yml 9221 192.168.X.X

[Install]
WantedBy=multi-user.target
```

Replace the IP address in that script with the IP of __this__ container

Save and exit

### Start pve_exporter service
`systemctl daemon-reload`

`systemctl enable pve_exporter`

`systemctl start pve_exporter`

Check that it's running with:
`systemctl status pve_exporter`

## Back on the Prometheus container

### Add the pve_exporter data to Prometheus
`nano /etc/prometheus/prometheus.yml`

Add another `job_name` after the line `- targets: ["localhost:9000"]`
```yml
  - job_name: 'proxmox'
    metrics_path: '/pve'
    params:
        # This is the ip address of the proxmox server
        target: ['192.168.X.X']
    static_configs:
      # This is the container running pve_exporter
      - targets: ["192.168.X.X:9221"]
```

Save and exit

### Restart Prometheus
`systemctl restart prometheus`

You should now be able to go to `http://<prometheus-container-ip>:9090/targets` and see two entries. The first one is there by default, it's the Prometheus container. The second one should be the Proxmox metrics.

---

## Setup Grafana


### Make new container
2cpu, 2gb ram, 8gb storage. Calling it 'grafana'.

### Setup apt for Grafana
`apt-get install apt-transport-https`

`apt-get install software-properties-common wget`

`wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key`

`echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | tee -a /etc/apt/sources.list.d/grafana.list`

### Install Grafana
`apt-get update`

`apt-get install grafana-enterprise`

### Start Grafana
`systemctl daemon-reload`

`systemctl enable grafana-server.service`

`systemctl start grafana-server`

### Check that it's running
`systemctl status grafana-server`

If everything looks good, you should be able to go to `http://<grafana-container-ip>:3000` and see the login screen

Login with username/password `admin` and create a new password.

## Configure Grafana with Prometheus

Once logged in to Grafana:
- Click the gear icon at the bottom left
- Click the "Add data source" button
- Select Prometheus
- In the URL field, add the Prometheus container's IP and port like: `http://192.168.X.X:9090`
- Click "Save and test"

## Set up the pve-exporter dashboard

- Click the Dashboards icon on the left menu and select `+ Import`
- In the textfield that says "Import via grafana.com" enter `10347` and click "Load"
- Select Prometheus as the data source and save

Now you should have all your Proxmox metrics in this Grafana dashboard.

There's a chance that some of the labels may be overlapping and look all messed up. If so, just click the dropdown arrow at the top of that section and click "Edit". Expand the "Options" accordion and change the textfield that says `{{name}}` to `{{id}}`.



Refs:
- https://samynitsche.de/3-monitoring-proxmox-with-prometheus-and-grafana
- https://blog.zwindler.fr/2020/01/06/proxmox-ve-prometheus/ (French)

This site is hosted on Cloudflare Pages, but I have another repo on the same Gitlab account that I want to put on a subdomain of pywkt.com. This turned out to be a really simple task that took me a really long time...

Host a Git Repo on a Subdomain with Cloudflare


This site is hosted on Cloudflare Pages, but I have another repo on the same Gitlab account that I wanted to put on a subdomain of pywkt.com. Since Cloudflare is already managing my DNS for the site, I assumed I would just add a CNAME record for pywkt.com that pointed to the `<repo>.pages.dev` URL that Cloudflare gives me when I host a repo on Pages. This is the correct idea, but I just went about it the wrong way.

---

The following instructions assume you already have a domain in your Cloudflare account. You don't have to purchase the domain through Cloudflare, but Cloudflare should be managing the DNS records for the domain you want to use.

- Log in to Cloudflare
- Click 'Pages' in the left hand menu
- Click the 'Create a project' button
- Select 'Connect to git'
- If your Git account isn't already connected, you will have to do that here. Once it's connected you'll see all the repos on that account.
- Select the repo you want to host on a subdomain
- When it asks for the domain you want to use, enter the domain _with_ the subdomain.

> __The root domain must already be connected to your Cloudflare account.__ For me, I entered something like: `whatever.pywkt.com`
- You'll have to either put in the build commands or have Cloudflare do it for you by choosing the type of site. For me it was a simple Create React App site so I just chose that.
- It'll also ask you to confirm that a CNAME record will be added to your domain's DNS. Confirm that and in a minute or two it should be good to go.

Simple.

The way I was originally trying to do it was through the domain's DNS options. I had my repo hosted already by Cloudflare at a `<repo>.pages.dev` URL from Cloudflare and I was trying to add a CNAME with that as the value, but when I would visit the subdomain on pywkt.com I kept getting Cloudflare timeout errors and couldn't figure out why. Turns out you do it from the start of setting up the repo. And that's that.

__Note:__ I may have the order of setting up build commands and confirming the DNS change mixed up. I just wrote this from memory, but will make a note to update that the next time I do this if it's backwards.

A lot of words on the journey from 43 words per minute to 100+ in about a year...

A Keyboard Story


![My custom keyboard with no name {800x300} {caption: The keyboard I use every day}](/images/title-of-post.jpg)

When I tell people that "keyboards" is a hobby of mine, the next words out of their mouths is usually "What kind of keyboards?". Then I say, "computer keyboards...". After that, their face says it all.

Usually they're a combination of surprised and curious. They more often than not know I'm a developer so they assume it's some insane keyboard required for my job. Like, without this keyboard they've never heard of, I wouldn't be able to do the work I do. Once I tell them that they're just "regular old keyboards" that type the same letters and numbers as every other keyboard they've ever used in their life, the "enthusiasm" dissipates and the only thing they're thinking is _"why?"_.

Then I go on and on about how _cool_ they are. About how they're just like any other keyboard, but they can _"do things"_ and you can program them to _"do things"_. The best part though is when I tell them the price. Something about seeing people's reactions when they hear that there's keyboards that cost more than their new iPhone. Or that there's a whole community of people even interested in keyboards at all. Something as mundane as a computer keyboard.

But I think that's what I love most about them. The fact that something so simple, so mundane, something that gets taken for granted by probably 99% of people every day, can have so many tiny nuances than you would never even think about unless you're privy to this world.

> It was a long road and cost a lot of money to get here, but in my opinion, it was totally worth it.

A few years ago I worked a job where 100% of my work could be done on a single laptop. Everything was done in Wordpress and a browser was really all I needed. February of 2020 that changed and I found myself in a role that involved using actual development skills/tools and the thirteen inch MacBook screen just wasn't going to cut it any more. My partner and I had been traveling quite a bit around that time and I started the new job while we were away for a few months. I had brought a portable monitor with me, so I had the screen part covered, and I had a mouse so I didn't have to use the trackpad on the laptop, but it was a bit awkward typing on the laptop keyboard and looking at the laptop screen and then also looking at the external monitor. I felt like I needed to be sitting further away, and for that to happen I needed a keyboard that wasn't attached to my screen.

Up until this point I didn't care about what keyboard I used. I typed "computer keyboard" in to Amazon and was a click away from having a working keyboard at the door in the morning. But then I saw there were several different types of keyboards I could get. I didn't want one loaded with RGB nonsense, but maybe there was something else I'd like to try. And after a few days of looking around, I settled on something totally different than what I had expected.

---

My first "custom" keyboard was a [Planck](https://drop.com/buy/preonic-mechanical-keyboard?defaultSelectionIds=967853) from Drop.com and I loved it. I really loved how small it was and it was also my first time using an [ortholinear](https://www.reviewgeek.com/70291/what-is-an-ortholinear-keyboard-and-should-you-use-one/) layout. It took about a week or two to get used to that, but eventually I grew to love it. I ordered the purple one, but eventually cerekote'd it white. I also knew absolutely nothing about switches, but I ordered some [Drop Halo (true)](https://drop.com/buy/drop-halo-switch-pack?defaultSelectionIds=965304%2C965305) and some random lube from Amazon, it wasn't Krytox, but looking back now, it really wasn't that bad either.

After using the Planck for a few months and really liking the format, I decided to try out other keyboards with a similar layout, so I got a [Plaid](https://keycapsss.com/diy-kits/90/plaid-gold-keyboard-kit-usb-c-mx/choc?c=17) from keycapsss.com. Basically the same thing as the Planck, but with through-hole components, so it looked cool. I built this one with Gateron Black Ink switches, which are still my favorite.

While using the Plaid as my daily, I really started to notice that over the years I had developed a slurry of bad typing habits. This is probably because I never "learned" to type. I've been using computers my entire life, seriously, when I was a kid my dad sold big chunky Apple computers so we _always_ had one in the house, but I always thought typing, as a skill, was boring. So for _years and years_ I did some mangled version of typing, but on typing tests I could never get over 43wpm and still had to look at the keys. Enter [Colemak](https://en.wikipedia.org/wiki/Colemak).

In 2019 I came across [keybr.com](https://keybr.com) from a search similar to "how to learn touch typing". I really can't speak highly enough of the site. Check it out if you're interested in learning a different layout and/or [touch typing](https://en.wikipedia.org/wiki/Touch_typing). I figured if I was going to learn to type, I may as well try an "efficient" layout rather than bother to correct the jacked-up thing I was already doing. After a few months, a shit-ton of practice, and having to jump between Colemak and Qwerty on a daily basis, I finally felt comfortable enough to go Colemak full time.

Of course the journey doesn't end there, I mean, if I'm loving ortho so much, maybe a split keyboard would be even better since I don't need to look at the keys anymore. So I bought a Nyquist.

It was basically just a split Planck/Plaid. Same layout and everything. While using the Nyquist as my daily I jumped in the group buy for the [Underscore 33](https://p3dstore.com/products/_33-pcb-extras-and-3dp-keyboard-case).

Again, loved it. It took a bit of getting-used-to, but once I got all the double-taps and combos set up it was smooth sailing. I seriously loved this damn keyboard, except for one thing; it wasn't split. I had gotten so used to having my hands at more of a distance while using the Nyquist. It felt like my wrists were cramped at a weird angle when I was typing now. I didn't want to change keyboards, but I also _needed_ a split one.

So, like any great DIY self hosting nut, I set out to design my own. How hard could it be? Day after day I followed the same damn tutorials on how to use KiCad. Studied schematics from other projects. Basically just learning all the things I needed to learn in order to get a working keyboard. Once I was pretty sure I had a PCB design ready I sent it to [JLC PCB](https://jlcpcb.com) and in a few weeks my prototype had arrived. For whatever reason I ordered the sample PCBs before I had both sides done and I just ordered samples of the left side, but to my absolute surprise, it worked! I was so damn excited.

After another few weeks designing the right side, and then a solid three days of non-stop work figuring out how to program the damn thing in QMK, it was done. I finally had the keyboard I wanted.

I originally designed it to be a stacked FR4 setup and used it like that for almost a year, but eventually when I got a 3d printer I designed a super basic case and printed that. The only problem now is that the original PCB I made was made to be screwed in to a stacked build and I don't have screw holes in the plate to attach it to the top of the case. And I didn't leave any room to sandwich it either. So as you could guess, It's time to make another version of the PCB/plate and finish this thing off. But for now, everything works great and I [love](20220906-pihole-udm-pro) it.

In the process of redoing my home network, I ran in to a problem where my Proxmox server wasn't getting an updated IP from the new router/subnet. This was due to the default Proxmox setup using a static IP and the new system couldn't access it...

Update the IP Address on a Proxmox Host


In the process of redoing my network I ran in to a problem where my Proxmox server wasn't getting an updated IP from the new router. Turn out this was happening because during the initial setup when you type in the IP for Proxmox to use, that's the one you're stuck with.

Previously I was using a `192.168.1.0/24` subnet, but now I need to get my Proxmox on something like `192.168.3.0/24` and since I can't access the old subnet it's not possible to SSH in to the Proxmox server and change it.

### Solution:
  - Connect a keyboard/monitor to the device hosting the Proxmox node and login as `root`
  - Edit the interfaces file: `nano /etc/network/interfaces`
  - Change the old IP address to whatever new one you want to use
  - Save and exit: `Ctrl + o`, `Ctrl + x`
  - Edit the hosts file: `nano /etc/hosts`
  - Change the old IP address to the same new one
  - Save and exit: `Ctrl + o`, `Ctrl + x`
  - Reload the network interfaces file: `ifreload -a`
  - Shutdown the device: `shutdown`

You can also reboot the system in the last step, but I had to move my server back to the rack in another part of the house.

Reference: [proxmox support forum](https://forum.proxmox.com/threads/help-with-changing-ip-address-reinstalling.55421)

An (unsuccessful) attempt to get WireGuard running directly on the UDM Pro.

Setting Up WireGuard on a UDM Pro


__These instructions are incomplete.__

> This was an attempt from the first day I set up the UDM and knew much less about WireGuard than I do now. From what I remember, I was 90% of the way there with this, but eventually decided to plug the pfSense box back in upstream and continue to use that as a whole network firewall/vpn. I'm just publishing these notes here in case it helps anyone trying to do something similar.

Once getting through the initial setup, we need to log in to the UDM via ssh.

The UDM uses an ssh-rsa key rather than whatever the usual SSH key is.

### Generate an rsa key
```bash
ssh-keygen -t rsa
```

### Copy the public key
```bash
cat ~/.ssh/id_rsa.pub
```

### Go in the network settings on the UDM and add that key to the SSH section at the bottom
`(Network > System > Network Device SSH Authentication)`

### Remove/comment out any existing 192.168.1.1 keys
```bash
nano ~/.ssh/known_hosts
```

### Update your ssh config
```bash
nano ~/.ssh/config
```

```bash
Host hostname_of_machine
HostName hostname_or_ip_address
User root
IdentityFile ~/.ssh/file.pem
IdentitiesOnly yes
PubkeyAcceptedAlgorithms +ssh-rsa
HostkeyAlgorithms +ssh-rsa
```

Now you should be able to SSH in to the UDM with an effin' bash shell. Here we go...

### Install the split-vpn package
```bash
curl -LSsf https://raw.githubusercontent.com/peacey/split-vpn/main/vpn/install-split-vpn.sh | sh
```

### Install WireGuard

Check for latest version [Here](https://github.com/tusc/wireguard-kmod/releases)

### Download WireGuard
```bash
curl -LJo wireguard-kmod.tar.Z https://github.com/tusc/wireguard-kmod/releases/download/v06-01-22/wireguard-kmod-06-01-22.tar.Z
```

### Extract tar file
```bash
tar -C /mnt/data -xvzf wireguard-kmod.tar.Z
```

### Make the script executable and run it
```bash
cd /mnt/data/wireguard
```

```bash
chmod +x setup_wireguard.sh
```

```bash
./setup_wireguard.sh
```

### Check that it installed
```bash
dmesg | grep wireguard
```

You should see something like this
```bash
[ 2758.032677] wireguard: WireGuard 1.0.20210124 loaded. See www.wireguard.com for information.
[ 2758.032680] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
```




Not sure if this will work because the guide says to do it on the UDM Pro SE, but let's try...
`curl -Lo /etc/systemd/system/setup-wireguard.service https://raw.githubusercontent.com/tusc/wireguard-kmod/main/src/boot/setup-wireguard.service`

Looks like the non-SE version doesn't have `systemctl`. Note to self: Look in to starting wireguard on boot...

### Generate wireguard keys
`wg genkey | tee privatekey | wg pubkey > publickey`


### Set permissions
`chmod 600 privatekey`

References:
  - https://di-marco.net/blog/it/2022-02-02-wireguad_and_split_vpn_on_unifi_dream_machine_pro_se/
  - https://github.com/peacey/split-vpn
  - https://github.com/tusc/wireguard-kmod/releases
  - https://www.hostifi.com/blog/wireguard-on-a-unifi-dream-machine-pro
  - https://www.nodinrogers.com/post/2022-03-15-wireguard-vpn-on-ubiquity-udm-pro/
  - https://florianmuller.com/setup-a-wireguard-vpn-on-unifi-dream-machine-udm-udm-pro-and-use-macos-as-a-client
  - http://dl-origin.ubnt.com/qsg/UDM-Pro/UDM-Pro_EN.html
  - https://www.ui.com/quick-start/udm-pro

Privacy, Linux, home automation and gardening are my main interests these days. I really just needed a place to put some things, so here we are.

Posts matching "monitoring":

Monitoring My UniFi Network with Prometheus and Grafana

Setting up Prometheus and Grafana on Proxmox LXCs