Tag: vim

How to set up Git to version control your Home Assistant configuration

Congratulations on making it this far!

You now have an installation of Hass.io running on your Raspberry Pi. This is the crucial point where you want to begin treating your installation with more care and professionalism.

There is a phrase in the DevOps world that applies here. “Treat your servers like cattle, not pets”. This means that you want your server (i.e. Raspberry Pi) to be homogeneous and easily replaceable (like cattle). You don’t want your server to be hand-built and hand-maintained (like pets).

This means that you should be able to throw away your server (i.e. flash it with a clean install) and use Infrastructure as Code (IaS) to configure it.

In the cloud development world, we would provision everything, including the server itself with code. In our installation, we have an actual piece of hardware (i.e. Raspberry Pi) and so we can’t do a complete IaS implementation.

However, we can version control our Hass.io configuration. Hass.io runs on a series of YAML files. These are simple JSON documents that declarative describe what they system should do. Hass.io determines how to do it based upon what is in the YAML files.

Hass.io does have a built-in way to “backup” your configuration; snapshots. On the Hass.io page, under the “SNAPSHOTS” tab, there is a button to take a snapshot. The downside of this is that it is still stored locally. You would have to then setup a system to copy this snapshot somewhere else. Therefore, I do not recommend this as the primary way you back up your configuration.

The best way, in my humble opinion, is to use Git & Github to back up your configuration files. Git is the de-facto standard for source control management and Github is a free, online repository for backing up, contributing and sharing out source code with the world.

Note that this is a rehash of the excellent instructions on the Home Assistant website.

Be warned, however, Git is not a simple system to learn and there are a huge number of options. However, since you will be doing very little multi-user editing of the configuration files, you are unlikely to run into the big issues with understanding git’s branching, merging, rebasing, etc. tools. A few simple commands should be sufficient to effectively use Git.

Setting up Github

First, we need to set up a Github account so we can store our configuration files. Navigate to the Github signup page and create an account.

Once you have an account, click on the New Repository button to create a new repo to store your configuration files.

Give it a meaningful name (example: homeassistant-config). The standard naming convention for Git repos is all lowercase letters separated by a dash. This is not required, but will make typing it easier.

Choose Public or Private. This is really up to you. If you want to make it easier to share your configuration with others, you can make it Public. If you don’t want to do this, you can make it Private.

Warning!!! Because you will be uploading your configuration to a “public” site, you must be careful not to expose secrets. We will go into this later, but setting the repo to “Private” is not enough to protect your secrets (passwords, IP addresses, etc.). Hass.io has built-in ways to deal with these.

Do not check the “Initialize this repository with a README” checkbox. Since we will already have a repo set up on our local machine, we want to upload it to a clean Github repo.

Do not add a .gitignore or license file yet. We want a clean, empty repo to start with and we will have a custom .gitignore file anyway.

Initialize the Git repo

Now that we can login to our Raspberry Pi and see our configuration files, we need to upload them to Git.

  • Launch PuTTY and connect to your Raspberry Pi (if you need a refresher, please see my previous blog post).
  • Now we are in the directory where the Hass.io configuration files are located inside the Docker image (if you are in the “home” directory for the Raspberry Pi, either see my previous blog post).
  • Make sure git is already installed (or install if needed)
which git
/usr/bin/git       <-- already installed
                   <-- not installed (will not return anything)

sudo apt-get update         <-- update your packages
sudo apt-get install git    <-- install git if needed

You will now need to create a .gitignore file. This is the most important file, both to prevent publishing important/sensitive data (IP addresses, passwords, etc) and to reduce what you upload to Github to a minimum.

whoami         <-- which user are you logged in as
root

pwd
/config         <-- make sure you are in this directory

vim .gitignore

Here is a copy of the excellent .gitignore file from the Home Assistant website. Feel free to customize as needed.

# Example .gitignore file for your config dir.
# A * ensures that everything will be ignored.
*

# You can whitelist files/folders with !, these will not be ignored.
!*.yaml
!.gitignore
!*.md

# Ignore folders.
.storage
.cloud
.google.token

# Ensure these YAML files are ignored, otherwise your secret data/credentials will leak.
ip_bans.yaml
secrets.yaml
known_devices.yaml

Now we need to “initialize” your git repository. This means we are going to tell git that this directory and all of its contents should be tracked as version controlled files (excluding the files/directories mentioned in the .gitignore file, of course)

git init       <-- initialize as a version controlled directory

git status     <-- see what files have been modified
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   configuration.yaml
        modified:   groups.yaml
        modified:   scene.yaml
        modified:   sensor.yaml
        modified:   switch.yaml

You are now seeing a listing of all the files and directories that git sees that have not been committed yet (your list will be different depending on what was set up in the directory).

We need to set some configuration options so git knows how to annotate our changes

git config user.email "dwight.k.schrute@dunder-mifflin.com" 
git config user.name "dwight"   

We need to “add” all the files and directories to the changeset so git saves their current state.

git add *

Now commit your changes so git will save them.

git commit -m "Initial checkin"

Excellent, now you have an initial git repo with a known setup that you can always refer to (or revert to if you break something).

Generate SSH key to connect between Raspberry Pi and Github

Now we need to “push” our changes to a central server to back them up. Git is a distributed version control system. This means it tracks and saves changes locally, but does not “upload” them somewhere else unless you tell it to. This was originally designed to allow lots of people to work independently (even when disconnected) and make it easy to upload and share code when network connectivity was restored.

The most secure way to connect to Github is via SSH key. Here is the link to the help documents on the Github site. We need to generate an SSH key locally, store it in the SSH agent on the Raspberry Pi and then upload it to Github.

Note: we have to generate and store the SSH key in the /config directory inside the Docker container. If it is stored anywhere else (like the default location, ~/.ssh/id_rsa, it will get deleted the next time the container is restarted.

  • Make sure you are in the /config directory
pwd
/config
  • Make a new directory to store the key, generate the ssh key and save it to disk. Save the file to .ssh/id_rsa. Enter a passphrase if you want to (you will be required to enter it each time unless you also set up ssh-agent, I’m not doing this as I feel it is overkill at this time).
mkdir .ssh       <- create a new "hidden" directory to store the key in

ssh-keygen -t rsa -b 4096 -C "dwight.k.schrute@dunder-mifflin.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): .ssh/id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in .ssh/id_rsa.
Your public key has been saved in .ssh/id_rsa.pub.

Now we need to upload the SSH key to Github so we can use it to authenticate our git repo pushes.You need to copy the public key from your local Raspberry Pi to Github (actual key redacted). If you are using Putty, you can just highlight the text starting with ssh-rsa and ending with your email address and it will be copied to the clipboard.

cat .ssh/id_rsa.pub
ssh-rsa .......................................................................................................................................................................................................................................== dwight.k.schrute@dunder-mifflin.com
  • In your Github.com account you created earlier, click on your profile picture in the upper-right hand corner and select Settings.
  • Select SSH and GPG keys
  • Click on New SSH key
  • Enter a Title: Home Assistant (or whatever will remind you what this key is used for)
  • Enter the Key: ssh-rsa…
  • Click Add SSH key
  • Enter your Github.com password to confirm

Push from Raspberry Pi to Github

Now you can “push” your configuration files from your Docker container to Github so they are backed up.

  • Navigate to your Github repo (click on the left-hand icon and select the repo you created earlier)
  • Click on the Clone or download button
  • Click on the Use SSH hyperlink in the right-hand corner if it is not already selected (the title of the dialog should say ‘Clone with SSH’)
  • Copy the url (git@github.com:username/repoName.git)
  • Set the remote repo where git will push your changes
git remote set-url origin git@github.com:dw/homeassistant-config.git
  • If you try to push now, you will get a permission denied error (-u says to use a new upstream repo for this branch, origin is the name of this upstream repo in git, master is the branch we are on). This is because you have stored your SSH key in a non-standard location. We need to tell git where to find the SSH key.
git push -u origin master
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
  • Git allows you to store configuration files locally, inside the repo itself. This is especially useful for our Docker container since we don’t want these changes to be lost when the container restarts. We need to tell Git where to find our SSH file
git config core.sshCommand "ssh -i /usr/share/hassio/homeassistant/.ssh/id_rsa -F /dev/null"

git push -u origin master           <-- push your changes to Github.com
Enumerating objects: 19, done.
Counting objects: 100% (18/18), done.
Delta compression using up to 4 threads
Compressing objects: 100% (11/11), done.
Writing objects: 100% (12/12), 1.26 KiB | 143.00 KiB/s, done.
Total 12 (delta 8), reused 0 (delta 0)
remote: Resolving deltas: 100% (8/8), completed with 5 local objects.

To github.com:dw/homeassistant-config.git
 * [new branch]      master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.

Hurray! You have successfully uploaded your initial configuration to Github!

How to set up the Hass.io Docker image for productivity

We need to add a few package to the default Docker image that hosts our Hass.io configuration files.

Docker only installs the packages specified in the Dockerfile that defines the image. This helps manage dependencies and keeps the image as small as possible.

However, we need some additional tools installed so we can easily modify the configuration files and version control them.

Specifically, we need vim & git. Sorry emacs fans, you are wrong here :). Vim is derived from the oldest text editors on the UNIX system and has been available on every Linux system since the beginning. Learning vim is a vital skill for any developer. Vim’s ability to chain together commands is incredibly powerful and makes editing very efficient for a power user.

Nowadays, everyone is using IDEs like VS Code, Visual Studio, IntelliJ, or even good old Notepad++. I will admit, I am mostly using VS Code both at work and at home. However, the first plugin I install is the VIM key bindings. The vim way of doing things is just too powerful and productive to give up (even if the GUI of VS Code is very helpful, it is not as powerful).

Since we will be SSH-ing in to the Raspberry Pi and it has a relatively low power processor, using vim is the most efficient and fastest way to get our work done. No shame if you want to RDP into your Raspberry Pi and install an IDE (or use SAMBA), but I prefer simple and fast.

  • Launch PuTTY and connect to your Raspberry Pi (if you need a refresher, please see my previous blog post).
  • Notice we are in the home directory of the “pi” user
pwd
/home/pi        <-- this is not a command, this is the expected output
  • We need to “login” to the Docker container that hosts Hass.io.
    • This command will sudo (elevate our privileges), docker (run a Docker command), exec (run a command in a Docker container), -it (start an interactive session), homeassistant (create the session on the homeassistant container, /bin/bash (open a Bash shell).
sudo docker exec -it homeassistant /bin/bash

Bash is the most popular Linux shell (command interpreter) and is another vital part of your professional software development toolkit.

After starting the Bash shell, we can see that we are in the “config” directory under “root” inside the container.

pwd
/config

If you “list” the directory, you can see the various configuration files Hass.io uses.

ls -aw 1
.
..
.august.conf
august.yaml
automations.yaml
binary_sensor.yaml
camera.yaml
climate.yaml
.cloud
configuration.yaml
customize.yaml
deps
device_tracker.yaml
.git
.gitignore
glances
groups.yaml
.HA_VERSION
home-assistant.log
home-assistant_v2.db
home-panel-config.json
home-panel.db
http.yaml
.ios.conf
ios.yaml
known_devices.yaml
light.yaml
logger.yaml
neato.yaml
notify.yaml
options.xml
OZW_Log.txt
panel_iframe.yaml
pyozw.sqlite
.ring_cache.pickle
ring.yaml
scene.yaml
scripts.yaml
secrets.yaml
sensor.yaml
.storage
switch.yaml
tts
ui-lovelace.yaml
.uuid
zwave.yaml
zwcfg_0xd53a21b8.xml
zwscene.xml

These are the YAML files that control your Hass.io installation. As you can see, there are lots of files (mine will have more than yours if you are just starting out).

However, if you tried to run git or vim now, you will get an error.

git
bash: git: command not found

vim
bash: vim: command not found

Again, this is because there is nothing installed in the Docker container that was not in the Dockerfile originally. So I don’t have to modify the default Dockerfile, let’s add the needed packages manually each time I login.

apk add git
apk add vim

Apk is a package management system used by our running container. We need to use the add command to install the two packages we need (technically, we don’t need vim since vi is installed by default, but that would just be barbaric :)).

Running these 3 commands each time we login to our Raspberry Pi to edit the configuration will get tedious, so let’s cause it to happen whenever we login. Exit the bash shell you are currently in (make sure you are in the home directory for the pi user).

exit
exit       <-- this is not a command, this is the expected output

pwd
/home/pi   <-- this is not a command, this is the expected output
  • Use vim to create a new file in the home directory of the pi user.
    • A .bash_profile is a “hidden” file that is executed whenever a new bash shell is instantiated (like when you login to the Raspberry Pi).
vim .bash_profile

Vim has 2 modes of operation (command & insert). This is the hardest concept to learn since this is the complete opposite of most of the WYSIWYG editors you are used to using in Windows (or emacs). Vim command mode is for moving around the editor window and manipulating text. Vim insert mode is for actually typing in text.

  • Make sure you are in insert mode (press i if you are not sure, it will say — INSERT — at the bottom of the screen). Enter the following text.
sudo docker exec -it homeassistant /bin/bash
  • Press Esc, then :wq. Congratulations! You have successfully gotten over the most asked question on StackOverflow; you have successfully exited vim.

Now close PuTTY and reopen. You will see that you are no longer logged into the default pi user anymore. You have been logged into the Hass.io Docker container.

Now we need to ensure vim and git are installed whenever you login (in a future article, I will figure out how to actually modify the Dockerfile and install these packages by default, but I’m not sure how to make sure I am always up to date with the Hass.io image, so it will have to wait).

  • Create a .bashrc file in the root home directory (it will get run whenever you login to the container and install the needed packages).
vi ~/.bashrc           <-- open vi (not vim, not installed yet)

apk add git            <-- install git
apk add vim            <-- install vim

:wq                    <-- save and quit vi

Now close PuTTY and reopen. You will see that you are no longer logged into the default pi user anymore. You have been logged into the Hass.io Docker container and the 2 most important tools have been installed for you (which is the command to see where an executable is located on the filesystem).

pwd
/config

which git
/usr/bin/git

which vim
/usr/bin/vim