From 2a067a2fe480443fe103a17301accc40d16fa11d Mon Sep 17 00:00:00 2001 From: Thomas Maurice Date: Wed, 11 Jan 2023 11:06:32 +0100 Subject: [PATCH] More content and playbooks * more readme.md content * imported basic roles * commented said basic roles * trimmed down playbooks to be understandable * update requirements.txt deps --- Makefile | 2 + README.md | 89 ++++++++++++++ ansible.cfg | 10 ++ base.yml | 9 ++ docker.yml | 4 + group_vars/all.yml | 14 +++ inventory | 10 ++ reboot.yml | 6 + requirements.txt | 10 ++ requirements.yml | 7 ++ roles/base/tasks/main.yml | 121 +++++++++++++++++++ roles/base/templates/hostname.j2 | 1 + roles/base/templates/hosts.j2 | 3 + roles/base/templates/motd.j2 | 9 ++ roles/ntp/files/ntp.conf | 60 +++++++++ roles/ntp/handlers/main.yml | 8 ++ roles/ntp/tasks/main.yml | 24 ++++ roles/root_user/tasks/main.yml | 28 +++++ roles/root_user/templates/authorized_keys.j2 | 4 + roles/vim/files/vimrc | 51 ++++++++ roles/vim/tasks/main.yml | 12 ++ upgrade.yml | 9 ++ 22 files changed, 491 insertions(+) create mode 100644 Makefile create mode 100644 README.md create mode 100644 ansible.cfg create mode 100644 base.yml create mode 100644 docker.yml create mode 100644 group_vars/all.yml create mode 100644 inventory create mode 100644 reboot.yml create mode 100644 requirements.txt create mode 100644 requirements.yml create mode 100644 roles/base/tasks/main.yml create mode 100644 roles/base/templates/hostname.j2 create mode 100644 roles/base/templates/hosts.j2 create mode 100644 roles/base/templates/motd.j2 create mode 100644 roles/ntp/files/ntp.conf create mode 100644 roles/ntp/handlers/main.yml create mode 100644 roles/ntp/tasks/main.yml create mode 100644 roles/root_user/tasks/main.yml create mode 100644 roles/root_user/templates/authorized_keys.j2 create mode 100644 roles/vim/files/vimrc create mode 100644 roles/vim/tasks/main.yml create mode 100644 upgrade.yml diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..92b9595 --- /dev/null +++ b/Makefile @@ -0,0 +1,2 @@ +install: + ansible-galaxy install -r requirements.yml diff --git a/README.md b/README.md new file mode 100644 index 0000000..1fd29d7 --- /dev/null +++ b/README.md @@ -0,0 +1,89 @@ +# ansible-polytech-2023 + +This is the repo that will serve as the support for the ASR practical work session of january. You should probably work in this directory and add code to it, it is going to be easier than creating a new ansible repository from scratch. You are also welcome to commit to this repository to checkpoint your work, as well as push it into any repository that polytech gives you access to. + +## Setup + +You will need to install a few things to get started, buckle up. + +### The virtualenv + +Since Ansible is written in python and we don't want to install it in the system, you will need to create a virtual environment. These are used to have your python stuff installed, without making them available system-wide, we are doing this to avoid polluting your lab machine with things that won't be used after today. + +To create the virtualenv you need to run the following: + +``` +$ python3 -m venv ~/.ansible-venv +# then you want to "activate" the venv, you will need to do this for every new term you open +$ . ~/.ansible-venv/bin/activate +``` + +:warning: you need to run `~/.ansible-venv/bin/activate` every time you want to open a new terminal and use ansible in it, otherwise it just won't work because the ansible binary won't be found. + +### Install ansible + +Install ansible via pip _after entering the venv_ + +``` +$ pip install -r requirements.txt +``` + +At this point you should have ansible installed. + +### Install the docker role + +Install the docker role using ansible galaxy (ansible galaxy is a sort of package manager for ansible). + +``` +$ ansible-galaxy install -r requirements.yml +``` + +At this point you should be good to go! + +### Generate an SSH key if you don't have one already + +You'll need an SSH key if you don't have one already + +If `ssh-keygen` complains about the key already existing, just reuse the existing key in case someone else needs it. + +``` +$ ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 -P '' +``` + +:warning: In real life, don't use `-P ''` because it creates your SSH key without a passphrase, it is ok for this lab, not for real life. + +Lastly, look into `group_vars/all.yml`, go at the end of the file and add the created _public_ key in the root_user.default_root_keys (from ~/.ssh/id_ed25519.pub, or any other keys you created before hand). When this is done, add it to the `~/.ssh/authorized_keys` in the `root` home folder on every one of your virtual machines. + +### One more thing, update your inventory + +You can now update your `inventory` file by modifying it with your new values (hostnames and ip addresses for the machines you'll be working with). + +### Read the roles to understand how everything works ! + +Ansible runs `playbooks`, which are collections of `roles` that in turn are a collection of `tasks`. `tasks` are instructions like "install this package", "copy this file", "create this directory", "install this service", "create this container" and so on and so forth. + +I have very much documented the example roles in `./roles` and I would _greatly_ encourage you to read them to understand how to do basic stuff in ansible such as copying a file, starting a service and so on. If you do not do that, you will be lost and won't understand anything that is coming at you. + +### Check everything works properly + +You should now be able to actually `run` ansible to execute the `base.yml` playbook. + +``` +$ ansible-playbook -vi inventory -l all base.yml +``` + +The `-i` flag specifies the inventory file to use, the `-l` file limits which hosts it applies to, either by hostname or group name, here we apply it to all the hosts. + +While you are at it I would recommend you install docker as well using the `docker.yml` playbook + +Good, now you are good to go ! + +## Install a database server + +To deploy Synapse and Mastodon, you need to deploy a database server. We are going to use Postgres in this lab. You will for this need to use the `community.postgres_db` module for this. The `community.*` modules are modules written by the community and available to everyone, you will encounter similar modules when you will want to start deploying docker container! + +For more details to do this, I refer you to [this link](https://stribny.name/blog/ansible-postgresql/) which will walk you through how to deal with this. + +## Create a certificate for our deployments + +We need to use a certificate to secure HTTPS communication, both the Mastodon and Matrix protocols require it. This can be done manually for the moment and automated later, as it is not super straightforward. I refer you to the [documentation on certbot + nginx](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04) I would recommend that you create a `mastodon.` and a `matrix.` certificate, then back them up somewhere safe as Letsencrypt has pretty aggressive rate limiting with regard to certificate creations. diff --git a/ansible.cfg b/ansible.cfg new file mode 100644 index 0000000..fadec6a --- /dev/null +++ b/ansible.cfg @@ -0,0 +1,10 @@ +[defaults] +remote_user = root +remote_port = 22 +host_key_checking = False +#hash_behaviour = merge +pipelining=True +fact_path = facts.d +gathering = implicit +gather_subset = all +inject_facts_as_vars = True diff --git a/base.yml b/base.yml new file mode 100644 index 0000000..bf23827 --- /dev/null +++ b/base.yml @@ -0,0 +1,9 @@ +--- +- hosts: all + roles: + - role: root_user + tags: root_user + - role: base + tags: base + - role: vim + tags: vim diff --git a/docker.yml b/docker.yml new file mode 100644 index 0000000..debda70 --- /dev/null +++ b/docker.yml @@ -0,0 +1,4 @@ +--- +- hosts: all + roles: + - geerlingguy.docker diff --git a/group_vars/all.yml b/group_vars/all.yml new file mode 100644 index 0000000..9947b69 --- /dev/null +++ b/group_vars/all.yml @@ -0,0 +1,14 @@ +--- +# Standard variables +os: "{{ ansible_system|lower }}" +arch: "{% if ansible_architecture == 'aarch64' %}arm64{% elif ansible_architecture == 'amd64' or ansible_architecture == 'x86_64' %}amd64{% elif ansible_architecture == 'armhf' %}armhf{% else %}{{ ansible_architecture }}{% endif %}" + +# Docker stuff +docker_apt_ignore_key_error: false +docker_install_compose: true +docker_apt_arch: "{{ arch }}" + +root_user: + default_root_keys: + - name: thomas-ed25519 + key: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPrw78OSJACq5MFXHrhAr2bPpnTNxwLE85mzij8gKmCs thomas@thonkpad diff --git a/inventory b/inventory new file mode 100644 index 0000000..32f70fe --- /dev/null +++ b/inventory @@ -0,0 +1,10 @@ +[all] +# Replace this with your hostname and IP address +your-db-host.local ansible_ssh_host="1.2.3.4" +your-docker-host.local ansible_ssh_host="5.6.7.8" + +[db] +your-db-host.local ansible_ssh_host="1.2.3.4" + +[docker] +your-docker-host.local ansible_ssh_host="5.6.7.8" diff --git a/reboot.yml b/reboot.yml new file mode 100644 index 0000000..bd59e96 --- /dev/null +++ b/reboot.yml @@ -0,0 +1,6 @@ +--- +- hosts: all + tasks: + - name: Reboot + reboot: + reboot_timeout: 3600 \ No newline at end of file diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..0d48dfc --- /dev/null +++ b/requirements.txt @@ -0,0 +1,10 @@ +ansible==7.1.0 +ansible-core==2.14.1 +cffi==1.15.1 +cryptography==39.0.0 +Jinja2==3.1.2 +MarkupSafe==2.1.1 +packaging==23.0 +pycparser==2.21 +PyYAML==6.0 +resolvelib==0.8.1 diff --git a/requirements.yml b/requirements.yml new file mode 100644 index 0000000..95b90a7 --- /dev/null +++ b/requirements.yml @@ -0,0 +1,7 @@ +--- +roles: + - src: geerlingguy.docker + version: 4.2.3 +collections: + - name: community.docker + version: 2.6.0 diff --git a/roles/base/tasks/main.yml b/roles/base/tasks/main.yml new file mode 100644 index 0000000..b0705b4 --- /dev/null +++ b/roles/base/tasks/main.yml @@ -0,0 +1,121 @@ +--- +# welcome to tbe base.yml role. As you can see, this is a collection of +# tasks, which in turn is basically a structure like this one +# name: "user friendly name of your task" +# a_module_name: +# a_module_parameter: value +# another_module_parameter: value2 + +# this installs packages, here we use the `apt` module +# because we are on a debian-based distribution. If we were +# using something like Fedora for instance, we would have used +# the `dnf` module. Then just pass as arguments the names of +# the packages you want to install just as you would pass them +# to `apt install`. +- name: "Install wanted packages" + apt: + name: + - apt-transport-https + - arptables + - bash-completion + - bridge-utils + - ca-certificates + - cmake + - coreutils + - curl + - dnsutils + - ebtables + - file + - gawk + - git + - glusterfs-client + - glusterfs-server + - gnupg + - htop + - iftop + - ifstat + - iputils-ping + - iptables + - iproute2 + - jq + - libsqlite3-dev + - libffi-dev + - libpython3-dev + - libssl-dev + - locales-all + - lsb-release + - lsof + - lvm2 + - mdadm + - minicom + - mtr-tiny + - netcat + - net-tools + - ntp + - open-iscsi + - openssl + - p7zip-full + - pwgen + - sqlite3 + - strace + - sudo + - sysstat + - telnet + - tcpdump + - tmux + - uuid-runtime + - unzip + - vim-nox + - wget + - wipe + - zip + # state: latest will update the package everytime the + # role is ran against a host + state: latest + # Update the cache before trying to update ? + # You generally want this because it's not updating itself + update_cache: true + # force update it if it's been updated for longer than an hour + cache_valid_time: 3600 + register: apt_res + # if running apt fails, retry to do it up to 5 times then give up and cry in the corner + retries: 5 + until: apt_res is success +# Same as above, except you *remove* packages instead of installing them +- name: "Remove unanted packages" + apt: + name: + - ntpdate + # note the value of `state` here + state: absent +# This renders a template, the `inventory_hostname` is a variable +# that is golbally available. It will map to the hostname you +# assigned your host in the `inventory` file +- name: "Hard set hostname" + template: + src: hostname.j2 + dest: /etc/hostname + owner: root + group: root + # you need to put it in string mode, and you need to + # have a leading `0`, otherwise ansible is going to interpret it weird + # more info https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html#parameter-mode + mode: "0644" +- name: "Setup motd" + template: + src: motd.j2 + dest: /etc/motd + owner: root + group: root + mode: "0644" +- name: "Setup hosts" + template: + src: hosts.j2 + dest: /etc/hosts + owner: root + group: root + mode: "0644" +# Note that you can also just `copy` files using the copy module: +# https://docs.ansible.com/ansible/latest/collections/ansible/builtin/copy_module.html#examples +# And finally you can create files and directories with the `file` module +# https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html#examples diff --git a/roles/base/templates/hostname.j2 b/roles/base/templates/hostname.j2 new file mode 100644 index 0000000..1ee9db5 --- /dev/null +++ b/roles/base/templates/hostname.j2 @@ -0,0 +1 @@ +{{ inventory_hostname }} \ No newline at end of file diff --git a/roles/base/templates/hosts.j2 b/roles/base/templates/hosts.j2 new file mode 100644 index 0000000..1e44be2 --- /dev/null +++ b/roles/base/templates/hosts.j2 @@ -0,0 +1,3 @@ +127.0.0.1 localhost +{{ ansible_default_ipv4["address"] }} {{ inventory_hostname }} + diff --git a/roles/base/templates/motd.j2 b/roles/base/templates/motd.j2 new file mode 100644 index 0000000..4e1e869 --- /dev/null +++ b/roles/base/templates/motd.j2 @@ -0,0 +1,9 @@ + This is {{ ansible_fqdn }} + +System : {{ ansible_distribution }} {{ ansible_distribution_version }} ({{ ansible_distribution_release }}) + +Kernel : {{ ansible_kernel }} {{ ansible_kernel_version }} +CPU(s) : {{ ansible_processor_cores }} +RAM : {{ ansible_memory_mb.real.total }}Mb +Architecture : {{ ansible_architecture }} +Address : {{ ansible_default_ipv4.interface }} - {{ ansible_default_ipv4.address }} diff --git a/roles/ntp/files/ntp.conf b/roles/ntp/files/ntp.conf new file mode 100644 index 0000000..7702785 --- /dev/null +++ b/roles/ntp/files/ntp.conf @@ -0,0 +1,60 @@ +# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help + +driftfile /var/lib/ntp/ntp.drift + +# Leap seconds definition provided by tzdata +leapfile /usr/share/zoneinfo/leap-seconds.list + +# Enable this if you want statistics to be logged. +#statsdir /var/log/ntpstats/ + +statistics loopstats peerstats clockstats +filegen loopstats file loopstats type day enable +filegen peerstats file peerstats type day enable +filegen clockstats file clockstats type day enable + +# Specify one or more NTP servers. + +# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board +# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for +# more information. +pool 0.ubuntu.pool.ntp.org iburst +pool 1.ubuntu.pool.ntp.org iburst +pool 2.ubuntu.pool.ntp.org iburst +pool 3.ubuntu.pool.ntp.org iburst + +# Use Ubuntu's ntp server as a fallback. +pool ntp.ubuntu.com + +# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for +# details. The web page +# might also be helpful. +# +# Note that "restrict" applies to both servers and clients, so a configuration +# that might be intended to block requests from certain clients could also end +# up blocking replies from your own upstream servers. + +# By default, exchange time with everybody, but don't allow configuration. +restrict -4 default kod notrap nomodify nopeer noquery limited +restrict -6 default kod notrap nomodify nopeer noquery limited + +# Local users may interrogate the ntp server more closely. +restrict 127.0.0.1 +restrict ::1 + +# Needed for adding pool entries +restrict source notrap nomodify noquery + +# Clients from this (example!) subnet have unlimited access, but only if +# cryptographically authenticated. +#restrict 192.168.123.0 mask 255.255.255.0 notrust + + +# If you want to provide time to your local subnet, change the next line. +# (Again, the address is an example only.) +#broadcast 192.168.123.255 + +# If you want to listen to time broadcasts on your local subnet, de-comment the +# next lines. Please do this only if you trust everybody on the network! +#disable auth +#broadcastclient diff --git a/roles/ntp/handlers/main.yml b/roles/ntp/handlers/main.yml new file mode 100644 index 0000000..7bddadd --- /dev/null +++ b/roles/ntp/handlers/main.yml @@ -0,0 +1,8 @@ +--- +# handlers are special tasks that you can invoke when something changes. +# in this example you want to invoke this restart handler when the +# configuration of the service changes for example. +- name: "Restart ntp" + service: + name: ntp + state: restarted diff --git a/roles/ntp/tasks/main.yml b/roles/ntp/tasks/main.yml new file mode 100644 index 0000000..205077b --- /dev/null +++ b/roles/ntp/tasks/main.yml @@ -0,0 +1,24 @@ +--- +- name: "Install ntp" + apt: + name: ntp + state: present +- name: "Configure ntp" + copy: + src: ntp.conf + dest: /etc/ntp.conf + owner: root + group: root + mode: "644" + # notify is going to invoke the "Restart ntp" handler defined in + # the roles/ntp/handlers/main.yml file. Basically it says that + # you want to restart the ntp service every time the configuratio + # changes + notify: "Restart ntp" +# Here you say that you want the NTP service to be restarted as well +# as enabled on boot. +- name: "ntp service" + service: + name: ntp + state: restarted + enabled: true diff --git a/roles/root_user/tasks/main.yml b/roles/root_user/tasks/main.yml new file mode 100644 index 0000000..4d91c30 --- /dev/null +++ b/roles/root_user/tasks/main.yml @@ -0,0 +1,28 @@ +--- +# Ensures the .ssh directory exists +- name: "creates the .ssh root directory" + file: + path: "/root/.ssh" + state: directory + owner: root + group: root + mode: 0700 +- name: "Install root SSH keys" + template: + src: authorized_keys.j2 + dest: /root/.ssh/authorized_keys + owner: root + group: root + mode: 0600 +# Delete users you don't need +# respectively you can also *add* users +# https://docs.ansible.com/ansible/latest/collections/ansible/builtin/user_module.html#ansible-collections-ansible-builtin-user-module +- name: "Delete usual cloud users user" + user: + name: "{{ item }}" + state: absent + remove: true + with_items: + - pi + - admin + - ubuntu diff --git a/roles/root_user/templates/authorized_keys.j2 b/roles/root_user/templates/authorized_keys.j2 new file mode 100644 index 0000000..03272f5 --- /dev/null +++ b/roles/root_user/templates/authorized_keys.j2 @@ -0,0 +1,4 @@ +{% for key in root_user.default_root_keys %} +# {{ key.name }} +{{ key.key }} +{% endfor %} diff --git a/roles/vim/files/vimrc b/roles/vim/files/vimrc new file mode 100644 index 0000000..ed75cfd --- /dev/null +++ b/roles/vim/files/vimrc @@ -0,0 +1,51 @@ +set nocompatible + +filetype on +filetype plugin indent on +syntax on + +set expandtab +set tabstop=4 +set softtabstop=4 +set shiftwidth=4 +set smarttab +set lazyredraw +set showmatch +set incsearch +set hlsearch +set number +set ruler +set autoindent +set smartindent +set wildmenu +set laststatus=2 +set cursorline +set showmode +set backspace=indent,eol,start +set mouse=a +set background=dark +set foldenable +set foldlevelstart=10 +set foldmethod=indent + +nnoremap :set invpaste paste? +set pastetoggle= +set showmode + +let mapleader="," + +nnoremap :nohlsearch +nnoremap za +map 10< +map 10- +map 10+ +map 10> +map = 10= +map s :%s/\s\+$// + +nnoremap tt :tabnew +nnoremap to :tabonly +nnoremap tc :tabclose +nnoremap tn :tabnext +nnoremap tp :tabprevious +nnoremap vs :vsplit diff --git a/roles/vim/tasks/main.yml b/roles/vim/tasks/main.yml new file mode 100644 index 0000000..5cdccff --- /dev/null +++ b/roles/vim/tasks/main.yml @@ -0,0 +1,12 @@ +--- +- name: Install vim + apt: + name: vim-nox + state: latest +# use the `copy` module to copy files to the remote host +- name: Configure vim + copy: + src: vimrc + dest: /etc/vim/vimrc + owner: root + group: root diff --git a/upgrade.yml b/upgrade.yml new file mode 100644 index 0000000..f3414ab --- /dev/null +++ b/upgrade.yml @@ -0,0 +1,9 @@ +--- +- hosts: all + tasks: + - name: "update apt cache" + apt: update_cache=yes force_apt_get=yes cache_valid_time=3600 + - name: "upgrade packages" + apt: upgrade=safe force_apt_get=yes + - name: "upgrade dist packages" + apt: upgrade=dist force_apt_get=yes