“Composting” Docker

Right now I’m fiddling with Docker and combing through their terrific, extensive docs. As I have a history of doing lately with this blog, I’ll talk about my own setup and the installation process, all the way to the Docker image I’ve created and messed with, and what’s next for my own knowledge.

New laptop

First, I recently purchased a machine from Free Geek so I’m going through the delightful process (really :D) of setting it up. When I volunteered at Free Geek (which I HIGHLY recommend, please ping me for details!!), the computer I received after building desktops for them for a while was 40G HDD, 512MB DDR2 RAM, and was capable of a wired-only connection to the internet. After a USB wifi antenna, the machine could do all the browsing I needed, which was all I did at the time. At the time, that kind of machine cost anywhere between $80-$130 if my memory serves me correctly. I believe this was about 2011.

Two days ago, I wanted a laptop that would be able to run VMs without too much trouble, and I wanted to spend $200 or less as I am unemployed. Five years after my volunteer stint, I got a machine with 8G ram DDR3, 250G SATA HDD, i5 dual-core processor, in a pretty giant old Dell Latitude E63340, for exactly $200. AMAZING. I’m astounded at how cheap it is. And what else has changed? It’s no longer Ubuntu that they ship the computers they clean, reassemble to spec, and sell or give away, it’s Linux Mint, specifically Edition 17.1. Thoughts so far are really just that it feels very Windows-y with its focus on the (not) Start menu, and you’ll get no complaints from me on this front. It’s Ubuntu with a coat of paint on top, and I’m not toooootally sure this is what I want to be running, but it definitely works and there is probably broadest reach of packages available for the end-user (not enterprise which is of course another red-colored ball of wax) so it will do, for now, and at some point I am sure I’ll explore other distros.

Docker

On to Docker! I’ve previously read up on Docker, and find its resource allocation methodology super interesting. I’ve used a lot of Vagrant/VMWare on my OSX box at my previous job, and while the mechanism to spin up/destroy boxes was pretty good, the amount of cruft left behind became frustrating at a work level – I had to clear out vast swaths of space with a machete on an irritatingly regular basis! Basically, Docker runs without a hypervisor, which in the spirit of keeping things in layperson’s terms, is a virtual machine management machine which is pretty heavy. That they’ve figured out a way around the heaviness of a hypervisor is a Big Deal. And as I understand it, it uses the resources of the host machine very intelligently. With the previous VM paradigm, you gave both disk and memory permanently (insomuch as a VM that you create & tear down relatively easily is permanent) to the machine, and the VM held onto that inaccessibly while in use. No such selfish tactics are employed by Docker and use of memory in particular is supposedly much more elastic.

Installation was just a few shell commands which worked with no fuss, I used this official site. The instructions are for Ubuntu 14.04 which Mint 17 is based on, and as I’ve covered above, all Ubuntu packages can be used on Linux Mint with little to no adaptation required.

sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates
sudo apt-get install linux-image-extra-$(uname -r)
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
echo 'deb https://apt.dockerproject.org/repo ubuntu-trusty main' >> /etc/apt/sources.list.d/docker.list
sudo apt-get install docker-engine
sudo apt-get update

Then, the Docker engine can be started! First start the service (this may be different if your linux distro uses systemctl, like a newer version of Fedora or RHEL/CentOS) & then make sure their dummy example works.

sudo service docker start
sudo docker run hello-world

Ok! Now we’ve got Docker installed. If you encounter anything funny here during installation I’d love to hear about it, please leave a comment! I can try to help.

There are two primary ways for the new end-user to continue forward. One of them is writing a Dockerfile. I leave that for its own blog post, but hold me to it! I want to write a few basic Dockerfiles for my own reference & I imagine they’ll be of some use to others! The second way I’ll mention here is via DockerHub, which you don’t even directly need to interact with to use! To me it’s reminiscent of pulling images down from VagrantCloud, but again, you don’t even have to go there to grab things. You can feed a few different common distros into the docker run command and it will pop you right into a container of that OS! It’s still rather magical at this point, so I’m still learning more about it so it becomes a bit less magical 😉

Getting into the box I wanted was as simple as plugging in the distro I wanted, Fedora, into the docker run command, like so:

sudo docker run -it fedora /bin/bash

What this led to for me, which I need to learn SO much more about, is an EXTREMELY spare version of Fedora that has very few executables I’m used to. Because it’s the latest version of Fedora, its package manager is dnf & not yum, but it still knows what you mean & permits installation “via” yum – but really it’s just aliased to /usr/bin/dnf, haha, which is fine.

Quick fun fact: dnf is an abbreviation of “Dandified Yellowdog Updater, Modified” – you can see the yum in there as the commonly used RHEL and RHEL-flavored linuxes’ package manager. To me, whenever I type dnf install -- it looks as though I’m typing “do not f&$%ing install” 😀

So, next steps, make the container persistent – it isn’t yet, and play around with Dockerfiles, and just do a little more spying on the produced container itself & probably try to do some babby’s frist load testing things in there & spy on the container as a process without the box & all its processes within!

See you next time! Would love to hear from you. If I’ve missed the mark on anything with this pretty chewy piece of technology please let me know, or if there’s anything you’d like to see me cover leave a comment!

btw the title of this is a joke which was made during the Docker-fiddling open space I held at Open Source Bridge 2016 🙂 if you were there, thanks for coming! Super fun discussion.

Postgres on Fedora

Note: This is a post from several months ago in the ol’ drafts bin and there’s a ton of information here, even though it is incomplete. I’m not running Fedora any more, but it’s possible this could help someone else, so I hit publish.

This shouldn’t be too long a post, but I’ve encountered something that does not really feel like it ought to be an edge case!

In trying to install PostgreSQL on Fedora 23 I ran into a few snags per the Other Linux Installation book published here using the download guide for RH-flavored Linuxes here.

Sidebar: In the Installation section of the introduction to Postgres (I know PG pretty well in the context of Puppet Enterprise, but I really want to expand that knowledge since I know a lot of people use & love it [/diatribe on why I’m doing this]) it says the following:

If you are installing PostgreSQL yourself, then refer to Chapter 15 for instructions on installation, and return to this guide when the installation is complete. Be sure to follow closely the section about setting up the appropriate environment variables.

That’s in section 1.1, on the Installation page, by the way. This might be why we can’t have nice things.

MOVING ALONG, the way which has worked for me to install Postgres is the following, thanks in much part to a) This Fedora Project doc and b) working knowledge of the su command.

$ sudo dnf install postgresql-server postgresql-contrib
 ... "are you sure you want to install y/N" ...

$ sudo systemctl enable postgresql
Created symlink from /etc/systemd/system/multi-user.target.wants/postgresql.service to /usr/lib/systemd/system/postgresql.service.

$ sudo postgresql-setup initdb
WARNING: using obsoleted argument syntax, try --help
WARNING: arguments transformed to: postgresql-setup --initdb --unit postgresql
 * Initializing database in '/var/lib/pgsql/data'
 * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log

$ sudo systemctl start postgresql

So, only after making the dummy initdb did the systemctl start command go through, but I noticed that there was still no pgsql executable, so I couldn’t actually use postgres yet. Finally, I catted the postgres log like so:

$ sudo cat /var/lib/pgsql/initdb_postgresql.log
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /var/lib/pgsql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    /usr/bin/postgres -D /var/lib/pgsql/data
or
    /usr/bin/pg_ctl -D /var/lib/pgsql/data -l logfile start

which, up at the top, points out that that this needs to be run with the postgres user, and after some fiddling, knowing that the -D flag needs to be before the /var/lib/pgsql/data path as designation. Also seeing su crap itself a number of times was irritating. To get into the postgres user, it was necessary to provide a password! I hadn’t set one, so I tried a couple easy guesses & couldn’t figure it out and a (very quick) DDG didn’t yield anything either, so I snuck around it with sudo -u postgres psql, which only asked me for the superuser password – A-OK! But Then! It then complained that it didn’t have permissions to get into ~, but reasonably got me into the postgres user’s prompt: HERE, FINALLY, I was able to run

postgres=# /usr/bin/postgres -D /var/lib/pgsql/data

Though… now I’m noticing that anything I type in there doesn’t even throw an error. It even offers ‘Type “help” for help,’ and yet when I do, with and without quotes, there is no output & no result. And I still don’t have a pgsql executable.

Ok, but there IS a result from which postgres, which is the nicely predictable /usr/bin/postgres. One success – I have a universally executable postgres! So when I run it with no arguments, hoping for more information from my sleuthing, I get some!

 $ postgres
postgres does not know where to find the server configuration file.
You must specify the --config-file or -D invocation option or set the PGDATA environment variable.

So that’s where the -D is necessary. Cool. But we also definitely don’t have the PGDATA env var
postgres -D /var/lib/pgsql/data.

Wow! There’s more to do here, and I don’t have time right now, so have SOME information, yet incomplete!