Setting Up A Chef Server in AWS

Introduction

I recently had need of setting up a Chef server for some internal work. Instead of setting up a hosted Chef server, I decided to try launching a machine inside AWS. My main reasoning behind this was that we wouldn’t need the Chef server up too frequently, and so having the ability to keep it up only some of the time, could save us some money over time. Of course, this brought all sorts of other consideration and problems when setting up the server.

Building The Server

After experimenting with about 5 different machines, I finally decided to launch the server on an Amazon Linux machine. The short story of why I went with this machine was simply compatibility. Amazon’s machine came with most of the prerequisites already installed, and had few conflicting packages already setup. Once the machine was launch, I simply followed the basic Chef server installation instructions.

# install chef-solo
curl -L https://www.opscode.com/chef/install.sh | sudo bash
# create required bootstrap dirs/files
mkdir -p /var/chef/cache /var/chef/cookbooks/chef-server
# pull down this chef-server cookbook
wget -qO- https://github.com/opscode-cookbooks/chef-server/archive/master.tar.gz | tar xvzC /var/chef/cookbooks/chef-server --strip-components=1
# GO GO GO!!!
chef-solo -o 'recipe[chef-server::default]'


Dealing with AWS Dynamics

As the main point of running Chef on this machine was to avoid higher costs, I didn’t want an elastic ip associated with this machine, as it would cost us money when the machine was down. As a result I wanted to setup a DNS using Amazon’s Route53. I used my co-worker Rich’s recent post about scripting Route53 to ensure that each time I launched the machine I was able to reach the machine in the same static way. This is VERY important to ensure that our clients and nodes can always reach our Chef server. To do this, I edited the hostname file to match our url assigned in Route53, and our route53 script.

vim /etc/hostname
[NEW HOSTNAME]
:wq

We now have our machine always pointing in the correct direction.

Fixing Security

When I first tried pushing a recipe into our Chef server, I got some weird errors from my client machine. After some research, I realized the problem was with my certificate. I ran the below commands

knife ssl fetch
knife ssl check

And I saw that the check was failing. Because I changed the hostname on the machine, I needed to update the certificate. First I needed to create a new key for our certificate

openssl genrsa -des3 -out chef_server.key 2048

Ensure you give this a nice long passphrase. However, in order to simplify our certificate process, I then wanted to create an insecure version of our key, one without a key. To do this I ran the below code, and entered my previous passphrase.

openssl rsa -in chef_server.key -out chef_server.key.insecure

I then decided to change the file around, so that the insecure key was our default key. Note that this is a security issue, and not something that should be normally done, and definitely not something to leave, ensure you change back your files once you are done with the process, and to be safe, the insecure key should be deleted.

mv chef_server.key chef_server.key.secure
mv chef_server.key.insecure chef_server.key

Now we want to create our CSR.

openssl req -new -key chef_server.key -out chef_server.csr

Finally, I wanted to self sign our certificate. This will cause security warnings when visiting the site, but is cheaper than submitting the csr to get a certificate. So to create our self signed certificate, I ran the below command

openssl x509 -req -days 365 -in chef_server.csr -signkey chef_server.key -out chef_server.crt

Notice that after a year our certificate will expire, and I will probably have to run through these steps again. Finally, I needed to install our certificates. Because this is not our typical webserver, I wanted to get these certificates where chef keeps it’s default certificates. I moved all of these files.

rm chef_server.key
mv chef_server.key.secure /var/opt/chef-server/nginx/ca/chef_server.key
mv chef_server.csr /var/opt/chef-server/nginx/ca/chef_server.csr
mv chef_server.crt /var/opt/chef-server/nginx/ca/chef_server.crt

Lastly, I needed to reconfigure chef to use our new certificate instead of the default one. To do this, I needed to supply this new key information to the chef server, and then reconfigure it. I added the below information into the chef configuration file /etc/chef-server/chef-server.rb

api_fqdn = "NEW HOSTNAME"
nginx['ssl_certificate'] = "/var/opt/chef-server/nginx/ca/chef-server.crt"
nginx['ssl_certificate_key'] = "/var/opt/chef-server/nginx/ca/chef-server.key"
nginx['server_name'] = "NEW HOSTNAME"

Then I re-ran the chef configuration

chef-server-ctl reconfigure

Finally, I needed to test out my new certificates. On my client machine, I re-ran the earlier fetch and check commands, and I saw everything passing. Then, when I tried pushing my cookbooks to the chef server voila everything worked perfectly.