My notes, things I have learned, and documentation I have created.

Platform: CentOS 6 x86_64

I knew when I implemented my split DNS solution years ago that I wanted to try to keep one recursive and open to use for queries. I wanted this more for my convenience (testing, and I could remember my address) when working on issues outside my own network. I knew at the time that I was taking a risk. It seemed like it was more about resources than security. I have two DNSs, but only the one was open to queries without restriction. Well, after probably about 10 years running like this, I finally had a need to add some ACLs and close up my convenient access. I was hit by several very long flurries of requests for isc.org. Because I have such limited bandwidth, it was quickly pretty obvious that something was wrong.

Initially, I looked to my gateway server. And this stumped me for a while, because I detected nothing wrong or any unusual traffic volume. However, a quick trace and look at the DNS logs on my secondary server revealed the requests. They would go on for hours and hours, but they would stop once in a while for a few hours before starting up again.

My initial approach was to block the IP address the queries were coming from, and hope to preserve my convenient access to my own recursive lookup server. I also found this interesting solution using iptables to block any requests to isc.org:

iptables -A INPUT -p udp -m string --hex-string "|03697363036f726700|" --algo bm --to 65535 -j DROP

But this does not seem practical to somebody who uses services provided by the Internet Systems Consortium, and does actually find myself on the site once in a while. While this solution does not really work for me in this case, it certainly enlightened me to adding this to my arsenal for some other situations. All you need to do is convert the URL to hex and create your iptables statement.

After watching the requests continue to come in despite being dropped at my firewall, I decided that it finally come to an end. I gave in an have added the ACLs to my external DNS servers.

In the /var/named/chroot/etc/named.conf:
...
acl "AllowToQuery" {
// Add subnets I trust to use my DNS for queries.
aaa.bbb.ccc.ddd/xx;
eee.fff.ggg.hhh/yy;
iii.jjj.kkk.lll/zz;
localhost;
};
...
// Added the ACL after DDOS attacked - repeated queries for isc.org.
allow-query { AllowToQuery; };
...

The allow-query statement was added to my external view, since I am running a split DNS. Having the ACLs setup this way eventually caused the request to stop.

This one presented quite a challenge for me. It seems that the magical solution is using the right versions. There are a number of ways (supposedly) to install ruby on rails. I started out trying to use ruby packages included in CentOS. I quickly realized that was not going to work. It kept running all kinds of issues with libraries missing or incorrect versions. So then, I opted to just build from source. Well, this got me much closer but I ended up having an openssl issue that I thought I resolved, but the gem command was having all kinds of issues. Something clearly was not right. So, I then went back to using RVM, which I had tried earlier. This time I started completely over with a new HPCloud CentOS 5.6 image. The following is what worked for me.

As I mentioned above, I started with a clean HPCloud CEntOS 5.6 image ( ami-000004d2 – local (CentOS 5.6 Server 64-bit) ).

# yum groupinstall “Development Libraries” “Development Tools”

The next step, of course, turned my 5.6 instance into a 5.7 instance:
# yum update
# shutdown -r now

Add the EPEL repository:

# cd /usr/local/src
# wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
# rpm -ivh epel-release-5-4.noarch.rpm
# yum install git

Remove certificate requirement for curl, when installing rvm:
# echo insecure >> ~/.curlrc

# bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer) # vigr ... rvm:x:500:root Update environment variables: # source /etc/profile.d/rvm.sh Install other dependencies for ruby determined by running rvm requirements:
# yum install -y gcc-c++ patch readline readline-devel zlib zlib-devel libyaml-devel libffi-devel openssl-devel make bzip2 autoconf automake libtool bison iconv-devel

# rvm install 1.9.2
# rvm use 1.9.2

Note: You can use the following command to make this version the system default:
# rvm use 1.9.2 –default

To verify that all is right at this point:
# type rvm | head -1
rvm is a function is the expected output.

# cd $(rvm gemdir)

The following will install the gem command:
# rvm rubygems current

This will allow you to install rails:
# gem install rails

This has allowed me (rails is not needed for this) to install hpcloud command to manage my Storage Object.

# wget http://build.hpcloud.com/sites/default/files/downloads/hpfog.tar
# tar -xvf hpfog.tar
# wget http://build.hpcloud.com/sites/default/files/downloads/hpcloud.tar
# tar -xvf hpcloud.tar
# gem install hpfog-x.x.x.gem
# gem install hpcloud-x.x.x.gem

# hpcloud
Tasks:
hpcloud account:setup # set up or modify your credentials
hpcloud acl