Hosting the entire Apt-Get Repository!

Here’s what I’ll do today: I’ll build a server that will hold the entire Aptitude Debian repository. That’s 86+ Gb of packages. Yay. However since I’m only getting the latest releases and only the ARM architecture, the number is now around 20 Gb. I’ve informed people at the Polar Institute I’ll be downloading a lot, and turns out the Linux servers could actually use an updated software repo! So after getting my plan green-lit, I decided to use an actual server for this task. So the server in question is a Dell PowerEdge R310 (we have 3 that aren’t being used for anything.) It has 8 HDD’s of 143 Gb on SAS, running on Raid 1. Basically all the drives have a redundant mirror drive, meaning I have 4 x 143Gb of storage, with Debian and some essentials already installed, I have about 550 Gb to work with. Marvelous!

First thing’s first, we need a web server on there. For that, I had two options: NginX or Apache. While I prefer NginX for it’s simplicity and ease of use. However, I have a lot of experience with Apache and I don’t feel like re-learning a lot of stuff; not because I’m lazy, I just don’t want to screw this part up! Let’s go ahead with the basics, after booting up the server in the terminal, I realized it was already set up to access the WWW. A quick ping to Google’s DNS to check:

$ ping 8.8.8.8

And sure enough, it ponged back! Next, just the usual:

$ sudo apt-get upgrade
$ sudo apt-get update

And wait for it to finish… After about 3 eternities, we get to work. The 8 drives are on 4 different partitions, so I’ll just use one exclusively for repos. I won’t detail this since ot’s pretty much up to you to do whatever with your storage! Just make sure you either put the www folder on there, or make a symbolic link.

Now, to download LAMP (Linux, Apache, MySQL, PHP). I actually only really need the first two. This part is pretty basic, let’s start with :

$ sudo apt-get install apache2 -y

Next, go to the IP of the Pi at [192.168.10.1], and… Success! Now we go to:

$ cd /var/www
$ sudo mkdir -p rasp
$ sudo mkdir -p ada

There’s 2 repositories I want to download, Raspbian’s and Adafruit’s (since I have a bunch of their hats and addons):

Now that we have those directories set up, let’s go ahead and start the download… Yikes:

$ cd rasp
$ sudo wget –mirror –convert-links http://archive.raspbian.org/raspbian/

$ cd ada
$ sudo wget –mirror –convert-links http://apt.adafruit.com/raspbian/

You can use rsync instead of wget, but since this is the initial download, there’s no risk of redundant downloads or such. If you update your repo, definitely go with rsync or purge your existing download first. The parameter —mirror is to download the source as a mirror image for offline use, and —convert-links is to convert any links to match my local directory!

one-eternity-later

Now that everything has downloaded locally – keeping in mind it takes HOURS – we have to think of an efficient way to link these; I don’t want to have to rely on an IP address since those will change on the fly. Instead, I’ll rely on a domain name assigned with a DNS, however that’s in a later step. Instead, let’s edit the host files for now. Let’s return to our trust Pi, and type in:

$ sudo nano /etc/hosts

And add the IP and hostname to my first raspberry pi: (note that this is the local IP to the server)

127.0.0.1                raspberrypi
172.148.7.14          ddu.repo.aq

The server actually has multiple IPs (4 to be precise) since the network card allows 4 simultaneous wired Ethernet connections (larger bandwidth). We’ll deal with the rest via DNS.

Fun fact! The TLD (Top Level Domain, e.g. .COM) for Antarctica is .aq, and since we’re not on the World Wide Web, DDU replaces WWW! 🙂

Now if I ping ddu.repo.aq I get a pong, perfect! The server is there. Test by navigating to:

raspindex
http://ddu.repo.aq/rasp/

Onto telling Aptitude where to fetch the files from… Apt-get fetches the source links from /etc/apt/sources.list, which reads:

deb http://mirrordirector.raspbian.org/raspbian/ jessie main contrib non-free rpi

let’s just comment that by adding # in front of it and add these two lines:

deb http://ddu.repo.aq/rasp/ jessie main contrib non-free rpi
deb http://ddu.repo.aq/ada/ jessie main

And voila! Alternatively, we could’ve set mirrordirector.raspbian.org/raspbian/ in the hosts file to follow the IP of the server, but it’s sloppy and unprofessional, let’s not do that. Hoping I don’t get any GPG key errors, I ran:

$ sudo apt-get update

And voila! Ran just fine, and was done in a jiffy (granted, everything was already up to date…) I’ll download the Debian and Ubuntu repo archives another time, for now, my Pi’s can update without the long and tedious network hopping to the raspbian mirrors. I even ran:

$ sudo apt-get install apache2 -y
$ sudo apt-get install php5 libapache2-mod-php5 -y
$ sudo apt-get install mysql-server php5-mysql -y

And it worked! Huzzah, LAMP on a Pi in a couple of minutes! #proud

I’ll have to work on getting a DNS to attribute ddu.repo.aq to the IP, because editing so many hosts files is going to bore me real fast… Tomorrow!

Until then,

Stay fluffy!

~Evie

 

[UPDATE]

If you have any issues with GPG keys and such, you can add the keys for both repos to your key-ring with:

$ wget -q https://archive.raspbian.org/raspbian.public.key -O – | sudo apt-key add –
$ wget -q https://apt.adafruit.com/apt.adafruit.com.gpg.key -O -| sudo apt-key add –

[HELPFUL LINKS]

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s