DarkNet

Hello Everybody,

I’ve spent considerable time making squid cache youtube.
I firstly tried http://eu.squid-cache.org/ConfigExamples/DynamicContent/YouTube using the webserver. However, a few problems where identified, firstly, you can only delete content based on date. I also identified that if the user fast forwards the video before it finishes downloading, it will start to save a nother copy of the video as the file size is now different. The method I used:

squid.conf:

acl store_rewrite_list url_regex ^http://(.*?)/get_video\?
acl store_rewrite_list url_regex ^http://(.*?)/videodownload\?
acl store_rewrite_list url_regex ^http://(.*?)/videoplayback\?
cache allow store_rewrite_list

# Had to uncomment this again, because I couln’d login to google mail using IE6 (firefox had no trouble):
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY

refresh_pattern (get_video\?|videoplayback\?|videodownload\?) 5259487 99999999% 5259487 override-expire ignore-reload ignore-private negative-ttl=0

storeurl_access allow store_rewrite_list
storeurl_access deny all

storeurl_rewrite_program /etc/squid/youtube.pl

youtube.pl
#!/usr/bin/perl
$|=1;
while (<>) {
@X = split;
$x = $X[0];
$_ = $X[1];
if (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?\&(itag=22).*?\&(id=[a-zA-Z0-9]*)/) {
print $x . “http://video-srv.youtube.com.SQUIDINTERNAL/” . $2 . “&” . $3 . “\n”;
# youtube Normal screen always HD itag 35, Normal screen never HD itag 34, itag=18 <–normal?
} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*)/) {
print $x . “http://video-srv.youtube.com.SQUIDINTERNAL/” . $2 . “&” . $3 . “\n”;

} else {
print $x . $_ . “\n”;
}
}

Hello World,

It’s only 9am and already today has been… interesting…

Today is bin day and I forgot to put the bin out, as such the half naked and freezing bin run was required early this morning.

In addition my older server DarkStar decided it was going to die last night. After getting up this morning I was presented with a segfaulting server at every command. Only command I could get to work was cat /proc/mdstat which showed a failed disk.

DarkStar has been made with RAID1+0 so a single failed disk shouldn’ t kill anything, however, on a reboot it didn’t comeback. After some tinkering I manually removed the failed disk and it booted!! but now it wants to fsck the drive.. which is going to take hours. As such I have aborted the fsck and are rapidly rsync’n the data to another server.

Later tonight we’ll complete the fsck and if all goes south I atleast have the data somewhere else. The replacement of DarkStar has been something on the cards for a number of months so I’ll migrate all data to the new server in the near future.

T

A number of IT professionals have been telling me of the virtues of cacti, a network graphing tool. I decided to take the jump and give it a crack. In my case I am graphing my internal network which consists of 2 x GS108T smart switches and a Billion 7800n router.

I was surprised to find that google failed to locate a working template to allow for the easy graphing of the billion. As such I used the existing template available on the cacti forums and modified it using the below MIBS

Downstream Attenuation: .1.3.6.1.2.1.10.94.1.1.3.1.5.3
Downstream SNR: .1.3.6.1.2.1.10.94.1.1.3.1.4.3
Downstream Speed: .1.3.6.1.2.1.10.94.1.1.3.1.8.3
Upstream Attenuation: .1.3.6.1.2.1.10.94.1.1.2.1.5.3
Upstream SNR: .1.3.6.1.2.1.10.94.1.1.3.1.4.3
Upstream Speed: .1.3.6.1.2.1.10.94.1.1.2.1.8.3

Net result is http://shell.dark.net.au:81/cacti/ username and password guest

T

As many of you are aware I am now a father, my daughter Keira Elizabeth Read was both 3rd June 2011.

Mother and Baby are both doing well

Keira Read age 1 day

I will be publishing a web gallery shortly!

Travis

Well the time has come to replace the old DarkNet Server.

New server spec as follows:

* AMD Athlon II Neo N36L Dual Core CPU
* 4gig RAM
* 4 x 2Tb Samsung HD204UI drives
* Software Raid5

Software: Ubuntu linux 10.04.2 LTS with KVM

Costs of hardware:
HP Microserver – $180
4gig Ram – $50
4 x 2Tb hard drives = $400
HP Remote Access card – $80

Total costs = $710 – Cheapest server I’ve ever built!

As per my previous post I’m still working on migrating email and web. I’ve been time poor of late dude to changes in my siutation.

T

Another electrical fault at Greenwood! This time it was a dead short in the bathroom power point. Thanks to my sparky mates we are back up and running. Downtime this outage was a day and a bit.

T

Thanks to Western Power I once again lost power at the location of the Greenwood server. This time Western Power replaced some cable in the street or similar resulting in no power for a few hours. Thanks to the redundant systems of DarkNet the server didn’t fail.

T

Greenwood server was upgraded from Gutsy to Hardy today. Other than a minor issue with mysql using 99% CPU no other issues where encounted. Once I return home we’ll do a kernel upgrade and hope for the best.
T

G’day All,

Yesterday we had a brief outage to services on the Greenwood Server. I’ve been having power stability issues for some time so finally got a sparky onsite who replaced some wiring.

I was also made aware of the fact that rebooting the server cleared a lot of memory. As such I have blown away vmware which was the main memory hog. vmware still runs on the colocated server which has much more memory avaliable.

T