Benchmarking Untangle Gateway

a work in progress


* This benchmark will be re-done to include more details (gateway's memory & CPU utilization)
* There have been a typo which mistakenly puts the filtering device where it won't be. Will be corrected later

We will NOT deploy a filtering appliance afterall. We'll be using Kaspersky Antivirus to enforce Web Filtering policies (due February). We'll deploy a proxy only, however I'll continue with this benchmark.

Untangle is a gateway software package that installs to x86 hardware. It functions like any gateway appliance from any company like Cisco, BlueCoat, ...etc. Only this one is built on top of Open Source projects running on Linux, giving it the power of guaranteed stability and the flexibility to mesh everything into a single platform that does wonders.

I have deployed this on a subsidiary company that has 120 employees. The machine has been running for about 7 months so far without any problems whatsoever. The total cost was 110 KD ($330 roughly) for the hardware: 1.8GHz dual core CPU, 2GB RAM, 80GB HDD, built-in 10/100Mbps NIC + video card + audio, 2x 1Gbit Linksys NICs.

Untangle offers its basic package for free. This contains the following components:

• Web-Filter


• Spam Blocker


• Spyware Blocker


• Phish Blocker


• Virus Blocker


• Layer-7 Filtering / Protocol Control


• Intrusion Prevention


• Firewall


• Attack Blocker


• VPN


• Reporting


• Routing and Quality of Service

The main reason why Untangle was chosen because it has Layer-7 filtering. This allows you to block certain programs from accessing the network, like Yahoo Messenger, MSN Messenger, ICQ, IRC, Jabber, Peer-to-Peer software, online games, ...etc.

Only the components in bold above were activated, because the rest were useless to our company since there existed other appliances that are doing their job.

Now, we're considering deploying this on the whole company for around 400 users. The stake got raised and we're comparing it against BlueCoat. So we'll need some commercial packages from Untangle to integrate it with our Active Directory Domain Controller using their AD Connector component, and a way to apply different rules to different people using Untangle's Policy Manager component.

Prices apart, since Untangle allows us to demo the commercial components for 14 days for free, we have done the integration with the domain controller and setup test policies for various domain users and everything worked perfectly in less than 10 minutes!

Now it's time to benchmark it and see if it can handle the stress.

Benchmark Details

Host Machine

AMD Dual-core 2.4GHz, 4GB RAM, 2x Gbit NICs, 80GB HDD.

Network Setup

EXT/WAN: 1x Gbit NIC is the external link which is connected to our LAN at work. The Internet link is 1Mbps.
INT/LAN: The 2nd NIC is a bridge to the internal network. I'm using a cross cable and hooking it directly to my laptop. (I've also tested it using a D-Link 5-port Gbit switch and got same results).

Caching: The Untangle box was NOT the DNS server (although it caches DNS requests).
As of this writing, Untangle does not offer a web-caching solution, though it's mentioned in their forums that one will emerge with following updates. Our tests didn't involve a web-caching scheme.

 

Deployment Options

Any appliance that we'll eventually go with will be either sitting between the firewall and the Internet router (bridged setup), or connected to the external switch like the firewalls are.

1) Bridged setup: Requires 2 Ethernet ports/interfaces; an internal and an external interfaces.
No need to change the users' settings nor the firewall's gateway. The external port gets a public IP while the internal one is bridged and holds no IP. Whatever traffic is flowing towards the router (the firewalls' gateway) must flow through the appliance.


2) Switch setup: Requires 1 Ethernet port. It doesn't require the users to change any settings, however, it does require the network admin to change the firewall's default gateway to point at the proxy server. Obviously, the proxy's gateway would be the router like in (1).

Software

Siege is an open-source and free HTTP stress, benchmark and load balance testing tool.
Siege was not used directly. The URLs fed to it were harvested using another nifty tool by the same author called sproxy.

Simulation Criteria

From our subsidiary company we got a max of 250 web transactions per minute. This translates to 4 requests per second.
Our test a bit more stressful than these numbers: 100 browser requests per 5 seconds (random). This translates to 20 requests per second.

The links that were used for testing are:
http://www.eicar.org/download/eicar_com.zip
http://www.eicar.org/download/eicar_com.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://www.eicar.org/download/eicarcom2.zip
http://wwww.cnn.com
http://www.gooogle.com
http://www.kooora.com
http://www.sectorx.org
http://www.fox.com
http://www.joecartoon.com
http://www.killfrog.com
http://www.yahoo.com
http://www.nbk.com
http://www.cakephp.org
http://www.tucows.com
http://www.download.com
http://www.youtube.com
http://www.islamway.com
http://www.jazeeraairways.com

The links are a mix to test the virus blocker when enabled, small file downloads, sites with many images, and sites that should be blocked by the web-filter. I repeated the same virus file to increase the probability of siege using it.
Note: the file eicarcom2.zip is a zip file that contains another zip file. The 2nd zip file contains a virus. This is to test if the Virus Blocker detects multiple packing. (It does).

Siege was run with the following parameters:

for i in `seq 1 5`; do siege -i -b -c 100 -t1m; done

-i, --internet: INTERNET, generates user simulation by randomly hitting the URLs read from the urls.txt file. This option is viable only with the urls.txt file.

-b, --benchmark: BENCHMARK, runs the test with NO DELAY for throughput benchmarking. By default each simulated user is invoked with at least a one second delay. This option removes that delay.

-c NUM, --concurrent=NUM: CONCURRENT, allows you to set the concurrent number of simulated users to num.

-t NUMm, --time=NUMm: TIME, allows you to run the test for a selected period of time. The format is "NUMm", where NUM is a time unit and the "m" modifier is either S, M, or H for seconds, minutes and hours.

This means that connections are established immediately without delay and we're simulating 100 browser requests on all URLs over the period of 1 minute, five times.

The test will be taken on 3 scenarios: Direct Connection, Without Antivirus, With Antivirus.

Results

Direct Connection: Connected directly without passing any filtering appliance.

Resp Time	Trans Rate	Concurrency	Succ. Trans	Failed Trans	Failure Rate
5.53	15	82	948	68	7.17%

Without Antivirus: Connected through Untangle appliance without antivirus blocker.

Resp Time	Trans Rate	Concurrency	Succ. Trans	Failed Trans	Failure Rate
4.4	20	86	1165	83	7.12%

With Antivirus: Connected through Untangle appliance using antivirus blocker.

Resp Time	Trans Rate	Concurrency	Succ. Trans	Failed Trans	Failure Rate
4.49	19.37	86.55	1170	99	8.46%

To be continued

Compiling SimpleScalar Simulator on Linux

SimpleScalar.com provides "fast, flexible and extensible infrastructure for hardware modeling and software analysis."

Their code was last updated in late 2003, but based on 1997 code-base. I got a lot of warnings and errors that prevented the compilation. With some help from twkm from #c on DALnet (IRC), I was able to complete the compilation process (though with warnings still).

I will mention what I have modified in the files, then I'll provide you with a script and .diff files to do the whole patching and compilation process automatically. This also includes the compilation of the SPEC2000int package (compiled against simplesim-ALPHA only).

The files simplesim-3v0d.tgz and SPEC2000int.tgz are hosted on this site (which also provides a guide).

Changes

Open simplesim-3.0/target-alpha/alpha.h: Shift the line 224 to line 239.
-> "extern enum md_opcode md_mask2op[];"
Move it such that it's above this line: "/* enum md_opcode -> description string */"

After extracting SPEC2000int package, the Makefile inside needs one line to point at the directory containing the simplesim-3.0 directory.
-> "/home/students//"
Here's a sample directory hierarchy to understand:
-/
--home
---username
----tmp
-----simplesim-3.0
Your path will be: /home/username/tmp

Scripts & Automagic!

1) Create a directory in your home directory and call it: simtmp
2) Create a file called machine.diff inside simtmp
3) Put the following inside it:

--- simplesim-3.0/target-alpha/alpha.h 2003-10-09 05:14:23.000000000 +0300
+++ simplesim-3.0-fixed/machine.h 2008-12-24 14:02:22.000000000 +0300
@@ -221,7 +221,6 @@
 #define MD_MAX_MASK  2048
 
 /* internal decoder state */
-extern enum md_opcode md_mask2op[];
 extern unsigned int md_opoffset[];
 extern unsigned int md_opmask[];
 extern unsigned int md_opshift[];
@@ -236,6 +235,8 @@
   OP_MAX /* number of opcodes + NA */
 };
 
+extern enum md_opcode md_mask2op[];
+
 /* enum md_opcode -> description string */
 #define MD_OP_NAME(OP)  (md_op2name[OP])
 extern char *md_op2name[];

4) Create a file called setup.sh inside simtmp and put this in it:

#!/bin/bash
echo Extracting simplesim
/bin/tar -xf simplesim-3v0d.tgz

echo Patching simplesim
cd `pwd`
patch -p0 < `pwd`/machine.diff

echo Extracting SPEC
/bin/tar -xf SPEC2000int.tgz

echo Creating results directory
mkdir Results

echo Compile simplesim ALPHA
cd simplesim-3.0
make config-alpha
make

echo Update SPEC Makefile
cd ../
THISDIR=`pwd`
cd SPEC2000int
sed '19s:/home/students/<YOUR USERNAME>/<PATH TO YOUR SIMPLESCALAR DIRECTORY>:'$THISDIR':' ./Makefile > specmake
rm Makefile
mv specmake Makefile

echo Compile SPEC
make

5) cd /home/username/simtmp
Make sure the two packages (simplesim & SPEC2000int) are downloaded here. Don't extract them!
6) chmod +x setup.sh
7) ./setup.sh

That's it. The setup file will extract, patch & compile the programs for you. The result of the spec2000int benchmark will be in a directory called Results in the etderr files.

Spreadsheets Maximum number of Rows

I had a spreadsheet (Excel sheet for you MS Office users) with over 253,000 entries. Since I run on Linux, I use OpenOffice.org's SpreadSheet program, which unfortunately supports a maximum of 65336 rows only.

A quick search got me to a comparison between multiple products that support more than that limit. OK, I lied. Two products only.

This Wikipedia page shows a nice table but people tend to overlook the tiny little IMPORTANT numbers used for special notes!!!

The products are: Microsoft Office 2007 and Gnumeric.

Unfortunately (yes, another unfortunate event), Gnumeric requires that you recompile it with max number of rows you desire. This can be a good thing if you look at it from the point that you can surpass the limit that Microsoft is boasting.

In all cases, opening such large sheets requires large memory. I'm guessing at least 1GB (for the data itself, not counting the program!). I'll test this claim on MS Office 2007.

Winter and Computer Hardware

As the cold sneaks under our blouses and trousers, we keep stocking ourselves with more clothes, and the more we put on, the more charged we get. Static Electricity is NOT your friend!

In case you didn't know, handling computer hardware while being charged with electricity will discharge the electricity into the piece you hold and most likely rendering it a paperweight. Be it RAM, CPU, Graphics Card, or Hard disk drive, unless you hold a piece of metal before touching the piece, you're dooming it!

You should also do the same before touching the fuel hose at the gas station. As much as it's fun to see an awkwardly dancing flaming show, the after-show smell is just obnoxious...

IT Consultant

As tempting as it was, I didn't open the email.

Zain e-Go 180

The new device reported by this blog, is Huawei's E180.

The older model was being sold for 60KD, while the new one is being sold for 45KD.

Here's my advice: If you already own the old model, stick to it as there's no need to get a newer one.

The so called "speed" of 14.4Mbps, is the network speed, not the Internet speed. The network here is the communication between your mobile device (the E180) and Zain's communication tower.

People with the older devices, that have a speed of 7.2 Mbps, would never get 7.2 Mbps of Internet speed, because again, that's the network speed only.

Also, the E180's link above doesn't mention anything about supporting 14.4 Mbps, as the blog claims (or Zain claims).

Plug & Play option is great. It also exists in the older model.

Support for Windows & Mac is great. The older has the same support, and it also supports Linux. I'm using mine on Kubuntu.

Choosing A Sub-notebook

My sister has been whining that the Lenovo laptop I got her is too big for her to carry around in college. Since she wanted something just to type word documents, do presentations and chat with, I showed her the Asus Eee PC models, and she loved them.

The available models were 7", 8.9" and 10" and luckily they sell these in Kuwait - Hawalli area - Ibn Khaldoun street. I took her so that she can see and relate to the sizes. The 10" was "big" so the debate was whether it's 7" or 8.9".

The difference between the 7" & 8.9" is the screen size and the disk capacity, but the dimensions of both are the same!

Linux or Windows?
Now, for a sub-notebook, you have very limited resources: 512MB-1GB of RAM, 8-30GB of disk space. And the choice of the Operating System makes a VERY big difference.

If you choose to go with the Windows model, you get Windows only, without Office and without an Anti-virus (which is a must). Office 2007 would kill the poor machine, and an anti-virus would make it unbearably slow.

So it's obvious that Linux is the proper choice for such small devices, and guess what? It does come with a set of applications pre-installed, including OpenOffice.org from which you can save as MS Word 2003 format, or export to PDF.

This doesn't apply to Asus only. For any device with limited resources (sub-notebook, PDA, tablet, Smartphone, Embedded devices) I hardly see how Windows would ever be a feasible solution.

Javascript Menu Fall Behind Flash: Fixed!

I used to face a problem before when browsing websites with flash animations under a menu, when the mouse goes over a menu item, the sub-menu drops behind the flash animation, and I can't click anything.

I thought it was caused by poor design of the website, where they could've used the Z-Index property in Cascading Style Sheets.

According to this page, z-index wouldn't even solve it, and the problem was actually caused by Flash Player 9 on Linux (only).

I grabbed a copy of Flash Player 10 and installed it, and now everything works as it should!

I run Kubuntu 8.04 and installed the package manually. Download the ".deb" package then run this command:

sudo dpkg --install install_flash_player_10_linux.deb

After the installation is over, restart firefox and the new flash player will be loaded.

Linksys WAP54G Access Point

I got a Linksys WAP54G AP and wanted to configure it for the first time.

The place where I want to deploy this AP has no DHCP server, and each PC has its own IP address assigned manually. This AP would replace a previous malfunctioning Linksys AP (same model).

Took the AP out of its package, hooked it with power and connected the supplied network cable to a PC, only to find out that the software that comes on the CD is telling me that I should connect it to a switch/router!

1) What if I want to deploy ONLY wireless using an ACCESS POINT?! NOT ALLOWED? MUST I HAVE A SWITCH? HMMM????

2) Their crap "auto-configure zomg pink ponies emo" software is for Windows only, and the manual on the "CD" (50 lines under "CD") is just the "quick" startup guide.

I had to drive back from Salmiya to industrial Ardhiya just to get to my office and download the FULL MANUAL, which is 2.2MB ONLY. COULDN'T YOU INCLUDE THAT IN THE CD??????!!!!!!

The "secret" default IP of the AP is: 192.168.1.245 -- I assumed it was 192.168.1.1, 192.168.0.1, 192.168.1.254, but NOOOO Linksys has to screw you up somehow...

Oh, and in the manual, they say that the default user is "admin" and password is "admin" -- only that doesn't work. You just need the password, and leave the user blank.

While reading, a phrase almost pushed me in a time wormhole: "Launch Internet Explorer or Netscape Navigator." -- Netscape???!!! ........

After deploying the crappy NEW AP, the signal was "Very Good" only... and sometimes the AP doesn't even show up and the connection gets dropped...

From now on I shall call these: LimpSys!

Model: WAP54G
Version: 3.1
Serial Number: MDG30Gxxxxxx and MDG30Hxxxxxx

Death of The Floppy Drive

Or not!

Update [Mar 7 2009]: Advanced timings corrected
Update [Feb 14 2009]: I finally got the machine to stabilize. The shop had given me 2 kits of different voltages and Corsair's support helped me tweak the box to stability. Muchos Gracias Corsair!

I got a new machine, with one of the extreme motherboards: EVGA nVidia 790i Ultra SLI MCP. It supports many features, including a built-in RAID controller, which is the focus of this post.

I assembled the box, and configured the BIOS to enable the RAID controller on two SATA ports. I reboot, configure a RAID 0 array, reboot again and insert the Windows XP Pro 64-bit with SP2 CD, and it couldn't detect the RAID array that I had just created.

Reading about this problem a few months back, I remember that I need a floppy drive (which I didn't buy for this new box) and a floppy diskette. And another computer, with Windows on it, to use the RAID floppy maker tool from the motherboard's CD.
I managed to salvage a floppy drive from another machine, which also happens to have a Windows partition (luckily). I copy the tool to it and use an old driver diskette. The tool formats the diskette and puts the drivers on it. Great, I'm good to go now.

Well, not quite. Windows Setup didn't see the drivers on the floppy, for some reason. So I take the floppy to the other machine, format it from Windows, and then copy the drivers to it manually (there was a directory of the drivers extracted). Windows happily saw the new drivers.

After installing Windows, I installed the drivers on the box from the mobo's CD. Then I install the driver updates which I had downloaded from the Internet earlier during assembly. The update tool detected that there were previous drivers, and the new ones will be installed on the next boot, and it rebooted, but never booted!
The "smart" update tool deleted the old drivers, which included the RAID drivers, rendering my array inaccessible. The only way to fix this is by booting to Windows Setup and run a repair on the existing partition, which I did. The Setup froze during file copying. I boot again to Setup and this time run a new installation.

There are two parties to blame here: EVGA, the motherboard manufacturer, and Microsoft, the Operating System maker.
1) Microsoft: Seriously, I'm using the latest ORIGINAL Windows CD, why the hell won't you update the Setup program to allow mass storage devices like USB Memory sticks? THIS IS 2008! 1995 Says hi!!!

2) EVGA: You soar losers! You could've included a floppy with the WORKING drivers on them! AND put a warning (even if tiny) on the cover that a floppy drive is required if RAID is to be used!
What if this was my first machine to be built? How the hell am I supposed to make the RAID floppy?!

And in case you're wondering, no, I didn't get to install Windows successfully yet (with drivers). Whenever I install the drivers from the CD, it reboots & the array doesn't boot anymore. 3 failed installations, 1 failed repair due to freeze & another failed repair due to driver installation.
I'm attempting a different approach: Create a striped array of ONE disk, install OS, then migrate using the software tool from the CD (read that it's possible from the nVidia RAID Guide).
I'll update on the result.

P.S.: The floppy drive is not dead, yet. You can thank Microsoft for that.

update:
It finally worked!
1) Create bootable stripe array of one disk ONLY
2) Install OS to disk, using drivers from floppy -- reboot
3) Install drivers & programs from CD -- reboot
4) Install downloaded updated drivers and programs -- reboot
5) Use MediaShield program to migrate your array to a 2nd stripe disk (will take time -- DO NOT REBOOT!)

I had lockup issues where Windows would freeze randomly. After doing some search, I ended up updating the BIOS (from EVGA website) to P07, but the RAMs caused some lockups too, so I had to keep the clock settings to "auto", while manually setting the memory FSB to 1333MHz (it was detected as 1066) -- after consulting with Corsair, I've manually set the memory frequency, voltage, timing and SPP voltage.

Using 2 different kits of Corsair CM3X2048-1333C9DHX. The printed timings are: 9-9-9-24, but the voltage on the version 3.2 kit is 1.7v while the 1.2 version had its voltage rate set as 1.6v.

Stable settings: Confirmed on Feb 14 2009
Stable timings at 1066MHz are: 9-9-9-24 (2T)
Advanced memory settings (auto):

tRRD = 4
tRC = 27
tWR = 10
tWTR = 14
tFAW = 17
tREF = 7.7us
tRFC = 59

Voltage rate for SPP & Memory:

SPP = 1.40v
Memory = 1.80v

According to Corsair's support: Both memory kits are warranted up to 2.0 volts, so no need to worry. Also, Corsair support asked me to set the SPP voltage to +0.2 the default value, which is 1.5v but I tried 1.4 first and it was working fine, so I didn't try 1.5v.

BIOS flashing utility info:

For 132-CK-NF79-6A61MB02C-00     Date: 26/6/2008
Flash type: Winbond W39V080A     Binary file name: nf79_p07.bin

After flashing new BIOS, when powering on, BIOS info:

Pheonix - AwardBIOS v6.00PG
6A61MB02  Release number 811N2P07
nVidia BIOS Version: 2.053.E8

MediaShield ROM info:

MediaShield ROM BIOS 9.85

Mesh Networks and Traffic Jams: A Double-Edged Solution

Intro
Being stuck in traffic jam is horrible. Being stuck in traffic jam with only a radio & nothing to listen to but rap music, is terrifying!

What if you could see traffic flow in real-time, before you leave and while driving? Sounds great, right? But what if you could see traffic flow AND be able to surf the Internet using the same inexpensive solution? THAT'S FREAKIN' AWESOME!

Background
Let's get realistic: In the United States, it's possible to get traffic reports using GPS units. This is done by attaching an antenna to receive radio transmissions from the respective service provider (either for free or by subscription).

This solution has been debated to not provide information security, as the FM radio transmissions can be spoofed and forged, producing false reports. There's also the monthly fee you have to pay to the service provider.

Proposed Solution
What I propose here, is creating a distributed mesh network, which can be tracked in real-time and provides Internet access at the same time. Each node is a mini-router that is configured to search for similar nodes of the same network, forming a bond and a redundant, anonymous, decentralized network.

Real-Life Application
Deploy a mini-router from Open-Mesh.com in each car and let it be pre-configured to join a network of the name "adrenalin" -- And let there be edge nodes (houses/flats/shops) with also a mini-router, but connected to the Internet.

Assume you're driving on an empty road and no one is in sight. Your router will not be able to connect with any node since none exist, and thus your path is free and there exist no traffic jam! (And unfortunately no Internet :( )

If you're driving in the city, there will be both cars and edge-nodes, hence your router will connect to one of these and you'll be able to see a topology representing the scatter of cars around, and you can determine congested routes and steer away from it, while at the same time, you gain the ability to access the Internet, thanks to the edge-nodes.

And guess what, the more jammed the traffic is, the more bandwidth is available for you to surf the Internet!

Open-Mesh.com provides a web-interface to see and manage all these nodes, and there exist projects to further enhance the application of these networks.
Go to the Network Status page and enter the Network Name as "test" (without double quotes). Leave the password field empty, and click sign in.

All mobile nodes (cars) are anonymous, and the edge-nodes can choose to be anonymous as well, protecting the privacy of people.

Solution Feasibility: Technicalities
* Each router has 2 interfaces: A public one and a private one. This means while your router is hooked to other routers, you connect to your own only, using a laptop or a PDA. The router has built-in firewall and encryption.

* The router runs on 5 volts only, and any car battery can handle it easily.

* If you have a computer in your car (carputer), you can hook it through LAN.

* The antenna is detachable, so it's easy to upgrade or relocate it.

Solution Feasibility: Mass Deployment
Let's assume this is a government sponsored project. When buying a new car, the person pays a subsidized sum to implement the mini-router along with the license plate fees.

According to open-mesh.com, a pack of 20 routers cost $799; that's $39.95 per mini-router (almost 11KD). The government could sell it for 4KD, and 1KD fee for the license plate, so the total is 5KD!

Local ISPs can join the party by offering free edge-nodes that allow Internet access. They make profit by displaying advertisements in a frame at top of the page. The profit made can be split with the government, to make up for the cost of the devices.

KFH Scam Sites

I was checking my ancient Hotmail (Windows Live) account and found an email in the "junk folder" from Kuwait Finance House (KFH) asking me to update my information. Well, the thing is that I'm not a KFH customer to begin with!

Here's a screenshot of how the email looked:


The registration link points at a scam website: kffhonline.com
A quick whois shows that the site is not related to the legitimate KFH:

Legitimate KFH records:

   Administrative Contact, Technical Contact:
      Finance House, Kuwait             ebusiness@KFH.COM
      Kuwait Finance House
      Kuwait Finance House -Almorgab
      Almorgab KW 24989
      KW
      +965 2439211 fax: +965-2448107


   Record expires on 07-Mar-2010.
   Record created on 06-Mar-1996.
   Database last updated on 6-Sep-2008 10:54:56 EDT.

   Domain servers in listed order:

   DNS.KFH.COM                  168.187.220.2
   DNS2.KFH.COM                 195.39.157.3

Scam KFH site:

Domain name: kffhonline.com

Administrative Contact:
   Whois Privacy Protection Service, Inc.
   Whois Agent (rhbxhmps@whoisprivacyprotect.com)
   +1.4252740657
   Fax: +1.4256960234
   PMB 368, 14150 NE 20th St - F1
   C/O kffhonline.com
   Bellevue, WA 98007
   US

These are the email headers:
X-Message-Delivery: Vj0zLjQuMDt1cz0wO2w9MDthPTA=
X-Message-Status: n:0
X-SID-PRA: KFH ONLINE
X-Message-Info: bKPJ5fID7nvr4q44yl8SAb7jxAnprY7AZ6DqJRTR/ubhk5PinpoWw0lC+PS7sSN+C7H+SA5wb6lYG+N+qpiX3w==
Received: from metrostone.net ([64.16.167.205]) by bay0-mc3-f9.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.2668);
Fri, 5 Sep 2008 21:30:05 -0700
Received: from User ([24.74.153.158]) by metrostone.net with Microsoft SMTPSVC(6.0.3790.3959);
Fri, 5 Sep 2008 17:19:25 -0400
From: "KFH ONLINE"
Subject: Update Your Records Please
Date: Fri, 5 Sep 2008 17:19:25 -0400
MIME-Version: 1.0
Content-Type: text/html;
charset="Windows-1251"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
Bcc:
Return-Path: kfh@online.kfh.com
Message-ID:
X-OriginalArrivalTime: 05 Sep 2008 21:19:25.0567 (UTC) FILETIME=[12C0B0F0:01C90F9D]


Firefox users already get a warning that this site is a scam site:


Also, if you change the link to provoke an incorrect link, you get a crap page like this:



I'm surprised they're targeting Arabs now, after seeing the eBay and PayPal scam emails.

If you're an admin for an email server, you can block the IP of the sender as shown in the headers, or if you wanna pick it up higher, deny the whole subnet that the host has reserved.

Extracting SMS Messages from Nokia NBU Backup File

I downloaded the latest Nokia PC Suite software for my ailing Nokia Communicator 9500, to backup everything, and to be able to read the SMS messages on the PC and delete the 1000+ SMS messages I have in my mobile...

After taking a full backup of everything, then another backup of SMS, Calendar and Contacts only, and while the phone was connected to the PC, I was able to view the messages, which made me think that the software downloaded them to the PC, so I proceeded to deleting some of them. After deleting about 200 or so, and having the mobile disconnected from the PC, I wasn't able to read them from the PC!!!
Apparently, the software was reading directly from the mobile and it had kept no copy, other than the backups that I took!

After 5 minutes of a panic attack followed by curses, I decided to restore the SMS messages from the backup to the phone. The software was kind enough to give me the choice to restore the SMS messages only, and not shove everything into my face, or the mobile's. The process took time, and the existing messages were not duplicated! But the process never finished; it got stuck at the end, so I canceled the operation but the messages were back.

I started looking for software to extract the messages from the .NBU backup file, which the Nokia software produced, since they're too retarded to present the option.
I was able to find only one piece of software that is both free and works! It's also able to extract the messages into one file (text, HTML, CSV, PDF, and more), or each message in a file. And to top it all, it can produce a table of contents of the messages!

The software's name is: ABC Amber NBU Converter. Here's a page with some description, but I don't know the software's original site, so I'm not sure if the linked page's file is safe from viruses. Make sure to scan it.

I'm glad Nokia didn't include the option to download the SMS messages to the PC, because it gave me yet another reason why I should dump Nokia altogether, and to find this beautiful piece of software.

Kudos to its creator(s).

Microsoft Windows XP End-User License Agreement

For the sake of amusement, I decided to read Microsoft's Windows XP Professional End User License Agreement.

I have read the EULA and decided to write some comments, or just highlight some points.

Head start:

YOU AGREE TO BE BOUND BY THE TERMS OF THIS
EULA BY INSTALLING, COPYING, OR OTHERWISE USING THE PRODUCT. IF YOU DO NOT AGREE, DO
NOT INSTALL OR USE THE PRODUCT; YOU MAY RETURN IT TO YOUR PLACE OF PURCHASE FOR A
FULL REFUND.

• Right. So, if you don't agree with ANY of the points, you're not allowed to use this piece of software.

---

Without prejudice to any other rights, Microsoft may cancel this EULA if you do not abide by the terms and conditions of this EULA, in which case you must destroy all copies of the Product and all of its component parts.

• I would like to point out that there's no mention of the word "refund."

---

The Product may not be used by more than two (2) processors at any one time on any single Workstation Computer.

• There's no mention of the word "cores," so by processor, they mean a physical chip. And God forbid you may have a dual processor motherboard and utilize both! I have seen some XP licenses that are valid for multiple CPUs. I don't know if they're referring to the whole computer or just a processor.

---

You may permit a maximum of ten (10) computers or other electronic devices (each a "Device") to connect to the Workstation Computer to utilize the services of the Product solely for File and Print services, Internet Information Services, and remote access (including connection sharing and telephony services). The ten connection maximum includes any indirect connections made through "multiplexing" or other software or hardware which pools or aggregates connections.

• So if you live in a dorm, you can have only 10 friends. If you get naughty and make more than 10 friens, they'll be upset to know that you can't permit them to access your fancy collections over the network. And if you think that you're smart enough to have the computers pass through a single gateway to appear as a single computer, then you're performing an illegal operation, my naughty friend. Microsoft shall spank you!

---

Except as otherwise permitted by the NetMeeting, Remote Assistance, and Remote Desktop features described below, you may not use the Product to permit any Device to use, access, display or run other executable software residing on the Workstation Computer, nor may you permit any Device to use, access, display, or run the Product or Product's user interface, unless the Device has a separate license for the Product.

• Is this still "my" computer?!

---

There are technological measures in this Product that are designed to prevent unlicensed or illegal use of the Product. You agree that we may use those measures.

• It's within their right to protect their software, but wait till you get to the good part of this deal, below.

---

Microsoft reserves all rights not expressly granted to you in this EULA.

• In other words: "All your base are belong to us!"

---

After upgrading, you may no longer use the product that formed the basis for your upgrade eligibility.

• So if I upgrade from Windows98 to XP, I'm no longer allowed to use Win98, which I paid for? Delightful.
  
• If I had to format, God forbid, how am I supposed to install XP without the right to install the base OS (Win98)?! (unless install doesn't count as using)

---

You may move the Product to a different Workstation Computer. After the transfer, you must completely remove the Product from the former Workstation Computer.

• I thought it wasn't allowed. Good to know it is.

---

You may not rent, lease, lend or provide commercial hosting services to third parties with the Product.

• To all of you little kids who had dreams of starting a tiny web-hosting service, now that you have an OS with a graphical interface and all, sorry to say that it's not going to happen! HAHA! In your face!

---

You agree that Microsoft and its affiliates may collect and use technical information gathered in any manner as part of the product support services provided to you, if any, related to the Product.

• "in any manner" -- yup! They lived to that promise alright! But I have a hunch we'll be reading about yet more exciting new ways :D

---

You acknowledge and agree that Microsoft may automatically check the version of the Product and/or its components that you are utilizing and may provide upgrades or fixes to the Product that will be automatically downloaded to your Workstation Computer.

• Does this mean, that by turning off the AutoUpdate service, I've violated the EULA?!
  
• Does this mean, that Microsoft can check my system, even when I have AutoUpdate turned off and never visited their update site/portal?!
  
• What if I don't want your fixes and updates (that might break certain parts of MY system)?

---

Content providers are using the digital rights management technology (“Microsoft DRM”) contained in this Product to protect the integrity of their content (“Secure Content”) so that their intellectual property, including copyright, in such content is not misappropriated. Owners of such Secure Content (“Secure Content Owners”) may, from time to time, request Microsoft to provide security related updates to the Microsoft DRM components of the Product (“Security Updates”) that may affect your ability to copy, display and/or play Secure Content through Microsoft software or third party applications that utilize Microsoft DRM. You therefore agree that, if you elect to download a license from the Internet which enables your use of Secure Content, Microsoft may, in conjunction with such license, also download onto your computer such Security Updates that a Secure Content Owner has requested that Microsoft distribute.

• Oh? So at some point of time, the "Secure content" which I have paid for, could be rendered useless, because the provider felt like it's not secure enough?
  
• I don't see the word "refund" in the text
  
• This sounds more like Digital Restrictions Management

---

NOT FOR RESALE SOFTWARE. Product identified as “Not for Resale” or “NFR,” may not be resold,
transferred or used for any purpose other than demonstration, test or evaluation.

• Hmmm... The original CD sitting in front me of Microsoft Windows Server 2003 R2 Enterprise Edition, has the following on it: "Not for retail or OEM Distribution. Not for resale." -- does this mean I may not use it in production environment and only for "demonstration, test or evaluation"?! I sure have paid for a production environment copy!
  Either the shipping elves sent me the wrong CD, or the EULA is mistaken...

---

11. LIMITED WARRANTY FOR PRODUCT ACQUIRED IN THE US AND CANADA.
Microsoft warrants that the Product will perform substantially in accordance with the accompanying materials for a period of ninety days from the date of receipt.
...
Any supplements or updates to the Product, including without limitation, any (if any) service packs or hot fixes provided to you after the expiration of the ninety day Limited Warranty period are not covered by any warranty or condition, express, implied or statutory.
...
This Limited Warranty is void if failure of the Product has resulted from accident, abuse, misapplication, abnormal use or a virus.
...
Microsoft’s and its suppliers’ entire liability and your exclusive remedy shall be, at Microsoft’s option from time to time exercised subject to applicable law, (a) return of the price paid (if any) for the Product, or (b) repair or replacement of the Product, that does not meet this Limited Warranty and that is returned to Microsoft with a copy of your receipt.
...
Outside the United States or Canada, neither these remedies nor any product support services offered by Microsoft are available without proof of purchase from an authorized international source.

• Microsoft guarantees you its Operating System (OS) for 3 months only, because God knows what kind of viruses might be unleashed to masses to hijack their precious Windows-based machines after 3 months. Microsoft surely can't be held liable for that! Nor its poorly designed and unsecure OS!
  
• Oh! Goody! Even their bug fixes aren't guaranteed! Fantabulous!
  
• You're a financial controller at a big firm and you're almost done from the year's financial records to close it down, and to your misfortune, got infected by a Trojan Horse which exploited a security hole in Microsoft Windows Firewall. This not-so-friendly Trojan has corrupted your precious OS and you no longer have access to your files. Your company doesn't close on time and falls into financial problems. Who's fault is this? Who's responsible to pay for the damages?
  
• Following the scenario above, you may not be so lucky to be granted Microsoft's love and get your $185 (imagine loss in millions). Or even better, they will give you another CD of the same OS! Yay! Oh .. I forgot to mention that you'll have to pay for shipping the CD you have to them first...
  
• What about the 3 months period? How come it's not available to those outside US & Canada?!

---

Except for the Limited Warranty and to the maximum extent permitted by applicable law, Microsoft and its suppliers provide the Product and support services (if any) AS IS AND WITH ALL FAULTS, and hereby disclaim all other warranties and conditions, either express, implied or statutory, including, but not limited to, any (if any) implied warranties, duties or conditions of merchantability, of fitness for a particular purpose, of reliability or availability, of accuracy or completeness of responses, of results, of workmanlike effort, of lack of viruses, and of lack of negligence, all with regard to the Product, and the provision of or failure to provide support or other services, information, software, and related content through the Product or otherwise arising out of the use of the Product. ALSO, THERE IS NO WARRANTY OR CONDITION OF TITLE, QUIET ENJOYMENT, QUIET POSSESSION, CORRESPONDENCE TO
DESCRIPTION OR NON-INFRINGEMENT WITH REGARD TO THE PRODUCT.

• It speaks for itself...

---

After these points, I don't know how you feel about your purchased copy of Windows, but it feels to me that I'm renting the software with a gun to my head, and my face glued with an awkward smile... (Glue made Microsoft, of course).

Freezing Internet Account: QualityNet: Not Possible (almost)

Updated: July 13th

I'm moving to a new place, and I had renewed my account one month before I found out that we're moving.

When I no longer was using the DSL at the old place, I called QualityNet support at 804444, and told them that I need to freeze my account and shift the service later to another phone line.
The representative said that it's not possible, but he was kind enough and logged the issue in the system for his superior to look at it and follow it up. He said that I'll be contacted with in 2 days.

I called the next day, just to be sure and make a reminder (since I'm losing days here), and they assured me that the problem was logged and I'll be contacted.

Indeed, the day after, I was contacted and spoken to a woman, who said that what I'm asking for is not possible. I argued that it's not logical and that I've been subscribed with them since August 2005! I then asked on whether it's possible to get a refund, but that wasn't possible either.
After some thinking, I asked if it's possible to transfer the amount available in my balance to another account; that is, convert the number of days that I have left in my service into money, then deposit it into another account.
She said she'll ask and see if that is plausible or not. The same girl called later and said it was OK, but I'd have to go over to their HQ and ask them to do it there.

Although we are able to come to a solution, I don't understand why it's not possible to freeze the account and hold the amount in it. I do know that the ISPs rent space from the Ministry of Communication where they install jumpers on the phone line, which is then calculated as part of the cost and the customer ends up paying for it.
This does not apply here, since the jumper is going to removed anyway (I'm moving out, remember?), so there's no cost!

If you're a QualityNet user or a staff member, I ask you to make a fuss about this, because it's ridiculous not be able to shift locations.

Unless they want us to renew month by month, just in case, and they benefit from the increase in price!

Update: I have went to QualityNet's HQ and the clerk suggested I ask an employee regarding freezing the account, and he said that such a thing is done as a courtesy only, and the maximum period is 1 month. It wasn't enough in my case, so I shifted the cash to another account.
When shifting the cash, I had to pay a certain amount so that the total covers a month, or a multiple. (not counted in days of subscription!)

Compiling AVIDemux on 64-bit Linux distro

AVIDemux is the best tool to rip DVDs, re-encode videos and edit them on the fly.

Sometimes it's a pain in the neck to compile stuff, especially on a 64-bit distro that has 32-bit compatibility libraries. Thankfully, guys at the Slamd64 forums are quite helpful in these matters.

I had an old copy of avidemux (version 2.3.0) which I worked on. I tried compiling the latest (version 2.4.1) but it requires "cmake" which I didn't have install, and didn't feel like going through the headache of finding its dependencies.

magic lines:

export LDFLAGS="-L/usr/lib64"
CFLAGS="-O2 -fPIC" ./configure --prefix=/usr \
--sysconfdir=/etc --libdir=/lib64 --includedir=/usr/include \
--with-jsapi-include=/usr/include/seamonkey-1.0.7/js/ --with-include-gettext
make -j4
make install

Without exporting LDFLAGS properly, you'll get an error like this:

/usr/lib/libXext.so: could not read symbols: File in wrong format

It was picking up the 32-bit lib instead of the 64-bit one. Exporting LDFLAGS properly solved this issue.

If you get something like this:

rm -f cs.gmo && : -c --statistics -o cs.gmo cs.po
mv: cannot stat `t-cs.gmo': No such file or directory

Make sure you have this in your configure line:

--with-include-gettext

* Found this in a RedHat archived text from 2004.

Compiling MPlayer on 64-bit Linux Distro

I had one H264 video which prevented the location bar to move and at some point, the audio would lead the video, and if I attempt to skip that part, it would play from start.

I thought it was Xine's fault for not being able to run the file properly, so I check whether there's a newer version or not, and there was so I updated, and Xine stopped working. No big deal, I had mplayer, but unfortunately it had the same old problems as Xine with that file.

Some guys suggested that the version I had of the H264 codec was compiled for a 32-bit distro, and it doesn't play well under 64-bit ones. That could be fixed with a recompile.

After some poking around at Slamd64 forums, I found some things that can help and this configure line made all the magic happen:

CFLAGS="-O2 -fPIC" ./configure --confdir=/etc --prefix=/usr --libdir=/usr/lib64 \
--with-extralibdir=/usr/lib --enable-gui --enable-largefiles \
--win32codecsdir=/usr/lib/codecs --codecsdir=/usr/lib/codecs \
--xanimcodecsdir=/usr/lib/codecs --realcodecsdir=/usr/lib/codecs

MPlayer was compiled fine and worked fine, but I still have problems with that bloody video file. I tried re-encoding it using avidemux on my laptop (32-bit Kubuntu), but avidemux kept crashing.

Oh well, MPlayer works fine now, at least.

Finding The Fastest Host

I play on Server 3 of Travian UAE, and the domain name s3.travian.ae points at 19 different hosts, and most are slow for me.

The lines below show how to get the server with the lowest average ping time. This assumes the servers will reply to ping requests (only one doesn't), and that you're not blocking ping requests from your end.

for i in $(host s3.travian.ae | awk '{print $4}'); do echo -n "$i:" >> trav.txt \
&& ping -qc5 $i | tail -n1 | cut -d/ -f5 >> trav.txt; done
sort -t: -k2 trav.txt | head -n1
rm trav.txt

Put the result in /etc/hosts and you're all set! This should save you time when querying for DNS, and wouldn't depend on your luck at which server you get (even though Travian employ round robin)

* This is not to be used with any website, since most have MX records (mail server records) and the script above doesn't work in such cases.

* You can remove the backslashes above and write the whole thing in one line. I put on multiple lines to make it readable.

The script is no longer useful since they moved to one server now. Seems like they distribute to multiples during registration period.

NFS Mapping and Mass Group Ownership Change

Right to the point: 2 Unix (AIX 5.3L) servers with NFS directory one needs to be mounted on the other. The catch? Both must have the same user & group, and both the user & group must have the same IDs.

Since the target server (NFS Client) must stay intact, I had to change the group ID (GID) on the other box, then apply the new ID to all the files & directories on the system.

I changed the GID from old to new using smitty, AIX's administration interface utility. Then I applied the following script to all files & directories on the NFS Server:

find / -group 206 -exec chgrp 320 {} \;

"find" will look for all files & directories with the old GID, which is 206*, and when found, it will run "chgrp" to change its group to 320.

* Since there is no longer a group name associated with the old GID, it shows as a number when "ls"

chgrp can take the group name: chgrp "groupname" -- but I used the numerical value anyway.

P.S.: I had a moronic idea of running chgrp recursively, that is:

find / -group 206 -exec chgrp -R 320 {} \;

Thank God, I had a wakeup call before running that, because if I did, it would change the GID of ALL files & directories under a directory that matched the old GID, so this would've changed directories & files that might have had a different GID.

After all that, mounting NFS worked like charm & I no longer had mapping issues. Mounting was done as:

mount blade6:/path/to/dir /path/to/dir

P.S.: mounting the directory, then running chown on it is a dumb idea, because it would change the GID on the NFS Server.

RAM and BIOS Upgrade

I've been contemplating buying new RAMs for months and been looking for the proper brand, price, capacity & speed.

My machine, code name: Adrenalin, has 2x 1GB of Kingston RAM; they're beautiful, fast & do the job, but they're just not quite enough for my kind of [ab]usage.

I tried buying from NewEgg.com and TigerDirect.com, and as newegg was clear that they won't accept VISA, AFTER registering & reaching the payment page, TigerDirect.com failed at it, and not only that, my questions via email went unanswered for days and when got a response, it was totally irrelevant to my questions!!! The only quick response I got from them, is when I emailed them to cancel my order, in which they did within 2 hours!

Luckily, I found the same piece I was looking for on Amazon, at the same price of TigerDirect; In fact, Amazon gets the item from TigerDirect themselves!

I was after Corsair XMS2 DHX 2x 2GB kit RAM, and though the CAS latency is higher than the Kingston that I had, it wouldn't make much of a difference to me.
This piece of RAM is absolutely amazing: Not only do you get high-grade RAMs, you also get an amazing heat-sink design and a life-time warranty!

I ordered 2 kits, totalling in 8GB of RAM to fill all my 4 DIMMs, and have been waiting for them to arrive for 2 weeks, and today I got them, went back home and started the upgrade ritual.

0) Open a shell and write down the uptime: "up 42 days, 10:52"
1) Download memtest+ ISO file & burnt it
2) Shutdown machine & keep power cord connected
3) Clean dust & wash fan filter
4) Open case and touch internal metal body to discharge static electricity
5) Remove old RAMs & grab a pocket-knife & start cutting the cover of new RAMs before checking if it can be opened by hand
6) Open RAM cover by hand
7) Enjoy the beauty & smell the fresh scent of the hardware pieces
8) Anxiously & carefully install new RAMs
9) Power up & enter BIOS to make sure correct values for RAM are detected
A) insert memtest+ media to thoroughly test all RAMs -- passed

After that, I booted into Windows (XP 64-bit) to check for BIOS updates, which I found, and downloaded manually after the ASUSupdate program failed to download the needed file.
It was straight forward: Open AsusUpdate & choose "Update From File" then select the BIOS .BIN file and click away. Almost.

There was an option to reset the BIOS settings to the factory default after flashing the new BIOS version, and since I didn't want to go through the whole configuration process, I unselected it and flashed the new BIOS. After rebooting, the machine wouldn't start. PANIC ATTACK!
I thought the BIOS is being flashed after a reboot, which is not what used to happen before, and waited for 2 minutes, yet nothing happened, so I shutdown & powered up again, and nothing happened.

I run to my sister's room to find that she's not there, but her laptop is; PERFECT! I snag it and surf away to Asus's website and grab the manual, then I find that I'm downloading the Chinese manual, so I cancel & hunt for the English manual.
After the manual downloaded & unpacked, the BIOS has a recovery mode for cases where flashing the BIOS goes wrong (EXCELLENT!) -- the problem is that the screenshots assume that the machine would boot and I'd be able to see text on the monitor, which I didn't. Failure.

I go back to my room, calculating the cost of sending the motherboard to Asus and the time I'd have to spend without my baby -- then I remembered the option which I unselected.
I remove the BIOS battery & unplug the power for a minute, then put the battery back & hook the power. Power up. IT'S ALIVE!
Seems like the nwe BIOS wasn't fond of the old BIOS's settings! Anyway, I configure my stuff again and all goes smooth.

Lessons learned: None.

-> Pictures.

Internet Shopping: Dreaming of Things You Will Never Purchase

The Internet provided means of reaching people without boundaries, beyond borders and resulted in a huge expansion in International freight & logistics services expansion.

However, post 9/11 events, the US & EU have "tightened security", along with credit card companies, and made everyone outside their region miserable, by requireing ridiculous set of rules that reduced (if not prevented) International trade over the Internet.

Many great websites like NewEgg.com & TigerDirect.com wee considered the best places to purchase computer hardware & accessories, because of the low prices compared to the local market (assuming the local market had it in the first place) and because of the wide variety of items, that they offer. Unfortunately, due to the reasons mentioned, those websites can't sell to anyone unless they verify his/her identity, by requesting name, VISA card, social security number & residence, and all this info must match that registered on the VISA, otherwise, you're considered a fraudster and denied transaction (if not reported to VISA & police as well!!)

The problem with such websites (I had personal experience with them), is that they do not mention that they accept American credit cards or paypal accounts only, and people like me end up going through the whole search-purchase-wait-disappointment process. It would help a lot, if they would mention that fact in their FAQ or at least in the payment page... It would save us the time and look for other sites, and them the time to attend to useless orders & emails.

In the past 2 years, our local market has become a monopoly where most shops agree on a certain price for all items, and the victim is the consumer, and the only way out is to purchase from the Internet, and now that famous websites are being locked out, we're forced to turn to non-famous websites that may in fact be a total rip-off. So how did the credit card companies reduce fraud? I have no idea.

So, where do I get my stuff from? Mostly Amazon.com, which for some reason, does accept International VISA and doesn't enforce ridiculous rules on the buyers.
The problem? They're not a computer house, and even though have some things, it's just not the place for the computer-holic to get his/her dosage of hardware & accessories, not to mention price differences, as they tend to be higher than websites like newegg.

If any of you have a way to purchase things from the US or EU without pleading & begging in emails to companies, please let me know!

Data Recovery: Hard Disk Surgery

A friend of mine brought a disk that wouldn't work when connected to a machine.

As soon as I hooked the disk to my box, it restarted and refused to start, with all lights blinking madly. Seems like a short-circuit caused that to heappen, so I replaced his case and put the disk in my case and this time: No short-circuit.

The OS booted, but running "lsusb" command showed nothing on the USB bus. When I replaced the case, the pins of his disk interface were a bit bent and I had to straighten them to be able to hook my case's PATA-to-USB (PATA = Parallel ATA) interface.

So, I thought, maybe the controller board itself, which has the PATA interface is damaged somewhere. Lucky for him, I had a disk of the same brand & model number, so I took mine out and put it on his disk. After doing that, I hooked it to the USB port on my machine, and still nothing on the USB bus.

After all easy & safe attempts were futile, the next step was to open the disk's case and look inside at what's really going on. During the previous attempts, I could hear a tiny sound of an attempt to read then failure, so hopefully now I'd be able to inspect further more.

After unscrewing all these tiny pesky screws with proper tools, it turned out that the disk's mechanical head was jammed. I tried nudging it and moving it with the screwdriver, but it seems it was stuck for good. This means, the whole disk housing is useless now, and the only way to get the data is by moving the disks from one hard disk to another. This was much tougher than I had anticipated and took me around 2 hours to carefully move the parts.

The disk heads rest at the edge of the disk, barely touching it, but they don't allow you to take the disks out. In order to do that, there's a white plastic piece, which has grooves where disks lay in, that should be removed first, in order to properly remove the disks. The heads, also partially rest on this plastic piece, and to take the piece out without damaging the heads, the heads must be moved to the inner-side of the disks -- This is by no means an easy task!!!

Pushing the heads by finger also got them bent. To be exact: The heads have small buds that hang by a very tiny piece of wire. These buds are the parts that touch the disks and read/write the data; the heads only work on moving these buds around the disks while the disks spin. I pushed one of the buds a bit to the top, to be able to push the heads safely to the inner part of the disk -- This was a major mistake. Pushing the bud inside rendered it dead and MY disk was no longer readable!!!

So, the whole salvation operation failed and resulted in 2 deaths. The only thing gained from this was experience on how to deal with these in the future, and the fact that I need a magnifier to be able to deal with the heads.

I should mention that I got my screwdrivers from: Al-Nisif warehouse, in Canada-Dry street, Shuwaikh. I'll get their exact info later.

Below are pictures I took while performing the operation.

Youtube not reachable, temporarily

Youtube stopped opening around 23:19, February 24^th.

At first I thought QualityNet has blocked Youtube, but they have no reason to. Further investigation showed that we got affected due to Pakistan blocking Youtube, and since we go through Pakistan, we got blocked as well.

Here's a traceroute of whole thing from QualityNet:

mj@mj-evil-station:~$ traceroute youtube.com
traceroute: Warning: youtube.com has multiple addresses; using 208.65.153.238
traceroute to youtube.com (208.65.153.238), 30 hops max, 40 byte packets
 1  dsldevice.lan (192.168.1.254)  81.140 ms  89.402 ms  100.769 ms
 2  62.150.109.1 (62.150.109.1)  199.651 ms  206.459 ms  261.711 ms
 3  rb02-internet.qualitynet.net (62.150.93.233)  371.742 ms  317.482 ms  353.407 ms
 4  jun-skb.qualitynet.net (62.150.200.5)  320.456 ms  225.162 ms  248.510 ms
 5  195.229.27.29 (195.229.27.29)  188.310 ms  298.991 ms  432.766 ms
 6  auh-emix-rb.emix.net.ae (195.229.31.3)  469.257 ms  299.604 ms  362.735 ms
 7  dxb-r2-ether-1-0-0.emix.net.ae (195.229.0.98)  406.317 ms 195.229.0.126 (195.229.0.126)  196.553 ms dxb-r2-ether-1-0-0.emix.net.ae (195.229.0.98)  269.163 ms
 8  195.229.0.181 (195.229.0.181)  355.469 ms  225.016 ms  226.647 ms
 9  pos2-0.ar01.hkg04.pccwbtn.net (63.218.61.137)  582.209 ms  514.871 ms pos2-1
.ar01.hkg04.pccwbtn.net (63.218.61.141)  550.151 ms
10  pos1-0.cr02.hkg04.pccwbtn.net (63.218.114.66)  392.507 ms  417.320 ms  373.513 ms
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  youtube.com.hk (208.65.153.238)  520.823 ms  418.294 ms  431.483 ms

mj@mj-evil-station:~$ traceroute 208.65.153.238
traceroute to 208.65.153.238 (208.65.153.238), 30 hops max, 40 byte packets
 1  dsldevice.lan (192.168.1.254)  45.403 ms  95.449 ms  99.379 ms
 2  62.150.109.1 (62.150.109.1)  179.348 ms  368.496 ms  342.110 ms
 3  rb02-internet.qualitynet.net (62.150.93.233)  369.112 ms  348.417 ms  253.605 ms
 4  jun-skb.qualitynet.net (62.150.200.5)  163.947 ms  367.323 ms  245.216 ms
 5  if-8-4.bb1.RSD-Riyad.teleglobe.net (66.198.126.45)  155.793 ms  41.123 ms  42.489 ms
 6  if-6-0.bb1.JSD-Jeddah.teleglobe.net (195.219.153.25)  48.791 ms  50.982 ms  103.248 ms
 7  if-11-0.mcore3.NJY-Newark.teleglobe.net (216.6.57.49)  430.598 ms  528.746 ms  583.103 ms
 8  if-2-0.core1.NTO-NewYork.teleglobe.net (216.6.57.66)  448.712 ms  545.164 ms 423.040 ms
 9  14.icore1.NTO-NewYork.teleglobe.net (216.6.82.2)  377.902 ms  403.390 ms  472.671 ms
10  209.58.26.30 (209.58.26.30)  575.457 ms  331.368 ms  379.346 ms
11  YOUTUBE-LLC.po1.401.ar2.SJC2.gblx.net (64.212.108.162)  645.748 ms  437.855 ms  420.526 ms
12  youtube.com.hk (208.65.153.238)  697.133 ms  447.682 ms  590.483 ms

From the above, you can see that the first attempt had a lot of unreachable hops, but they passed through the second time. This happened because I was attempting traceroute the first time while the routes to youtube were being cut/changed. The time between the 2 attempts was less than 10 minutes.

While writing this post, youtube seemed to work again. Some say it's not loading fully or some functions are not working.

Just checked from a FastTelco user, who was unable to view youtube at all. Here's a traceroute:

Tracing route to youtube.com [208.65.153.251]
over a maximum of 30 hops:

  1     1 ms     1 ms     1 ms  192.168.1.1
  2   724 ms   724 ms   716 ms  89.203.7.254
  3   670 ms   682 ms   786 ms  tec-core-so32.fasttelco.net [62.215.0.94]
  4   696 ms   752 ms   971 ms  62.215.0.57
  5   997 ms   748 ms   422 ms  78.159.161.9
  6   463 ms   192 ms   125 ms  195.229.29.133
  7   143 ms    73 ms   132 ms  dxb-emix-ra.ge1302.emix.ae [195.229.31.67]
  8   117 ms   100 ms    77 ms  195.229.0.181
  9   245 ms   246 ms   386 ms  t2a6-p5-3.uk-lon2.eu.bt.net [166.49.160.161]
 10   281 ms   267 ms   250 ms  t2c1-ge7-0.uk-lon2.eu.bt.net [166.49.176.43]
 11   374 ms   321 ms   249 ms  t2c1-p3-0.uk-glo.eu.bt.net [166.49.208.98]
 12   250 ms   250 ms   244 ms  t2c1-p9-1.uk-eal.eu.bt.net [166.49.208.125]
 13   275 ms   271 ms   265 ms  t2c1-p5-0-0.us-ash.eu.bt.net [166.49.164.65]
 14     *        *        *     Request timed out.
 15     *      292 ms   260 ms  10.33.112.70
 16   300 ms   272 ms   317 ms  10.33.254.141
 17   379 ms   399 ms   378 ms  10.33.254.117
 18   503 ms   388 ms   510 ms  10.33.176.114
 19   383 ms   337 ms   330 ms  youtube.com.hk [208.65.153.251]

More details on the ban from ZDNet.

High File Compression

I've been toying with 7-zip yesterday to see how far I could compress a file.

From the tests I did, 7-zip is capable of doing 7088:1 compression ratio. Here's the run:

mj@mj-evil-station:~/share$ dd bs=1073741824 count=10 if=/dev/zero | 7z a -mx9 -si haha.zip

7-Zip 4.43 beta  Copyright (c) 1999-2006 Igor Pavlov  2006-09-15
p7zip Version 4.43 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,1 CPU)

Updating archive haha.zip

Compressing  [Content]                                                  0%
10+0 records in
10+0 records out
10737418240 bytes (11 GB) copied, 2465.64 seconds, 4.4 MB/s

mj@mj-evil-station:~/share$ ls -l
-rw-r--r-- 1 mj     mj        1514800 2008-02-19 13:03 haha.zip

As you can see, I stuffed 10GB (not 11, notice the block size parameter) into a 1.4MB file. This was on a laptop. At home, my dual-core box stuffed 100GB into 14.4MB in 3 hours, 39 minutes & 30 seconds. (while watching a video file, consuming some CPU cycles)

I could probably optimize it more, by tweaking parameters of 7-zip, unfortunately it requires a lot of RAM, which I don't have (limited to 2GB only).

The tests done were simply stuffing zeroes in a text file, and then compressing the file. This is an ideal case, since all characters are the same, hence producing the highest rate of compression (provided you tweak your program to the max).

I have a couple of machines at work that are quad-core with 6GB of RAM, unfortunately they have Windows installed, so I don't have a device to pump zeroes into 7-zip directly, instead, I have to create a file of the size I need, then have 7-zip compress it; quite inconvenient.

Making Bootable USB Drives Recognizable on Windows, Again

A couple of weeks back, I "borrowed" my sister's 256MB USB flash memory and put Slax on it.

Now, my sister wants her "USB" back, which I tried to explain that if I give her just the USB part it won't work as intended. As funny as I was, she didn't think so, and she wanted the WHOLE thing back. And working.

Windows didn't recognize the whole device and couldn't even see it as a removable disk, so formatting was not an option, yet.

Using a Windows XP bootable CD and trying to use fixmbr failed, because the recovery console failed to see the USB drive.

I fixed it on Linux, by using "dd" to write zeros on the disk (/dev/sdf in my case), then used fdisk (not cfdisk) to write a DOS partition table, then created a new primary partition with type "Windows 95 FAT" (b).

root@adrenalin:~# dd if=/dev/zero of=/dev/sdf
dd: writing to `/dev/sdf': No space left on device
517121+0 records in
517120+0 records out
264765440 bytes (265 MB) copied, 110.158 s, 2.4 MB/s

root@adrenalin:~# fdisk /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF
disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by
w(rite)

Command (m for help): p

Disk /dev/sdf: 264 MB, 264765440 bytes
9 heads, 57 sectors/track, 1008 cylinders
Units = cylinders of 513 * 512 = 262656 bytes

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help): o
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by
w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1008, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1008, default 1008):
Using default value 1008

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): b
Changed system type of partition 1 to b (W95 FAT32)

Command (m for help): p

Disk /dev/sdf: 264 MB, 264765440 bytes
9 heads, 57 sectors/track, 1008 cylinders
Units = cylinders of 513 * 512 = 262656 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1        1008      258523+   b  W95 FAT32

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.

The disk is fixed, but doesn't have a filesystem. You can either make one in Linux, or slap it on a Windows machine & format it.

To make a FAT32 filesystem on Linux, run:

mkfs.vfat -F 32 /dev/sdf1

GMail Acting Up

A couple of friends have told me that when they sign in to their gmail account, they get other people's inbox! Hitting refresh, shows yet another inbox.

The good news is that whenever clicking on an email message, or a label, or a directory, the session expires and you get logged off.

The bad news is that people can see emails in your inbox and can see the emails of those who sent you the emails.

I searched around and so far, no one has reported this!

K.The Kuwaiti had reported this issue here.

The problem seems to be facing those using FastTelco ISP connection, both home & businesses. Seems like they're using transparent proxies & caching data.
I confirmed it with those who told me about these symptoms, and they were using FT connections.

As for the post at 248AM blog, I don't agree with the suggestions in that post and have posted a correction.

Here's my my take at the issue:

"Tor was abused and embassy information was stolen through it. The same goes for proxies. They're simply someone else's machine.

By the way, even when an ISP is using a cache or a transparent proxy, it should NOT cache SSL (encrypted) pages. It can't. Simply because the session exists between the user (you) and the server (gmail, for example) and the cache server has nothing to see.

Try accessing GMail using this link: https://mail.google.com instead of using gmail.com -- This will work because gmail.com will only use SSL to authenticate then redirect you to your inbox using HTTP, where if you use https://mail.google.com, it will keep the SSL session and log you to your inbox with an encrypted channel.

The same applies for banking sites using SSL."

Update: Tuesday, Feb 19^th: According to those who were facing the problems, they have stopped now.

RAID Expansion and The Beauty of XFS

Existing Hardware Setup

My workstation has 4 SATA2 hard disk drives installed: 3x 320GB & 1x 80GB. The first 3 are on RAID arrays, while the last is a standalone partition with Windows to play games once in a while. The RAID arrays is managed by the Linux kernel, which is known as software RAID.

Existing Storage Capacity

The total storage capacity of the array was 592GB, of which 50GB was free. As I haven't finished my work on the NAS box, I had no place to store my future stuff, so I needed storage urgently, and had to add another disk into the existing array. As I hope to get my NAS box soon, I bought only one 320GB HDD.

Stage 0: HDD Installation & Partitioning

Since I had no RAID controllers, adding the hard disk while the PC was running was out of the question. So, after turning off the box, I added the new hard disk, and booted the box. Then, I partitioned the new hard disk into 2 partitions: boot & data, where boot occupied 100MB only.
This is different from the previous 3 HDDs, since they had a swap partition of 2GB per disk. I didn't add a swap partition in the new one. (And yes, there's 2GB of unused space).
This is how the new disk looked after being partitioned:

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1   *           1          12       96358+  fd  Linux raid autodetect
/dev/sde2              13       38913   312472282+  fd  Linux raid autodetect

Another way to look at it would be:

Name    Flags    Part Type   FS Type                     Size (MB)
------------------------------------------------------------------
sde1    Boot      Primary    Linux raid autodetect           98.71
sde2              Primary    Linux raid autodetect       319971.62

* The first print was produced using fdisk, the second using cfdisk.

Stage 0.5: Remove Shadowed Mounts

My /boot is a RAID1 array that is mounted over the existing /boot, and it's mounted as read-only. So, sense I'll be expanding the root filesystem (/) I thought of removing all un-needed filesystems, just in case!

Stage 1: Closer Than Veins

Now it's time to add partitions to their RAID arrays. If you don't wish to suffer like I did, change the minimum speed limit as follows:

echo '150000' > /proc/sys/dev/raid/speed_limit_min

Then, I added the boot partition to the array:

mdadm /dev/md0 -a /dev/sde1
mdadm --grow /dev/md0 --raid-disks=4

The first line will add the disk as a spare one only, and not an active part of the array. The second line makes it part of the array and mirrors the data to the new member of the RAID1 array.

Then, I added the data partiton to the RAID5 array, in a similar matter to the previous partition:

mdadm /dev/md1 -a /dev/sde2
mdadm --grow /dev/md1 --raid-disks=4

Because the RAID5 array is big, rebuilding it takes a lot of time. Mine took about 8.5 hours! The rebuilding process starts as soon as you execute the 2^nd line. You can monitor the process of rebuilding the array by:

cat /proc/mdstat

You'll get something looking like these:

[=>...................]  reshape =  7.7% (24075264/310472064) finish=583.2min speed=8184K/sec
[===>.................]  reshape = 19.8% (61607172/310472064) finish=582.5min speed=7116K/sec

Neat, right? Indeed, but the work isn't done yet!

Stage 2: Command and Conquer

Occupying the newly available free space is the last thing to do. After roughly 8.5 hours of rebuilding the array, it's finally online and ready to be abused by yours truly.
As the title of this post suggests, I use XFS filesystem, and one great feature that it offers is the ability to grow the filesystem online: No need to unmount the filesystem.
Quoting the manual page:

The filesystem must be mounted to be grown (see mount(8)). The existing contents of the
filesystem are undisturbed, and the added space becomes available for additional file
storage.

This is how things went:

root@adrenalin:~# xfs_growfs /
/dev/root: No such file or directory
Usage: xfs_growfs [options] mountpoint

Options:
        -d          grow data/metadata section
        -l          grow log section
        -r          grow realtime section
        -n          don't change anything, just show geometry
        -I          allow inode numbers to exceed 32 significant bits
        -i          convert log from external to internal format
        -t          alternate location for mount table (/etc/mtab)
        -x          convert log from internal to external format
        -D size     grow data/metadata section to size blks
        -L size     grow/shrink log section to size blks
        -R size     grow realtime section to size blks
        -e size     set realtime extent size to size blks
        -m imaxpct  set inode max percent to imaxpct
        -V          print version information
root@adrenalin:~# xfs_growfs -t /etc/mtab /
meta-data=/dev/md/1              isize=256    agcount=45, agsize=3449690 blks
         =                       sectsz=4096  attr=0
data     =                       bsize=4096   blocks=155236032, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096
log      =internal               bsize=4096   blocks=16384, version=2
         =                       sectsz=4096  sunit=1 blks
realtime =none                   extsz=262144 blocks=0, rtextents=0
data blocks changed from 155236032 to 232854048
root@adrenalin:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md/1             889G  543G  347G  62% /
/dev/sdd1              67G   22G   46G  33% /crap
/dev/md/0              92M   16M   71M  19% /boot
root@adrenalin:~# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md1 : active raid5 sde2[3] sdc3[2] sdb3[1] sda3[0]
      931416192 blocks level 5, 128k chunk, algorithm 2 [4/4] [UUUU]

md0 : active raid1 sde1[3] sdc1[2] sdb1[1] sda1[0]
      96256 blocks [4/4] [UUUU]

unused devices: 
root@adrenalin:~# cat /etc/raidtab
raiddev                 /dev/md0
raid-level              1
nr-raid-disks           4
nr-spare-disks          0
persistent-superblock   1
chunk-size              32

device                  /dev/sda1
raid-disk               0
device                  /dev/sdb1
raid-disk               1
device                  /dev/sdc1
raid-disk               2
device                  /dev/sde1
raid-disk               3

raiddev                 /dev/md1
raid-level              5
nr-raid-disks           4
nr-spare-disks          0
persistent-superblock   1
parity-algorithm        left-symmetric
chunk-size              128

device                  /dev/sda3
raid-disk               0
device                  /dev/sdb3
raid-disk               1
device                  /dev/sdc3
raid-disk               2
device                  /dev/sde2
raid-disk               3

MacBook Air vs. Lenovo X300

A while ago, a few friends showed me a video on youtube on Apple's release of their new ultra-thin & sexy MacBook Air.

Later on, Gizmodo had some *leaked* info & specs on Lenovo's ultra-thin Thinkpad X300.

Thinkpad X300	MacBook Air
+Battery is swappable	-Battery can't be changed
+Can add 2nd battery	-Can not add another battery
+Built-in DVD writer	-CD/DVD: Sold as an accessory
+Built-in fingerprint reader	-No fingerprint reader
+Max of 4GB RAM @ 667MHz	-Max of 2GB RAM @ 667MHz
+Max CPU speed: 2.0GHz, 800MHz FSB, 4MB L2 Cache	-Max CPU speed: 1.8GHz, 800MHz FSB, 4MB L2 Cache
+Max resolution of: 1440x900 @ 128 DPI	-Max resolution of: 1280x800
+#USB ports: 3	-#USB ports: 1
+#Mini PCI-Express: 2 Full, 1 Half	-#Mini PCI-Express: None
+Weight: 1.13 kg / 2.5 lbs	-Weight: 1.36 kg / 3 lbs
+Antennas: Bluetooth, WiFi 802.11n, WiMAX, GPS, WWAN	-Antennas: Bluetooth, WiFi 802.11n
+Wired network: Gigabit Ethernet port	-Wired network: Sold as an accessory
+Enclosure material: Magnesium-alloy for a sturdy & durable cage	-Enclosure material: Aluminum
+Built-in camera	+Built-in camera
-Video output: VGA	+Video output: DVI, VGA
-HDD: 64GB Solid State Disk	+HDD: 64GB Solid State Disk, 80GB PATA
-Looks: Sturdy, Rigid	+Looks: Cute, Sexy
-31.8x23.1x2.34 cm	+32.5x22.7x1.94 cm
Keyboard illumination: Backlit keyboard	Keyboard illumination: Backlit keyboard with sensor

Seems like the Apple crowd are sacrificing a lot here...

* Lenovo's laptop isn't out for consumers yet.
** Thinkpad's are IBM laptops. Lenovo handles them now, instead of IBM.