Upgrade Log 3

The last one for this batch! Everything arrived, assembled, and finished without much problem.

Windows 10 is even more annoying than ever. Disabling Cortana now must be done using Group Policy. Great. I have to slowly live it up because this is the future of Windows and I don’t see myself using another operating system for desktop for foreseeable future.

Also, don’t disable universal app background process if you want a functional start menu search.

<insert a bunch of other tweaks here>

Up next

Closest upgrade I can think of is getting an extra 6+To drive so I have 6 drives raidz2 instead of current 5 which is quite a waste. I’m not sure how to migrate the data though. That’ll cost about 25k?

And I remembered about my netbook only having 2Gio of RAM. Surely can be upgraded to 8Gio for maximum lulz. Or just more useful. I remember it’s much more usable when it’s running on 4Gio of RAM. I don’t exactly remember when and why it’s only 2 now. It already has SSD so the RAM upgrade would pretty much max out upgrades for this system. Not counting higher capacity/performance SSD because I don’t think it won’t make much difference apart of having more storage – faster SSD won’t help the slow CPU much. 5k for RAM.

After that, I can certainly use more storage for my main desktop. A 1To SSD would certainly be nice. A bit expensive at 33k.

With storage out of the way (and moves the 525Go drive to office desktop), I think my office server can also use some storage upgrade. Just like current home server, it can certainly use two more drives for optimum raidz2. That means a controller, HDD cage, and one extra HDD (because I already have one spare 3To HDD). The total would be about 51k.

There’s VGA card upgrade for main desktop but I’m still not sure about that. I don’t really need it but certainly would be nice! Let’s pretend it’ll cost 40k for whatever card at that budget whenever the upgrade is happening.

Talking about VGA card, there’s also a would-be-nice upgrade for my office desktop VGA. It’s currently running GT730 which is not quite fast. Limited to 45W, current choice is limited to GT1030 at 10k.

At this point there isn’t much left to be upgraded. So let’s upgrade the server RAM to 32Gio from currently pitiful 12Gio. I would like to pretend it’s cheap but it really isn’t even now. I was pretty lucky last time getting two sticks of 8Gio for just 10k but it won’t happen often. So maybe about 25k I’d be willing to spend.

I think there is no more after this. I probably won’t reach this far until at least next year or even later anyway and something may break in the meantime, requiring change of plan.

  1. (5k) RAM: 8Gio PC3-12800S
  2. (25k+) Storage: 6+To HDD
  3. (33k) Storage: 1To SSD
  4. Storage:
    • (4k) Controller: LSI SAS 9212
    • (7k) Misc: HDD Cage 2 5.25″ to 3 3.5″
    • (15k?) Storage: 3+To HDD
  5. (40k) VGA card: ???
  6. (10k) VGA card: GT1030 (or better)
  7. (25k) RAM: 32Gio PC3-12800E

Total: 164k.

…maybe this will happen sooner than expected ( ゚◡゚)

VirtualBox again

Due to reasons I swapped my web server (was in VM) and VM box role. Or more like web server is now bare metal and functions as VM box. It’s downgraded from real virtualization (Hyper-V) to VirtualBox though.

That said, FreeBSD doesn’t seem to be quite fully functional on Hyper-V thanks to performance hit and slow disk detection on boot causing failure.

Yeah, I’ve had enough of it so I decided to swap the role and run the web server directly on the server instead.

On the bright side, I can use zfs for everything and no more stupid SSD setup (it was unbalanced 500 and 250 GB disks).

VirtualBox seems to be flaky as always though apparently once it’s setup correctly it runs without much trouble. Also thankfully VirtualBox on FreeBSD has complete scripts for running it headless and autostart on boot.

This was also my main setup for quite a long time years ago and I sure hope it’s at least as stable as it was back then! That said, the initial setup seems to be flakier. Oh well.

Memo and stuff.

Ruby 2.3 on FreeBSD 11

Compiling Ruby on FreeBSD is not quite simple.

  • make sure to tell it to also find libraries in /usr/local
  • and tell configure script to find OpenSSL in /usr because the later version isn’t quite compatible with latest ruby yet

So here’s the configure line

CFLAGS=-I/usr/local/include \
CPPFLAGS=-I/usr/local/include \
LDFLAGS=-L/usr/local/lib \
./configure --prefix="/opt/ruby23" --disable-install-doc --with-openssl-dir=/usr \
&& make \
&& make install

PSA: Never trust external X-Forwarded-For

For god knows how long, proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; is one of the line usually included in nginx config snippet for proxying to a unicorn (Rails) backend.

…which is something you should never do unless you have another load balancer in front of the nginx being configured.

That line basically tells nginx to append $remote_addr to whatever X-Forwarded-For value currently set. It is only useful when your nginx is behind other load balancer which set up its own (hopefully correctly) X-Forwarded-For. It should be set explicitly to $remote_addr for any external-facing proxy. Otherwise fun things will happen.

FreeBSD pkg (manual) upgrade thingy

For working with locked packages (the ones which must be upgraded through compilation because of using custom options)

#!/bin/sh
# filename: pkg-lock-outdated

pkg query -e '%k = 1' %o | while read pkgorig; do
  pkg version -ovL => -O "$pkgorig"
done

The script above is to list locked packages which need upgrading. And to upgrade everything at once (and sit in front of PC waiting for whole process)

#!/bin/sh

listfile="/tmp/pkgforupgrade.$(date '+%Y%m%d%H%M%S')"
pkg-lock-outdated | cut -f 1 -d '<' > "$listfile"

while read <&3 outdated; do
  pkg unlock "$outdated"
  portmaster "$outdated"
  pkg lock "$outdated"
done 3< "$listfile"

rm -f "$listfile"

There’s another alternative of unlocking all packages at once, run batched portmaster, and lock them all again.

WordPress 3.5 Initial Impressions

This blog has been updated to WordPress 3.5. Initial impressions:

  • everything is less rounded now
  • Twenty Twelve (the new theme) sucks for following reasons:
    • uses custom font: causes this blog unreadable until fonts are loaded (at least in chrome)
    • image header: positioned below navigation menu (and the header text is above the menu)
    • no more search bar in navigation menu
  • the new media uploader looks useful

Overall there isn’t much change (what should I expect here) but the new theme sucks. At least to be used on this blog.

PostgreSQL authentication quick start

Connecting to PostgreSQL from command line can be a bit confusing.

For starter, just like MySQL, psql command defaults to connecting to socket instead of tcp. To make matter confusing, most PostgreSQL installation defaults to ident (also called peer)authentication for socket connection: it basically matches current user’s username (ssh login, etc) with PostgreSQL equivalent.

So, instead of using this to login from root to PostgreSQL superuser (usually named postgres or pgsql):

# psql -U postgres

you do this (assuming sudo installed):

# sudo -u postgres psql

The configuration for this is located in pg_hba.conf of PostgreSQL data (or config in Debian) directory (/etc/postgresql/$version/main in Debian, /usr/local/pgsql/data in FreeBSD, /opt/PostgreSQL/$version/data in EnterpriseDB PostgreSQL).

To switch to password based authentication for all methods just replace ident (or peer) with md5 in respective lines and reload/restart the service. Don’t forget to set password for postgres user first before changing this otherwise you won’t be able to connect. You can then connect using psql to any user using password.

Windows 8 Initial Impressions

6 hours with Windows 8.

  • Metro Modern UI is shit (the applications)
  • The Start Menu (or whatever it’s called now) is ok-ish though
  • The lock screen is awesome with its customizable background image
  • The global one is difficult to change though
  • File association was updated and doesn’t work properly in at least IrfanView and 7-Zip
  • Compressed folder still exists and can be disabled the same way as Windows 7
  • So is Aero Shake. Really though, there is no more Aero but there’s still Aero Shake? Is it some kind of joke?
  • The window border is thick-ish but I guess OK for now (quick google also showed some border customizer)
  • Hyper-V is in it as promised. Works as expected. Most OS need Legacy Network Adapter (except Ubuntu and SLES) or installation of the integration driver for some OSes. The biggest plus is it works at system level – VMs can be automagically started as Windows starts
  • All applications I usually use works without problem
  • It’s not move to (top right) corner but move to corner and move down a bit
  • Updates (from Microsoft Update) which requires license agreement doesn’t work (at least on my PC)
  • There’s no discernible performance difference
  • Yes, Everything also works
  • Also, flat
  • Windows Explorer is a bit better: no more intrusive action buttons above file list (moved to the top)
  • Remote desktop work OK. There’s no more classic style but I think the new one (Modern UI) doesn’t use too much bandwidth (being flat and all)
  • WRT RDP, the 2X Client I usually use in Android doesn’t work with Windows 8. The good old Remote RDP Lite works though (and with CyanogenMod’s plain keyboard, the physical keyboard input works again)

I think that’s it for now.

Zeropaste – the featureless pastebin

Tonight, when trying to compile Rubinius with Rubinius, I got some errors and wanted to report to relevant party. As usual, the log of what happened is required so I used my usual pastebin – pastie.org to send the logs. But then I noticed that the “Raw” link in it doesn’t provide an actual raw file anymore. What the fuck. It is now a html disguised as txt.

So I decided to whip up a new pastebin (because the world need one more pastebin) which doesn’t have any actual features (like tag highlighting, etc). I also learned the way to create shortest RESTful path possible (read: ‘/’).

There may or may not be more features coming. Developed in Rubinius because I can. Using mysql because of where it’ll be deployed at (see below).

I’ll get around deploying this soon after reinstalling VPS running this blog to Ubuntu or Debian. Running yum in a 128 MB box is suffering.

[ Source Code ]

Ruby 1.9, Rails, and UTF-8

(main purpose of this post is to link this “server error” page of rubygems.org)

The relevant issue in Rails Issue Tracker (3789). AFAICT, there are few ways to “fix” (read: workaround) this:

  • Modify the relevant Rack code to handle this crap
  • Create additional middleware to intercept (how?) the request (tried, either didn’t work or horribly inefficient)
  • Extend rack before it is started

Well, they all sucks. Hopefully someone comes up with actual working solution for this.

Oh, there’re another solutions:

  • Use REE 1.8 (really?)
  • Use JRuby in 1.8 mode
  • Use Rubinius (rbx 2.0 where?) in 1.8 mode

Um, yeah.

Update: I figured out how to “fix” it. Check it out in Moebooru (requires this).

Rails: read_multi and dalli

Be careful when using read_multi with dalli: it may return nil-valued key instead of the correct key.

The issue is tracked here and thanks to this I dropped the read_multi usage in moebooru and used the much simpler (and most likely slower) single fetch (per entry) instead. There’s alternative way to use it – do a read_multi and refetch whatever missing/nil-keyed but apparently I’m too lazy to do it.

Disabling Upstart Service in Ubuntu (11.04+)

Took me few weeks to find out that this one-liner does wonder:

echo manual >> /etc/init/mysql.override

(the line above is to disable mysql, obviously. And must be done as root)

The answer is on first hit (as of this post’s writing) of googling “ubuntu disable service” but you need to scroll down a bit and ignore shitload of crappy, outdated explanations to find that small gem.

Unfortunately doesn’t apply to previous LTS. Or does it?

zpool hourly status check

I’m setting up cron job for a storage server using ZFS. There’s zpool status -x but it returns “all pools are healthy” (or “no pools available”) on no error and prints errors to stdout (instead of stderr), rendering it annoying for cron job.

#!/bin/sh

set -e
set -u

zstatus="`zpool status -x 2>&1`"
case "${zstatus}" in
  "all pools are healthy"|"no pools available")
    return 0
  ;;
  *)
    printf "%sn" "${zstatus}" >&2
    return 1
  ;;
esac

Put it in a file (e.g. /root/bin/zpool-status-cron), make it executable, and add it to crontab.

bcrypt in Debian

WARNING: using method below will lock yourself out when using emergency console since whatever crypt it’s using surely doesn’t understand bcrypt (as I experienced myself). Additionally, this solution won’t add bcrypt support to other applications using crypt interface like proftpd unless it’s started by preloading libxcrypt.so first (also from my own experience).

As much as Drepper want to pretend bcrypt is wrong solution, it actually gives one benefit: ease of switch to Linux. Some systems use bcrypt by default or configurable to use it. On other case, there might be time where you need system’s (or applications using system’s) crypt to handle bcrypt passwords from external system (usually web applications).

It’s quite difficult to enable bcrypt support in RHEL based distro as there is no libxcrypt and pam_unix2 packages available. Thankfully it’s available in Debian (and derivatives) in package libpam-unix2.

The README.Debian says to modify files in /etc/pam.d but if I remember it correctly, it confused apt PAM handling system or whatever. Fast forward few weeks, I discovered a better way to use it by creating PAM configuration in /usr/share/pam-configs. Since it’s mostly equivalent to normal pam_unix, I just copy and modify the file using this (long-ass) oneliner sed:

sed -e 's/pam_unix.so/pam_unix2.so/g;s/^Name: Unix authentication$/Name: Unix2 authentication/;s/pam_unix2.so obscure sha512/pam_unix2.so obscure blowfish rounds=8/;s/ nullok_secure//' /usr/share/pam-configs/unix > /usr/share/pam-configs/unix2

Then execute pam-auth-update, select Unix2 authentication and deselect Unix authentication. Don’t forget to update passwords for all other users as well or they won’t be able to login since pam_unix2 doesn’t recognize sha based hashes.

Actually, change all other users password to use md5 first before replacing the PAM with pam_unix2.

Update 2012-04-01: Removed nullok_secure since it isn’t supported.

Update 2012-06-09: Added warning.

FreeBSD is Rolling Release (the ports)

Don’t get tricked by the “release” system. Apart of the base system, FreeBSD perfectly qualifies as rolling release. I guess it’s also why the binary package management sucked so badly. You won’t find how to upgrade certain packages using binary method in their Ports’ UPDATING page.

Here’s the example:

20120225:
  AFFECTS: users of archivers/libarchive
  AUTHOR: glewis@FreeBSD.org

  libarchive has been updated to version 3.0.3, with a shared library bump.
  This requires dependent ports to be rebuilt.

  # portmaster -r libarchive
  or
  # portupgrade -r archivers/libarchive

You would think the dependent packages got version bump to ensure their proper dependency – but they didn’t. Instead you had to recompile everything depending on it.

And then there’s another case:

20120220:
  AFFECTS: users of graphics/libungif
  AUTHOR: dinoex@FreeBSD.org

  libungif is obsolete, please deinstall it and rebuild all ports using
  it with graphics/giflib.

  # portmaster -o graphics/giflib graphics/libungif
  # portmaster -r giflib
  or
  # portupgrade -o graphics/giflib graphics/libungif
  # portupgrade -rf giflib

Of course, ArchLinux kind of managed to do it but that’s a purely binary rolling release Linux distro. The maintainer worked hard to ensure such kind of thing get handled properly by all their users which mostly use binary packages. FreeBSD on other hand tried to claim capable of both but it really isn’t (unless I missed something).

I’m intending to contact pkgng creator to ask his opinion about this but have yet to do it…

Removing Annoying Speaker Static Noise

I’m not sure which sound cards are exhibiting this problem but at least it is in my system (onboard Realtek HD – Intel DH61BE motherboard running Windows 7 x64). It’s been annoying me since like forever and finally tonight I decided to actually solve the problem.

As it turns out, the solution is quite simple: disable PC Beep channel. A quick google showed this hit quite a bit of people and apparently this is the reason (or at least related).

On related note, apparently I’ve did this before and then completely forgotten. This is why I wrote it this time.

nginx – gzip all the text

During my migration to other server, I recreated some of my configs and enabled gzip compression for most file types. Here’s the relevant config:

gzip on;
gzip_vary on;
gzip_disable "msie6";
gzip_comp_level 6;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/xml application/xml application/json application/x-javascript text/javascript text/css;

It should cover most text-based content one will ever serve over the web. Probably.

On Comment System

I just migrated to Disqus – a fully managed comment system for websites. I saw it first used in Engadget. It didn’t work quite good at the time (or at least I didn’t have good memory on it) but it’s quite wonderful now.

One of the problem I’m having when leaving comment on other blogs is it’s difficult to track which posts and sites I’ve left a comment on. Some sites (like this site used to be) provide “Notify via E-mail on Replies” option but it’s clunky at best and with dozens, hundreds of sites out there it’s quite impossible to track them all. Not to mention you have to visit each blogs to unsubscribe from notification.

Then come Disqus – it’s a centralized comment system which allows any website to use their service and let the users enjoy one-stop interface to manage all their comments on various websites. It uses JavaScript to embed the comment interface on a page – not the best way but I guess it’s acceptable now with emergence of smartphones and tablets which actually capable of rendering JS.

Replacing WordPress’ comment system with Disqus is quite easy. The official WordPress plugin provides everything to migrate comments quickly and easily. It can even keep the comments synced with local database – allowing quick way out in case Disqus goes evil™.

I doubt anyone still read this blog (and blog is so 2009) but well, here it is.

Server changes, etc

Apparently now this blog is blazing fast. I’m not sure why but I sure can feel the difference.

I’ve done lots of changes on this site so I’m not sure which one did the trick most:

  • Not using WordPress Multisite
  • Moved to 64-bit OS
  • gzip-ed everything (er, most things)
  • Used MySQL 5.5

Yeah, those are the changes. Also this site moved to all my four VPS’. Last (this) one is in Hostigation. Hopefully the won’t have the network problem like the one occurred sometime last month anymore. I enabled collectd with ping plugin so I’ll have concrete data on what’s happened next time there’s network problem.

…and as it turns out, I set the expiration overly aggressive resulting in caching even the main page. Epic fail on my side.

nginx/php single config for SSL and non-SSL connection

This morning I noticed I haven’t upgraded WordPress MU Domain Mapping plugin to the latest version. It supposedly brings better SSL support. And after upgrading I couldn’t log in to my mapped domain blogs (e.g. this blog). Wasn’t it a great way to start my morning?

After some digging, I found out the problem was because I don’t have one PHP(?) parameter – HTTPS – passed properly. It should set to True whenever one is using SSL connection otherwise there’s no way the PHP process can know if the connection is secure or not. Previous version of WPMUDM have a bug in which skips SSL check but in turn enables using HTTPS even without such parameter. Decided it’s my fault (I believe it would completely breaks phpMyAdmin), adding the parameter then I did.

But it’s not that simple: I’m using unified config for both my SSL and non-SSL connection’s PHP include. Splitting the config would make the duplication worse (it’s already relatively bad as it is) so that’s not an option. Using the evil if is also not a solution since it doesn’t support setting fastcgi_param inside it.

Then the solution hit me. The map module – a module specifically made for things like this and to avoid usage of if. I tested it and indeed worked as expected.

Here be the config:

...
http {
  ...
  map $scheme $fastcgi_https {
    https 1;
    default 0;
  }
  ...
  server {
    ...
    location ~ .php$ {
      ...
      fastcgi_param HTTPS $fastcgi_https;
    }
...

And WordPress MU Domain Mapping is now happy.

Update 2012-02-20: nginx version 1.1.11 and up now have $https variable. No need to have that map anymore.

This pile of crap called OpenLDAP

In attempt to learn THE directory service called LDAP, I tried to setup OpenLDAP in Scientific Linux. The install went all right and slapd can be immediately started without much problem. Except that the config is one big mystery and there’s not even a rootpw defined by default. Being a complete newbie in LDAP thingy, I decided to build configuration and all from zero.

…except that it’s not actually trivial. Most examples/tutorials are for OpenLDAP prior to 2.4 which still uses slapd.conf which was obsoleted in favor of configuration in meta-format using LDAP’s ldif. Instead of one nice config, instead we have directories called cn=config etc inside slapd.d. Someone must’ve been into Linux too much (xxx.d – Linux users sure love “modularizing” their configs).

Anyway, the example in manual page of slapd-config doesn’t even work because the include syntax was wrong (should be file:///etc/… instead of /etc/…) and even after fixing that there still an error:

[root@charlotte openldap]# slapadd -F /etc/openldap/slapd.d -n 0 -l initman.ldif
str2entry: invalid value for attributeType olcSuffix #0 (syntax 1.3.6.1.4.1.1466.115.121.1.12)
slapadd: could not parse entry (line=626)
_#################### 100.00% eta   none elapsed            none fast!
Closing DB...
[root@charlotte openldap]# slaptest
slaptest: bad configuration file!

The example from the guide also gives exact same error.

In short, I kind of given up and tried to follow the “Quick Start” from the very same guide. Instead of using slapd.d format, it still uses slapd.conf format despite it being a guide for 2.4. Seems like following a pattern, the config example also spit out error:

[root@charlotte openldap]# vi slapd.conf
[root@charlotte openldap]# slaptest
/etc/openldap/slapd.conf: line 2:  invalid DN 21 (Invalid syntax)
slaptest: bad configuration file!

So much for an example. Few attempts later at both methods, I gave up and wrote this post.

WordPress Multisite with nginx (subdomain/wildcard domain ver.)

WordPress Multisite, previously known as WordPress MU (Multi User), is an application which allows hosting multiple WordPress blogs with just one installation. Instead of creating copies of WordPress for each users’ blogs, one can use one installation of Multisite to be used by multiple users, each with their own blogs. Personal/custom domain is also possible as used for this blog (this blog’s master site is genshiken-itb.org). Too bad, the official documentation only provided guide for installing on Apache. If you haven’t known, I usually avoid Apache – I simply more proficient with nginx. Of course, this blog is also running on nginx therefore it’s perfectly possible to run WordPress Multisite on nginx.

At any rate, reading the official documentation is still a must and this post will only cover the nginx version of Apache-specific parts (namely Apache Virtual Hosts and Mod Rewrite and .htaccess and Mod Rewrite) and only for subdomain install. Subdirectory one will or will not follow some time later.

Assuming you have working nginx and php-cgi (with process manager like php-fpm or supervisord), for starter you’ll want to create a specific file for this WP install. Let’s say this file named app-wordpress.conf. Obviously you have put WordPress installation somewhere in your server. In this example I put the files in /srv/http/genshiken-itb.org/.php/wordpress. Its content:

client_max_body_size 100m;
root /srv/http/genshiken-itb.org;
location /. { return 404; }
location / {
  root /srv/http/genshiken-itb.org/.php/wordpress;
  index index.php;
  try_files $uri $uri/ /index.php?$args;
  rewrite ^/files/(.*) /wp-includes/ms-files.php?file=$1;
  location ~ .php$ {
    try_files $uri =404;
    fastcgi_pass unix:/tmp/php-genshiken.sock;
    fastcgi_read_timeout 600;
    fastcgi_send_timeout 600;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $request_filename;
    include fastcgi_params;
  }
}

Simple enough. Actually, it’s exact same with normal WordPress install except one extra line:

rewrite ^/files/(.*) /wp-includes/ms-files.php?file=$1;

And you’re set. Note that I set fastcgi timeouts higher than default to work around the slow performance of Amazon EC2 Micro Instance. Should only needed on network upgrade and massive blog import.

Anyway, in your main nginx.conf file, put:

server {
  listen 80;
  server_name *.genshiken-itb.org;
  include app-wordpress.conf;
 }

In proper place. The usefulness of separate file for WordPress configuration will become apparent once you want to tweak performance for some blogs. I’ll explain that later if I feel like to.